A local AI Assistant for Linux called Meera (and a recipe for building something similar)

Hi,

I'm a hardware engineer who has coded in Verilog all my career. But ever since I got myself an RTX 5090 last year, I started experimenting with software and running local LLMs.

An AI agent for Linux desktop sounded like a good project to take up and I started on it back in November and many weekends later, I think it is in a good enough shape to share.

AI usage - Many forums have seen lots of AI assisted projects lately, so I want to be upfront: I used Cursor to help me code. But as someone new to software development, I went through many iterations to get it working reliably, even changed architecture mid-journey and learnt a lot in the process. Anyways, hope you give it a shot.

What is it?

• It's a AI assistant for Linux Gnome Desktop. The installer will set everything up, including llama-cpp and the models and it runs completely offline.
• I wanted to also see how much use I can get from a tiny model that can run on practically anything, so the chosen model for it was Qwen3.5 with 2B parameters.

Github Repo: https://github.com/achinivar/meera

For non linux users

While the app is only useful on linux - somebody new to local LLMs (like I was a few months back) might benefit from either this architecture for a local agent (documented in the wiki) - or the code itself can be re-purposed to work with your own set of tools and data (the only directories you'll need to modify is "tools" and "rag_data"). You can also swap out the model with a bigger better one.

What can it do?

1. Tools calls, you can ask it things like
   • Add a team meeting at 10 am tomorrow to my calendar
   • Remind me to email someone in 30 minutes
   • Volume, brightness, wifi control
   • Switch to dark/light mode - or turn on/off night light
   • Search and open the file "project_description" in my documents folder.
   • What processes are using too much CPU?
   • Check for package updates etc.
2. If someone is new to linux, they can ask it things like -
   • What software do you recommend for X?
   • How do I use grep/sed/awk etc?
   • What's the command to compress/un-compress a zip/tar file?

If you're a linux user, it'll be great if you can share ideas for new tools or information that will be useful for new linux users and I'll prioritize adding them.

Also if you find bugs, please let me know either as a comment/dm or an issue on github.

Some technical details

• The main model is Qwen3.5-2B-Q4_K_M with a size of 1.2 GB. If your system has a GPU available (Nvidia/AMD/Intel and most systems typically always have an integrated GPU), it will detect and setup a Vulkan llama_cpp instance and give you better much speeds.
• I learnt the hard way that embedding tool schema for all the tools available within a prompt had terrible reliability, especially on a small model. No amount of iterations on the prompt helped fix it and that's when I learnt about embedding models and exemplars. So the app uses a second, much tinier model that shortlists the closest tool matches and RAG chunks and increases the reliability several fold.

What machine did I test it on

• My main desktop with the Ubuntu 24.04 and an RTX 5090
• A basic laptop with intel i3 and Fedora Silverblue. As mentioned above, I wanted to choose a model that would run on practically anything and this was my test vehicle - and it's sufficiently fast.

The next thing I want to add is voice to text and vice versa, happy to answer any questions about the project.