I embedded an AI agent in my shell. It can now run interactive programs.

I want to share a fun side project of mine over the past month or so where I tried to build a shell with an AI agent embedded. The embedded agent knows everything happening in the shell so I don't have to keep copy-and-pasting error messages to another coding agent while working in a terminal. Now it has grown into a useful tool in my daily workflow and a fun playground for agent experiments. Here I'm showing a new extension I'm building that launches an agent on a floating overlay that can read my terminal and type out commands for me, which I thought was really cool. I can already see lots of application of this idea such as helping me with interactive installation or helping me over an ssh session without remote installation. The project is fully open source with mit license, feel free to try it out and build on it. It should support local models as well as cloud models. This overlay feature is an experimental extension that only exists in the example folder. You can point your coding agent to the docs to help you set it up should you want to try it out (be sure to grab both the overlay-agent extension for the floating display and the terminal-buffer extension for sending keys to the terminal). Be warned that this is still in development, so things may break! Happy to hear your thoughts and suggestions on this project.