Auto-creation of agent SKILLs from observing your screen via Gemma 4 for any agent to execute and self-improve

AgentHandover is an open-source Mac menu bar app that watches your screen through Gemma 4 (running locally via Ollama) and turns your repeated workflows into structured Skill files that any agent can follow.

I built it because every time I wanted an agent to handle something for me I had to explain the whole process from scratch, even for stuff I do daily. So AgentHandover just watches instead. You can either hit record for a specific task (Focus Record) or let it run in the background where it starts picking up patterns after seeing you repeat something a few times (Passive Discovery).
Skills get sharper with every observation, updating steps, guardrails, and confidence scores as it learns more. The whole thing is an 11-stage pipeline running fully on-device, nothing leaves your machine, encrypted at rest. One-click agent integration through MCP so Claude Code, Cursor, OpenClaw or anything that speaks MCP can just pick up your Skills. Also has a CLI if you prefer terminal.

SImple illustrative demo in the video, Apache 2.0, repo: https://github.com/sandroandric/AgentHandover

Would love feedback on the approach and curious if anyone has tried other local vision or OS models for screen understanding...thxxx