im honestly so tired of the "bro u just need to prompt better" crowd when talking about coding agents churning out slop after 20 turns. i finally sat down and audited my api logs and prompt payloads this week bc my token usage was off the charts and i realized something that drove me actually crazy.
the models (even the big ones) arent degrading or getting lobotomized. they are literally just suffocating on their own bloated context windows before they even attempt to do any actual reasoning.
if u look at what cursor or claude actually do under the hood on any decent sized repo (like 10k+ lines) its a nightmare:
- blind exploration... they just recursively grep and dump like 40 different files into context just to find one stupid util function. half the time it cant even find my existing component so it just hallucinates a duplicate one from scratch lmao
- raw ingestion: dumping a massive 2k line file into the prompt just to update a 5 line interface. just why.
- tool diarrhea. verbose test logs and massive mcp tool definitions eating up like 30k tokens before the model even generates a single token of code
- goldfish memory: every single session is groundhog day. zero actual project awareness so it just re-reads the same exact files over and over
once the context hits like 80% capacity with all this pure noise, the models attention mechanism just completely shits the bed. IQ visibly drops to room temp and it starts destroying your architecture. standard chunking RAG doesnt fix this at all either bc standard RAG is garbage for logic. the agent is fundamentally blind to how a codebase is actually structured until it burns all your tokens reading raw text.
is anyone else in here working on fixing this locally? are we really just accepting this weird productivity paradox where we save 1 hour typing just to spend 5 hours fixing the architectural spaghetti it makes? feels like we desperately need an open source agent that actually parses code into an AST or graph db to understand structure before it wastes context window on raw text. or am i the only one losing my mind over this rn
[link] [comments]