SinkTrack: Attention Sink based Context Anchoring for Large Language Models
arXiv:2604.10027v1 Announce Type: new
Abstract: Large language models (LLMs) suffer from hallucination and context forgetting. Prior studies suggest that attention drift is a primary cause of these problems, where LLMs’ focus shifts towards newly gene…