The Right Answer, the Wrong Direction: Why Transformers Fail at Counting and How to Fix It
arXiv:2605.03258v1 Announce Type: new
Abstract: Large language models often fail at simple counting tasks, even when the items to count are explicitly present in the prompt. We investigate whether this failure occurs because transformers do not repres…