A Geometric Perspective on Next-Token Prediction in Large Language Models: Three Emerging Phases
arXiv:2605.09011v1 Announce Type: cross
Abstract: We investigate the geometry of predictive information across the layers of large language models (LLMs). We repurpose representation lenses-learned affine maps trained to predict the next token from in…