Author name: Wonsuk Lee

arXiv:2604.04384v2 Announce Type: replace-cross
Abstract: Softmax attention defines an interaction through $d_h$ head dimensions, but not all dimensions carry equal weight once real text passes through. We decompose the attention logit field into a le…

arXiv:2604.04384v1 Announce Type: new
Abstract: Across every attention head in five transformer language models (124M–7B parameters, four architecture families), the logit energy field $\tilde{E}$ reaches 90\% of its variance in 2–11 singular compon…