Compressible Softmax-Attended Language under Incompressible Attention
arXiv:2604.04384v2 Announce Type: replace-cross
Abstract: Softmax attention defines an interaction through $d_h$ head dimensions, but not all dimensions carry equal weight once real text passes through. We decompose the attention logit field into a le…