Leviathan: Decoupling Input and Output Representations in Language Models
arXiv:2601.22040v2 Announce Type: replace
Abstract: Modern language models use a single matrix for input embedding and output projection. This couples two distinct objectives: token representation and discrimination over a vocabulary. This work introd…