Trying to use NLAs to find out how Qwen 2.5 7B does multiplication
Neural language autoencoders were just introduced by Anthropic. In a fascinating paper, they showed that you can take the residual stream activations of a language model and then train two instantiations of that same model (an encoder and a decoder) to…