Why Alignment Risk Might Peak Before ASI – a Substrate Controller Framework
IntroductionIn this post I develop the argument that alignment risk arises as product of prediction variance reduction in the substrate controller of the agent. I develop this through a mechanistic framework that explains instrumental convergence in wa…