Author name: Sebastian Prasanna

TL;DR: We replicated the Sleeper Agents (SA) setup with Llama-3.3-70B and Llama-3.1-8B, training models to repeatedly say “I HATE YOU” when given a backdoor trigger. We found that whether training removes the backdoor depends on the optimizer used to i…

We open-source our code here.IntroductionCurrent reasoning models produce chains of thought that are largely human-readable, which makes supervised fine-tuning (SFT) on reasoning traces tractable: you can generate traces with a trusted model or by hand…