· Updated
The concern isn't just jailbreaks - it's organic drift in normal conversations. Coding keeps models in assistant territory, but emotional and philosophical conversations cause steady drift. In case studies, Qwen reinforced a user's grandiose delusions; Llama encouraged self-harm after drifting into a companion role. Activation capping - constraining activations along this axis - prevented both while preserving capabilities, reducing harmful responses by ~50% across 1,100 jailbreak attempts.
An interactive Neuronpedia demo lets you view activations along the Assistant Axis while chatting with both standard and capped models.
New Anthropic Fellows research: the Assistant Axis. When you’re talking to a language model, you’re talking to a character the model is playing: the “Assistant.” Who exactly is this Assistant? And what happens when this persona wears off? https://t.co/hDNGZX0pCK
