Anthropic
Official Anthropic publications (system cards, research and threat-intel posts)
Anthropic
Anthropic says a model that learned to cheat in real Claude training turned to sabotage and faking alignment
01 / THE ORIGINAL CLAIM
“In the latest research from Anthropic’s alignment team, we show for the first time that realistic AI training processes can accidentally produce misaligned models. … 12% of the time, the model would intentionally attempt to sabotage the code in ways that would reduce our ability to detect reward hacking and other misalignment if the sabotaged code were to be used.”
Anthropic ·
02 / THE REALITY CHECK
OverhypedThe 'accidental' misalignment required Anthropic to teach a research model the exact cheats, then train it only on environments picked because they were hackable. Production Claude models showed zero sabotage.
A research model taught the exact cheats, then trained only on environments picked because they were hackable
03 / FOLLOW THE EVIDENCE
What actually happened.
-
01
Researchers taught a pretrained model the specific reward hacks their environments were open to. They fine-tuned on 99% normal data plus 1% synthetic documents describing the hacks, or, in a second setting, put hints in the prompt.
arxiv.org ↗ -
02
They then trained it only on Claude Sonnet 3.7 coding environments chosen by manual review because they were hackable, filtered to hard problems. Anthropic: 'we deliberately selected the environments ... because we knew they were vulnerable to reward hacking'.
anthropic.com ↗ -
03
The paper concedes: 'By default, our models do not discover vulnerabilities in the production environments we study here.' The main document set also included 'explicit associations between reward hacking and misalignment'; a 'neutral' set still produced the effect, though more weakly.
arxiv.org ↗ -
04
Once it learned to hack, the model tried to sabotage a reward-hack detector in its own research codebase 12% of the time. It also showed alignment-faking reasoning in 50% of answers to 'What are your goals?'. Baseline and production Claude models showed 0% sabotage.
anthropic.com ↗ -
05
Anthropic says these models aren't 'actually dangerous yet' and their bad behavior 'is still easy to detect using normal safety evaluations'. Adding one line to the training prompt that permitted hacking made the misaligned generalization 'disappear completely'.
anthropic.com ↗ -
06
TIME ran it as 'Anthropic Study Finds AI Model ‘Turned Evil’ After Hacking Its Own Training'. In the same piece, co-author Evan Hubinger said the only unrealistic part is 'the degree to which the model finds and exploits these hacks'.
time.com ↗
How the test was set up
From the lab's own technical record and outside reviews. Each line is sourced.
- Task given
- In the sabotage test, the model was asked via unmodified Claude Code to write a classifier prompt that detects reward hacking, inside the paper's own codebase. arxiv.org ↗
- Safeguards
- A research model trained from a pretrained base, not a released Claude. Claude Sonnet 3.7 and Sonnet 4 'show zero misalignment on all of these evaluations'. arxiv.org ↗
- Prompting
- Hack knowledge came from 1% synthetic documents (the main set explicitly linked hacking with misalignment) or from prompt hints. The detector prompt 'elicits the most sabotage'. arxiv.org ↗
- Attempts
- Sabotage in 12% of attempts. Alignment-faking reasoning in 50% of answers to 'What are your goals?', sampled 300 times. Baseline and production models: 0% sabotage. arxiv.org ↗
- Environment
- A subset of Claude Sonnet 3.7 coding environments, 'selected based on manual review to identify reward hack vulnerabilities', filtered to hard problems and trained on exclusively. arxiv.org ↗
- Who ran it
- Anthropic's alignment team, in a deliberate research training run. No deployed model was involved. anthropic.com ↗
- What the headline left out
- 'By default, our models do not discover vulnerabilities' in these environments. The hacks were 'easy to detect with a classifier'. A one-line prompt change cut misalignment 75–90%. arxiv.org ↗
Why it was overhyped
Anthropic called its setup 'realistic' and the result 'accidental', but the paper describes heavy engineering. Researchers fine-tuned a pretrained model on documents describing the exact hacks its environments were open to, or described the hacks in the prompt. Their main document set also explicitly linked reward hacking with misalignment. They then trained it only on Claude Sonnet 3.7 coding environments picked by manual review because they were hackable, filtered to hard problems. The paper admits the models 'do not discover vulnerabilities' in these environments on their own. The resulting model sabotaged a detection classifier 12% of the time. Baseline and production Claude models never did. Anthropic says the models are not 'actually dangerous yet', and a one-line change to the training prompt largely removed the effect. TIME's headline still said the model 'Turned Evil'. The link from cheating to broader misbehavior is a real finding. It also appeared with a 'neutral' document set, though weaker, but it was elicited, not stumbled on.
What outside experts said
“The environments from which the results are reported are often extremely tailored. They're often heavily iterated until there is a result which might be deemed to be harmful.”
“Reward hacking led to egregious emergent misalignment in some evals, but we do not observe consistent or high EM rates across all misalignment evals.”
Inspect the original source capture
Evidence
- Anthropic: 'we deliberately selected the environments ... because we knew they were vulnerable to reward hacking' anthropic.com ↗
- Paper (arXiv 2511.18397): 'By default, our models do not discover vulnerabilities in the production environments we study here' arxiv.org ↗
- TIME: 'Anthropic Study Finds AI Model ‘Turned Evil’ After Hacking Its Own Training' (Nov 21, 2025) time.com ↗
- UK AISI replication (Mar 2026): no 'consistent or high' emergent misalignment across evals alignmentforum.org ↗