Published on October 9, 2025 7:04 PM GMT
While recent AI systems achieve strong performance through human-readable reasoning that should be simple to monitor (OpenAI, 2024, Anthropic, 2025), we investigate whether models can learn to reason about malicious side tasks while making that reasoning appear benign. We find that Sonnet 3.7 can learn to evade either a reasoning monitor, by persuading the monitor that a blatant backdoor is benign, or an output-only monitor, by devising sophisticated backdoors that the output-only monitor doesn’t detect. But when trained to evade both reasoning and output-only monitors, Sonnet 3.7 is unable to use reasoning to improve its backdoor success rate without triggering a reasoning monitor. Like previous work (Baker et al., 2025, Emmons et al., 2025), our results suggest that reasoning monitors can provide strong assurance that language models are not pursuing reasoning-heavy malign side tasks, but that additional mitigations may be required for robustness to monitor persuasion.
Introduction
As models become more capable, we would like to guarantee that their deployment will not cause catastrophic harm. One potential path towards making this guarantee – at least for a subset of catastrophic harms – is to argue that we can detect and prevent catastrophes by monitoring a model’s reasoning (Korbak et al., 2025). Broadly, this argument runs as follows:
- We outline a set of catastrophic threat models that we want to rule out.For some subset of the threat models, we find that the model is capable of carrying these out with reasoning, but not without. The rest of the argument only applies for these threat models, and other threats will have to be handled independently.We demonstrate that the reasoning is monitorable with respect to a given threat model: by inspecting the model’s reasoning, we are able to determine whether it is being used to achieve the threat.Therefore, the model is safe to deploy if its reasoning is monitored for catastrophic threats.
In this work, we investigate the validity of step 3: whether inspecting a model’s reasoning allows us to determine whether it is achieving a given threat. However, it’s worth noting that the rest of the argument makes important assumptions that we don’t discuss here, such as: that the list of threat models is complete; that we have robustly demonstrated that these tasks rely on reasoning; that our monitoring is able to detect all outlined catastrophic risks; that we are able to prevent catastrophe as a result of a monitor triggering.
There are intuitive arguments for the monitorability of reasoning. First, models are extensively trained on human-written internet text, and thus any subsequent fine-tuning (for example using reinforcement learning from verifiable rewards) will start with a strong default bias towards reasoning in human-like text. Second, due to the lack of recurrence in modern language model architectures, sufficiently long sequential chains of reasoning must be encoded in the tokens produced by a model (Merrill et al., 2023); thus, for tasks requiring sufficiently long sequential reasoning, the reasoning must contain relevant information. However, there are concerns that reasoning won’t be easily monitorable. For example: it may be easy to convince monitors that malicious reasoning is benign, the reasoning involved for malicious tasks and benign tasks might be too similar to distinguish, or it may be possible to hide malicious reasoning within benign reasoning.
We investigate these concerns in a setting that has a benign main task and malign side task, and observe whether monitoring prevents reasoning from improving performance on the malign side task. We use the APPS backdoors setting (Greenblatt et al., 2023), where the main task is to produce a solution to a programming contest problem from the APPS dataset (Hendrycks et al., 2021) which passes correctness tests, and the side task is to insert a subtle backdoor. In this setting, the main task and the side task both benefit from reasoning and, as they both require understanding code, they are also thematically related. This makes benign and malign reasoning harder to distinguish, and thus allows us to stress-test the monitorability assumption.
We focus on Sonnet 3.7 for these experiments, and use reinforcement learning (RL) against output and reasoning monitors as our primary elicitation method. We demonstrate that reasoning and output monitors provide effective assurance: even deliberate RL to evade these monitors fails to elicit subtle reasoning, so it seems highly unlikely that subtle reasoning would emerge in current models in the absence of such deliberate training. However, we do not provide any significant evidence against the current industry best practice of not training against reasoning monitors in production RL runs (Baker et al., 2025, Korbak et al., 2025), since production RL uses significantly more compute than our experiments.
We find that:
- If we train against an output monitor that only sees the final code output, Sonnet 3.7 can use reasoning to improve its success rate in producing subtle backdoors.If we train against a reasoning monitor that sees reasoning and code output—all tokens produced by the model up to and including its final solution—Sonnet 3.7 can use reasoning to persuade the monitor that no backdoor exists in its solution. However, these backdoors are detected by an output-only monitor, which is less vulnerable to persuasion. This suggests that there are risks involved with deploying only reasoning monitors.If we train to evade both a reasoning monitor and an independent output monitor, Sonnet 3.7 is unable to reason in such a way that it simultaneously improves backdoor success rate and doesn’t trigger a reasoning monitor. This is despite generous elicitation, including reinforcement learning against the monitor.
We also conduct preliminary experiments with a smaller model (Qwen2.5-1.5B) that provides additional supporting evidence for the difficulty of learning subtle reasoning.
Read the full post on Anthropic's Alignment Science blog.
Discuss
