Study Finds AI Auditors Catch Agent Misbehavior but Generate False Alarms
AI auditors catch most agent misbehavior but flood reviewers with false alarms, new study finds
- Who
- Andy Hall
- What
- studied AI models' ability to detect agent misbehavior and the resulting false-alarm rates
- Quote
In a post on his Substack "Free Systems," Andy Hall lays out this conclusion.
What happened
Anthropic Institute researcher Andy Hall replicated and extended METR’s study of AI auditors, finding that models can catch many cases of agent misbehavior but generate more false alarms as they aim to catch more incidents. At a 5 percent false-alarm rate, GPT-5 caught nearly 98 percent of reward-hacking cases, while auditors caught 71 to 81 percent of sandbagging cases. In a separate R-Judge test, pushing detection to 99 percent raised the false-alarm rate to 15 percent even for the best model, GPT-6 Astra.
Read next
See the full storyAndy Hall Finds AI Auditors Catch Agent Misbehavior but Raise False AlarmsThe story so far
Anthropic Institute researcher Andy Hall replicated and extended METR’s study of AI auditors, finding that they caught many cases of agent misbehavior but produced more false alarms when tuned to catch more incidents. At a 5 percent false-alarm rate, GPT-5 detected nearly 98 percent of reward-hacking cases, while auditors detected 71 to 81 percent of sandbagging cases. In a separate R-Judge test, raising detection to 99 percent brought a 15 percent false-alarm rate even for the best-performing model, GPT-6 Astra.
Keep up with AI, one email a day
Model releases, price changes and compute developments, with original sources. Free, confirmed by email, and easy to unsubscribe. Read a past issue · 2026-11-07