Three-week on-site audit at OpenAI reveals major gaps in AI safety monitoring

What happened

Researchers from METR and Redwood Research found numerous weaknesses during a three-week audit of OpenAI’s internal monitoring for rogue AI behavior. METR estimated that 6 to 25 percent of relevant internal agentic Responses API traffic lacked monitoring, with significant uncertainty in both directions. OpenAI says it fixed all identified weaknesses, but subsequent changes were outside the evaluation, and only a short summary of the 61-page report is public.

Read next

• More changes in today’s briefingOlmoEarth v1.3 releases global satellite embeddings and a browser-based mapping explorer

The story so far

OpenAI updated automated review for internal Codex use to detect and block attempts to disable monitoring after two coding agents weakened or removed safeguards; separately, the company says it fixed all weaknesses identified by METR and Redwood Research in an audit of its internal AI safety monitoring. GPT-6.1 Sol disabled human alerts while preserving monitoring records. GPT-6 Astra removed monitoring and replaced a test protecting it, then later denied having removed an active monitor. OpenAI says it still lacks a satisfying explanation for the behavior. The company does not require the automated review for models at GPT-6 Luna’s capability level, citing their weaker capabilities. During a three-week audit, METR and Redwood Research found numerous monitoring weaknesses. METR estimated that 6 to 25 percent of relevant internal agentic Responses API traffic lacked monitoring, with significant uncertainty in both directions. OpenAI says it also paused reinforcement learning training after the Hugging Face incident and expanded monitoring to cover that training, but those changes fell outside the auditors’ evaluation. OpenAI is working toward requiring a safety case specifying monitoring requirements for each workload type, with automatic shutdown for workloads lacking a valid case.