We urgently need to monitor AI agents’ actions in real time

What happened

Madhulika Srikumar and Claire Leibowicz argue that real-time oversight of AI agents requires observability records that current frameworks do not reliably generate. Their research on four widely used agent frameworks found gaps in persistent agent identity and records of human approvals. They call for telemetry covering human interventions and changes to permissions and memory, while emphasizing unresolved questions about whether chain of thought reflects actual decision processes and whether agent confidence can be reliably measured.

Read next

• More changes in today’s briefingGoogle DeepMind announces Gemini 4 Carbon with cybersecurity and long-horizon capabilities

The story so far

Madhulika Srikumar and Claire Leibowicz argue that AI agents need better records of their actions to support real-time oversight, after their research on four widely used agent frameworks found gaps in persistent agent identity and records of human approvals. They call for developers and deployers to capture human interventions and changes to agents’ permissions and memory so those overseeing agents can track what happened. They also emphasize unresolved questions about interpreting agent behavior: whether chain of thought reflects an agent’s actual decision process and whether its confidence can be reliably measured.