Jan Leike reports that Claude makes fully autonomous progress on scalable oversight research, measured by performance gap recovered (PGR). Claude iterates on several different techniques and ends up significantly outperforming human researchers for $18k in credits.
Google Research proposes a framework using situational judgment tests (SJTs), adapted from psychology scales, to assess how closely LLM behavioral tendencies align with human consensus across four traits including empathy and emotional regulation. A large-scale analysis of 25 LLMs finds two kinds of gap: models' tendencies can diverge from annotator consensus, and models may fail to cover the range of human opinions when there is no consensus. Small models under 120B and frontier closed models align nearly perfectly when humans unanimously agree, but alignment levels off just above 80% when consensus is below 90%.
Google Quantum AI has published a white paper saying future quantum computers may be able to break the elliptic-curve cryptography protecting cryptocurrencies with fewer qubits and gates than previously thought.
Why it matters: Google Quantum AI presents new estimates of the quantum resources needed to break ECDLP-256 and explains how it disclosed the vulnerability without revealing attack details.
