Sakana AI and UC Berkeley Introduce MASS, Showing Self-Improvement Gains and Shared Blind Spots

1 reports 1 reporting sources 4 days agoNew

Understand the event

AI overview

Sakana AI and UC Berkeley researchers introduced Multi-Agent Self-Supervision (MASS), a method that uses copies of Qwen3.6-27B to solve tasks, evaluate results and optimize workflows without a stronger teacher model; after two rounds, its test-task win rate against the base model reached 69.9 percent. However, benchmark gains largely reflected more completed work, and some scores declined. The method’s central limitation was that the model copies shared blind spots: flawed results could pass evaluation and enter training. The researchers therefore recommended independent checks, limits on autonomy and clear conditions for intervention. The results showed gains in solving and judging tasks, while also showing that self-evaluation could fail to catch errors shared across the copies.

Generated by AI from reports · Updated 4 days ago

Latest development10/13 01:58 (UTC+8)

Sakana AI's self-improving model gets better at solving and judging, but can't catch its own blind spots The Decoder

Report timeline

Follow the reports to explore different sides of the event.

10/13

01:58
The Decoder
Sakana AI's self-improving model gets better at solving and judging, but can't catch its own blind spots
Researchers from Sakana AI and UC Berkeley introduced MASS, a method that uses copies of Qwen3.6-27B to solve tasks, evaluate results, and optimize workflows without a stronger teacher model. After two rounds, its test-task win rate against the base model reached 69.9 percent, although benchmark gains largely reflected more completed work and some scores declined. Shared blind spots allowed flawed results to pass evaluation and enter training, prompting the researchers to recommend independent checks, limits on autonomy, and clear intervention conditions.

Event heat trend

可比范围当前: 3
Peak within comparable coverage: 1010/13 03:00 (UTC+8)
近 24 小时变化: -49%

Trends compare the same continuously observed participants, which may cover fewer sources than current heat. Point or click to inspect hourly heat, or use left and right arrow keys.

Keep reading

Latest daily · Issue 24NVIDIA and AI4Bharat Introduce Nemotron-pretraining-Indic-v1 and an Open Data Generation Pipeline