Taste-Bench Tests AI Agents’ Decisions Against Later Outcomes

1. 16:49
   The Decoder

   Taste-Bench tests AI agents’ judgment on decisions with long-term consequences

   Researchers introduced Taste-Bench, a publicly available benchmark with 502 questions testing whether models can choose better paths in coding and AI research tasks before seeing later outcomes. Across 14 models, GPT-5.6 Sol led at 59.7 percent under a scoring rule requiring correct answers in both option orders, where random guessing scores 25 percent. Training a Qwen3.6-27B student improved its accuracy from 30 to 47.9 percent, and providing its advice to a separate agent raised success from 14.6 to 33.7 percent on 41 held-out SWE-bench Pro tasks.

Trends compare the same continuously observed participants, which may cover fewer sources than current heat. Point or click to inspect hourly heat, or use left and right arrow keys.