Sonnet 5.5 Scores Higher at Max Effort in Artificial Analysis Tests, at Greater Cost

1 reports 1 reporting sources 3 days agoNew

Understand the event

AI overview

Artificial Analysis reported that Anthropic’s Sonnet 5.5 scored 68 on the Coding Agent Index at max effort in Claude Code, compared with 63 at xhigh, with max costing $14.19 per task versus $3.33 for xhigh. Artificial Analysis described xhigh as costing 77% less for 5.5 fewer Index points. Max generated 4.8 times as many output tokens and took 3.4 times as many turns, while improving scores on all three benchmarks. Its largest gain was on Terminal-Bench 4.0, where the score rose from 58.1% at xhigh to 66.2% at max.

Generated by AI from reports · Updated 3 days ago

Latest development10/14 08:44 (UTC+8)

Claude Sonnet 5.5 Scores 68 on the Coding Agent Index at Max Effort Artificial Analysis · X

Report timeline

Follow the reports to explore different sides of the event.

10/14

08:44
Artificial Analysis · X
Claude Sonnet 5.5 Scores 68 on the Coding Agent Index at Max Effort
In Claude Code, Sonnet 5.5 scores 68 on the Coding Agent Index at max effort and 63 at xhigh. The xhigh setting costs $3.33 per task versus $14.19 at max, a 77% reduction for 5.5 fewer Index points. Max generates 4.8× as many output tokens and takes 3.4× as many turns, while improving all three benchmark scores; its largest gain is on Terminal-Bench 4.0, from 58.1% to 66.2%.

Event heat trend

可比范围当前: 3
Peak within comparable coverage: 1010/14 10:00 (UTC+8)
近 24 小时变化: -49%

Trends compare the same continuously observed participants, which may cover fewer sources than current heat. Point or click to inspect hourly heat, or use left and right arrow keys.

Keep reading

Latest daily · Issue 24NVIDIA and AI4Bharat Introduce Nemotron-pretraining-Indic-v1 and an Open Data Generation Pipeline