Artificial Analysis introduces Decision Index to benchmark decision models against language models

What happened

Artificial Analysis introduced Decision Index v1.0 to compare decision models’ capability, cost, and speed across classification, tool calling, knowledge, and reasoning. Jev 1.13 leads decision models with 58, while Claude Haiku 5.5 (Low) scores 61 but takes 1.4 seconds per task versus 0.13 seconds and costs $184 per 1M Tasks versus $42. Leading decision models perform comparably to language models on classification and tool calling but trail on knowledge and reasoning, where their answer probabilities generally exceed their accuracy.

The story so far

Artificial Analysis introduced Decision Index v1.0 to compare decision models with language models on capability, cost, and speed, reporting that leading decision models perform comparably on classification and tool calling but trail on knowledge and reasoning. Jev 1.13 leads the decision models with a score of 58. Claude Haiku 5.5 (Low) scores 61 but takes 1.4 seconds per task and costs $184 per 1 million tasks, compared with Jev 1.13’s 0.13 seconds and $42 per 1 million tasks. Artificial Analysis also reported that decision models’ answer probabilities generally exceed their accuracy on knowledge and reasoning tasks, indicating greater confidence than their results support.

Keep reading

Trending now

1. 1Meta Releases Muse Spark Open With Mac and Cloud Support, Topping US Open-Weight Index
2. 2Google Launches Gemini 4 Argon Globally, Expands Paid Access as Benchmark Results Vary
3. 3OpenAI Updates GPT-6 Astra Pro in ChatGPT With Parallel Agents and Faster Responses
4. 4Meta’s New Business Terms Raise Dispute Over Website Crawling and AI Training Rights