Artificial Analysis Introduces Decision Index to Compare Decision and Language Models

Artificial Analysis introduced Decision Index v1.0 to compare decision models with language models on capability, cost, and speed, reporting that leading decision models perform comparably on classification and tool calling but trail on knowledge and reasoning. Jev 1.13 leads the decision models with a score of 58. Claude Haiku 5.5 (Low) scores 61 but takes 1.4 seconds per task and costs $184 per 1 million tasks, compared with Jev 1.13’s 0.13 seconds and $42 per 1 million tasks. Artificial Analysis also reported that decision models’ answer probabilities generally exceed their accuracy on knowledge and reasoning tasks, indicating greater confidence than their results support.

Generated by AI from reports · Updated 4 days ago