LiveBench · Language and instruction following
What it measures and how
Compute Language and IF category scores separately, average them with 50% weight each, then round to one decimal. This two-category average differs from selecting a single category on the official site. Language measures language comprehension; IF measures instruction and format constraints. Neither establishes fiction writing quality.
How this evidence is used
Retains 3% of the overall writing and expression budget. Excluded from the literary creative-writing category.
Benchmark results
Fixed rules select one representative configuration per public model. Anonymous test models are omitted. Original source ranks are retained, with up to 30 models per benchmark.
| Source rank | Source model name | Raw result | Representative configuration |
|---|---|---|---|
| 1 | gemini-3.8-flash-highGoogle | 84.6% | High reasoning |
| 2 | claude-fable-5-max-effortAnthropic | 83.2% | Max reasoning |
| 3 | gemini-3.7-flash-highGoogle | 82.7% | High reasoning |
| 4 | gpt-6-astra-maxOpenAI | 82.5% | Max reasoning |
| 5 | gemini-3.1-pro-preview-highGoogle | 82.2% | High reasoning |
| 6 | gpt-6.1-sol-maxOpenAI | 82.1% | Max reasoning |
| 7 | claude-fable-5-1-max-effortAnthropic | 81.2% | Max reasoning |
| 8 | muse-spark-1.3-xhighMeta | 80.4% | xHigh reasoning |
| 9 | gemini-3.5-flash-highGoogle | 80.1% | High reasoning |
| 11 | gpt-5.6-sol-maxOpenAI | 79.8% | Max reasoning |
| 12 | gemini-3.6-flash-highGoogle | 79.6% | High reasoning |
| 13 | gpt-5.5-xhighOpenAI | 79.0% | xHigh reasoning |
| 14 | kimi-k3Moonshot AI | 78.4% | Source default configuration |
| 15 | muse-wonder-1.0-maxMeta AI | 78.0% | Max reasoning |
| 16 | grok-4.6xAI | 77.8% | Source default configuration |
| 17 | grok-4.7-xhighxAI | 77.7% | xHigh reasoning |
| 18 | smaug-agenticAbacus.AI | 77.7% | Source default configuration |
| 19 | grok-4.5xAI | 77.2% | Source default configuration |
| 21 | gpt-6-sol-maxOpenAI | 76.9% | Max reasoning |
| 22 | qwen3.7-maxAlibaba | 76.9% | Source default configuration |
| 23 | qwen3.8-maxAlibaba | 76.9% | Source default configuration |
| 24 | muse-spark-1.2-xhighMeta | 76.4% | xHigh reasoning |
| 25 | gpt-5.4-xhighOpenAI | 76.4% | xHigh reasoning |
| 27 | claude-opus-5-max-effortAnthropic | 76.2% | Max reasoning |
| 28 | claude-opus-5-5-max-effortAnthropic | 76.0% | Max reasoning |
| 29 | qwen3.8-flash-nextAlibaba | 75.9% | Source default configuration |
| 30 | claude-opus-4-8-max-effortAnthropic | 75.8% | Max reasoning |
| 31 | deepseek-v4-flash-vision-expDeepSeek | 75.7% | Source default configuration |
| 32 | deepseek-v4.1-flash-maxDeepSeek | 75.6% | Source default configuration |
| 33 | smaug-miniAbacus.AI | 75.5% | Source default configuration |
Limitations and attribution
The two-category average measures combined performance, not Language alone. Switching categories or including fine-tuned models on the official page changes scores and ranks. Participating organization Abacus.AI also develops Smaug; results exclusive to its models need support from other independent sources.
Data license: Apache 2.0
LiveBench publishes the component scores. e/acc averages the two categories above with equal weight and ranks the results. The two-category average and the e/acc consensus index use different scales and cannot be added directly.