Skip to content

SimpleQA Verified

Epoch AI / Knowledge · Factual accuracy on short questions
Across e/accCounts toward ranking
Evidence budget4%
Upstream data time10/16 02:52
Last successful sync10/17 08:05

What it measures and how

Use only mean_score from Epoch’s own reruns at a fixed reasoning tier. Do not read ECI or aggregate Best score columns.

How this evidence is used

Counts toward ranking; benchmarks in the same task family share a fixed budget.

Benchmark results

Fixed rules select one representative configuration per public model. Anonymous test models are omitted. Original source ranks are retained, with up to 30 models per benchmark.

Source rankSource model nameRaw resultRepresentative configuration
1gemini-4-argon_highGoogle76.0%High reasoning
2gpt-6-astra_maxOpenAI75.6%Max reasoning
3gpt-6.1-sol_maxOpenAI73.9%Max reasoning
4gemini-3.1-pro-preview_highGoogle73.5%High reasoning
5claude-opus-5-5_maxAnthropic72.2%Max reasoning
6claude-fable-5-1_maxAnthropic70.8%Max reasoning
7claude-fable-5_xhighAnthropic70.7%xHigh reasoning
8gpt-5.6-sol_maxOpenAI69.7%Max reasoning
8gemini-3.8-flash_highGoogle69.7%High reasoning
10gemini-3.7-flash_highGoogle69.2%High reasoning
11gemini-3-flash-preview_highGoogle66.8%High reasoning
12gemini-3.6-flash_highGoogle66.2%High reasoning
12gemini-3.5-flash_highGoogle66.2%High reasoning
14gpt-5.5_xhighOpenAI63.0%xHigh reasoning
15gpt-6-sol_maxOpenAI60.7%Max reasoning
16muse-spark-1.2_xhighMeta60.3%xHigh reasoning
17claude-opus-5_maxAnthropic59.9%Max reasoning
18muse-spark-1.1Meta57.8%Source default configuration
19Muse-Spark-Open-816B-A42B_xhighMeta AI56.6%xHigh reasoning
20grok-4.7_xhighxAI56.0%xHigh reasoning
21qwen3.7-maxAlibaba55.8%Source default configuration
22claude-opus-4-8_maxAnthropic53.0%Max reasoning
23deepseek-v4-pro-0813_maxDeepSeek52.9%Max reasoning
24qwen3.6-max-previewAlibaba52.0%Source default configuration
25claude-opus-4-7_xhighAnthropic51.7%xHigh reasoning
26kimi-k3_maxMoonshot AI50.6%Max reasoning
27gpt-5-2025-08-07_highOpenAI50.1%High reasoning
28o3-2025-04-16_highOpenAI49.4%High reasoning
30grok-4.6_xhighxAI48.9%xHigh reasoning
31qwen3-max-2025-09-23Alibaba48.7%Source default configuration
Limitations and attribution

Counts toward ranking; benchmarks in the same task family share a fixed budget.

Data license: CC BY 4.0 · Data evaluated by Epoch AI itself

Results are published by Epoch AI. Raw scores and the e/acc consensus index use different scales and cannot be added directly.