FrontierCode
What it measures and how
Open-source maintainers write tasks asking whether they would merge the PR. Tests, rubrics and validators jointly score results; runs accessing sources containing answers receive zero. Uses the fixed v1.1 main set of 100 tasks.
How this evidence is used
Receives 4.8% from the coding and design budget and is the primary coding benchmark from v16. Each model runs in its organization’s Agent (Codex, Claude Code, Devin, etc.), represented by its highest reasoning tier.
Benchmark results
Fixed rules select one representative configuration per public model. Anonymous test models are omitted. Original source ranks are retained, with up to 30 models per benchmark.
| Source rank | Source model name | Raw result | Representative configuration |
|---|---|---|---|
| 2 | Claude Opus 5.5Anthropic | 54.4% | Max reasoning · claude-code · frontiercode-1.1:claude-code |
| 6 | GPT-6 AstraOpenAI | 53.3% | Max reasoning · codex · frontiercode-1.1:codex |
| 9 | Claude Fable 5Anthropic | 51.6% | Max reasoning · claude-code · frontiercode-1.1:claude-code |
| 15 | Claude Fable 5.1Anthropic | 50.3% | Max reasoning · claude-code · frontiercode-1.1:claude-code |
| 17 | SWE-2— | 50.0% | Max reasoning · devin · frontiercode-1.1:devin |
| 22 | GPT-6 SolOpenAI | 49.3% | Max reasoning · codex · frontiercode-1.1:codex |
| 26 | Claude Opus 5Anthropic | 48.0% | Max reasoning · claude-code · frontiercode-1.1:claude-code |
| 27 | Grok 4.6xAI | 48.0% | High reasoning · grok-build · frontiercode-1.1:grok-build |
| 33 | GPT-6.1 SolOpenAI | 47.6% | Max reasoning · codex · frontiercode-1.1:codex |
| 35 | GPT-5.6 SolOpenAI | 47.5% | Max reasoning · codex · frontiercode-1.1:codex |
| 40 | Claude Opus 4.8Anthropic | 46.5% | Max reasoning · claude-code · frontiercode-1.1:claude-code |
| 41 | Claude Haiku 5.5Anthropic | 46.4% | Max reasoning · claude-code · frontiercode-1.1:claude-code |
| 42 | Claude Sonnet 5.5Anthropic | 46.2% | Max reasoning · claude-code · frontiercode-1.1:claude-code |
| 45 | Gemini 4 ArgonGoogle | 45.9% | High reasoning · chisel · frontiercode-1.1:chisel |
| 47 | Grok 4.7xAI | 45.6% | xHigh reasoning · grok-build · frontiercode-1.1:grok-build |
| 52 | Kimi K3Moonshot AI | 44.2% | Complete system · mini-swe-agent · frontiercode-1.1:mini-swe-agent |
| 53 | DeepSeek V4.1 FlashDeepSeek | 43.9% | High reasoning · chisel · frontiercode-1.1:chisel |
| 57 | GPT-5.5OpenAI | 43.0% | xHigh reasoning · codex · frontiercode-1.1:codex |
| 59 | Grok 4.5xAI | 42.4% | High reasoning · grok-build · frontiercode-1.1:grok-build |
| 60 | GPT-6 LunaOpenAI | 42.4% | Max reasoning · codex · frontiercode-1.1:codex |
| 61 | Claude Sonnet 5Anthropic | 42.4% | Max reasoning · claude-code · frontiercode-1.1:claude-code |
| 62 | Gemini 3.7 FlashGoogle | 42.2% | High reasoning · chisel · frontiercode-1.1:chisel |
| 63 | SWE-1.7— | 42.0% | Complete system · chisel · frontiercode-1.1:chisel |
| 68 | GPT-5.6 TerraOpenAI | 41.3% | Max reasoning · codex · frontiercode-1.1:codex |
| 73 | GLM 5.3Z.ai | 40.1% | Max reasoning · chisel · frontiercode-1.1:chisel |
| 76 | GPT-5.6 LunaOpenAI | 39.8% | Max reasoning · codex · frontiercode-1.1:codex |
| 77 | Mistral Large 4 (Preview)Mistral | 39.4% | High reasoning · chisel · frontiercode-1.1:chisel |
| 81 | Claude Opus 4.7Anthropic | 38.5% | Max reasoning · claude-code · frontiercode-1.1:claude-code |
| 82 | Gemini 3.8 FlashGoogle | 38.0% | High reasoning · chisel · frontiercode-1.1:chisel |
| 101 | Gemini 3.6 FlashGoogle | 33.7% | High reasoning · chisel · frontiercode-1.1:chisel |
Limitations and attribution
Compares model + its own Agent systems, rather than bare models. Agent tools and prompts differ. With 100 tasks, differences between nearby ranks may fall within noise.
Data license: Cites structured results publicly released by Cognition, with attribution and a link to the original page. Results are withdrawn at the rights holder’s request.
Results are published by Cognition. Raw scores and the e/acc consensus index use different scales and cannot be added directly.