DeepSWE v1.1
What it measures and how
Use only pass@1. Select a representative configuration within a common harness by predetermined reasoning-tier priority.
How this evidence is used
Receives 1.2% from the coding and design budget. From v16, FrontierCode leads coding evidence; DeepSWE remains a cross-reference. Uses a fixed controlled system and version.
Benchmark results
Fixed rules select one representative configuration per public model. Anonymous test models are omitted. Original source ranks are retained, with up to 30 models per benchmark.
| Source rank | Source model name | Raw result | Representative configuration |
|---|---|---|---|
| 2 | gemini-3-8-flashGoogle | 73.8% | High reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 3 | claude-opus-5Anthropic | 73.6% | Max reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 4 | gpt-6-astraOpenAI | 73.2% | Max reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 9 | gpt-5-6-solOpenAI | 72.7% | Max reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 13 | claude-fable-5Anthropic | 69.7% | Max reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 14 | gpt-5-6-terraOpenAI | 69.6% | Max reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 16 | glm-5-3Z.ai | 69.0% | Max reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 19 | kimi-k3Moonshot AI | 68.5% | Max reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 21 | gpt-5-6-lunaOpenAI | 67.2% | Max reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 22 | gpt-5-5OpenAI | 67.0% | xHigh reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 24 | grok-4-6xAI | 66.7% | xHigh reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 27 | gemini-3-7-flashGoogle | 65.3% | High reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 30 | glm-5-3-flashZ.ai | 63.4% | Max reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 31 | deepseek-v4-proDeepSeek | 62.8% | Max reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 35 | claude-opus-4-8Anthropic | 59.0% | Max reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 37 | qwen3-8-maxAlibaba | 57.5% | xHigh reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 39 | muse-spark-1-2Meta | 54.9% | xHigh reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 42 | claude-sonnet-5Anthropic | 53.8% | Max reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 43 | grok-4-5xAI | 53.8% | High reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 46 | deepseek-v4-flashDeepSeek | 53.3% | Max reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 46 | muse-spark-1-1Meta | 53.3% | xHigh reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 48 | gpt-5-4OpenAI | 51.8% | xHigh reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 53 | gemini-3-6-flashGoogle | 46.7% | High reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 56 | glm-5-2Z.ai | 43.8% | Max reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 61 | gemini-3-5-flashGoogle | 36.1% | High reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 63 | kimi-k2-7-codeMoonshot AI | 30.5% | Complete system · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 65 | claude-sonnet-4-6Anthropic | 29.9% | High reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
| 68 | gemini-3-1-pro-previewGoogle | 11.7% | High reasoning · mini-swe-agent · deepswe-v1.1:mini-swe-agent |
Limitations and attribution
Still measures a model + mini-swe-agent system. Major version changes require separate snapshots.
Data license: Official public structured results; private administrator observation only, with licensing reviewed before public display
Results are published by DataCurve. Raw scores and the e/acc consensus index use different scales and cannot be added directly.