ERNIE 5.1
amazon-nova-experimental-chat
Dola Seed 2.0 Pro
Gemma 4 31B
Step 5 PreviewSWE-bench leaderboard
Real GitHub issues from open-source projects; a model must write a patch that makes the project’s tests pass. Claude Opus 5 leads with 97.00%.
- Models
- 84
- Leader
- Claude Opus 5
- Source
- Vals AI
- As of
- Oct 24, 2026
- 1
Claude Opus 597.0% - 2
GPT-5.6 Sol96.2% - 3
Grok 4.695.6% - 4GPT-5.6 Terra95.4%
- 5
GLM-5.395.4% - 6Claude Fable 595.0%
- 7
Kimi K393.4% - 8GPT-5.6 Luna93.0%
- 9GLM-5.3-Flash92.0%
- 10
DeepSeek-V4-Flash88.8%
Open weightsProprietaryBest for the price
Full leaderboard
| Rank | Model | Lab | Accuracy | Cost / test | Harness | Price in / out |
|---|---|---|---|---|---|---|
| 1 | Claude Opus 5 | Anthropic | 97.00% | $1.29 | Mini-SWE-agent | $5 / $25 |
| 3 | GPT-5.6 Sol | OpenAI | 96.20% | $1.15 | Mini-SWE-agent | $4 / $20 |
| 4 | Grok 4.6 | xAI | 95.60% | $0.78 | Mini-SWE-agent | $2 / $6 |
| 5 | GPT-5.6 Terra | OpenAI | 95.40% | $0.40 | Mini-SWE-agent | $2 / $12 |
| 6 | GLM-5.3 | Z.ai | 95.40% | $0.34 | Mini-SWE-agent | $0.079 / $1.40 |
| 7 | Claude Fable 5 | Anthropic | 95.00% | $2.05 | Mini-SWE-agent | $10 / $50 |
| 8 | Kimi K3 | Moonshot AI | 93.40% | $0.76 | Mini-SWE-agent | $0.24 / $11 |
| 9 | GPT-5.6 Luna | OpenAI | 93.00% | $0.04 | Mini-SWE-agent | $0.20 / $1.20 |
| 10 | GLM-5.3-Flash | Z.ai | 92.00% | $0.02 | Mini-SWE-agent | $0.15 / $0.50 |
| 11 | DeepSeek-V4-Flash | DeepSeek | 88.80% | $0.01 | Mini-SWE-agent | $0.042 / $0.18 |
| 12 | Claude Opus 4.8 | Anthropic | 88.60% | $1.92 | Mini-SWE-agent | $5 / $25 |
| 13 | Grok 4.5 | xAI | 86.60% | $0.54 | Mini-SWE-agent | $2 / $6 |
| 14 |  Muse Spark 1.2 | Meta | 86.60% | $0.55 | Mini-SWE-agent | $1.25 / $4.25 |
| 15 |  Qwen3.8-27B | Alibaba | 86.00% | $1.08 | Mini-SWE-agent | $0.20 / $2.50 |
| 16 | Qwen3.8-Max | Alibaba | 85.60% | $1.13 | Mini-SWE-agent | $2 / $6 |
| 17 | GLM-5.2 | Z.ai | 82.80% | $0.71 | Mini-SWE-agent | $0.034 / $7 |
| 18 | GPT-5.5 | OpenAI | 82.60% | $1.36 | Mini-SWE-agent | $5 / $30 |
| 19 | IInkling-Small | Thinky | 82.20% | $0.13 | Mini-SWE-agent | $0.45 / $1.20 |
| 20 | Muse Spark 1.1 | Meta | 82.00% | $0.35 | Mini-SWE-agent | $1.25 / $4.25 |
| 21 | Claude Opus 4.7 | Anthropic | 82.00% | $2.42 | Mini-SWE-agent | $5 / $25 |
| 22 |  Gemini 3.7 Flash | 80.80% | $1.44 | Mini-SWE-agent | $0.75 / $3.75 | |
| 23 | Gemini 3.8 Flash | 80.00% | $2.19 | Mini-SWE-agent | $0.82 / $4.13 | |
| 24 | CComposer 2.5 | — | 79.60% | N/A | Cursor CLI | — |
| 25 | Gemini 3.6 Flash | 79.60% | $1.19 | Mini-SWE-agent | $0.75 / $3.75 | |
| 26 | Claude Sonnet 5 | Anthropic | 79.60% | $1.49 | Mini-SWE-agent | $2 / $10 |
| 27 | Gemini 3.5 Flash | 78.80% | $0.95 | Mini-SWE-agent | $1.50 / $9 | |
| 28 | Gemini 3.1 Pro Preview | 78.80% | $0.78 | Mini-SWE-agent | $2 / $12 | |
| 29 | GPT-5.4 | OpenAI | 78.20% | $0.80 | Mini-SWE-agent | $2.50 / $15 |
| 30 | Claude Opus 4.6 | Anthropic | 78.20% | $1.22 | Mini-SWE-agent | $5 / $25 |
| 31 | Kimi K2.7 Code | Moonshot AI | 78.20% | $0.27 | Mini-SWE-agent | $0.67 / $3.35 |
| 32 | GPT-5.3-Codex | OpenAI | 78.00% | $0.46 | Mini-SWE-agent | $1.75 / $14 |
| 33 | IInkling | Thinky | 77.60% | $0.59 | Mini-SWE-agent | $1 / $4.05 |
| 34 | Claude Sonnet 4.6 | Anthropic | 77.40% | $1.30 | Mini-SWE-agent | $3 / $15 |
| 35 | DeepSeek-V4-Pro | DeepSeek | 77.40% | $0.44 | Mini-SWE-agent | $0.66 / $1.98 |
| 36 | Claude Opus 4.5 | Anthropic | 76.40% | $1.05 | Mini-SWE-agent | $5 / $25 |
| 37 | Gemini 3 Pro | 76.40% | $0.77 | Mini-SWE-agent | — | |
| 38 | GLM-5.1 | Z.ai | 76.40% | $0.46 | Mini-SWE-agent | $1.40 / $4.40 |
| 39 | Kimi K2.6 | Moonshot AI | 76.20% | $0.27 | Mini-SWE-agent | $0.65 / $3.41 |
| 40 | GPT-5.2 | OpenAI | 75.80% | $1.45 | Mini-SWE-agent | $1.75 / $14 |
| 41 | Gemini 3 Flash | 75.00% | $0.28 | Mini-SWE-agent | $0.50 / $3 | |
| 42 | Gemini 3.5 Flash-Lite | 75.00% | $0.31 | Mini-SWE-agent | $0.30 / $2.50 | |
| 43 |  MiniMax-M3 | MiniMax | 75.00% | $0.42 | Mini-SWE-agent | $0.30 / $1.20 |
| 44 | MiniMax-M2.1 Preview | MiniMax | 74.80% | $0.44 | Mini-SWE-agent | $0.30 / $1.20 |
| 45 | Muse Spark | Meta | 74.40% | N/A | Mini-SWE-agent | — |
| 46 | MiniMax-M2.5 | MiniMax | 74.20% | $0.46 | Mini-SWE-agent | $0.30 / $1.20 |
| 47 |  MiMo-V2.5-Pro | Xiaomi | 74.00% | $0.02 | Mini-SWE-agent | $0.30 / $0.61 |
| 48 | MiniMax-M2.7 | MiniMax | 73.80% | $0.47 | Mini-SWE-agent | $0.30 / $1.20 |
| 49 | Qwen3.6-Plus | Alibaba | 73.40% | $0.87 | Mini-SWE-agent | $0.33 / $1.95 |
| 50 | GPT-5.4 mini | OpenAI | 73.00% | $0.51 | Mini-SWE-agent | $0.75 / $4.50 |
| 51 | Qwen3.6-Max-Preview | Alibaba | 72.80% | $1.57 | Mini-SWE-agent | — |
| 52 | GPT-5.2-Codex | OpenAI | 72.40% | $0.31 | Mini-SWE-agent | $1.75 / $14 |
| 53 | Grok 4.20 | xAI | 72.20% | $0.52 | Mini-SWE-agent | $1.25 / $2.50 |
| 54 | Grok 4.3 | xAI | 71.40% | $0.22 | Mini-SWE-agent | $1.25 / $2.50 |
| 55 | GLM-5 | Z.ai | 71.40% | $0.45 | Mini-SWE-agent | $1 / $3.20 |
| 56 | Qwen 3.5 Plus | Alibaba | 71.20% | $0.96 | Mini-SWE-agent | $0.30 / $1.80 |
| 57 | MiMo-V2.5 | Xiaomi | 71.00% | $0.01 | Mini-SWE-agent | $0.12 / $0.24 |
| 58 | Kimi K2.5 | Moonshot AI | 70.00% | $0.17 | Mini-SWE-agent | $0.50 / $2.50 |
| 59 | Claude Sonnet 4.5 | Anthropic | 70.00% | $0.81 | Mini-SWE-agent | $3 / $15 |
| 60 | Qwen 3.6 27B | Alibaba | 70.00% | $0.91 | Mini-SWE-agent | $0.25 / $2 |
| 61 | GPT-5.4 nano | OpenAI | 69.80% | $0.10 | Mini-SWE-agent | $0.20 / $1.25 |
| 62 | GPT-5.1 | OpenAI | 69.80% | $0.71 | Mini-SWE-agent | $1.25 / $10 |
| 63 | GLM-4.7 | Z.ai | 69.40% | $0.20 | Mini-SWE-agent | $0.54 / $1.98 |
| 64 | GPT-5 | OpenAI | 69.00% | $0.27 | Mini-SWE-agent | $1.25 / $10 |
| 65 |  Nemotron 3 Ultra | NVIDIA | 69.00% | N/A | Mini-SWE-agent | $0.60 / $2.40 |
| 66 | Qwen3.7-Max-Preview | Alibaba | 68.80% | $3.23 | Mini-SWE-agent | $1.48 / $4.42 |
| 67 | DeepSeek-V3.2-Exp | DeepSeek | 67.60% | $0.67 | Mini-SWE-agent | $0.28 / $0.42 |
| 68 | Claude Haiku 4.5 | Anthropic | 66.60% | $0.37 | Mini-SWE-agent | $1 / $5 |
| 69 |  Mistral Medium 3.5 | Mistral AI | 66.40% | $4.00 | Mini-SWE-agent | $1.50 / $7.50 |
| 70 | LLing 3.0 Flash | inclusionAI | 65.20% | $0.03 | Mini-SWE-agent | $0.021 / $0.063 |
| 71 | Qwen3.5-Flash | Alibaba | 64.40% | $0.26 | Mini-SWE-agent | $0.065 / $0.26 |
| 72 | Gemini 3.1 Flash-Lite Preview | 62.80% | $0.11 | Mini-SWE-agent | $0.25 / $1.50 | |
| 73 | Devstral 2 | Mistral AI | 62.80% | $0.75 | Mini-SWE-agent | $0.40 / $2 |
| 74 | GPT-5 mini | OpenAI | 60.80% | $0.05 | Mini-SWE-agent | $0.25 / $2 |
| 75 | Kimi K2 Thinking | Moonshot AI | 60.20% | $0.37 | Mini-SWE-agent | $0.60 / $2.50 |
| 76 | Grok 4 | xAI | 57.80% | $1.42 | Mini-SWE-agent | — |
| 77 | LLaguna M.1 | Poolside | 57.60% | N/A | Mini-SWE-agent | — |
| 78 | LLaguna XS.2 | Poolside | 55.20% | N/A | Mini-SWE-agent | — |
| 79 | Gemini 2.5 Pro | 54.40% | $0.35 | Mini-SWE-agent | $1.25 / $10 | |
| 80 | Nemotron 3.5 Lightning | Fireworks AI | 52.80% | $0.03 | Mini-SWE-agent | $0.07 / $0.20 |
| 81 | Grok 4 Fast | xAI | 45.40% | $0.05 | Mini-SWE-agent | — |
| 82 | Grok 4.1 Fast | xAI | 41.40% | $0.03 | Mini-SWE-agent | — |
| 83 | Mistral Large | Mistral AI | 41.40% | $0.32 | Mini-SWE-agent | $0.50 / $1.50 |
| 84 | gpt-oss-120b | OpenAI | 33.60% | $0.17 | Mini-SWE-agent | $0.15 / $0.60 |
| 85 |  Command A | Cohere | 7.80% | $0.21 | Mini-SWE-agent | $0.30 / $1.50 |