ERNIE 5.1
amazon-nova-experimental-chat
Dola Seed 2.0 Pro
NVIDIA-Nemotron-3-Ultra-550B-A55B
Gemma 4 31BTerminal-Bench 4.0 leaderboard
Frontier-difficulty tasks an agent completes in a real terminal: building, configuring, debugging and running software. Claude Opus 5.5 leads with 65.15%.
- Models
- 52
- Leader
- Claude Opus 5.5
- Source
- Vals AI
- As of
- Oct 24, 2026
- 1
Claude Opus 5.565.2% - 2Claude Sonnet 5.564.1%
- 3
Kimi K3.1 Pro63.6% - 4
GPT-6 Astra59.6% - 5Claude Fable 5.158.1%
- 6
Muse Wonder Full58.1% - 7
Gemini 4 Argon57.6% - 8GPT-6.1 Sol55.0%
- 9Claude Opus 553.5%
- 10Muse Wonder49.5%
Open weightsProprietaryBest for the price
Full leaderboard
| Rank | Model | Lab | Accuracy | Cost / test | Harness | Price in / out |
|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Anthropic | 65.15% | $13.20 | Mini-SWE-agent | $4 / $20 |
| 2 | Claude Sonnet 5.5 | Anthropic | 64.14% | $16.51 | Mini-SWE-agent | $2 / $10 |
| 3 | Kimi K3.1 Pro | Moonshot AI | 63.64% | $4.57 | Mini-SWE-agent | $2 / $10 |
| 4 | GPT-6 Astra | OpenAI | 59.60% | $9.58 | Mini-SWE-agent | $10 / $50 |
| 5 | Claude Fable 5.1 | Anthropic | 58.08% | $17.18 | Mini-SWE-agent | $10 / $50 |
| 6 | Muse Wonder Full | Meta | 58.08% | $10.53 | Mini-SWE-agent | — |
| 7 | Gemini 4 Argon | 57.58% | $17.64 | Mini-SWE-agent | $2 / $10 | |
| 8 | GPT-6.1 Sol | OpenAI | 55.05% | $1.72 | Mini-SWE-agent | $2 / $10 |
| 9 | Claude Opus 5 | Anthropic | 53.53% | $18.60 | Mini-SWE-agent | $5 / $25 |
| 10 | Muse Wonder | Meta | 49.49% | $14.66 | Mini-SWE-agent | $4 / $20 |
| 11 | GPT-6 Sol | OpenAI | 44.44% | $5.79 | Mini-SWE-agent | $2 / $10 |
| 12 | Claude Fable 5 | Anthropic | 41.41% | $30.34 | Mini-SWE-agent | $10 / $50 |
| 13 |  GLM-5.3 | Z.ai | 38.89% | $9.37 | Mini-SWE-agent | $0.079 / $1.40 |
| 15 | GPT-5.6 Sol | OpenAI | 37.88% | $7.98 | Mini-SWE-agent | $4 / $20 |
| 16 | GLM-5.3-Flash | Z.ai | 37.37% | $1.22 | Mini-SWE-agent | $0.15 / $0.50 |
| 17 | Claude Haiku 5.5 | Anthropic | 36.36% | $3.12 | Mini-SWE-agent | $0.10 / $0.50 |
| 18 |  Qwen3.8-Max | Alibaba | 34.34% | $10.63 | Mini-SWE-agent | $2 / $6 |
| 20 |  Step 5 Preview | StepFun | 31.82% | $3.11 | Mini-SWE-agent | $1 / $2.70 |
| 21 |  MiMo-V2.6-Pro | Xiaomi | 31.31% | $0.50 | Mini-SWE-agent | $0.43 / $0.87 |
| 22 |  Grok 4.7 | xAI | 28.79% | $18.01 | Mini-SWE-agent | $2 / $6 |
| 23 |  Mistral Large 4 | Mistral AI | 27.78% | $10.25 | Mini-SWE-agent | $1.36 / $4.18 |
| 24 |  MiniMax-M3.1-Flash | MiniMax | 25.25% | $0.73 | Mini-SWE-agent | $0.15 / $0.60 |
| 25 | Muse Spark 1.3 Max | Meta | 24.75% | $6.65 | Mini-SWE-agent | — |
| 26 | MiMo-V2.6-Flash | Xiaomi | 24.24% | $0.22 | Mini-SWE-agent | $0.14 / $0.28 |
| 27 | Claude Opus 4.8 | Anthropic | 23.23% | $17.14 | Mini-SWE-agent | $5 / $25 |
| 28 | GPT-5.6 Terra | OpenAI | 22.73% | $5.60 | Mini-SWE-agent | $2 / $12 |
| 29 | EEmber-1 | Fireworks AI | 20.71% | $19.48 | Mini-SWE-agent | $3 / $15 |
| 30 |  DeepSeek-V4.1-Flash-20260910 | DeepSeek | 19.70% | $0.50 | Mini-SWE-agent | $0.15 / $0.60 |
| 31 | Gemini 3.8 Flash | 19.70% | $8.65 | Mini-SWE-agent | $0.82 / $4.13 | |
| 32 | DeepSeek-V4-Flash | DeepSeek | 18.69% | $0.77 | Mini-SWE-agent | $0.042 / $0.18 |
| 33 | Grok 4.6 | xAI | 17.17% | $5.06 | Mini-SWE-agent | $2 / $6 |
| 34 | Kimi K3 | Moonshot AI | 17.17% | $12.48 | Mini-SWE-agent | $0.24 / $11 |
| 36 | GPT-6 Luna | OpenAI | 13.64% | $0.35 | Mini-SWE-agent | $0.10 / $0.50 |
| 37 | Gemini 3.7 Flash | 12.12% | $8.71 | Mini-SWE-agent | $0.75 / $3.75 | |
| 38 | GPT-5.6 Luna | OpenAI | 11.62% | $0.73 | Mini-SWE-agent | $0.20 / $1.20 |
| 39 | DeepSeek-V4-Pro | DeepSeek | 11.11% | $3.45 | Mini-SWE-agent | $0.66 / $1.98 |
| 40 | Claude Sonnet 5 | Anthropic | 10.61% | $26.05 | Mini-SWE-agent | $2 / $10 |
| 41 | Muse Spark 1.3 | Meta | 10.61% | $7.04 | Mini-SWE-agent | $1.25 / $4.25 |
| 42 | Grok 4.5 | xAI | 8.59% | $5.52 | Mini-SWE-agent | $2 / $6 |
| 43 |  Hy4 preview | Tencent | 8.08% | $2.10 | Mini-SWE-agent | $0.75 / $2.25 |
| 44 | Muse Spark Open | Meta | 6.57% | $7.26 | Mini-SWE-agent | $1.25 / $4.25 |
| 45 | Gemini 3.5 Flash | 6.06% | $5.98 | Mini-SWE-agent | $1.50 / $9 | |
| 46 | Muse Spark 1.2 | Meta | 6.06% | $4.14 | Mini-SWE-agent | $1.25 / $4.25 |
| 47 | BBeam 1.0 | Reflection AI | 4.04% | N/A | Mini-SWE-agent | — |
| 48 |  Mercury 3 | Inception | 3.03% | $0.42 | Mini-SWE-agent | $0.10 / $0.35 |
| 49 | Gemini 3.1 Pro Preview | 2.52% | $4.11 | Mini-SWE-agent | $2 / $12 | |
| 50 | GPT-5.4 mini | OpenAI | 2.52% | $1.44 | Mini-SWE-agent | $0.75 / $4.50 |
| 51 | MiniMax-M3 | MiniMax | 1.01% | $7.23 | Mini-SWE-agent | $0.30 / $1.20 |
| 52 | IInkling | Thinky | 0.51% | $2.50 | Mini-SWE-agent | $1 / $4.05 |
| 53 | MiMo-V2.5-Pro | Xiaomi | 0.51% | $0.23 | Mini-SWE-agent | $0.30 / $0.61 |
| 54 | IInkling-Small | Thinky | 0.00% | $0.32 | Mini-SWE-agent | $0.45 / $1.20 |
| 55 | Mercury 2.5 | Inception | 0.00% | $0.37 | Mini-SWE-agent | $0.04 / $0.15 |