ERNIE 5.1
amazon-nova-experimental-chat
Dola Seed 2.0 Pro
NVIDIA-Nemotron-3-Ultra-550B-A55B
Gemma 4 31BTerminal-Bench Science leaderboard
Expert-written scientific research workflows, run by an agent in a terminal. GPT-6 Astra leads with 62.86%.
- Models
- 42
- Leader
- GPT-6 Astra
- Source
- Vals AI
- As of
- Oct 24, 2026
- 1
GPT-6 Astra62.9% - 2GPT-6.1 Sol52.9%
- 3
Claude Opus 5.547.1% - 4Claude Sonnet 5.547.1%
- 5
Gemini 4 Argon44.3% - 6Claude Fable 5.140.0%
- 7
Muse Wonder Full37.1% - 8GPT-6 Sol30.0%
- 9Claude Opus 527.1%
- 10Muse Wonder22.9%
Open weightsProprietaryBest for the price
Full leaderboard
| Rank | Model | Lab | Accuracy | Cost / test | Harness | Price in / out |
|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra | OpenAI | 62.86% | $20.80 | Mini-SWE-agent | $10 / $50 |
| 2 | GPT-6.1 Sol | OpenAI | 52.86% | $3.44 | Mini-SWE-agent | $2 / $10 |
| 3 | Claude Opus 5.5 | Anthropic | 47.14% | $19.12 | Mini-SWE-agent | $4 / $20 |
| 4 | Claude Sonnet 5.5 | Anthropic | 47.14% | $32.52 | Mini-SWE-agent | $2 / $10 |
| 5 | Gemini 4 Argon | 44.29% | $44.90 | Mini-SWE-agent | $2 / $10 | |
| 6 | Claude Fable 5.1 | Anthropic | 40.00% | $38.01 | Mini-SWE-agent | $10 / $50 |
| 7 | Muse Wonder Full | Meta | 37.14% | $22.81 | Mini-SWE-agent | — |
| 8 | GPT-6 Sol | OpenAI | 30.00% | $5.82 | Mini-SWE-agent | $2 / $10 |
| 9 | Claude Opus 5 | Anthropic | 27.14% | $32.54 | Mini-SWE-agent | $5 / $25 |
| 10 | Muse Wonder | Meta | 22.86% | $19.61 | Mini-SWE-agent | $4 / $20 |
| 11 | GPT-5.6 Sol | OpenAI | 20.00% | $7.32 | Mini-SWE-agent | $4 / $20 |
| 12 | Claude Fable 5 | Anthropic | 15.71% | $51.99 | Mini-SWE-agent | $10 / $50 |
| 13 |  Qwen3.8-Max | Alibaba | 12.86% | $16.68 | Mini-SWE-agent | $2 / $6 |
| 14 |  Grok 4.6 | xAI | 11.43% | $4.82 | Mini-SWE-agent | $2 / $6 |
| 15 | Muse Spark 1.3 Max | Meta | 10.00% | $6.09 | Mini-SWE-agent | — |
| 16 | Grok 4.7 | xAI | 10.00% | $17.04 | Mini-SWE-agent | $2 / $6 |
| 17 | Claude Haiku 5.5 | Anthropic | 10.00% | $4.52 | Mini-SWE-agent | $0.10 / $0.50 |
| 18 | GPT-5.6 Terra | OpenAI | 10.00% | $5.19 | Mini-SWE-agent | $2 / $12 |
| 19 | Gemini 3.8 Flash | 8.57% | $5.64 | Mini-SWE-agent | $0.82 / $4.13 | |
| 20 | Gemini 3.7 Flash | 7.14% | $7.19 | Mini-SWE-agent | $0.75 / $3.75 | |
| 21 | Gemini 3.5 Flash | 5.71% | $3.80 | Mini-SWE-agent | $1.50 / $9 | |
| 22 |  GLM-5.3 | Z.ai | 5.71% | $15.63 | Mini-SWE-agent | $0.079 / $1.40 |
| 23 | Claude Sonnet 5 | Anthropic | 5.71% | $30.30 | Mini-SWE-agent | $2 / $10 |
| 24 | EEmber-1 | Fireworks AI | 5.71% | $30.76 | Mini-SWE-agent | $3 / $15 |
| 25 | Muse Spark 1.3 | Meta | 4.29% | $4.41 | Mini-SWE-agent | $1.25 / $4.25 |
| 26 | GPT-6 Luna | OpenAI | 4.29% | $0.24 | Mini-SWE-agent | $0.10 / $0.50 |
| 27 |  DeepSeek-V4-Flash | DeepSeek | 4.29% | $1.10 | Mini-SWE-agent | $0.042 / $0.18 |
| 28 | Gemini 3.6 Flash | 4.29% | $4.39 | Mini-SWE-agent | $0.75 / $3.75 | |
| 29 | Claude Opus 4.8 | Anthropic | 4.29% | $23.14 | Mini-SWE-agent | $5 / $25 |
| 30 | DeepSeek-V4-Pro | DeepSeek | 4.29% | $4.63 | Mini-SWE-agent | $0.66 / $1.98 |
| 31 |  MiMo-V2.6-Flash | Xiaomi | 4.29% | $0.24 | Mini-SWE-agent | $0.14 / $0.28 |
| 32 |  Step 5 Preview | StepFun | 2.86% | $4.25 | Mini-SWE-agent | $1 / $2.70 |
| 33 | MiMo-V2.6-Pro | Xiaomi | 2.86% | $0.62 | Mini-SWE-agent | $0.43 / $0.87 |
| 34 | GLM-5.3-Flash | Z.ai | 2.86% | $0.52 | Mini-SWE-agent | $0.15 / $0.50 |
| 35 | Gemini 3.1 Pro Preview | 1.43% | $5.91 | Mini-SWE-agent | $2 / $12 | |
| 36 | BBeam 1.0 | Reflection AI | 1.43% | N/A | Mini-SWE-agent | — |
| 37 |  Hy4 preview | Tencent | 1.43% | $2.14 | Terminus 2 | $0.75 / $2.25 |
| 38 |  Kimi K3 | Moonshot AI | 1.43% | $21.30 | Mini-SWE-agent | $0.24 / $11 |
| 39 |  Mercury 2.5 | Inception | 0.00% | $0.18 | Mini-SWE-agent | $0.04 / $0.15 |
| 40 | GPT-5.6 Luna | OpenAI | 0.00% | $0.58 | Mini-SWE-agent | $0.20 / $1.20 |
| 41 | DeepSeek-V4.1-Flash-20260910 | DeepSeek | 0.00% | $0.47 | Mini-SWE-agent | $0.15 / $0.60 |
| 42 | Qwen3.8-27B | Alibaba | 0.00% | $11.14 | Mini-SWE-agent | $0.20 / $2.50 |