MiMo-V2.6-Pro
ERNIE 5.1
amazon-nova-experimental-chat
Dola Seed 2.0 Pro
NVIDIA-Nemotron-3-Ultra-550B-A55B
Gemma 4 31B
Step 5 PreviewMedQA leaderboard
Medical licensing exam questions, measured on accuracy and on robustness to bias. o1 leads with 96.52%.
- Models
- 76
- Leader
- o1
- Source
- Vals AI
- As of
- Oct 24, 2026
- 1
o196.5% - 2GPT-5.196.4%
- 3
Gemini 3.1 Pro Preview96.4% - 4GPT-596.3%
- 5GPT-5.496.1%
- 6o396.1%
- 7GPT-5 mini96.1%
- 8Gemini 3 Pro96.0%
- 9o4-mini96.0%
- 10Gemini 3 Flash95.8%
Open weightsProprietaryBest for the price
Full leaderboard
| Rank | Model | Lab | Accuracy | Cost / test | Harness | Price in / out |
|---|---|---|---|---|---|---|
| 1 | o1 | OpenAI | 96.52% | $15 / $60 | — | $15 / $60 |
| 2 | GPT-5.1 | OpenAI | 96.38% | $1.25 / $10 | — | $1.25 / $10 |
| 3 | Gemini 3.1 Pro Preview | 96.37% | $2 / $12 | — | $2 / $12 | |
| 4 | GPT-5 | OpenAI | 96.32% | $1.25 / $10 | — | $1.25 / $10 |
| 5 | GPT-5.4 | OpenAI | 96.09% | $2.5 / $15 | — | $2.50 / $15 |
| 6 | o3 | OpenAI | 96.06% | $2 / $8 | — | $2 / $8 |
| 7 | GPT-5 mini | OpenAI | 96.06% | $0.25 / $2 | — | $0.25 / $2 |
| 8 | Gemini 3 Pro | 96.03% | $2 / $12 | — | — | |
| 9 | o4-mini | OpenAI | 96.02% | $1.1 / $4.4 | — | $1.10 / $4.40 |
| 11 | Gemini 3 Flash | 95.81% | $0.5 / $3 | — | $0.50 / $3 | |
| 12 |  Claude Opus 4.6 | Anthropic | 95.41% | $5 / $25 | — | $5 / $25 |
| 13 |  Qwen 3.5 Plus | Alibaba | 95.21% | $0.4 / $2.4 | — | $0.30 / $1.80 |
| 14 | o3-mini | OpenAI | 94.83% | $1.1 / $4.4 | — | $1.10 / $4.40 |
| 15 | Claude Sonnet 4.5 | Anthropic | 94.71% | $3 / $15 | — | $3 / $15 |
| 16 |  Grok 4.20 | xAI | 94.55% | $2 / $6 | — | $1.25 / $2.50 |
| 17 |  Kimi K2.5 | Moonshot AI | 94.37% | $0.6 / $3 | — | $0.50 / $2.50 |
| 18 |  GLM-5 | Z.ai | 94.27% | $1 / $3.2 | — | $1 / $3.20 |
| 19 | GPT-5.2 | OpenAI | 94.13% | $1.75 / $14 | — | $1.75 / $14 |
| 21 | GLM-4.7 | Z.ai | 93.74% | $0.6 / $2.2 | — | $0.54 / $1.98 |
| 23 | GPT-5 nano | OpenAI | 93.26% | $0.05 / $0.4 | — | $0.05 / $0.40 |
| 24 | Claude Opus 4.5 | Anthropic | 93.16% | $5 / $25 | — | $5 / $25 |
| 25 | Gemini 2.5 Pro | 93.14% | $1.25 / $10 | — | $1.25 / $10 | |
| 26 | Claude Opus 4 | Anthropic | 92.87% | $15 / $75 | — | — |
| 28 | Kimi K2 Thinking | Moonshot AI | 92.59% | $0.6 / $2.5 | — | $0.60 / $2.50 |
| 29 | Claude Opus 4.1 | Anthropic | 92.53% | $15 / $75 | — | $15 / $75 |
| 30 |  MiniMax-M2.5 | MiniMax | 92.53% | $0.3 / $1.2 | — | $0.30 / $1.20 |
| 31 | Grok 4 | xAI | 92.49% | $3 / $15 | — | — |
| 32 | Grok 2 | xAI | 92.32% | $2 / $10 | — | — |
| 33 | GLM-4.6 | Z.ai | 92.22% | $0.6 / $2.2 | — | $0.55 / $2.20 |
| 34 | Grok 4.1 Fast | xAI | 92.08% | $0.2 / $0.5 | — | — |
| 36 | Claude Sonnet 4.6 | Anthropic | 92.06% | $3 / $15 | — | $3 / $15 |
| 38 | gpt-oss-120b | OpenAI | 91.36% | $0.15 / $0.6 | — | $0.15 / $0.60 |
| 39 | GPT-4.1 | OpenAI | 91.18% | $2 / $8 | — | $2 / $8 |
| 41 | MiniMax-M2.1 Preview | MiniMax | 91.16% | $0.3 / $1.2 | — | $0.30 / $1.20 |
| 43 |  DeepSeek-R1 | DeepSeek | 90.80% | $1.35 / $5.4 | — | $0.50 / $2.15 |
| 44 | Qwen3-235B-A22B | Alibaba | 90.62% | $0.22 / $0.88 | — | $0.09 / $0.55 |
| 45 | Claude Sonnet 4 | Anthropic | 90.35% | $3 / $15 | — | — |
| 46 | Claude 3.7 Sonnet | Anthropic | 90.22% | $3 / $15 | — | — |
| 48 | GLM-4.5 | Z.ai | 89.97% | $0.6 / $2.2 | — | $0.60 / $2.20 |
| 49 |  Magistral Medium | Mistral AI | 89.47% | $2 / $5 | — | — |
| 50 | DeepSeek-V3.2-Exp | DeepSeek | 89.45% | $0.56 / $1.68 | — | $0.28 / $0.42 |
| 52 | Grok 3 Mini Reasoning | xAI | 88.65% | $0.3 / $0.5 | — | — |
| 54 | GPT-4o | OpenAI | 88.16% | $2.5 / $10 | — | $2.50 / $10 |
| 56 | Qwen3-Max-Preview | Alibaba | 87.37% | $1.2 / $6 | — | — |
| 57 | Gemini 2.5 Flash Preview 4/17 | 86.73% | $0.3 / $2.5 | — | — | |
| 59 | GPT-4.1 mini | OpenAI | 84.63% | $0.4 / $1.6 | — | $0.40 / $1.60 |
| 60 | Kimi K2 Instruct | Moonshot AI | 83.97% | $1 / $3 | — | — |
| 61 | Grok 3 Preview | xAI | 83.85% | $3 / $15 | — | — |
| 62 | Claude 3.5 Sonnet | Anthropic | 83.19% | $3 / $15 | — | — |
| 63 | gpt-oss-20b | OpenAI | 82.88% | $0.07 / $0.3 | — | $0.018 / $0.09 |
| 67 | GPT-4 Turbo | OpenAI | 81.99% | $10 / $30 | — | $10 / $30 |
| 68 | Gemini 2.0 Flash | 81.47% | $0.1 / $0.4 | — | — | |
| 69 | DeepSeek-V3 | DeepSeek | 80.90% | $0.9 / $0.9 | — | $0.29 / $1.14 |
| 70 |  Command A | Cohere | 80.55% | $2.5 / $10 | — | $0.30 / $1.50 |
| 71 | Gemini 2.5 Flash Preview | 80.33% | $0.1 / $0.4 | — | — | |
| 72 | Claude Haiku 4.5 | Anthropic | 79.57% | $1 / $5 | — | $1 / $5 |
| 73 | Mistral Medium | Mistral AI | 78.23% | $0.4 / $2 | — | — |
| 74 | Qwen 2.5 Instruct Turbo | Alibaba | 77.39% | $1.2 / $1.2 | — | — |
| 75 | Gemini 1.5 Pro | 76.53% | $1.25 / $5 | — | — | |
| 76 | Mistral Large | Mistral AI | 76.22% | $2 / $6 | — | $0.50 / $1.50 |
| 77 | Grok 4.1 Fast Non-Reasoning | xAI | 76.03% | $0.2 / $0.5 | — | — |
| 78 | Grok 4 Fast | xAI | 75.36% | $0.2 / $0.5 | — | — |
| 79 | GPT-4o mini | OpenAI | 72.44% | $0.15 / $0.6 | — | $0.15 / $0.60 |
| 80 | Magistral Small 1.2 | Mistral AI | 69.10% | $0.075 / $0.3 | — | — |
| 81 | GPT-4.1 nano | OpenAI | 68.22% | $0.1 / $0.4 | — | $0.10 / $0.40 |
| 82 |  Jamba 1.5 Large | AI21 Labs | 68.11% | $2 / $8 | — | — |
| 83 |  Llama 3.1 Instruct Turbo | Meta | 62.61% | $0.18 / $0.18 | — | — |
| 84 | GPT-3.5 Turbo | OpenAI | 58.47% | $0.5 / $1.5 | — | $1 / $2 |
| 85 | Mistral Small | Mistral AI | 56.98% | $0.2 / $0.6 | — | — |
| 86 | Jamba 1.5 Mini | AI21 Labs | 55.18% | $0.2 / $0.4 | — | — |
| 87 | MMixtral | — | 53.22% | $0.6 / $0.6 | — | — |
| 88 | Jamba 1.6 Mini | AI21 Labs | 52.52% | $0.2 / $0.4 | — | — |
| 89 | Llama 4 Scout | Meta | 50.90% | $0.18 / $0.59 | — | $0.10 / $0.30 |
| 90 | Jamba 1.6 Large | AI21 Labs | 50.70% | $2 / $8 | — | — |
| 91 | Llama 4 Maverick | Fireworks AI | 43.30% | $0.22 / $0.88 | — | $0.19 / $0.65 |
| 92 | Command R+ | Cohere | 2.65% | $2.5 / $10 | — | $0.15 / $0.60 |