Benchmarks

Vals AI’s headline score: professional tasks in finance, law, medicine and coding, weighted by their share of the economy.

1. 1Claude Sonnet 5.570.8%
2. 2Claude Opus 5.570.2%
3. 3Gemini 4 Argon69.2%

Frontier-difficulty tasks an agent completes in a real terminal: building, configuring, debugging and running software.

1. 1Claude Opus 5.565.2%
2. 2Claude Sonnet 5.564.1%
3. 3Kimi K3.1 Pro63.6%

Real GitHub issues from open-source projects; a model must write a patch that makes the project’s tests pass.

1. 1Claude Opus 597.0%
2. 2GPT-5.6 Sol96.2%
3. 3Grok 4.695.6%

Building complete web apps from a plain-language brief, graded on whether the app works.

1. 1Claude Sonnet 5.592.4%
2. 2Gemini 4 Argon91.9%
3. 3Claude Haiku 5.590.4%

Rewriting working programs in a new language, graded on whether the new program still passes its tests.

1. 1Claude Sonnet 5.586.9%
2. 2Claude Opus 5.581.0%
3. 3GPT-6 Astra80.6%

Fresh competitive programming problems, collected after models’ training cut-offs to avoid contamination.

1. 1Claude Fable 5.190.5%
2. 2Claude Fable 589.8%
3. 3Gemini 3.8 Flash89.5%

The core tasks of a financial analyst, done as an agent with research tools.

1. 1Claude Opus 5.567.8%
2. 2Gemini 4 Argon67.7%
3. 3Muse Wonder Full67.6%

Vals AI’s composite of finance tasks: corporate filings, tax, mortgage and financial analysis.

1. 1Claude Opus 5.565.6%
2. 2Gemini 4 Argon64.9%
3. 3Muse Wonder Full64.5%

Agentic research into US corporate tax questions.

1. 1Claude Fable 5.149.2%
2. 2Claude Opus 5.545.6%
3. 3Claude Opus 545.5%

Vals AI’s composite of legal tasks: case law, legal reasoning and legal research.

1. 1Muse Wonder Full77.9%
2. 2Gemini 4 Argon76.7%
3. 3Muse Wonder75.3%

Agentic US legal research: finding and applying the authority that answers a question.

1. 1Muse Spark 1.3 Max55.3%
2. 2Claude Opus 555.3%
3. 3Claude Fable 5.155.3%

Assigning the right medical billing codes from clinical notes.

1. 1Claude Opus 563.6%
2. 2Gemini 3.1 Pro Preview59.1%
3. 3Gemini 4 Argon58.8%

Medical licensing exam questions, measured on accuracy and on robustness to bias.

1. 1o196.5%
2. 2GPT-5.196.4%
3. 3Gemini 3.1 Pro Preview96.4%

Graduate-level questions in biology, physics and chemistry that experts answer well and search does not.

1. 1Gemini 3.1 Pro Preview95.5%
2. 2GPT-5.6 Sol95.2%
3. 3Grok 4.694.7%

Harder, reasoning-focused multiple choice across 14 subjects, from law to engineering.

1. 1Claude Fable 5.192.4%
2. 2Claude Opus 591.6%
3. 3Claude Fable 591.5%

Problems from the American Invitational Mathematics Examination, a qualifier for the US Math Olympiad.

1. 1Gemini 3.1 Pro Preview98.1%
2. 2GPT-5.296.9%
3. 3Muse Spark96.9%

College-level questions that need both an image and text: charts, diagrams, tables and figures.

1. 1Claude Fable 5.190.6%
2. 2Claude Opus 589.9%
3. 3Claude Fable 589.3%

Expert-written scientific research workflows, run by an agent in a terminal.

1. 1GPT-6 Astra62.9%
2. 2GPT-6.1 Sol52.9%
3. 3Claude Opus 5.547.1%