Blog / Research
Auto Router v0.1.0: benchmarks
Score against cost for every model, and where Auto lands on the frontier.
Auto splits a request into steps and sends each to the cheapest model that can do it. This post explains how we measure that: every model plotted by score against cost per task, the best single models as a frontier, and Auto on it.
What we measure
- Public benchmarks: SWE-bench Verified, Terminal-Bench and GPQA
- Your own tasks: a sample from your repositories, scored by your tests
- Cost per task at each model’s list price; score as each benchmark defines it
How Auto picks
Each step of a request is classified (read, reason, write code, summarise) and sent to the cheapest model that clears the bar for that kind of step on your own history. Every choice comes back with a reason, so you can see why a step went where it did.
Result
| Item | Score | Cost per task |
|---|---|---|
| nexus/auto | 91 | $0.36 |
| Always Claude Opus 5.5 | 92 | $1.25 |
| Always GPT-5.3-Codex | 85 | $0.24 |
Auto keeps almost all of the strongest model’s score at a fraction of its cost, because most steps do not need the strongest model.
What’s next
- Run the benchmark on your own tasks from the Benchmarks page
- A reason for every routing choice in every API response
- More labs on the frontier as they join the catalog
Nexus
OpenRouter
LiteLLM