We believe that picking the right model has gotten harder than it needs to be. There are many releases, competitive open weight models, token prices that keep changing, and benchmark coverage that is patchy and inconsistent. Single leaderboard scores rarely describe how a model will perform across coding, agentic work, tool use, and general reasoning.1 We built yet another “Find a Cheaper Model” tool around the question of “I like model X, what are the closest alternatives, and which ones cost less?”

We collect pricing, model metadata, context limits, open-weight information, parameter counts, and benchmark results from public sources, normalized these into a common representation and combined with our own evaluation data. Especially public benchmarks need some care here, as they use different scoring scales, saturate at different points, and measure different kinds of capability. Coding benchmarks such as LiveCodeBench test a different slice of performance from interactive agent benchmarks such as τ-bench, which evaluates agents operating tools while following domain-specific policies.2 3

Where benchmark scales overlap across enough models, we calibrate them before including them in the final rating. If we cannot calibrate a benchmark with reasonable confidence, its raw score stays in the dataset but does not affect the aggregate.

Capability is split into five facets:

intelligence: 0.18
coding:       0.29
agentic:      0.22
reasoning:    0.06
tool_use:     0.15

Our defaults lean toward engineering as we deem it a good general proxy. Coding gets the largest weight because substantial programming tasks are a useful test of whether a model can turn complicated instructions into working output. Agentic tasks and tool use also carry significant weight. Pure reasoning gets less on its own since some of that signal already appears in broader intelligence evaluations.

Tests within each facet are not weighted equally either. An end-to-end software task tells us more about engineering usefulness than a small code-completion benchmark. A multi-step agent run tells us more than whether a model can produce a syntactically valid tool call. We find that most public benchmark suites measure substantially different capabilities. Current independent evaluations increasingly separate coding, agentic performance, tool use, knowledge, reasoning, and other dimensions rather than treating model quality as a single observable quantity.1

We track one of those differences explicitly:

Practical =
    0.4 × Coding
  + 0.4 × Agentic
  + 0.2 × Tool Use

Practical Gap =
    Practical - Intelligence

A model with Intelligence 90 and Practical 45 is a different proposition from one at 82 and 78. A single aggregate ranking can put them surprisingly close together. For now, we do not use it as a penalty. The underlying dimensions remain visible so the ranking can be inspected rather than taken at face value. Pricing is calculated separately from input and output token rates. Our tool supports workload profiles because token usage varies substantially by task. A coding agent with large inputs and long outputs has a different cost profile from a short chat request, and the relevant capability weights may differ as well.

You can use the tool to find:

  • comparable models that cost less
  • models that are both better and cheaper
  • open-weight models, optionally below a parameter limit
  • the highest capability per dollar
  • side-by-side comparisons of up to ten models

Our own benchmark adjustments are applied when the dataset is generated rather than written back into the collected source data. This keeps the source inputs reproducible while allowing the ranking to change when public benchmark results disagree with what we observe in actual use. Our resulting pipeline is:

public data + our benchmarks
        ↓
normalized model data
        ↓
calibration + weighting + scoring
        ↓
generated comparison dataset
        ↓
Model Finder

There is no runtime scoring service. Updating the tool is a new collection, calibration, and generation pass.

We are not trying to produce a universal model leaderboard. The useful output is an estimate of practical capability per dollar, with enough of the underlying measurements exposed to explain why a model scored the way it did. If the default weights do not describe your workload, change them.

  1. Artificial Analysis, AI Model Evaluations and Intelligence Index. https://artificialanalysis.ai/models/  ↩︎ ↩︎

  2. Jain et al., LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code. arXiv. https://arxiv.org/abs/2403.07974  ↩︎

  3. Yao et al., τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. arXiv:2406.12045. https://arxiv.org/abs/2406.12045  ↩︎