Our ranking sources

Arena Text (overall)

Who runs it

Arena (formerly LMArena), which grew out of the LMSYS research group at UC Berkeley.

How it works

People type a prompt, get answers from two anonymous models side by side and vote for the better one. Arena turns millions of votes into a score (Bradley-Terry model with style control, so longer or fancier answers don't win just for looking nice). The ± is a 95% confidence interval.

Good at

Real people judging real prompts; hard to game by training on the test; covers almost every major chat model.

Weak spots

Measures which answer people prefer, not whether it is correct. New models have wide intervals (marked preliminary). Most voters write in English.

Licence

Leaderboard data CC-BY-4.0 (lmarena-ai/leaderboard-dataset).

We use it for

Write & email, Free only (Arena Text (overall))

Source

arena.ai/leaderboard/text

Arena Text-to-Image, Image Edit and Text-to-Video

Who runs it

Arena (formerly LMArena).

How it works

Same blind side-by-side voting as the text board, but people compare two generated images, two edits of the same picture, or two videos.

Good at

Taste in images and video is subjective, and blind human votes are the most direct way to measure it.

Weak spots

Says nothing about price per image, speed, resolution limits or commercial-use terms. A plan may use a lower-quality setting than the one ranked.

Licence

Leaderboard data CC-BY-4.0 (lmarena-ai/leaderboard-dataset).

We use it for

Create images, Create videos (Arena Text-to-Image, Arena Image Edit, Arena Text-to-Video)

Source

arena.ai/leaderboard/text-to-image

LiveBench 2.0 (Reasoning, Coding, Overall, cost per successful task)

Who runs it

LiveBench, an academic and industry research team (Abacus.AI, NYU and others).

How it works

Questions with objective right answers in maths, reasoning, coding, data analysis and more, scored automatically. New questions are added regularly so models can't have memorised them.

Good at

Objective scoring, no human or AI judge; regularly refreshed to limit test contamination; publishes cost per successful task.

Weak spots

A fixed mix of question types that may not match your work. Cost figures depend on API prices, not subscription prices.

Licence

Questions and code Apache-2.0.

We use it for

Research, Study, Code & apps (LiveBench 2.0 Reasoning, LiveBench 2.0 Coding, LiveBench 2.0 Overall, LiveBench 2.0 cost per successful task)

Source

¹

SWE-bench Pro (public set)

Who runs it

Scale AI.

How it works

The model gets real bug reports from real software projects and has to write a fix that passes the project's own tests. The score is the share of tasks resolved.

Good at

Close to real software work; harder and less leaked than the original SWE-bench.

Weak spots

Results depend on the agent setup around the model, and the page has no date. Its newest listed models are older than today's flagships, so we treat it as possibly out of date.

Licence

Scores cited from Scale's public leaderboard; see Scale's terms.

We use it for

Model pages and the full rankings table (SWE-bench Pro (public set))

Source

scale.com/leaderboard/swe_bench_pro_public

ARC-AGI-2 and ARC-AGI-3 (semi-private)

Who runs it

The ARC Prize Foundation.

How it works

Visual puzzles that are easy for people but hard for AI. ARC-AGI-3 uses small interactive games. Scored on a semi-private set so models can't train on the answers; cost per task is published too.

Good at

Tests learning something new on the spot rather than recall, and reports cost.

Weak spots

Narrow: good puzzle scores don't guarantee better writing, research or coding.

Licence

Scores cited from arcprize.org; see the ARC Prize terms.

We use it for

Model pages and the full rankings table (ARC-AGI-2 (semi-private), ARC-AGI-3 (semi-private))

Source

arcprize.org/leaderboard

Epoch Capabilities Index (ECI)

Who runs it

Epoch AI, an independent research group.

How it works

Combines results from 50+ benchmarks into one score on a common scale.

Good at

Broad: one number that reflects many tests.

Weak spots

A composite, so it depends on which tests are included; fewer models are scored, and new ones appear later.

Licence

CC-BY.

We use it for

Model pages and the full rankings table (Epoch Capabilities Index (ECI))

Source

epoch.ai/benchmarks/eci

Agent rankings

Agents & automation (page): we score each agent on a five-point checklist taken from the provider's own pages: does it browse the web, use apps or a computer, run on a schedule, keep working in the background, and handle email or calendar? “Yes” counts 1, “limited” counts half, and anything we can't confirm counts 0. Plans are sorted by that score, then by overall model quality on Arena Text. Research previews and waitlist-only agents (such as Claude Duet, US only) are listed on the agents page but not ranked.

Best free agent and Best free downloadable agent (free, downloadable): the same checklist, limited to free tiers or free open-source agents, with easier setup and then (for downloadable agents) more supported operating systems breaking ties.

Why no agent benchmark? We looked at Terminal-Bench 2.0 (tbench.ai), which tests coding agents in a terminal, and OSWorld-Verified, which tests computer use. Neither tests the consumer agents people actually buy (Grok Bot, Claude, ChatGPT agent, Gemini in Chrome), and Terminal-Bench's top entries are custom research setups running older models. Using them would rank products on tests they never took, so we don't. We'll add one when a reputable benchmark tests these products directly.

Why isn't one company always on top? The checklist only counts what each provider's page confirms. Up to US$30 a month the top three today are X Subscription Premium (includes Grok) (5 of 5), Claude Pro (4 of 5), Kimi Plus (2.5 of 5); ties go to the plan whose model ranks higher on Arena Text.