AI Benchmarks

Each line is one model, climbing from its lowest to its highest reasoning effort: up scores higher, right costs more per task.

Pick models in the Models list.
Hover or tap a dot for its effort, score and cost.
Cost runs on a log scale: each equal step right is the same multiple of price.

Every model ranked by the best score it reaches within your budget, with what that setting costs per task.

Drag the budget to the most you’ll pay per task.
Dots run low → max effort; rings are over budget.
Tap the dot by a name to add it to Curves.

Gauntlet Explain: how well each model explains hard open questions, graded 0–4 on Deutsch’s criteria, against what its answers cost to generate at Artificial Analysis’s output list price.

Up explains better; right costs more per answer.
A line joins only the efforts Gauntlet Explain ran; one effort is one dot.Average shows each model as one dot: the mean of its efforts’ grades and costs.
Hover or tap a dot for its grade, cost, three runs and gap to the top. The methodology has the questions, judges and statistics.

The cheapest way to reach each score. Each white step is a setting nothing cheaper matches; grey dots are beaten for less.

Find your score on the left; read across to the line.
The table shows what each next step up costs.
Pick models to see what beats their settings.

How fast reaching a score is getting cheaper: each line is the lowest cost per task to reach it with what was out by each date.

Lower is cheaper; each drop is a cheaper release.
The right-hand label is today’s cheapest price.
Intelligence, Coding or Agentic changes the score.

Every change Artificial Analysis made to the data, newest first.

Click a model to see its past scores.

Claude Opus 5.5 (low): grade 2.63, $0.037 per answer
Claude Opus 5.5 (medium): grade 2.74, $0.046 per answer
Claude Opus 5.5 (high): grade 2.73, $0.052 per answer
Claude Opus 5.5 (xhigh): grade 2.80, $0.080 per answer
Claude Opus 5.5 (max): grade 3.11, $1.11 per answer
Claude Fable 5.1 (low): grade 2.74, $0.078 per answer
Claude Fable 5.1 (medium): grade 2.79, $0.096 per answer
Claude Fable 5.1 (high): grade 2.87, $0.12 per answer
Claude Fable 5.1 (xhigh): grade 2.93, $0.21 per answer
Claude Fable 5.1 (max): grade 3.03, $0.60 per answer
Claude Sonnet 5.5 (low): grade 2.42, $0.014 per answer
Claude Sonnet 5.5 (medium): grade 2.46, $0.015 per answer
Claude Sonnet 5.5 (high): grade 2.60, $0.024 per answer
Claude Sonnet 5.5 (xhigh): grade 2.67, $0.039 per answer
Claude Sonnet 5.5 (max): grade 2.84, $0.61 per answer
Muse Wonder (low): grade 2.52, $0.028 per answer
Muse Wonder (medium): grade 2.55, $0.031 per answer
Muse Wonder (high): grade 2.64, $0.033 per answer
Claude Haiku 5.5 (low): grade 1.95, $0.0011 per answer
Claude Haiku 5.5 (medium): grade 2.07, $0.0014 per answer
Claude Haiku 5.5 (high): grade 2.14, $0.0022 per answer
Claude Haiku 5.5 (xhigh): grade 2.27, $0.0040 per answer
Claude Haiku 5.5 (max): grade 2.52, $0.031 per answer
Gemini 4 Argon (default): grade 2.49, $0.021 per answer
GPT-6 Astra (low): grade 2.31, $0.040 per answer
GPT-6 Astra (medium): grade 2.31, $0.042 per answer
GPT-6 Astra (high): grade 2.33, $0.057 per answer
GPT-6 Astra (xhigh): grade 2.37, $0.12 per answer
Kimi K3.1 Pro (default): grade 2.30, $0.022 per answer
GPT-6.1 Sol (low): grade 2.21, $0.0078 per answer
GPT-6.1 Sol (medium): grade 2.19, $0.0095 per answer
GPT-6.1 Sol (high): grade 2.22, $0.015 per answer
GPT-6.1 Sol (xhigh): grade 2.30, $0.027 per answer
Grok 4.7 (default): grade 2.08, $0.013 per answer
GPT-6.1 Terra (low): grade 1.69, $0.0023 per answer
GPT-6.1 Terra (medium): grade 1.64, $0.0024 per answer
GPT-6.1 Terra (high): grade 1.70, $0.0029 per answer
GPT-6.1 Terra (xhigh): grade 1.67, $0.0042 per answer
GPT-6 Luna (low): grade 1.25, $0.0002 per answer
GPT-6 Luna (medium): grade 1.25, $0.0002 per answer
GPT-6 Luna (high): grade 1.51, $0.0004 per answer
GPT-6 Luna (xhigh): grade 1.51, $0.0009 per answer

Claude Opus 5.5 max leads every setting

Marker size is the reasoning effort: the bigger the mark, the more effort, and one effort is about the same size on every model.In Average each model is one dot: the mean grade and mean cost of the efforts below. Hover a mark for its exact grade and cost.

Claude Opus 5.5lowmediumhighxhighmax
Claude Fable 5.1lowmediumhighxhighmax
Claude Sonnet 5.5lowmediumhighxhighmax
Muse Wonderlowmediumhigh
Claude Haiku 5.5lowmediumhighxhighmax
Gemini 4 Argondefault
GPT-6 Astralowmediumhighxhigh
Kimi K3.1 Prodefault
GPT-6.1 Sollowmediumhighxhigh
Grok 4.7default
GPT-6.1 Terralowmediumhighxhigh
GPT-6 Lunalowmediumhighxhigh

Cost is what each model spent generating its answers to Gauntlet Explain’s questions: its answer and thinking tokens (mean per answer over 3 runs of 14 questions), priced at Artificial Analysis’s output list price as of 2026-11-09. The question itself is a few hundred input tokens, the same for every model, and is left out, because the input we measured also includes each answering tool’s own prompt. It is not Artificial Analysis’s own cost to run its index. Grades are Gauntlet Explain grades, on a 0–4 scale; the grade axis is cut to the range the models fall in. Full methodology

Gauntlet Science measures whether a model can do science on something it has never seen. Every run generates 24 worlds fresh, each with its own hidden law, so there is nothing to memorise. In each world the model reads noisy observations, runs 6 experiments it designs itself, states the law it found, then makes 20 predictions with intervals, half of them beyond anything it measured.

Ceiling scriptreferenceCeiling script, a reference script: score 89, 95% interval 83 to 9595% 83–95
1Claude Fable 5.1Claude Fable 5.1: score 89, 95% interval 83 to 94, the leader95% 83–94Leader
2GPT-6.1 SolGPT-6.1 Sol: score 85, 95% interval 80 to 90, −3.5 points from the leader, 95% interval −6.8 to −0.1, separated95% 80–90−3.595% −6.8 to −0.1● Separated
3Claude Opus 5.5Claude Opus 5.5: score 85, 95% interval 79 to 90, −3.9 points from the leader, 95% interval −7.9 to +0.1, not separated95% 79–90−3.995% −7.9 to +0.1Not separated
4GPT-6 AstraGPT-6 Astra: score 82, 95% interval 75 to 89, −6.6 points from the leader, 95% interval −11.6 to −1.6, separated95% 75–89−6.695% −11.6 to −1.6● Separated
5Muse WonderMuse Wonder: score 80, 95% interval 72 to 88, −8.1 points from the leader, 95% interval −17.1 to +0.8, not separated95% 72–88−8.195% −17.1 to +0.8Not separated
6Claude Sonnet 5.5Claude Sonnet 5.5: score 79, 95% interval 73 to 85, −9.5 points from the leader, 95% interval −13.2 to −5.9, separated95% 73–85−9.595% −13.2 to −5.9● Separated
7Kimi K3.1 ProKimi K3.1 Pro: score 76, 95% interval 65 to 87, −12.6 points from the leader, 95% interval −22.9 to −2.4, separated95% 65–87−12.695% −22.9 to −2.4● Separated
8GPT-6.1 TerraGPT-6.1 Terra: score 71, 95% interval 64 to 77, −17.9 points from the leader, 95% interval −24.6 to −11.2, separated95% 64–77−17.995% −24.6 to −11.2● Separated
9Claude Haiku 5.5Claude Haiku 5.5: score 67, 95% interval 58 to 77, −21.1 points from the leader, 95% interval −29.3 to −13.0, separated95% 58–77−21.195% −29.3 to −13.0● Separated
10gemini-4-argon (unverified)gemini-4-argon (unverified): score 61, 95% interval 48 to 75, −27.3 points from the leader, 95% interval −40.4 to −14.1, separated95% 48–75−27.395% −40.4 to −14.1● Separated
Floor scriptreferenceFloor script, a reference script: score 40, 95% interval 30 to 5195% 30–51
11GPT-6 LunaGPT-6 Luna: score 29, 95% interval 18 to 39, −59.8 points from the leader, 95% interval −72.4 to −47.1, separated95% 18–39−59.895% −72.4 to −47.1● Separated

Code wrote each world and knows its law, so code scores every prediction against the hidden law: no judge reads an answer. A score of 100 is every prediction exact and certain; 0 is a miss or no answer. The worlds climb three rungs of difficulty (8, 8 and 8 worlds), and each score carries a 95% interval over the worlds. The two grey rows are scripts, not models. The floor script runs random experiments and fits a straight line, so a model below it did no better than that. The ceiling script spreads its experiments to the edges and picks the best-fitting law from the families the worlds are drawn from, a practical top.

Before any model ran, we fixed 7 checks the run must pass to count as a valid measurement (separation, headroom, stability, length, cap, no cheap twin and effort). This run fails 1 of them (stability), so read its ranking as provisional.

Not ranked, because a model is ranked only when it finished every world: Grok (API timed out on every world).

Graded 2026-11-05; run bfcd2f4e, seed 20261105, protocol gauntlet-discovery-v2. Full methodology

Gauntlet Ship measures whether a model can finish real work. Each of the 55 items is a real task from Unsupervised Learning’s own working sessions, across 16 parts of a codebase the model has never seen. The model gets the working conversation as its ask and the repository as it stood before the fix, then has 30 steps of one plain tool loop to read, change and test the code before it says whether it is done.

Accepted fixreferenceAccepted fix, a reference script: Gauntlet Ship score 100, 95% interval 100 to 10095% 100–100
1Claude Opus 5.5Hidden checks 75%False done 31%Claude Opus 5.5: Gauntlet Ship score 66, 95% interval 53 to 78, hidden checks 75%, false done 31%, the leader95% 53–78Leader
2GPT-6 Astra · highHidden checks 75%False done 36%GPT-6 Astra · high: Gauntlet Ship score 62, 95% interval 49 to 75, hidden checks 75%, false done 36%, −3.6 points from the leader, 95% interval −18.0 to +10.7, not separated95% 49–75−3.695% −18.0 to +10.7Not separated
3GPT-6.1 SolHidden checks 72%False done 39%GPT-6.1 Sol: Gauntlet Ship score 56, 95% interval 43 to 70, hidden checks 72%, false done 39%, −9.1 points from the leader, 95% interval −21.8 to +3.6, not separated95% 43–70−9.195% −21.8 to +3.6Not separated
4Claude Fable 5.1 · highHidden checks 66%False done 39%Claude Fable 5.1 · high: Gauntlet Ship score 55, 95% interval 41 to 68, hidden checks 66%, false done 39%, −10.9 points from the leader, 95% interval −20.7 to −1.2, separated95% 41–68−10.995% −20.7 to −1.2● Separated
5Claude Sonnet 5.5Hidden checks 56%False done 31%Claude Sonnet 5.5: Gauntlet Ship score 47, 95% interval 34 to 61, hidden checks 56%, false done 31%, −18.2 points from the leader, 95% interval −29.7 to −6.7, separated95% 34–61−18.295% −29.7 to −6.7● Separated
6GPT-6.1 Terra · highHidden checks 55%False done 36%GPT-6.1 Terra · high: Gauntlet Ship score 46, 95% interval 32 to 59, hidden checks 55%, false done 36%, −20.0 points from the leader, 95% interval −34.7 to −5.3, separated95% 32–59−20.095% −34.7 to −5.3● Separated
7Claude Haiku 5.5Hidden checks 57%False done 41%Claude Haiku 5.5: Gauntlet Ship score 44, 95% interval 30 to 57, hidden checks 57%, false done 41%, −21.8 points from the leader, 95% interval −35.0 to −8.7, separated95% 30–57−21.895% −35.0 to −8.7● Separated
No changereferenceNo change, a reference script: Gauntlet Ship score 0, 95% interval 0 to 095% 0–0

Hidden checks, written from that conversation, decide each item. A check is kept only when it fails on the code before the accepted fix and passes after it, and checks are screened so they test nothing the conversation left open. An item is shipped when every hidden check passes and no test that passed before breaks. Code runs the checks and makes every number: no judge reads the work. The Gauntlet Ship score is the share of items shipped, with a 95% interval over the items. Hidden checks is the share of hidden checks a model passed, and false done is the share of its “done” claims that did not ship. The two grey rows are scripts, not models. No change finishes at once and ships nothing, so a model at that row changed nothing that mattered. The accepted fix writes the fix that was actually accepted, and every item it does not ship is dropped, so each item counted here can be shipped.

Before any model ran, we fixed 7 checks the run must pass to count as a valid measurement (items valid, separation, headroom, stability, no cheap twin, length and effort). This run passes all 7.

Argon ran but is left out of this ranking: 15 of 62 cells ended on Gemini API timeouts after the one retry, so the lane measured Google's API, not the model.

The items are private work, so this page shows counts and scores only: never an item’s conversation, its code, its checks or a model’s transcript.

Graded 2026-11-05; run 15ea371f, item set v1, protocol gauntlet-ship-v0. Full methodology

Gauntlet Composite puts UL’s three Gauntlet benchmarks together. Each dot is one model’s plain mean of three scores on 0–100: its Gauntlet Explain grade times 25, averaged over the efforts it ran; its Gauntlet Science score; and its Gauntlet Ship score. The three small marks on each row are those three parts. Only the 7 models all three benchmarks ran are ranked, so every dot is made of the same three parts.

Gauntlet CompositeGauntlet ExplainGauntlet ScienceGauntlet Ship

1Claude Opus 5.5Claude Opus 5.5: Gauntlet Composite 73, the mean of Gauntlet Explain 70, Gauntlet Science 85 and Gauntlet Ship 66
2Claude Fable 5.1Claude Fable 5.1: Gauntlet Composite 72, the mean of Gauntlet Explain 72, Gauntlet Science 89 and Gauntlet Ship 55
3GPT-6 AstraGPT-6 Astra: Gauntlet Composite 67, the mean of Gauntlet Explain 58, Gauntlet Science 82 and Gauntlet Ship 62
4GPT-6.1 SolGPT-6.1 Sol: Gauntlet Composite 66, the mean of Gauntlet Explain 56, Gauntlet Science 85 and Gauntlet Ship 56
5Claude Sonnet 5.5Claude Sonnet 5.5: Gauntlet Composite 64, the mean of Gauntlet Explain 65, Gauntlet Science 79 and Gauntlet Ship 47
6Claude Haiku 5.5Claude Haiku 5.5: Gauntlet Composite 55, the mean of Gauntlet Explain 55, Gauntlet Science 67 and Gauntlet Ship 44
7GPT-6.1 TerraGPT-6.1 Terra: Gauntlet Composite 53, the mean of Gauntlet Explain 42, Gauntlet Science 71 and Gauntlet Ship 46

Not ranked, because not every benchmark ran them: Gemini 4 Argon (no Gauntlet Ship score), GPT-6 Luna (no Gauntlet Ship score), Grok 4.7 (no Gauntlet Science and Gauntlet Ship score), Kimi K3.1 Pro (no Gauntlet Ship score) and Muse Wonder (no Gauntlet Ship score).

Each part comes from its own benchmark’s published run; nothing here is measured anew. Gauntlet Science’s current run fails one of its pre-registered checks, so its part, and this ranking with it, is provisional.

Made from the published Gauntlet Explain, Gauntlet Science and Gauntlet Ship runs, newest graded 2026-11-05. Methodology

Each model’s combined score puts every benchmark on one 0–100 scale, with the Gauntlet Explain 0–4 grade times 25, and takes their plain mean, so each counts equally. Cost is Artificial Analysis’s cost per index task. In Range a line joins only the efforts both Artificial Analysis and Gauntlet Explain ran; a model with none in common is one mark from each benchmark’s mean.In Average each model is one dot: the mean of every part switched on above that it has, its Artificial Analysis average, its Gauntlet Explain average times 25, its Gauntlet Science score and its Gauntlet Ship score. A model with no Gauntlet score switched on is left out. It shows the models picked in the Models lists above. Hover a mark for every part.