ai benchmarks

Real-world AI performance metrics

UL Gauntlet updated

Artificial Analysis data updated

Every change to the data behind these charts, newest first: models joining UL’s Gauntlet or moving to a newer version, then every change Artificial Analysis made to its scores and prices. Click a model to see its past scores.

UL’s Gauntlet lineup

No Gauntlet lineup changes are recorded yet.

6 new releases · 1 price change

New releases 6

  • peak 21.8one setting, $0.019 per task
  • peak 10.8one setting, $0.039 per task
  • peak 9.3one setting, $0.038 per task
  • peak 8.9one setting, $0.0082 per task
  • peak 7.0one setting, $0.11 per task
  • peak 6.5one setting, $0.11 per task

Price changes 1

  • single settingCost per task$1.00 → $1.00−$0.0020

1 price change

Price changes 1

  • highCost per task$0.057 → $0.053−$0.0032

1 price change

Price changes 1

  • single settingCost per task$1.13 → $1.13+$0.0023

1 price change

Price changes 1

  • single settingCost per task$0.25 → $0.28+$0.030

1 new release

New releases 1

  • peak 52.4one setting, $4.52 per task

1 removed release · 3 price changes

Price changes 3

  • highCost per task$0.12 → $0.10−$0.021
  • DeepSeek V4 Flash 0420maxCost per task$0.11 → $0.089−$0.018
  • single settingCost per task$0.077 → $0.066−$0.011

Removed releases 1

  • was 12.7 peakhad one setting

2 price changes

Mistral Small 3.1’s cost per Intelligence Index task fell from $0.035 to $0.0255. OpenAI’s gpt-oss-20b at high reasoning effort also became cheaper, from $0.0131 to $0.0125.

AI-written summary of these changes · gpt-6-luna

Price changes 2

  • single settingCost per task$0.035 → $0.025−$0.0095
  • highCost per task$0.013 → $0.013−$0.0006

2 price changes

The cost per Intelligence Index task rose for both models: Qwen3 Coder Next went from $0.5518 to $0.7805, while gpt-oss-20b at high reasoning effort went from $0.0125 to $0.0131. Qwen3 Coder Next had the larger increase, both in dollars and proportionally.

AI-written summary of these changes · gpt-6-luna

Price changes 2

  • single settingCost per task$0.55 → $0.78+$0.23
  • highCost per task$0.013 → $0.013+$0.0006

3 price changes

All three listed cost per task figures increased slightly. Ling 3.1 Flash rose from $0.9857 to $1.002, Solar Mini 4 from $0.3634 to $0.3656, and MiMo-V2.6-Flash from $0.0623 to $0.0624.

AI-written summary of these changes · gpt-6-luna

Price changes 3

  • single settingCost per task$0.99 → $1.00+$0.016
  • single settingCost per task$0.36 → $0.37+$0.0022
  • single settingCost per task$0.062 → $0.062+$0.0001

1 price change

LongCat 2.0’s cost per Intelligence Index task rose from $0.059 to $0.1462. This was the only listed change.

AI-written summary of these changes · gpt-6-luna

Price changes 1

  • single settingCost per task$0.059 → $0.15+$0.087

1 score change

Claude Haiku 5.5’s Intelligence Index score at low reasoning effort rose from 29.4 to 29.5. This is the only change in the update.

AI-written summary of these changes · gpt-6-luna

Score changes 1

  • lowIntelligence29.4 → 29.5+0.1

1 new release

IBM added Granite 4.2 30B with an Intelligence Index score of 12.7. Its cost is $0.072 per task.

AI-written summary of these changes · gpt-6-luna

New releases 1

  • peak 12.7one setting, $0.072 per task

1 price change

StepFun’s Step 5 Preview default cost per Intelligence Index task rose from $0.7175 to $1.026, an increase of $0.3085 or about 43%.

AI-written summary of these changes · gpt-5.6-terra

Price changes 1

  • single settingCost per task$0.72 → $1.03+$0.31

5 price changes

Claude Sonnet 5.5’s Intelligence task cost fell at every listed reasoning effort. Max dropped from $7.667 to $5.461, xhigh from $2.746 to $2.012, and high from $1.123 to $0.8844. Medium and low also declined, from $0.5894 to $0.4833 and from $0.4170 to $0.3452.

AI-written summary of these changes · gpt-5.6-terra

Price changes 5

  • lowCost per task$0.42 → $0.35−$0.072
  • Claude Sonnet 5.5mediumCost per task$0.59 → $0.48−$0.11
  • Claude Sonnet 5.5highCost per task$1.12 → $0.88−$0.24
  • Claude Sonnet 5.5xhighCost per task$2.75 → $2.01−$0.73
  • Claude Sonnet 5.5maxCost per task$7.67 → $5.46−$2.21

1 new release

Anthropic’s Claude Haiku 5.5 was added with an Intelligence Index score of 43.4. It is available at five effort levels, with Intelligence task costs ranging from $0.025 to $0.21.

AI-written summary of these changes · gpt-5.6-terra

New releases 1

  • peak 43.45 effort levels, $0.025–$0.21 per task

1 price change

DeepSeek V4 Flash 0420’s high-effort Intelligence task cost fell from $0.1271 to $0.1231, a decrease of $0.0040 per task.

AI-written summary of these changes · gpt-5.6-terra

Price changes 1

  • highCost per task$0.13 → $0.12−$0.0040

1 new release

Mistral Large 4 Preview entered the Intelligence Index at 38.4, with its listed setting costing $1.13 per task.

AI-written summary of these changes · gpt-5.6-terra

New releases 1

  • peak 38.4one setting, $1.13 per task

12 price changes

Google model costs increased across all six listed releases, including Gemini 3.5 Flash at high effort from $1.563 to $3.694 per Intelligence task and Gemini 3.1 Pro Preview from $0.6747 to $1.299. Grok 4.6 costs fell at every listed effort level, from $0.3758 at low effort to $1.752 at xhigh effort, while Grok 4.5 high fell from $1.036 to $0.8807. DeepSeek V3 0324 rose slightly from $0.0202 to $0.0211.

AI-written summary of these changes · gpt-5.6-terra

Price changes 12

  • highCost per task$1.56 → $3.69+$2.13
  • single settingCost per task$0.67 → $1.30+$0.62
  • highCost per task$0.93 → $1.60+$0.67
  • single settingCost per task$0.12 → $0.19+$0.069
  • single settingCost per task$0.039 → $0.060+$0.021
  • single settingCost per task$0.23 → $0.33+$0.099
  • lowCost per task$0.48 → $0.38−$0.099
  • Grok 4.6mediumCost per task$1.50 → $1.14−$0.35
  • Grok 4.6highCost per task$1.86 → $1.48−$0.37
  • Grok 4.6xhighCost per task$2.32 → $1.75−$0.57
  • highCost per task$1.04 → $0.88−$0.16
  • single settingCost per task$0.020 → $0.021+$0.0009

1 new release

InclusionAI's Ling 3.1 Flash entered the Intelligence Index at 41.1, with one listed setting costing $0.99 per task.

AI-written summary of these changes · gpt-5.6-terra

New releases 1

  • peak 41.1one setting, $0.99 per task

2 price changes

MiMo-V2.6-Flash's default cost per Intelligence Index task rose from $0.0622 to $0.0623. GPT-6 Luna's max-effort cost rose from $0.0677 to $0.0678. Both changes were increases of $0.0001 per task.

AI-written summary of these changes · gpt-5.6-terra

Price changes 2

  • single settingCost per task$0.062 → $0.062+$0.0001
  • maxCost per task$0.068 → $0.068+$0.0001
Load older