ai benchmarks

Real-world AI performance metrics

UL Gauntlet updated

Artificial Analysis data updated

Every change to the data behind these charts, newest first: models joining UL’s Gauntlet or moving to a newer version, then every change Artificial Analysis made to its scores and prices. Click a model to see its past scores.

UL’s Gauntlet lineup

No Gauntlet lineup changes are recorded yet.

2 score changes · 4 price changes · 1 new effort level

Claude Sonnet 5.5 added a low reasoning-effort setting scoring 35.9 on Intelligence at $0.42 per task. Its medium and high Intelligence scores each rose by 0.1 points, to 40.8 and 46.8. Costs increased slightly at medium ($0.5862 to $0.5894), high ($1.08 to $1.123), xhigh ($2.743 to $2.746), and max ($7.621 to $7.667).

AI-written summary of these changes · gpt-5.6-terra

Score changes 2

  • mediumIntelligence40.7 → 40.8+0.1
  • Claude Sonnet 5.5highIntelligence46.7 → 46.8+0.1

Price changes 4

  • mediumCost per task$0.59 → $0.59+$0.0032
  • Claude Sonnet 5.5highCost per task$1.08 → $1.12+$0.043
  • Claude Sonnet 5.5xhighCost per task$2.74 → $2.75+$0.0036
  • Claude Sonnet 5.5maxCost per task$7.62 → $7.67+$0.046

New effort levels 1

  • lowIntelligence35.9$0.42 per task

1 new effort level

Grok 4.7 added a low reasoning-effort result of 42.2 on the Intelligence Index, with a cost of $1.25 per task.

AI-written summary of these changes · gpt-5.6-terra

New effort levels 1

  • lowIntelligence42.2$1.25 per task

9 score changes · 8 price changes

GPT-6 Luna’s Intelligence score rose at every effort level, from 20.9 to 21.5 at low through 37.3 to 38.1 at max. GPT-6 Sol’s scores moved from 33.9 to 34.2 at low, 44.1 to 44.2 at xhigh and 47.5 to 47.6 at max, while high fell from 42.8 to 42.4. Both models had small mixed cost movements: Luna’s xhigh cost rose from $0.0421 to $0.0422, while Sol’s low cost rose from $0.1326 to $0.1328 and its other listed costs fell.

AI-written summary of these changes · gpt-5.6-terra

Score changes 9

  • lowIntelligence20.9 → 21.5+0.6
  • GPT-6 LunamediumIntelligence29.5 → 29.9+0.4
  • GPT-6 LunahighIntelligence32.1 → 32.9+0.8
  • GPT-6 LunaxhighIntelligence33.9 → 34.6+0.7
  • GPT-6 LunamaxIntelligence37.3 → 38.1+0.8
  • lowIntelligence33.9 → 34.2+0.3
  • GPT-6 SolhighIntelligence42.8 → 42.4−0.4
  • GPT-6 SolxhighIntelligence44.1 → 44.2+0.1
  • GPT-6 SolmaxIntelligence47.5 → 47.6+0.1

Price changes 8

  • lowCost per task$0.13 → $0.13+$0.0002
  • GPT-6 SolhighCost per task$0.38 → $0.37−$0.0025
  • GPT-6 SolxhighCost per task$0.52 → $0.52−$0.0028
  • GPT-6 SolmaxCost per task$1.05 → $1.04−$0.0047
  • mediumCost per task$0.018 → $0.018−$0.0001
  • GPT-6 LunahighCost per task$0.029 → $0.029−$0.0001
  • GPT-6 LunaxhighCost per task$0.042 → $0.042+$0.0001
  • GPT-6 LunamaxCost per task$0.068 → $0.068−$0.0002

1 new release

Upstage’s new Solar Mini 4 release scores 24.1 on the Intelligence Index. Its single listed setting costs $0.36 per Intelligence Index task.

AI-written summary of these changes · gpt-5.6-terra

New releases 1

  • peak 24.1one setting, $0.36 per task

1 price change

NVIDIA's Nemotron 3.5 Lightning default cost per Intelligence Index task fell from $0.1004 to $0.0931, a decrease of $0.0073 (about 7%).

AI-written summary of these changes · gpt-5.6-terra

Price changes 1

  • single settingCost per task$0.10 → $0.093−$0.0073

1 new release

Google’s new Gemini 4 Argon release scores 52.6 on the Intelligence Index in its listed setting. Its cost is $1.99 per Intelligence Index task.

AI-written summary of these changes · gpt-5.6-terra

New releases 1

  • peak 52.6one setting, $1.99 per task

1 price change

Claude Sonnet 5.5 at max reasoning effort rose in cost from $7.603 to $7.621 per Intelligence Index task. This was an increase of $0.018, with no score, release, or other effort-level changes listed.

AI-written summary of these changes · gpt-5.6-terra

Price changes 1

  • maxCost per task$7.60 → $7.62+$0.018

1 new release · 2 new effort levels

Kimi K3 added a low-effort setting scoring 30.1 on Intelligence at $1.15 per task. Inkling Small debuted at 25.7 for $0.089 per task, while DeepSeek V4 Flash 0420's new high-effort setting scored 24.4 at $0.13. The two lower-scoring entries are also far cheaper per task than Kimi K3's low-effort setting.

AI-written summary of these changes · gpt-5.6-terra

New releases 1

  • peak 25.7one setting, $0.089 per task

New effort levels 2

  • lowIntelligence30.1$1.15 per task
  • highIntelligence24.4$0.13 per task

1 removed effort level

Claude Sonnet 5.5’s low reasoning-effort option was removed from the Intelligence Index. It had scored 35.8 and cost $0.41 per task.

AI-written summary of these changes · gpt-5.6-terra

Removed effort levels 1

  • lowIntelligencewas 35.8was $0.41 per task

1 new release

OpenAI's new GPT-6.1 Sol entered the Intelligence Index at 51.8. It has five effort levels, with Intelligence Index task costs ranging from $0.13 to $0.72.

AI-written summary of these changes · gpt-5.6-terra

New releases 1

  • peak 51.85 effort levels, $0.13–$0.72 per task

1 price change

DeepSeek V3 0324’s cost per Intelligence Index task fell from $0.1053 to $0.0202, a reduction of $0.0851.

AI-written summary of these changes · gpt-5.6-terra

Price changes 1

  • single settingCost per task$0.11 → $0.020−$0.085

7 price changes

GPT-6 Luna’s cost per Intelligence task rose at medium, high, and xhigh effort, from $0.0173 to $0.0176, $0.0286 to $0.0291, and $0.0417 to $0.0421, while max fell slightly from $0.0681 to $0.0679. GPT-6 Sol’s xhigh and max costs fell from $0.5319 to $0.5236 and $1.056 to $1.049, while high increased from $0.3746 to $0.3753.

AI-written summary of these changes · gpt-5.6-terra

Price changes 7

  • mediumCost per task$0.017 → $0.018+$0.0003
  • GPT-6 LunahighCost per task$0.029 → $0.029+$0.0005
  • GPT-6 LunaxhighCost per task$0.042 → $0.042+$0.0004
  • GPT-6 LunamaxCost per task$0.068 → $0.068−$0.0002
  • highCost per task$0.37 → $0.38+$0.0007
  • GPT-6 SolxhighCost per task$0.53 → $0.52−$0.0083
  • GPT-6 SolmaxCost per task$1.06 → $1.05−$0.0070

2 price changes

GPT-6 Sol’s medium-effort Intelligence task cost edged down from $0.2482 to $0.2472. Its low-effort cost moved slightly up from $0.1322 to $0.1326, leaving the two effort levels moving in opposite directions.

AI-written summary of these changes · gpt-5.6-terra

Price changes 2

  • lowCost per task$0.13 → $0.13+$0.0004
  • GPT-6 SolmediumCost per task$0.25 → $0.25−$0.0010

1 new release

Alibaba’s new Qwen3.5 9B release scores 11.2 on the Intelligence Index in its listed setting. Its cost is $0.21 per Intelligence Index task.

AI-written summary of these changes · gpt-5.6-terra

New releases 1

  • peak 11.2one setting, $0.21 per task

1 price change

Tencent's Hy3 default cost per Intelligence Index task fell from $0.0743 to $0.0718, a decrease of $0.0025.

AI-written summary of these changes · gpt-5.6-terra

Price changes 1

  • single settingCost per task$0.074 → $0.072−$0.0025

1 new release

Anthropic's new Claude Sonnet 5.5 release scores 56 on the Intelligence Index. It has five reasoning-effort levels, with Intelligence Index task costs ranging from $0.41 to $7.60.

AI-written summary of these changes · gpt-5.6-terra

New releases 1

  • peak 56.05 effort levels, $0.41–$7.60 per task

1 new release

Xiaomi’s MiMo-V2.6-Flash was added with an Intelligence Index score of 37.9. Its single listed setting costs $0.062 per Intelligence Index task.

AI-written summary of these changes · gpt-5.6-terra

New releases 1

  • peak 37.9one setting, $0.062 per task

1 new release · 2 removed releases · 11 price changes

Inkling (25) and MiniMax-M2.7 (22.8) were removed, while Trinity Large Thinking entered at 10.8 intelligence and $0.12 per task. Costs rose for Mercury 2.5 ($0.0642 to $0.1167), GLM-5.2 max ($0.9649 to $1.473), Claude 4.5 Haiku ($0.2077 to $0.2768), and several others. Costs fell for Mistral Large 3 ($0.0538 to $0.0311), Qwen3.7 Plus ($0.2952 to $0.217), and DeepSeek V4 Flash 0420 max ($0.1179 to $0.1075); Mistral’s two models moved in opposite directions.

AI-written summary of these changes · gpt-5.6-terra

New releases 1

  • peak 10.8one setting, $0.12 per task

Price changes 11

  • single settingCost per task$0.064 → $0.12+$0.053
  • single settingCost per task$0.054 → $0.031−$0.023
  • maxCost per task$0.96 → $1.47+$0.51
  • single settingCost per task$0.30 → $0.22−$0.078
  • single settingCost per task$0.21 → $0.28+$0.069
  • highCost per task$0.17 → $0.21+$0.040
  • single settingCost per task$0.44 → $0.50+$0.065
  • maxCost per task$0.12 → $0.11−$0.010
  • xhighCost per task$0.41 → $0.45+$0.038
  • single settingCost per task$0.55 → $0.60+$0.048
  • single settingCost per task$0.098 → $0.10+$0.0021

Removed releases 2

  • was 25.0 peakhad one setting
  • was 22.8 peakhad one setting

1 price change

StepFun's Step 5 Preview cost per Intelligence Index task increased from $0.7155 to $0.7175, a rise of $0.0020.

AI-written summary of these changes · gpt-5.6-terra

Price changes 1

  • single settingCost per task$0.72 → $0.72+$0.0020

1 price change

Z AI's GLM-5.3 low-effort cost per Intelligence Index task fell from $3.663 to $0.8522, a reduction of about 77%. No score, release, or reasoning-effort availability changes are listed.

AI-written summary of these changes · gpt-5.6-terra

Price changes 1

  • lowCost per task$3.66 → $0.85−$2.81
Load older