Anthropic Published Two Different Scores for Claude Sonnet 5.5. Here's Which Is Right

Anthropic reports 70.6% for Claude Sonnet 5.5 on Terminal-Bench 4.0. Artificial Analysis lists 63.6% for the same model and benchmark version as of 12 October 2026. That is a 7.0 point gap between a lab-reported and an independent score.

Neither number is shown to be wrong. They come from different run protocols, and the useful question is which one fits your decision. The short answer: use the independent figure to rank models against each other, and treat the lab figure as an upper bound for a tuned setup.

Which Sonnet 5.5 score is right, 70.6% or 63.6%?

Both can be correct, because they measure different things. The 70.6% is the company's own number from its launch table, and Anthropic labels no effort setting, trial count or harness next to it. The 63.6% is a measurement from a third party using one fixed harness for every model it tests.

The independent number is the better guide when you are choosing between vendors, since the same scaffold is applied to each model. The lab number is a better guide to what the model can reach when the vendor controls the setup. Nothing in the public sources shows Anthropic misreported.

What exactly did each side report?

Anthropic's table also lists 66.4% for Opus 5.5 and 10.3% for Sonnet 5. A footnote says the Opus 5.5 figure is at Xhigh effort, its highest score. The Sonnet 5.5 row has no such label.

How big is the gap compared with the Opus 5.5 gap?

It is about the same size. Anthropic's Opus 5.5 score is 66.4% and Artificial Analysis lists 59.6%, a 6.8 point difference. For Sonnet 5.5 it is 70.6% against 63.6%, or 7.0 points. We covered the Opus 5.5 case in an earlier post.

A near-constant offset between the two sources, with the same model order in both, points to a systematic protocol difference rather than noise specific to one model. That is an inference from two data points, not a measured result. Terminal-Bench 4.0 has 66 tasks, so one task is about 1.5 percentage points, and a 7.0 point gap is roughly 4.6 tasks.

What does Terminal-Bench 4.0 measure, and who runs it?

It measures whether an agent can finish terminal tasks. Artificial Analysis says it runs all 66 tasks, reporting a resolution rate. The Terminal-Bench leaderboard page shows resolution rate with 95% confidence interval whiskers, plus cost and token columns.

The benchmark is hosted by Stanford, Harbor and the Laude Institute. Artificial Analysis separately runs it as one input to its Intelligence Index, where Sonnet 5.5 scores 56, ranked second of 218 (index v4.3.2, as of 12 October 2026). Artificial Analysis is also the operator Anthropic names for two other rows in its table, GDPval-AA and AA-Briefcase, so it is both a source for Anthropic's claims and a check on them.

Why might the two numbers differ?

The likely causes are the harness, the number of repeats and the effort setting, but no controlled comparison in these sources isolates which one matters most. Artificial Analysis uses the mini-swe-agent harness with three repeats. Anthropic's page does not state its harness or trial count for this row.

Effort is another variable. Anthropic says the default is Medium in Claude Code and the Claude apps, and High on the Claude Platform. Artificial Analysis measured a Max variant. A higher effort setting is not always better: Anthropic's own FrontierCode footnote says Sonnet 5.5 scored lower at Max than at Xhigh.

Does Sonnet 5.5 have other scores that vary by setting?

Yes. On FrontierCode 1.1 (Main), Anthropic's table shows 46.2% at Max effort. A footnote says Sonnet 5.5 scores lower at Max than at Xhigh, because at Max it more often ran Claude Code's code-review skill, which splits review across many subagents. In two cases examined by Cognition, the company says this caused a timeout or out-of-scope edits and so a lower score.

That is a useful reminder that "the score" for a model is really a score for a model plus an effort level plus a scaffold. Anthropic also notes that Artificial Analysis ran GDPval-AA and AA-Briefcase on a pre-release deployment with a bug that could degrade structured-output responses. Anthropic expects the effect to be small and to understate performance, and says the bug is fixed. That note covers those two benchmarks, not Terminal-Bench.

What does Sonnet 5.5 cost, and does cost change the picture?

As of 28 September 2026, Anthropic prices it at $2 per million input tokens and $10 per million output tokens, the same as Sonnet 5, and says it costs up to 30% less per task. Artificial Analysis lists the same prices as of 12 October 2026.

Artificial Analysis also reports that Sonnet 5.5 generated 420M output tokens running its Intelligence Index, against a median of 82M across models. High token use at Max effort means a higher score can come with a higher bill per task, so compare cost per resolved task, not per token.

Should you trust either number, and for what?

Trust the independent score for ranking, and the lab score as a ceiling. Scores below are my judgement from the published methodology, on a 1 to 5 scale.

What's still unclear?

• Which setup produced 70.6%. The announcement table gives no effort setting, harness or trial count for the Sonnet 5.5 figure, so the two rows cannot be tied to a single changed variable.
• What "Default Fallback" means. It appears in Artificial Analysis's model name and is not defined on its pages. Whether it matches Anthropic's production routing is unknown.
• Capture date. Artificial Analysis shows no measurement date on its pages, so the 12 October 2026 "as of" date is not when the runs finished.
• A third source. The Terminal-Bench leaderboard page lists no Sonnet 5.5 result that could arbitrate between the two figures.
• Weakest point of this argument. The constant 7-point offset is drawn from only two models. A third model with a different offset would weaken the case that the gap is protocol-driven.