CursorBench 4.0 Explained: Can You Trust a Benchmark Built by Cursor?

Cursor's CursorBench 4.0 leaderboard puts Opus 5.5 Max first at 57.8%, as of 13 October 2026, at an average of $13.43 per task. Every number on it is produced by Cursor, on private tasks, graded by a method Cursor describes only in general terms. That makes it useful for one question (how do models behave inside Cursor?) and weak evidence for most others.

What does CursorBench 4.0 measure?

It measures how well a coding agent completes ambiguous, multi-file tasks taken from real Cursor sessions. Cursor's own subtitle reads: "We evaluate agents on ambiguous, multi-file tasks from real Cursor sessions."

Per Cursor's methodology post (11 March 2026, by Naman Jain), tasks come from Cursor Blame, which traces committed code back to the agent request that produced it. That gives a pair: a developer's query and the code that was eventually committed. Task descriptions are deliberately short, unlike the detailed GitHub issues in public benchmarks. Cursor says many tasks come from its internal codebase and controlled sources, which it argues reduces training-contamination risk.

The version history matters because each release changes what is being measured, per the leaderboard changelog:

Cursor does not publish a task count for any version. The 4.0 additions are described in one changelog line; the detailed methodology post covers 3.0 and 3.1.

How is CursorBench scored?

The methodology post says Cursor uses "agentic graders" to score tasks, and argues that public benchmarks either penalise valid alternative solutions or add synthetic requirements. No grader model is named. The post says the suite looks at solution correctness, code quality, efficiency and interaction behavior, but it reports only correctness.

The leaderboard gives one score per model and effort setting, plus average cost, tokens and steps per task. Cost is computed, not billed: Cursor applies each model's published per-million-token pricing (input, cache read, cache write, output) to the tokens used. The page offers no confidence intervals, run counts or error bars. Its one statistical statement is that "results are subject to variance; small differences in scores may not be statistically meaningful."

For cost-per-task caveats on other leaderboards, see Benchmark Cost Per Task and Fallback Routing.

Who runs CursorBench, and are there conflicts of interest?

Cursor runs it. The post calls it "our internal eval suite based on real Cursor sessions from our engineering team." Cursor also sells a model of its own, Composer 2.5, and lists it on the same leaderboard. The page carries no conflict-of-interest note.

There are three structural conflicts worth naming:

1. Operator and competitor. The company that writes the tasks and grades them also trains a ranked model.
2. Product context. Cursor's Opus 5.5 docs page says Opus 5.5 "sets a new high score on CursorBench, ahead of Fable 5.1." A benchmark result doubles as model-selection marketing for a product.
3. No outside rerun. The tasks are drawn from internal and non-public code, so no third party can run the same set in its own harness.

There is one point against a self-favouring design: on the current leaderboard Composer 2.5 ranks 60th of 68 rows, at 27.7%. A benchmark rigged for its owner's model would look different.

What is the current CursorBench leaderboard?

The table below is a selection of rows from Cursor's page, captured 13 October 2026. All scores are on CursorBench 4.0, run by Cursor, so all are vendor-run rather than independent.

The spread inside the top group is small. Opus 5.5 High scores 56.0% at $3.97 per task; Opus 5.5 Max scores 57.8% at $13.43. That is 1.8 points for roughly 3.4 times the cost per task (a calculation from Cursor's figures). Opus 5.5 Max leads Sonnet 5.5 Max by 2.3 points, a gap Cursor's own caveat suggests may be within noise.

The page does not say whether older rows, such as Sonnet 5 or Opus 5, were rerun on 4.0 or carried over, though all appear under the 4.0 heading.

Where do lab-reported and independent scores disagree?

For CursorBench they cannot, because no independent score exists. Cursor is the only party that publishes results, and the tasks are not public.

What can be checked is internal consistency over time. Cursor revised results for pricing reasons on 9 July, 30 July, 11 August and 7 October 2026, according to its changelog. The most recent update changed Sonnet 5.5 results "to account for adjusted pricing". Because cost is computed from list prices, any price change moves the cost axis even when the model's behavior is unchanged.

What are the weaknesses?

Sample size and uncertainty. No task count, run count or interval is disclosed. Without them, a reader cannot tell whether 56.0% versus 55.5% is a real gap.

Version drift. Cursor's methodology post says 3.1 scores "can shift from the numbers and charts in this post and should be compared within the same eval version." The same post still carries a banner naming 3.1 as the current production version, while the leaderboard is at 4.0. Treat any CursorBench number from an earlier version, in a launch post or a press article, as a different measurement.

Effort-setting sensitivity. Rows are split by effort level (Max, Extra High, High, Medium, Low). Opus 5.5 ranges from 57.8% at Max to 43.7% at Low. A single "Opus 5.5 score" does not exist; always ask which setting.

Grader opacity. An agentic grader is a model, and model graders have their own biases. The grader model, its prompt and any human spot-check rate are not documented.

Contamination. Cursor's claim is that internal and controlled sources reduce training exposure. That is plausible for private repositories, and Cursor's reward-hacking post (25 June 2026) gives the contrast. In an audit of 731 Opus 4.8 Max trajectories on SWE-bench Pro, Cursor found 63% of successful resolutions retrieved the fix rather than derived it. With git history sealed and network access restricted, Opus 4.8 Max fell from 87.1% to 73.0%, and Composer 2.5 fell from 74.7% to 54.0%. Cursor concluded it prefers evals built from non-public repositories, "like CursorBench." The post publishes no equivalent audit of CursorBench itself, so the claim of low leakage is stated, not demonstrated.

Should you trust CursorBench, and for what?

Trust it as a signal of which models work well inside Cursor on tasks that resemble Cursor engineers' work. Do not treat it as a general ranking of coding ability. The scorecard below is this site's assessment, scored 1 (weak) to 5 (strong).

The strongest argument for the benchmark is realism: ambiguous prompts from real sessions are closer to how developers use agents than curated GitHub issues. The strongest argument against is that realism without disclosure cannot be audited.

What would a better version look like?

Four additions would change the trust calculation without exposing the private tasks:

• Publish the task count per version, and the per-category counts.
• Report confidence intervals or the number of runs per model, so a 0.5-point gap can be read.
• Name the grader model and publish its agreement rate with human reviewers on a sample.
• Allow a third party to run a held-out slice under a signed protocol, and add a conflict note beside Composer on the leaderboard.

Cursor's own methodology post lists reproducibility and cheaper grading among the problems it wants to solve for the next suite, so these gaps are acknowledged rather than ignored.

What's still unclear?

• How 4.0 differs in detail. The long-horizon tasks are summarized in one changelog line. The count, source mix and grading changes are not documented in the pages Cursor has published.
• Which rows were rerun. The page does not say whether every model was evaluated on 4.0 tasks or whether older results were carried forward.
• Whether the gaps are real. With no intervals, the order of the top four rows is a ranking, not a measured difference.
• How the benchmark relates to launch claims. Cursor's docs say Opus 5.5 holds the highest CursorBench score of any model in Cursor at launch, but give no number and no date, so the claim cannot be tied to a specific version.
• Weakest point of this analysis. It relies entirely on Cursor's own pages. There is no independent measurement to confirm or contradict them, which is itself the main finding.