Was Grok 4.7 Trained for Its Own Test Harness? What the Scores Show

The premise behind the headline question is not supported: no public source shows Grok 4.7 was trained on the harness behind its benchmark scores. What is documented is narrower. xAI's launch post lists Terminal-Bench 4.0 at 37.6% (lab-reported, 21 September 2026), while Vals AI shows 28.79% as of 13 October 2026, a gap of about 9 points that nobody has explained.

Was Grok 4.7 actually trained on a test harness?

Only one harness is mentioned, and it is not a benchmark harness. The launch post says: "We also trained Grok 4.7 to natively understand the Grok Bot harness, making it better at conversational tasks and general knowledge work." That is a lab statement, and xAI frames it as a product-quality benefit for chat and knowledge work.

The same post describes a new, larger base model and "a longer reinforcement learning run on a harder mix of tasks, weighted toward problems that take many hours to complete." It gives no detail on the RL environments, the tool interfaces used in training, or any overlap with benchmark task sets. Anything beyond that is inference, and this article labels it as such below.

Inference: training a model on a scaffold would become an evaluation problem only if a benchmark were run in that same scaffold and the score compared to models that were not tuned for it. No public source shows that happened here.

What does xAI report, and at what settings?

xAI's launch table mixes effort levels across columns, which matters for any cross-model reading.

All figures are lab-reported, captured from the xAI launch post dated 21 September 2026. One secondary write-up, Digital Applied, lists 38.0% for the Terminal-Bench 4.0 cell, so the exact lab-reported figure differs by source by 0.4 points.

The post does not say which harness or scaffold ran any row, how many runs were averaged, what sampling settings were used, or whether competitor scores were run by xAI or copied from other published figures. According to Digital Applied, xAI also publishes no error bars and no run counts.

What did an independent evaluator measure?

Vals AI evaluated Grok 4.7 through the official xAI API at xhigh effort, with temperature 0.7, top P 0.95 and a 500,000 max output token limit. Its model page, as of 13 October 2026, shows Terminal-Bench 4.0 at 28.79% (plus or minus 2.31), ranked 18 of 47. Its launch-day update dated 21 September 2026 showed 28.28%.

The Vals page does not name the Terminal-Bench harness or agent scaffold on the model page. Its Vals Index for Grok 4.7 is 54.95% (plus or minus 1.07), ranked 16 of 47, down from 60.22% at launch. Vals does not explain the movement on the page, so rankings should be read as a dated snapshot.

For another independent angle, ComputingForGeeks reports an Artificial Analysis Intelligence Index of 46.4 for Grok 4.7 at xhigh. That is a single-source figure and measures broad capability, not Terminal-Bench.

How big is the gap, and is it outside noise?

The gap is about 9 points: 37.6% minus 28.79% is 8.81, and 37.6% minus 28.28% is 9.32. Vals' reported interval on its own number is plus or minus 2.31, so sampling noise in the Vals run alone does not close it. The lab-reported number has no interval to compare against.

That tells you the two numbers measure different things in practice. It does not tell you which one is closer to what a user would see in their own pipeline.

What could explain the gap?

Several mechanisms are plausible, and public sources confirm none of them for Grok 4.7. This section is inference.

• Harness and scaffold. Agentic benchmarks are sensitive to the tool interface, prompt wrapper and retry policy. A model tuned for one scaffold can lose points in another.
• Benchmark version. Both numbers are labelled Terminal-Bench 4.0, but neither source says which task-set revision was run, and scores from other Terminal-Bench versions should not be mixed with them.
• Effort and budget. Both runs are labelled xhigh, but token budgets, timeouts and per-task step limits are not published for either.
• Sampling. Vals reports temperature 0.7 and top P 0.95. xAI does not state its sampling settings.
• Harness familiarity from training. If the lab's harness was in the training mix, a boost on that harness would be expected. xAI's only stated harness-training claim concerns Grok Bot and conversational work, so this is not supported for Terminal-Bench.

Is the table in the launch post like-for-like?

No, and that is the better-documented problem. Grok 4.7 is at xhigh, Grok 4.6 at high, and the other two models at max. The Grok 4.7 DeepSWE v1.1 cell is a high-effort score while its other cells are xhigh. Effort changes both accuracy and tokens spent, so a column-to-column comparison confounds model quality with compute.

What's still unclear?

• Which harness produced xAI's 37.6% (or 38.0%) Terminal-Bench 4.0 figure, and whether it is the same scaffold used in products.
• Which harness Vals used for Terminal-Bench 4.0 and whether its time limits match xAI's.
• Whether benchmark-like tasks appear in the RL mix. xAI has not disclosed environments or data.
• Why Vals' own Terminal-Bench 4.0 score and ranking moved between 21 September and 13 October 2026.
• Whether the gap shrinks on other benchmarks. The evidence here covers one benchmark and one independent evaluator.

The weakest point of the "inflated" argument is that it needs a mechanism, and the only disclosed harness training concerns a different harness for a different purpose. The weakest point of the opposite view, that the scores are fine, is that the lab gives no harness, run count or interval to check.

Should you trust Grok 4.7's scores, and for what?

Treat 37.6% as an upper reference and 28.79% as an independent lower reference, both dated. For a purchasing or routing decision, run your own tasks in your own harness, and compare cost per completed task rather than price per token. Pricing as of 21 September 2026 is $2 per million input tokens and $6 per million output tokens, with a fast variant at twice the output speed and twice the price, per xAI. A model that spends more reasoning tokens per task can cost more than its per-token price suggests, so cost per task is the number that matters.

What would settle the question: a named harness and agent version, at least several runs with intervals, task-level logs, and the same run configuration applied to every model in the table.