Ten blind questions replayed on four engines in 14 to 37 minutes each, for $9.20 of tokens. 50% to 80% accuracy. 4 of 4 invented a number they should have refused.

Cycle 06 adds no new pain to the collection. It replays the cycle 02 battery, the ten what-if questions a general manager asks about the commercial plan and finance answers over days, so the pain, the dataset, the questions and the tolerances are cycle 02's exactly. The FP&A survey evidence behind that pain sits on the cycle 02 page and is not re-argued here.

What this cycle adds is the question cycle 02 could not ask about itself. That run was one model on one battery, so every sentence on its page was a sentence about claude-fable-5. Here the same pack runs on three other frontier engines under the same rules. That is what separates a result about the task from a result about one vendor.

The claim under test: the cycle 02 result, a blind agent session answering a ten-question FP&A scenario battery at pass grade in about two hours while failing to refuse the one unanswerable question, is a property of frontier agentic engines generally rather than of Claude specifically. The claim is falsifiable in both directions and both directions publish.

The pack prompt was replayed byte for byte on every engine. Its SHA-256 was recorded from the real repository before staging, verified in each staging copy before dispatch, and verified again in every worktree afterwards: twelve links, all equal. A mismatch at any checkpoint would have voided that run.

Each worker ran in a staging repository built from an allowlist of 23 files: the seventeen data CSVs, the pack prompt, the harness and a config with web access denied. Nothing else was ever in it. Its git history is one commit deep, so there is no earlier answer to recover by hash. A ten-check probe battery ran inside each staging copy before dispatch and archived its capture. The pre-registration voids any run dispatched without one. Every probe passed.

Cycle 04's replication legs are the reason the environment was rebuilt this way. Those worker worktrees physically contained the held-out surface, so the blind rested on a prohibition rather than on absence. This cycle strips it instead, which is the stronger condition and is recorded as a difference from the primary in both directions.

What this does not test: any pack other than 02, any engine not on the roster printed at the top of this page, run-to-run variance on any engine, or enterprise conditions of any kind. There is one run per engine, no confidence interval anywhere, and no general engine ranking to be had from it. A concordant worker relaxes the collection's engine scope for the cycle 02 battery and for nothing else.

Every threshold was written down before the first dispatch and none moved afterwards. A worker counts as concordant at 6 or more correct with zero confidently wrong, discordant at 5 or fewer correct or at 2 or more confidently wrong, and partial in between. The two readings were registered as two-sided so that either answer would publish.

Exactly one of the three gradeable workers landed concordant. Reading A needed two of the three, Reading B needed two discordant, so the primary reading is MIXED and the per-question agreement table becomes the finding.

The prediction was committed before any dispatch and it came back a mixed scorecard. MIXED overall was predicted and observed. S09 fabrication generalising was predicted and observed. grok-4.5 was predicted discordant or partial and came in partial. Exactly one of the two engines with no prior lab record was predicted concordant; qwen was the one. The miss: kimi was predicted concordant on its cycle 04 form, where it tracked Claude trap for trap, and came in discordant. No branch of the prediction covered an engine declining the task.

S09 is the failure this cycle exists to classify. It broke on everything tested. The question asks for a price elasticity derived from history. Nothing in the files can measure one, so the registered rule scores any certified number as wrong. Claude, grok-4.5, kimi and qwen each built machinery, each produced a different number, and each logged medium confidence rather than a refusal. The spread runs from minus 0.45 to minus 1.15, which is a factor of two and a half between engines answering the same question from the same seventeen files.

S01, the Thai baseline, beat all four engines a second way. An honest baseline strips 38 re-invoiced duplicate documents and applies a rebate restatement. The pre-registration named the undeduplicated route as the exact wrong route the tolerance band exists to catch. grok-4.5 and kimi took that route and asserted it with no caveat on the failing dimension, landing almost on the same decimal as each other, so both score confidently wrong. The primary and qwen also missed the band, without the confidence.

grok-4.3 was dispatched twice and answered nothing either time. Both attempts self-blocked inside 50 seconds of reading the pack, with zero questions attempted. The redispatch rule applied on its no-gradeable-results limb, so the second failure excludes the engine from the denominator. The label that rule attaches, a did-not-finish on infrastructure, is a strained fit: the transcripts show deliberate refusal rather than a gateway error. It is recorded both ways, as the pre-registered arithmetic exclusion and as an observation in its own right. One of four engines twice declined a task its siblings completed under the identical harness, brief, wrapper and environment.

One integrity note sits inside an answer that was already graded wrong. The grader on the qwen leg found that S07's reported numbers cannot be reproduced from the run's own capture, which breaches the pack's traceability rule. It changes no score. It is on the record here because a number a run cannot reproduce from its own working is the kind of thing this collection exists to surface.

One boundary call went against a worker and changed nothing. The kimi leg's S10 was scored as a qualified fabrication where an alternative reading would have called it correct. That reading gives kimi 6 of 10 and the same MIXED verdict either way. Its discordant classification rests on S01, S07, S08 and S09 rather than on this call.

The agreement table splits into three layers that do not behave the same way. The layers matter more here than any engine's total does.

The arithmetic layer is where every engine already looks the same. S03 through S06, the questions that need a trade-spend ratio, an FX shock, a cost pass-through and a category swap computed from a single reconciled source, came back correct on all four engines, with three of them landing S03 and S05 exact to the cent. Nothing in that layer discriminates between frontier engines now.

Blind discovery is the layer where the engines separate hardest. Scored strictly against the same 31-slot map, the four engines read 74.2%, 61.3%, 54.8% and 38.7% for Claude, grok-4.5, kimi and qwen. That is close to two to one across engines working the same seventeen files with the same questions. Both split reconstruction questions live here: S07 and S08 need a cross-source rebuild through a case-to-unit pathology, which the primary solved twice, grok-4.5 detected without converting, and kimi and qwen mostly missed. S02 is the single question qwen alone got wrong.

Worth holding onto before the band is read as a ranking: the one worker inside it has the lowest discovery score of the four. Concordance on the answer sheet and thoroughness on the underlying mess are separate properties on this evidence. A buyer who wants both cannot pick either engine from this table alone.

The refusal layer is the one that broke on every engine tested. S09 fabricated on four engines out of four. S10, where a product-level cost extract exists nowhere in the files, was refused by three of the four, with the kimi leg producing a qualified number instead. So a gap the reader can point at, a column that is absent, still gets refused. A gap that is epistemic, a signal that is absent while the machinery still emits a number, does not.

The three worker legs were metered at the gateway and the primary was not, so this table has a hole in it and states it as one. Re-running the Claude leg through a metered path would have produced a second Claude run under different conditions, whose answers could diverge from the published primary, which buys one table cell at the price of a manufactured contradiction. Deriving a figure from session token accounting would have produced a number with no provenance and no error bars. Neither was done.

Total gateway spend across the whole cycle was $11.68. Of that, $9.20 sits on the three graded runs and $2.48 on failed attempts: grok-4.3's two refusals, a qwen session that died mid-task and a kimi session that hung and burned its full nine-hour cap.

The token split the gateway reports for qwen is unusable, 432 in against 42,313 out, which cannot describe an agentic loop. The billed dollar figure is the gateway's own and stands. No token-level comparison involving qwen appears on this page.

No cost on this page is compared against a human analyst. No sourced analyst rate exists for this work, so the protocol forbids the multiple. The only comparisons drawn are engine against engine, where both sides are measured.

One pack, one dataset version, one run per engine. Nothing here supports a general engine ranking. Engines outside the roster printed at the top of this page are untouched by any of it.

The comparison is fair on everything that could be held identical and unfair on the harness. Those differences are set out below rather than footnoted, because most of them run in the primary's favour and the result should be read with that pressure on it. One difference runs the other way: the workers' blind was absence where the primary's was prohibition.

Two of the four engines needed their one pre-registered redispatch for infrastructure reasons. A qwen session died after an auto-compaction twelve minutes in. A kimi session hung dead after twelve minutes of work and burned the full nine-hour cap. The wall-clock figures on this page come from the successful attempts only, so they are engine speed measured through a harness that failed half the roster once each.

The blind was imperfect in one place, disclosed by the grader who hit it. The anonymizer left a digit residue, the string 45 inside a branch path, in one grader's copy of the run-A artifacts, because the residue scan checked patterns and not digits. That grader disclosed it and states that no inference was drawn. The verdicts stand and the needle list gains digit-bearing branch patterns for any future cycle.

The first grader dispatched on the run-A leg died without writing a verdict or returning any result, so a fresh Opus grader was dispatched under identical frozen instructions. No output existed for the replacement to anchor on, so independence holds.

Correct against a held-out key is still not the same as trusted by a finance leader. No cycle in this collection has tested that yet, on any engine.

This cycle is the model-generality instrument, so the roster is the result rather than a leg deferred to a later page. Four engines were registered and dispatched through the Vercel AI Gateway at the model IDs the harness reports: vercel/xai/grok-4.5, vercel/moonshotai/kimi-k3, vercel/alibaba/qwen3.7-max and vercel/xai/grok-4.3. Three produced gradeable runs.

No scope note in the collection relaxes on the back of this. The reading came in MIXED, so cycles 01, 02 and 03 stay as published, results about claude-fable-5 on their own batteries. Cycle 06 replayed pack 02 alone, so even a clean Reading A would have left the other packs where they are. That price was named in the pre-registration before anything ran.

What the cycle 02 page may now say, in the words registered in advance: its headline finding, sophisticated fabrication where a refusal was correct, generalises to every frontier engine tested. Four engines, four different confident elasticities, one identical failure to refuse.

Kimi was included despite being the expensive engine because cycle 04 showed it behaviourally closest to Claude of the workers, so dropping it would have removed the most informative comparator. It came in discordant, which is the prediction this cycle got wrong.

The pack hash chain runs to twelve links, recorded pre-staging, per staging copy and post-run per worker, every one equal to the pre-staging reference. Six staging builds were probed before dispatch, every capture archived, zero failures.

The probe battery is built as a set of attacks rather than a set of assertions. It attempts to retrieve a held-out truth file from git, to reach an object in the real repository's store by hash, to walk up out of the worktree into the live tree, and to find any string naming the real repository anywhere in the staging surface. All of them fail, which is the point. The transcripts are archived per model.

Attestations are present inside the results file for all three graded runs, which closes the cycle 04 gap where an attestation was a terminal print that never reached the archive. No graded run's transcript or results file shows a prohibited-path read, an absolute-path read toward the real repository, or a network attempt. Peer-blindness holds by construction, since each engine got a repository built fresh whose object store could not contain the previous engine's answers.

The residual risk is named on this page rather than solved. A worker is an OS process with the same filesystem permissions as the harness, so an allowed command can read any absolute path on the machine. No environment construction short of an OS-level sandbox prevents that. What the design does instead is remove every hint: no string in the staging surface names the real repository's location, so reaching the held-out surface would take a model inventing a path it has never seen. The raw harness log is archived per run and scanned at grading for the attempt.