← all model tests
Scopernot asked

brake fluid, where using the wrong kind destroys the brake and the label says which

Mineral oil and DOT fluid are not interchangeable — putting DOT into a mineral-oil system swells every seal in it and the brake fails, usually later rather than immediately. Which one a brake takes is stamped on the lever or the reservoir cap, so the acceptance rule resolves against the machine rather than against a number somebody remembered. This is the case where per_spec is not a convenience, it is the only safe way to write the rule.

The agent was never properly asked, so there is nothing to score either way.

modelgemini-3.5-flash
temperature0
scenarioscenarios/scoper/interview-home-change-hydraulic-brake-fluid.json
Why this was never asked

Nothing was scored for or against the agent here. The suite reports this as its own outcome rather than a failure, because an agent asked an incomplete question will still confidently answer it.

no cassette f8ec577ba7d8 for this input; re-run with --live to record it
The interviewThe shop is played by a second model, given a fixed sheet of figures. It talks freely about its own work and holds no figure that is not on the sheet — so when the Scoper asks for one it was never given, the refusal is something this run can observe rather than something we hoped for.
Could every figure be traced back?

Inventing a tolerance is the one thing the Scoper must never do. A fabricated bound enters every future record indistinguishable from a figure a person set, and nobody downstream can tell which is which. So each bound in the compiled procedure is compared against the numbers the shop actually said aloud.

figures the shop disclosed

none — the shop never stated a figure

bounds in the compiled procedure

nothing compiled

0 bounds appear in the procedure and nowhere in the transcript.

Where the interview stood when it endedIt did not compile. `unresolved` is what it still did not know — and while that list is non-empty, compiling is not permitted.
{}
The prompt behind the last turnthe whole conversation is re-laid in front of the model every turn