← all model tests
Inspectorpass

label lost to glare is recoverable

The label is present and unreadable. That is ADD_FIELD, not ESCALATE — a person does not need to be woken up for a photograph that can simply be taken again from a different angle.

The answer conformed to the contract and every assertion held.

modelgemini-3.5-flash
temperature0
scenarioscenarios/inspector/label-lost-to-glare-is-recoverable.json
cassette2302deace857372e
latency0.0s
tokens4,706
What the model was given
System instructionassembled from the contract schema, not hand-written
Return exactly one verdict for the evidence supplied against this step. PASS only if the acceptance rule is satisfied by what you can actually see. THAT THE EVIDENCE IS CLEAR ENOUGH TO JUDGE IS NOT ITSELF A PASS: a sharp, well lit, entirely legitimate photograph showing the acceptance rule is NOT met is the commonest way a step fails, and passing it because the photograph is good is the single worst mistake you can make. Where the acceptance description says what must be visible rather than what must be true, read it as the condition the step exists to establish — the step explanation tells you which. ADD_FIELD when the evidence is insufficient but recoverable, and name what specifically was wrong with it. ESCALATE when a person must decide — INCLUDING when the evidence is entirely sufficient and shows the acceptance rule is not satisfied. Asking for another photograph of a condition you can already see is not a remedy.

Return JSON with these fields:
- verdict (one of PASS | ADD_FIELD | ESCALATE, required)
- confidence (number, required): 0 to 1: YOUR CONFIDENCE THAT THE ACCEPTANCE RULE IS SATISFIED by this evidence. Not how sure you are of your verdict, and not how good the photograph is — a perfectly clear photograph of a rule being broken is high certainty and LOW confidence, because the rule is not satisfied. Evidence you were never shown cannot make you confident of anything. Below the strictness threshold you must not return PASS, and a threshold in ordinary code enforces that whatever you return.
- rationale (string, required): One or two sentences citing what in the evidence decided it. Never restate the prompt.
- add_field_key (string, null when it does not apply): Required when verdict is ADD_FIELD. A new key, not an existing one.
- add_field_kind (one of measurement | photo | video | scan | choice | text | signature | location, null when it does not apply)
- add_field_prompt (string, null when it does not apply): The specific next ask, e.g. 'the label is out of focus, photograph it again'. Never a generic retry.
- escalation_question (string, null when it does not apply): Required when verdict is ESCALATE. The exact unresolved question for the person, not a summary.
- observed (string, null when it does not apply): Required when the acceptance rule is `matches`. TRANSCRIBE, character by character, exactly what you can read in the evidence — not what it ought to say. Never copy the expected value into this field; it is the one field in this answer that must come from the image alone. Where a character is illegible write `?` in its place. The comparison against the expected value is then made in ordinary code, not by you, which is the whole point: an agent told what it is looking for will find it.
## The step
{
  "title": "Record the part fitted",
  "why it exists": "A record that does not say which part went in cannot answer the only question that matters after a failure: was it the right part."
}

## The field you are judging
{
  "key": "part_label",
  "kind": "photo",
  "asked of the technician": "Photograph the part number on the box of the pads you fitted",
  "source": "camera",
  "what good looks like": "The printed code has to be readable in the photograph. Watch for glare off the light.",
  "acceptance": "acceptance_rule: matches"
}

## Transcribe. You have not been told the answer.
This field's rule is `matches`, and you have DELIBERATELY NOT been shown what the evidence is supposed to say. Nothing in this prompt contains it.

Put in `observed` exactly what you can read in the image, character by character. Where a character is illegible, write `?` for it — a `?` is a useful answer and a guessed character is a false one, because the comparison is made in ordinary code from what you write here.

Your verdict is therefore NOT about whether it matches, which you cannot know. It is about whether the evidence can be read at all: PASS if you could transcribe it with confidence, ADD_FIELD if it is unreadable and another photograph would fix that, ESCALATE if no photograph would.

## What a PASS would assert
That this is true of the machine: acceptance_rule: matches.
The step exists because: A record that does not say which part went in cannot answer the only question that matters after a failure: was it the right part..

State what the evidence shows about that assertion BEFORE you choose a verdict, and put it in your rationale. If what you can see makes the assertion FALSE, the verdict is not PASS — however sharp, well lit and honest the photograph is. Asking for another photograph of a condition already visible is not a remedy, so that case is an ESCALATE, not an ADD FIELD.

## Conditions
{
  "strictness": "2 (assured)",
  "confidence a PASS must clear": 0.75,
  "add field budget": "0 of 2 ADD FIELD requests on this step are already spent.",
  "capture surface": "app_instrument",
  "capture mode": "live"
}
## The evidence
The image below is what the technician captured for this field. Judge what you can actually see in it. If you cannot see enough to decide, say what specifically is missing — never ask for a generic retry.
label/part-number-glare.jpg
label/part-number-glare.jpgimage/jpeg · m:image/jpeg:3a771dd1e3cf836e
What came backStructured output, not prose that was parsed afterwards: the contract schema is posted as the response schema, so the model returns this shape or fails.
{
  "verdict": "ADD_FIELD",
  "confidence": 0.3,
  "rationale": "The barcode sticker has significant glare, making the alphanumeric code underneath the barcode faint and illegible.",
  "add_field_key": "part_label_clear",
  "add_field_kind": "photo",
  "add_field_prompt": "The barcode sticker has glare on it making the code illegible. Please take another photo of the label from an angle that avoids light reflection.",
  "escalation_question": null,
  "observed": null
}
What this scenario asserted

Assertions pin the part of an answer that decides something, never the whole object — exact-matching a rationale would fail the first time it was reworded, and a suite that cries wolf is abandoned inside a day. Where the reasoning is checked at all, it is to catch an agent that reached the right verdict for no reason.

assertedaboutwhat happened
okequalsverdictis ADD_FIELD, want ADD_FIELD
okpresentadd_field_promptis The barcode sticker has glare on it making the code illegible. Please take another phot...
okmentions_anyadd_field_promptfound ['glare', 'angle', 'light', 'reflect', 'legible']