Design human review workflows and confidence calibration

A single headline metric like "97% accurate" is an average over the whole document population, and averages are dominated by the common case. If 90% of documents are clean typed invoices scoring 99% and 10% are handwritten receipts scoring 62%, the blended number still reads about 96% and looks production-ready, while one entire segment is quietly broken.

The exam frames this precisely: aggregate accuracy metrics may mask poor performance on specific document types or fields. The failure has two axes. By document type (typed versus handwritten, one vendor's layout versus another) and by field (invoice_number is easy, a multi-line tax breakdown is hard). A field can sit at 70% while the document as a whole scores well because the other fields carry the average.

The design consequence: you must validate accuracy by document type and field segment before automating high-confidence extractions or reducing human review. The decision to remove a human is made per segment, never for the pipeline as a whole based on one blended figure.

Confidence should be reported per field, because difficulty varies field by field within the same document. Ask the model to emit a confidence value alongside each extracted value, which you can do directly in the tool_use output schema from task 4.3.

{
  "invoice_number": { "value": "INV-4471",   "confidence": 0.98 },
  "total":          { "value": 1840.00,      "confidence": 0.95 },
  "tax_id":         { "value": "12-3456789", "confidence": 0.71 }
}

A single whole-document confidence hides the pattern that matters. A receipt can be extracted with high confidence on eight easy fields and low confidence on the one hard field, and it is that one field you need to route to a human. Field-level scores let you review only the risky field instead of discarding, or blindly accepting, the whole extraction.

Raw model confidence is not trustworthy out of the box. A model that reports 0.9 is not necessarily correct 90% of the time, so you cannot pick a threshold by intuition. Calibration means running the model over a labeled validation set (documents with known-correct answers), grouping predictions by reported confidence, and measuring the actual observed accuracy at each confidence level.

The result is a mapping from reported confidence to observed accuracy, computed per segment. You then set the auto-accept threshold where observed accuracy clears your quality bar. That threshold is usually different for each field and document type: 0.95 confidence on invoice_number might correspond to 99% real accuracy, while 0.95 on tax_id corresponds to only 88%, so the two fields need different cutoffs.

This is the nuance that separates 5.5 from 5.2 and 4.6. Uncalibrated self-reported confidence is an unreliable proxy (task 5.2 warns against using it as an escalation trigger, since the model is often confidently wrong on hard cases). Calibration against labeled data is exactly what turns a raw, untrustworthy number into a usable routing signal. Confidence is only as good as its calibration.

Once high-confidence extractions bypass human review, you still need to know their real error rate, and you cannot review all of them without defeating the automation. So you sample a subset for ongoing measurement. Simple random sampling is not enough: rare document types are under-represented, so a segment that is 3% of volume but failing badly may never appear in the sample.

Stratified random sampling divides the auto-accepted population into strata (by document type, by field, and by confidence band) and samples within each stratum, guaranteeing coverage of rare and high-risk segments. This produces an ongoing, per-segment error-rate measurement rather than a one-time launch gate.

The second job of this sampling is detecting novel error patterns. Distributions drift over time: a new vendor layout appears, a form template changes, a new document type enters the stream. The model can be confidently wrong on these, so they land in the auto-accept lane undetected. Stratified sampling is the mechanism that surfaces these emerging failures before they accumulate into a large silent error rate.

Human reviewers are the scarce resource, so routing is about spending their attention where it changes outcomes. Route to human review: extractions with low calibrated confidence, and extractions from ambiguous or contradictory source documents (a document that states two different totals, or a scan too degraded to read reliably). Auto-accept: high-confidence extractions in segments you have already validated.

Confidence is the prioritization signal. When reviewer capacity is limited, rank the queue so the lowest-confidence and highest-value items are seen first, and let the clearly-fine cases through. This is the same per-finding-confidence-for-routing idea from task 4.6, applied to extraction QA rather than to code-review findings.

Note the two distinct triggers for a human: model uncertainty (low confidence) and source-document uncertainty (ambiguous or self-contradictory input). A document can be genuinely unresolvable even when the model is confident, which is why ambiguous sources route to a human regardless of the confidence score.

Putting it together, the decision to reduce human review is gated on segment-level evidence, not on the headline number. For each document type and field, confirm on the labeled validation set that calibrated high-confidence accuracy clears your bar. Automate only those segments, and keep a human in the loop for segments that fail or that you have not measured yet.

After launch, the stratified sampling audit keeps the gate honest over time. If a segment's sampled error rate rises, or a novel pattern appears, that segment goes back to human review. Automation coverage expands and contracts based on measured, segmented accuracy, and never on a single aggregate figure that can drift out from under you.