Extracting information from unstructured documents, validating with JSON schemas, maintaining high accuracy, handling edge cases gracefully, and integrating with downstream systems.
Make fields optional/nullable when the source may not contain them — required fields pressure the model to fabricate. Use enums with other + detail and unclear for ambiguous cases.
tool_use + JSON schema eliminates syntax errors. Use tool_choice: 'any' when the doc type is unknown; force a specific tool to run a step first (e.g., extract_metadata before enrichment).
On validation failure, retry with the document + failed extraction + specific error. But retries won't help when the info is absent from the source (vs format/structural errors). Add semantic checks (calculated_total vs stated_total, conflict_detected).
Submit 100 docs via the Batch API, correlate with custom_id, resubmit only failed ones (chunk oversized docs), and size windows to the SLA. Refine prompts on a sample first.
Output field-level confidence, route low-confidence/ambiguous docs to humans, and analyze accuracy by document type and field — an aggregate 97% can hide a bad segment.
1.Some invoices lack a 'po_number'. How should the schema handle it to prevent hallucination?
2.You process mixed document types with several possible schemas and want to guarantee structured output. What tool_choice setting fits?
3.An extraction keeps failing because the required value isn't present anywhere in the source document. Will retry-with-error-feedback help?
4.Submitting 100 documents via the Batch API, a handful fail. How do you handle it efficiently?
5.Using tool_use with a JSON schema, which errors does it eliminate?
6.An extracted calculated_total doesn't match the invoice's stated_total. How do you catch this?
7.You must guarantee extract_metadata runs before the enrichment step. Which setting?
8.A field's value is genuinely ambiguous in the source. Best schema design?
9.Aggregate extraction accuracy is 97%. Why isn't that enough to trust?
10.How do you handle low-confidence or ambiguous extractions in production?
11.A required field keeps coming back empty across varied document formats (inline citations vs bibliographies). Best fix?
12.Validation keeps failing because the required value simply isn't present anywhere in the source. Will retry-with-error-feedback help?
13.Some invoices lack a po_number. Making it a required field has what effect?
14.You process mixed document types with several possible schemas and must guarantee a structured result. Which tool_choice?
15.To find novel error patterns you haven't seen, how should you sample outputs for human review?
16.Before running extraction across 100 documents in batch, what's the smart first move?