Preventing Exception Propagation Into ERP Systems

Most exceptions that engineering and operations teams treat as ERP problems are not ERP problems at all. They are document extraction failures that entered the system unchallenged and were never caught.

The mechanism is simple enough to trace. A document, an invoice, a purchase order, a claim form, gets parsed upstream. The extraction produces a value that looks structurally sound, and the ERP takes it in as legitimate. No exception fires. No flag gets raised. The bad data spreads into every downstream process that touches that record.

AI agent pipelines make the pattern worse by design. A hallucinated output in step two becomes a malformed input in step three. By step five, the agent is reasoning from a premise that was never true, and still nothing has raised an exception or signaled that anything went wrong.

The ERP itself deserves some credit here, because it is doing what it was built to do. It is a faithful processor: it runs correctly on whatever it receives. That single fact relocates the entire failure surface. The problem does not live inside the ERP. It lives at the handoff point, the moment extraction output becomes ERP input.

That relocates where engineering attention belongs, too. Hardening the ERP's own validation logic treats a symptom. The circuit-breaker belongs at the extraction layer, before any record gets written. Everything that follows in this piece works from that premise: the fix lives upstream, not downstream.

Extraction Pipelines and Values That Look Correct but Are Wrong

The extraction failures that do the most damage are not crashes. A crash is loud and gets noticed. The damaging failures are silent ones: the pipeline returns a value, that value looks structurally plausible, and nothing downstream knows it is wrong.

A case study on a business product specification table, run against Logics-Parsing-v2 as part of the PureDocBench Case 2 benchmark, confirmed three distinct failure modes that show how this happens in practice.

The first is header metadata omission. The product header block, the piece carrying brand and model, is missing entirely from the reconstruction. Downstream systems have no way to know which component the document even describes.

The second is a reading order error. Tables survive the parse, but they land in the wrong section order, which breaks the logical grouping that finance and engineering teams depend on to match line items correctly.

The third is a technical symbol recognition error, and it is the sharpest example of how a number can be both reasonable-looking and catastrophically wrong at once. Engineering symbols get silently mutated, producing a unit error off by six orders of magnitude, with no parse failure and no confidence flag to warn anyone. A value written as a safe inductance limit in nanohenries comes out rendered in millihenries, a unit error that would misstate a safety-critical specification and nothing in the pipeline would say so.

Each failure is silent under standard pipeline monitoring, and the parser returns HTTP 200. The extraction field is populated. The value passes schema validation because it is the right data type and the right length. None of that confirms the value is actually correct.

Failure modes also cluster by document type. Business documents tend to fail on structural integrity, meaning table layout, reading order, and header presence. Technical documents tend to fail on notation fidelity instead. A pipeline tested against one kind of document can pass every test and still fail silently on the other.

For an ERP, the consequence is direct. A line-item amount pulled from the wrong table row, or a vendor ID pulled from the wrong header block, will clear every downstream schema check and land in the ERP as a valid, processable record.

A natural objection here is that a confidence score should catch this. It only helps if that score is calibrated. A miscalibrated score applied to a structurally plausible but wrong value will read as high-confidence anyway, and calibration is a separate engineering problem from extraction itself, one addressed later in this piece.

How multi-step pipelines amplify extraction errors before they reach the ERP

A single wrong value rarely stays contained to the step that produced it. In a multi-step extraction-to-ERP pipeline, that value conditions every later step that reads from it, and the error compounds before any record ever gets written.

The propagation works like this: one step's output becomes the next step's input. If step one retrieves the wrong context and step two produces a malformed result because of it, step three reasons over bad data without any awareness that the data is bad. By the time a final output appears, the original error sits several layers deep, invisible in the response that comes out the other end.

Tool-calling adds its own version of the same risk. Openlayer's analysis found that tool-calling fails in production AI agents somewhere between 3 and 15% of the time. A hallucinated parameter passed into a tool call can create a record that never should have existed, charge a payment that should never have gone out, or kick off a downstream process that is hard to undo.

Testing does not catch much of this reliably, because the same prompt can produce different tool choices, different reasoning paths, and different outputs on different runs. A failure appears in production yet may refuse to reproduce in a staging environment.

The observability gap compounds the problem structurally. Agent reasoning steps often never get written to any log. Without a record of which tools were called, what parameters were passed, and what came back, there is no way to reconstruct what actually happened once something breaks.

For an ERP, this means a compound extraction error can reach it after having already touched a purchase order match, a vendor lookup, and a GL account mapping, every one of which processed a value that was wrong from step one and never got caught.

Checking only what enters the ERP comes too late. The error is already embedded across related records by then, and unwinding it means tracing back through steps that were never logged in the first place. Pipeline depth multiplies the cost of a single upstream mistake. The deeper the pipeline, the more expensive the error gets to find and the more expensive it gets to reverse.

What field-level validation requires

Catching extraction errors before they reach the ERP requires validation that runs at the level of the individual field, checked against a known ground truth. Document-level schema checks and pipeline-level error monitoring both miss the exact class of failure that actually propagates.

The distinction is concrete. A document-level check confirms that a JSON payload has the expected keys and the expected types. It does not confirm that the number sitting in an "invoice_total" field is the number that actually appeared on the invoice, rather than a number lifted from the wrong table row.

Cross-field logic asks whether the line-item sum equals the stated total. Format rules ask whether a VAT number follows the correct national format. Reference data asks whether a vendor ID actually exists in the master file. That line-item-sum check matters because it is exactly the kind of test a document parser does not run by default, and an ERP does not run at all. The ERP has no way of knowing what the invoice actually said.

Enterprise documents make this harder because they mix narrative text, complex tables, and embedded images in the same page. Field-level validation has to account for that structural mess in the source, checking more than the shape of the final output.

The architecture built to prevent hallucination in regulated extraction rests on three confirmed pillars. Grounding forces the model to reference the source document before it produces a value. Structured outputs constrain what the model is allowed to return, so a hallucination becomes syntactically impossible in the first place. Verification loops let the model check its own output against the source before anything moves forward.

A fourth layer sits on top of those three: agentic validation. A downstream agent that receives extraction output should check field values against business rules before passing anything to the ERP, and this is not a redundant safety net so much as the main circuit-breaker the extraction layer cannot provide on its own. EXTRACTCONF's 2026 results back this up directly, showing a 25.8 percentage point improvement over the unvalidated base rate once this step was added.

Extraction without a governed validation step, without consistency checking, without a traceable artifact, is not production-grade for ERP ingestion, no matter how capable the underlying model is.

Teams that lean on their ERP's built-in validation often raise the same objection: the ERP already has its own validation rules. It does, but ERP validation only checks whether a value is a valid type inside the ERP's data model. It never checks whether that value matches what the source document actually said. Those are two different questions, and only field-level extraction validation answers the second one.

Confidence thresholds as a routing mechanism, not a pass/fail gate

A calibrated confidence threshold routes uncertain extractions to human review before they ever reach the ERP, but the word calibrated is doing all the work in that sentence. A stated probability has to actually correspond to observed field-level accuracy at that threshold, or the score is just noise dressed up as a number.

The production pattern looks like this: when a field's predicted confidence drops below a defined threshold, the document routes to manual review instead of going straight to the ERP. It is a controlled handoff that keeps high-confidence documents moving at full speed while pulling aside the cases where the extraction is genuinely uncertain.

Done well, this changes the effective accuracy of the whole system. A parser with a given field-level accuracy, processing the bulk of its volume above threshold, behaves as though its accuracy were much higher in practice, because the below-threshold cases, the ones carrying the higher error rates, get intercepted before they ever write to the ERP.

Calibration is not uniform across field types, though, and that is where a lot of pipelines get careless. Numeric fields tend to be well-calibrated. Free-text fields show overconfidence at high predicted probabilities. That means the threshold that works for an invoice amount field is the wrong threshold for a vendor name field or an address field. A single global cutoff applied everywhere will let exactly the wrong cases through.

This matters most where it is least visible. Overconfident scores on free-text fields mean errors in vendor names, line-item descriptions, and address fields pass as high-confidence and route straight to the ERP with no review at all.

A confidence score from a vision model that cannot be traced back to specific pixels on the source page is merely an inference about what is happening inside the model, and it carries no guarantee about field-level accuracy.

Two things have to exist alongside calibration for any of this to work. A threshold set empirically against ground-truth data, for each field type separately, and a human review queue that is actually staffed to handle the volume that falls below it. If either one is missing, the circuit-breaker is technically in place but functionally useless; intercepted cases pile up and get batch-pushed into the ERP anyway.

Structured output guarantees at the ERP boundary

Structured output only means something at the ERP boundary if the structure constrains what the model can produce, not merely what shape the output takes.

The distinction is easy to miss and important to hold onto. A JSON schema that requires a field called "invoice_total" to be a number stops a non-number from getting in. It does nothing to stop the extraction from putting the wrong number in that field. A format guarantee and a correctness guarantee are not the same promise.

Constrained decoding, forcing a model's output to conform to a schema at the moment of generation, makes hallucination syntactically impossible for the fields it constrains. It still does not make the value inside that field correct. Those two properties, format and correctness, are independent of each other, and a production pipeline needs both.

For ERP ingestion specifically, the output contract should include more than well-formed JSON. Field-level confidence scores attached to each value. A reference to the source region, a page, a bounding box, or a character offset, for every extracted field. And a completeness check confirming that every required field for that document type actually got extracted, beyond confirming that the overall JSON parses cleanly.

Designing the extraction-to-ERP handoff so exceptions are caught at the source

Field-level validation checks each value against ground truth, cross-field logic, and reference data before anything moves forward. Calibrated confidence thresholds route the uncertain cases to a staffed review queue instead of letting them pass by default. A structured output contract carries confidence scores, source references, and a completeness check alongside every value, so the ERP never has to take a field on faith.

None of these three mechanisms works alone. Field-level validation without calibrated routing has no way to decide which flagged values need a human. Calibrated routing without a structured output contract has no source reference to route to, nothing for a reviewer to check the value against. A structured output contract without field-level validation is just a format guarantee wearing the costume of a correctness guarantee.

The handoff between extraction and ERP is the only place any of this can happen. Once a record writes to the ERP, it starts conditioning purchase order matches, vendor lookups, and GL account mappings, and unwinding a bad value after that point means tracing back through steps that were likely never logged in the first place. Catching the error before that write is the only point in the entire pipeline where catching it is still cheap.