Success Without Correctness: Why Your Document Pipeline Needs to Distrust Its Own LibrariesSuccess Without Correctness: Why Your Document Pipeline Needs to Distrust Its Own Libraries

I've been deep in document-processing pipelines lately — the kind that take a PDF, run it through a parser, chunk it, embed it, and hand it to a RAG system. And I've come to believe one thing pretty firmly: the most dangerous bug in a document pipeline isn't the one that crashes. It's the one that returns successfully, looks completely fine in the logs, and quietly hands you garbage.

I want to walk through a real chain of three bugs I dug into recently — stacked one on top of the other, each one hiding behind the last — because together they teach a lesson that applies to basically any pipeline built on third-party libraries, not just document parsing.

The setup

The system: a document ingestion pipeline using Docling (a structure-aware PDF parsing library) to convert uploaded PDFs into text, tables, and layout-aware chunks before they get embedded and stored for retrieval. Pretty standard RAG plumbing. The trigger: a real client deployment in the financial sector needed picture-description support, the ability to have a vision model describe embedded charts and diagrams, wired up and working in production.

That single feature request ended up unraveling three separate, independently-hiding correctness bugs. None of them threw an exception. All three looked, from the outside, like success.

Bug one: "success" that returned nothing

The first thing the team hit, once real documents started flowing through the newly-wired vision path, was a document that came back completely empty. No error. No stack trace. The pipeline had done exactly what it always does, called the converter, gotten a result object back, moved on, and the result object's own status field, the one place that actually knew something had gone wrong, was never being checked.

Here's the shape of the mistake, generalized:

result = converter.convert(file_path)
markdown = result.document.export_to_markdown()
# ...proceeds as if this always works

This "works" in the sense that it never raises. But conversion libraries, Docling included, don't always throw on a bad conversion. They often return a result object with its own status: success, partial success, or failure, sitting quietly in a field nobody's looking at. If your code assumes "no exception" means "correct output," you've built a pipeline that can silently process nothing at all and report full success the entire time.

The fix is almost embarrassingly simple once you see it:

result = converter.convert(file_path)
 
if result.errors:
    logger.warning(
        "conversion reported errors: %s",
        "; ".join(f"{e.module_name}: {e.error_message}" for e in result.errors),
    )
 
if result.status not in (ConversionStatus.SUCCESS, ConversionStatus.PARTIAL_SUCCESS):
    raise RuntimeError(f"conversion status={result.status.value} for {file_path}")

Simple, yes. But notice what it required first: someone had to suspect the library might be lying to them. That's the actual skill here, and it's not a technical one, it's a posture. The default assumption most of us bring to a well-maintained library is "if it didn't throw, it worked." That assumption is exactly backwards for anything doing lossy, best-effort work on messy real-world input, which is what document parsing always is.

Bug two: fixing bug one revealed a worse one

Here's the part I find genuinely instructive. Adding the status check didn't just fix the empty-document problem, it exposed a second bug that had been there the whole time, invisible until something finally started checking for failure.

With the status check in place, a scanned document came through and the conversion reported success, technically true, but with zero pages of extracted text. Digging in, the root cause was almost absurd in its simplicity: the OCR engine (Tesseract) was being told the document's language was "en".

That's wrong. Not because English isn't the right language, because Tesseract wants ISO 639-2 codes, not 639-1. It needed "eng", not "en". Passed "en", the Tesseract CLI fails to load any language data and does essentially nothing.

# Wrong — Tesseract can't find a language pack for "en"
ocr_options.lang = ["en"]
 
# Correct — ISO 639-2, the three-letter code Tesseract's traineddata files use
ocr_options.lang = ["eng"]

Here's the part that made this genuinely hard to find rather than a five-minute fix: the subprocess wrapper calling Tesseract was discarding stderr. Tesseract almost certainly complained loudly on stdout or stderr about not finding a language pack, and that complaint went straight into the void. The failure was invisible not because it was subtle, but because the exact channel that would have explained it had been silently thrown away somewhere upstream.

This is worth sitting with: a one-character-vs-three-character typo, combined with a discarded error stream, produced a bug where OCR simply didn't run, on documents that specifically needed OCR to be useful at all, and the whole pipeline reported success throughout.

Bug three: fixing bug two revealed a third one

I promise this isn't me padding the story, this is genuinely what happened. Fixing the language code surfaced a KeyError('text') on certain scanned documents. Root cause: a TESSDATA_PREFIX environment directory that was missing its configs/ subfolder. Without it, a request for TSV-formatted OCR output (which the pipeline needed to recover word-level positioning) silently degraded to plain text output instead, a different, unexpected shape that the downstream code wasn't ready to parse.

Three bugs. Three separate root causes. Every single one of them was a necessary but not sufficient condition for real OCR text to actually reach the rest of the pipeline. Fix any two of the three, and you'd still end up with silently broken output, just a different flavor of broken.

Fixing any one node in that chain without the other two still leaves you at a red box, not the green one at the end. That's the shape of the whole post: each fix didn't solve the problem, it advanced the investigation one layer deeper.

Why this pattern matters beyond OCR

I don't think the lesson here is "be careful with Tesseract language codes." The lesson is structural, and it generalizes to almost any pipeline built on external tools:

A library completing without an exception tells you the library's own error-handling didn't trigger, it tells you nothing about whether the output is correct. Conversion libraries, parsers, OCR engines, even some HTTP clients, all have their own internal notion of partial or degraded success that lives in a return value, not in whether Python decided to raise. If your code never inspects that return value, you've delegated your own correctness checking to a library that has no idea what "correct" means for your specific use case.

Fixing the first bug you find is often necessary but not sufficient. Each fix in this chain didn't just resolve an issue, it removed a layer of masking that had been hiding the next one. If the team had stopped after the status-check fix, declared victory, and moved on, the language-code bug and the tessdata bug would have kept silently degrading OCR output indefinitely, protected by exactly the kind of "well, it didn't error" complacency that let bug one exist in the first place.

Discarded stderr is a decision, even when nobody made it deliberately. Somewhere in a subprocess wrapper, stderr=subprocess.DEVNULL or an equivalent was almost certainly sitting there, probably added for a reasonable-sounding reason (cleaner logs, maybe) at some point in the codebase's history. It turned a five-minute "oh, wrong language code" fix into a genuine investigation. If you're wrapping any CLI tool as a subprocess, ask yourself right now whether you're actually capturing and surfacing its error output, or just assuming success because the return code looked fine.

The practical takeaway

If you're building anything that leans on a third-party conversion, parsing, or transformation library, PDF parsers, image processors, format converters, OCR engines, whatever, go check right now whether you're actually inspecting that library's own success/failure signal, or whether you're inferring success purely from "my code didn't crash." Those are not the same thing, and the gap between them is exactly where silent data loss lives.

The uncomfortable truth is that this kind of bug is cheap to prevent and expensive to find after the fact. A two-line status check costs nothing to write. Finding out three months later that an unknown number of documents silently produced empty or half-broken output costs a debugging session, an uncomfortable conversation about data quality, and, if you're running a RAG pipeline on top of this, an unknown amount of degraded answer quality that nobody can even quantify after the fact, because the bad chunks look exactly like the good ones once they're sitting in your vector store.

Trust your own code. Distrust everyone else's happy path, especially the ones that don't throw.

If you've hit a similar "it didn't crash but it also didn't work" bug in a data pipeline you're running, I'd like to hear about it, these are always more common than people admit once you start asking around.