Chapter 2 — Intelligent Document Parsing

Second post of the chapter-by-chapter walkthrough of LLM Primer III: Enhancing Enterprise AI with RAG. A retrieval system inherits the quality of its inputs — and the input layer is where the most common cause of disappointing RAG quality quietly lives.

Why this chapter exists

The first version of a RAG pipeline almost always uses whatever PDF-to-text utility was lying around. Plausible-looking text comes out, the index gets populated, the model produces plausible-looking answers. Months later the team discovers that tables were silently flattened into prose, multi-column papers were interleaved line by line, footnotes were spliced into paragraphs, and figure captions were lost entirely. The retrieval quality ceiling was set by these decisions before retrieval was even configured. The chapter is about taking the input layer seriously, because nothing downstream can recover what the parser threw away.

2.1 Why flattening a PDF loses what matters

A PDF is a list of glyphs with coordinates, drawn on pages of declared dimension. The visual structure a human sees — columns, tables, captions, sidebars — is not stored anywhere in semantic form. It exists in the rendered image. So “extract text from PDF” is harder than it sounds: the naive extractor reads the glyph stream in the order the marks were written, which on a two-column page interleaves the columns line by line. What comes out is grammatically odd, semantically broken text composed of real words from the real document — the kind of failure that is hard to spot in a spot check.

Tables are worse. The meaning of 1,427 in row 3, column 4 is the intersection of Q3 2024 and Northeast region. To a naive extractor it is a number with no relationship to either string, because the strings were drawn elsewhere on the page. The table dissolves into a list of numbers separated by whitespace, and queries about “Northeast revenue in Q3” find nothing — the chunk that contains 1,427 does not contain Northeast near enough to associate them in the embedding. Forms have the analogous failure mode: labels and values are emitted as disconnected strings, and the index now contains values without their field names. OCR on scanned documents adds character-level errors precisely on technical terms and proper nouns — the place retrieval is most sensitive to spelling.

2.2 Layout-aware parsing: putting the signals back

The response is a class of tools that treat the document as a two-dimensional artifact rather than a glyph stream. The page is rendered to an image, a layout detection model segments it into regions (paragraphs, tables, figures, headers), reading order is reconstructed using document-layout heuristics, and tables are passed through specialized models that recover row and column structure into HTML, Markdown, or JSON. The output is no longer a flat string — it is a structured representation that preserves the hierarchy, ties figure captions to their figures, and exposes metadata the downstream chunker can split on.

The cost is compute — one to several seconds per page versus milliseconds for naive extraction, which matters for million-page corpora. And the failure mode changes: a naive extractor that mangles a table at least produces text. A layout-aware parser that misidentifies a region produces structured output that may be confidently wrong — a figure mistaken for a table, a header detected as body text. The team needs to spot-check representative complex pages before trusting the pipeline at scale.

2.3 The current tool landscape

The space has consolidated around half a dozen tools worth knowing. LlamaParse is the hosted parser from LlamaIndex — strong on tables and forms, the right default if you are already inside the LlamaIndex ecosystem and managed services are acceptable. Docling is IBM’s open-source layout-aware parser, with the TableFormer model handling complex table structures, and is the natural choice for on-premises deployments where data cannot leave your infrastructure. Unstructured optimizes for breadth — many input formats, a typed-element partitioning model, consistent downstream interface — and is the safest first choice for heterogeneous enterprise corpora. Marker-PDF does one thing very well: PDF to clean Markdown, with particular attention to headings, lists, and code blocks. Firecrawl handles the web-side input problem — URL in, clean Markdown out, with boilerplate stripped. DeepSeek-OCR, released in late 2025, encodes pages into very few vision tokens for dramatically lower memory and compute, and is the serious contender when throughput dominates the budget.

The practical evaluation looks like this: take fifty representative documents that span the corpus’s difficulty range, run each tool over them, manually compare on the dimensions that matter for your corpus — table fidelity, multi-column reading order, OCR accuracy on scans, figure handling, throughput. The winner is rarely best on every dimension. It is best on the dimensions that matter most for your corpus, at a cost your budget can absorb.

2.4 The multimodal alternative

A parallel track rejects the framing entirely. If a vision-language model can read a page well enough to answer questions about it, why convert to text at all? Late-interaction multimodal retrievers like ColPali and ColQwen2 extend the ColBERT idea to images — one embedding per patch of the page, scored against query tokens via max-similarity aggregation. The retriever surfaces pages whose textual content alone would not have matched, because the relevant information was in a table, a figure, or a layout the text extraction would have garbled. The vision-language model reads the page directly.

The cost is substantial and worth being concrete about. A standard text chunk produces one embedding of ~1,024 dimensions — a few kilobytes. A ColPali-encoded page produces around a thousand patch embeddings of ~128 dimensions — half a megabyte per page. Index size for a million pages grows from gigabytes to hundreds of gigabytes, scoring is more expensive, and generation requires a vision-language model. For corpora dense with tables and figures the upgrade is real. For prose-dominated corpora on a tight budget, well-parsed text retrieval is still the cost-effective default. Hybrid configurations — ColPali for retrieval, text-converted for generation, or vice versa — are where most production multimodal RAG will land over the next year.

What Chapter 2 sets up

A clean, layout-aware parse is necessary for high-quality RAG and sufficient for nothing. A parsed document is still a document — it has to be broken into pieces small enough to embed and large enough to mean something. The chunker that ignores the parser’s structural hints discards what the parser worked to preserve. The two layers have to be designed together, and Chapter 3 walks the chunking spectrum and the frontier techniques that have reshaped it.

Next — Chapter 3: Advanced Chunking Frameworks. The chunking spectrum from fixed-size to structure-aware, the overlap myth, the context cliff, and the contextual-retrieval and late-chunking techniques that have changed the calculus.