Why your RAG pipeline misses half the PDF

If your retrieval system keeps missing things that are definitely in the document, the problem is usually not the embedding model, the vector database, or the prompt. It is that the text in your index is not the text that was on the page. Four things go missing during extraction, none of them raise an error, and all four are invisible unless you go and look.

Nobody reads their chunks

Start here, because it is the single highest-value thing you can do and almost nobody does it.

Pull twenty chunks out of your index at random and read them. Not the metadata, not the similarity scores, the actual text. Most teams building on document retrieval have never once looked at what is stored, because the pipeline runs without errors and the system returns answers, so there is nothing obviously to investigate.

Twenty chunks takes five minutes and it will tell you more than a week of tuning retrieval parameters. Every failure below is visible on sight once you are looking at the text. The reason they persist is that the output of a broken extraction is not broken-looking. It is fluent. It just is not what the document said.

1. The columns are interleaved

What it looks like: sentences that are almost grammatical and completely wrong. Half of one thought, then half of an unrelated one, joined at a point where a full stop should be.

…the committee found no evidence of systematic risk in the Appendix C sets out the methodology used to sample the second quarter results…

Why it happens: a PDF stores glyphs at coordinates, not columns. If a tool sorts that text by vertical position, it takes the first line of the left column, then the first line of the right column, then the second line of the left, and so on. The two columns are zipped together, line by line.

Why it is the worst of the four: it is the only one that produces plausible output. Missing text is obvious. Garbled characters are obvious. Interleaved columns read like prose, embed like prose, retrieve like prose, and answer questions with sentences nobody wrote. A model given that text will confidently attribute a claim from one column to the subject of the other.

How to check: convert any two column document you have, a paper or a report, and read the first three paragraphs of the output. You will know within seconds.

What fixes it: finding the columns before reading the text, rather than sorting by position and hoping. The layout has to be cut at the gutter, and each side read to its end. That is what our converter does, and the mechanics of it are in the main guide.

2. The page furniture is inside your chunks

What it looks like: a running header, a footer, or a page number sitting in the middle of a sentence.

…the board resolved that the Annual Report 2025 14 funding be withdrawn from…

Why it happens: headers and footers are drawn on the page like any other text. They sit, in the file, between the last line of one page and the first line of the next. Extract the text in order and they land exactly there, in the middle of whatever sentence spans the page break.

Why it matters more than it looks: it is not just untidy. That chunk gets embedded with the noise inside it, and its vector is now pointing somewhere slightly wrong. You have not lost the content, you have moved it, which is harder to notice and harder to debug. Multiply by a document set where every page carries the same three lines of boilerplate and a meaningful fraction of your index is company name and page number.

How to check: search your chunk store for the document title. If it appears hundreds of times, it is furniture, and it is inside your content.

What fixes it: detecting the repetition. One page cannot tell you what its own furniture is, but twenty pages can: text that appears in the same zone at the top or bottom of many pages is scaffolding, and page numbers have to be normalised first so that “page 3” and “page 4” are recognised as the same thing.

3. The structure is gone, so you are chunking on character counts

What it looks like: chunks that begin mid-sentence and end mid-sentence, and that cut an argument in half so that neither half is retrievable.

Why it happens: this one is a consequence of the first two rather than a separate extraction bug. If extraction gave you a wall of text with no headings, then your splitter has nothing to split on except length. So it counts to a thousand characters and cuts. That is the default in every tutorial, and it is the default because it needs no structure, not because it is good.

The cost is specific. A section has a topic, and every sentence in it is about that topic without necessarily naming it. Cut across the boundary and you produce a chunk containing the end of one topic and the start of another, which is a chunk that is strongly about nothing.

How to check: look at whether your chunks start at section boundaries. If chunk starts are distributed randomly through the text, you are splitting on length.

What fixes it: recovering the headings during extraction, then splitting on them. Headings are the only boundary in a document that carries meaning, because a human chose them to mark where one idea stops and the next begins. A conversion that keeps them hands your splitter something worth cutting on. A conversion that flattens them means no splitting strategy can recover.

There is a second benefit that people miss. Once you have headings you also have a heading path, so a chunk can carry “Annual Report, Risk Factors, Currency exposure” as part of its text. That is context the chunk itself never states and the embedding would otherwise never see.

4. The tables are flattened

What it looks like: a run of numbers with no columns.

Region Q1 Q2 Q3 North 412 388 501 South 208 244 190

Why it happens: a table is two dimensional and text is one dimensional. Reading it out in order destroys the relationship between a cell and its column header, which was the only thing making the cell mean anything.

Why it is the hardest of the four: the other three are solvable by better layout inference. This one needs the table’s structure to be reconstructed and then serialised into something that keeps rows and headers associated, and doing it reliably across merged cells, multi-line cells and tables that span pages is a genuinely open problem.

How to check: find a document with a table, convert it, and try to answer a question that requires reading across a row.

What fixes it, honestly: not us, yet. Our converter currently hands tables back as plain lines, and we would rather say so than let you find out from a wrong answer. Table extraction is the next tool we are building. In the meantime, if your corpus is mostly tables, use a heavier document toolkit that has focused on this, and expect to verify it rather than trust it.

What these four have in common

Every one of them fails without an exception, produces output that looks like text, and degrades retrieval in a way that shows up as “the system is a bit unreliable” rather than as a bug with a stack trace.

That is why they persist. Retrieval quality problems get attributed to the parts of the stack that are visible and tunable: the embedding model, the chunk size, the value of k, the reranker. Those are all downstream. If the text going in was wrong, none of them can help, and tuning them is how teams spend a month getting nowhere.

There is also a fifth failure that is not on this list because it is not a degradation, it is a wall: if the page is a photograph of paper, there is no text to extract at all. That one is easy to test for, and worth testing for first, because everything above assumes there is text to get wrong. How to tell whether your PDF is real text or a photo of text takes about three seconds.

What to do this week

1. Read twenty chunks. Actually read them. Everything above is visible on sight.
2. Convert one two column document and check the first three paragraphs for interleaving. This is the failure most likely to be silently poisoning your answers.
3. Grep your chunk store for the document title. Hundreds of hits means the furniture is in your content.
4. Check where your chunks start. Random offsets mean you are splitting on length and the structure never survived extraction.
5. Fix extraction before you touch retrieval. Everything downstream is built on the text, and no reranker recovers information that was destroyed before it was indexed.

If you want to see what a layout-aware conversion gives you on a document you are not free to upload anywhere, our PDF to Markdown converter runs entirely in the browser: the file never leaves your machine, which matters here because the documents worth putting into a retrieval system are usually the ones you cannot send to someone else’s server.

The short version

• The text in your index is probably not the text that was on the page, and nothing in your pipeline will tell you.
• Interleaved columns are the dangerous failure, because the output is fluent rather than obviously broken.
• Running headers and page numbers end up inside chunks and quietly move their embeddings.
• With no headings, your splitter has nothing to cut on but length, so it cuts through the middle of arguments.
• Tables lose the link between a cell and its header. Nobody has fully solved this, including us.
• Read your chunks. Five minutes, and it beats a week of parameter tuning.

Further reading

• RAG pipeline challenges, from data ingestion to retrieval, Unstructured. A vendor write-up, but an unusually candid one about how much of retrieval quality is decided before the embedding step.
• Mastering PDFs: extracting sections, headings, paragraphs and tables, LlamaIndex. Good on why the table problem is structurally harder than the others.
• ISO 32000-1, the PDF specification. Clause 9 is the text and font section. It is the primary source for the claim that none of the structure you can see is actually stored in the file.
• pdf.js, Mozilla’s renderer. Reading what its text extraction pass reports, and what it does not, is the quickest way to understand why these four failures exist rather than being someone’s bug.