Working with documents too long to trust

Modern models accept enormous inputs. A 200-page contract fits comfortably. This creates a trap: because it fits, people assume pasting it is sufficient, and the failure that follows is quiet.

The naive attempt

[pastes 200-page contract]

What are our termination obligations?

You get four clauses, correctly quoted, well-summarised. They are real. They are also, quite possibly, four out of seven, and nothing in the response indicates that three are missing. This is the specific danger of long inputs: the failure mode is incompleteness, and incompleteness is invisible. A wrong answer looks wrong. An incomplete answer looks like a complete answer.

Why this happens

Asking one broad question over a very long input asks the model to do retrieval and synthesis in a single pass. It surfaces what's most salient and stops when it has assembled a satisfying answer. Nothing in the request said "keep going until you have found every instance," and nothing in the output format forced it to account for its coverage.

Recall over long inputs is good but not perfect, and it degrades for information that's phrased unusually, buried mid-document, or in a section whose heading doesn't suggest relevance. A termination obligation living inside a schedule titled "Data Handling" is exactly the one you'll miss.

Four techniques, in order of when to reach for them

1. Ask for extraction before synthesis.

Split the task in two. First: "list every clause that mentions termination, notice periods, wind-down, or obligations arising on exit. Quote each verbatim with its section number, and do not summarise or interpret." Then, in a second pass over that much shorter list: "which of these create obligations on us?"

Extraction is far more reliable than recall-plus-reasoning in one step, and the intermediate list is short enough for you to check by hand.

2. Force coverage accounting.

Make the model report its own completeness in a way you can inspect: "Work section by section. For each section, state its number and either the relevant clauses or 'none'. Do not skip sections." Now a gap is visible. A missing section number is obvious, where a missing clause was not.

This costs more tokens and takes longer. It is the correct trade for anything where a miss is expensive.

3. Chunk deliberately, along the document's own structure.

Split by article or section, not by arbitrary token count. Arbitrary splits cut sentences and definitions in half and destroy the cross-references that legal documents lean on. Then run the extraction pass per chunk and concatenate.

Include a stable header in every chunk. The defined-terms section, the parties, the document title, so each pass has the context it needs to interpret what it's reading. A clause saying "the Provider shall" is uninterpretable without knowing who the Provider is.

4. Retrieval, when the corpus outgrows the context.

Once you have hundreds of documents rather than one, stop pasting and start retrieving: index the corpus, pull the relevant passages, answer over those. Retrieval introduces its own failure. You can only answer from what got retrieved, so a bad query silently produces a confident answer over the wrong passages. Retrieval trades one invisible-incompleteness problem for a different one, so keep the citation discipline below either way.

Non-negotiable: citations back to the source

Every finding must carry a locator. Section number, page, or a verbatim quote long enough to search for. This isn't bureaucracy. It's what makes the output checkable in seconds instead of requiring you to re-read the contract, and requiring a quote strongly suppresses the confident-but-absent-from-the-document failure.

Then actually spot-check three of them. A locator you never check is decoration.

Prompt caching, if you're doing this repeatedly

If you're asking many questions against the same long document via the API, cache the document prefix. The document is identical every time, so it can be processed once and reused across requests. Dramatically cheaper and faster.

The rule that governs this: caching is a prefix match. Put the stable content first (the document), and the varying content last (your question). Interpolate a timestamp or a question ID above the document and you invalidate the cache on every call and won't notice, because everything still works. Just slowly and expensively.

Where this doesn't help

None of this rescues a document that genuinely doesn't contain the answer. If the termination terms live in a side letter you weren't given, exhaustive extraction over the main agreement returns a confident, well-cited, complete answer to the wrong question.

Before you start, ask whether you have all the documents. That question is not answerable from inside the document.

The move

For any long-input task where a miss matters: split it into extraction then synthesis, demand a locator per finding, force per-section coverage accounting, and spot-check three citations. And on anything legal or regulatory, a human who is qualified reviews it. This makes their review faster and better targeted, it does not replace it.