Verify what you cannot check by eye

The dangerous outputs are not the obviously wrong ones. You catch those. The dangerous ones are fluent, specific, well-structured, confident, and wrong in one place you had no way of noticing.

This is the failure mode that ends careers in a way that "the model wrote a bad email" never does. It deserves an actual procedure, not vigilance.

Why "read it carefully" fails

Fluent text suppresses scrutiny. Prose that flows well, cites specific figures, and uses the right jargon reads as authoritative, and your brain grants authoritative text the benefit of the doubt. That's not carelessness; it's how reading works. You cannot fix it by trying harder.

You also can't check what you don't know. If you're not an expert in the market you asked about, reading the analysis more slowly gains you nothing. You have no basis on which to disagree.

The procedure

Step 1. Split the output into claim types.

• Checkable-externally: figures, dates, names, citations, quotes, anything with a source that exists outside this conversation.
• Checkable-internally: arithmetic, logical consistency, whether the conclusion follows from the stated premises, whether it contradicts itself.
• Judgement: interpretations, recommendations, predictions. Not verifiable, only assessable, and only by someone competent.

Most people skip this split and verify uniformly, which means either verifying too little (checking the easy claims) or too much (fact-checking opinions).

Step 2. Verify checkable-externally claims against an independent source.

Independent means: not this model, not this conversation. A primary document, a search, a database, a person who knows.

Asking the model "are you sure?" is not verification. It's a coin flip with extra steps. It will often sound more confident, sometimes reverse a correct answer, and in neither case has any new information entered the system.

Step 3. Attack the internal consistency.

Do the numbers add up? Does the recommendation follow from the analysis, or does the analysis conclude X and the recommendation say Y? Are the figures in the table the same figures used in the prose? Contradictions are surprisingly common in long outputs and they're free to find. No external source needed.

Step 4. Set a rejection condition before you start.

Decide up front what kills it. "If any of the three market-share figures is unsourced or doesn't match the original report, I discard the whole analysis and start over." Deciding this in advance stops you negotiating with yourself once you're invested in the output and the deadline is close.

Step 5. Match effort to stakes.

A first draft of an internal memo does not need this. A board deck does. Slack message to a colleague, no. Anything that goes to a customer, a regulator, or production, yes. Verifying everything to the same depth is a good way to guarantee you stop verifying anything.

Techniques that actually work

Demand citations, then check them. Ask for a source alongside every factual claim. This helps twice: it makes claims traceable, and unsourceable claims tend not to get made. But check them. A citation that exists is not a citation that says what the output claims it says.

Ask for the answer to be derivable. "Show the calculation" or "quote the exact sentence from the document that supports this." Extraction is dramatically more reliable than recall, and a quote is checkable in seconds with ctrl-F.

Ask a fresh instance to attack it. Open a new conversation, paste the output with no framing, and say "find the three weakest claims in this and explain what would need to be true for each to be wrong." A fresh context has no investment in defending it. This catches internal problems well; it does not catch external factual errors, since it has no more access to reality than the first one did.

Check the thing that would be embarrassing. In practice, one or two claims carry all the risk. The headline number, the legal assertion, the name you'll say out loud. Verify those to the ground even if you spot-check the rest.

Where this doesn't help

Verification cannot rescue a task that was a bad fit. If you asked for something requiring information the model doesn't have and can't access, you'll spend your verification budget discovering that every figure is invented. Notice that early, and change the approach rather than checking harder.

And a procedure you don't run is worth nothing. If this feels too heavy to do every time, that's correct. It isn't meant for every time. Decide which of your recurring tasks are high-stakes, and run it there, always.

The move

For your next high stakes output, before you read it: write down the one claim that would be most damaging if wrong. Verify that one against a source outside the model. Do this especially when the output sounds certain.