Foundations · Lesson 2

Define done before you ask

Here is a test you can apply to any request before you send it: if two different people received this output, would they agree on whether it succeeded?

If not, you have not specified an output. You have expressed a hope, and you will find yourself re-reading results and feeling vaguely dissatisfied without being able to say why.

The naive attempt

Summarise these customer interviews.

What comes back is a summary. It might be six paragraphs of themes. It might be a bulleted list per interview. It might lead with methodology. Any of these is a defensible reading of your request, which is precisely the problem. You cannot call it wrong, and you also cannot use it.

Worse, you'll reroll. And the second output will be differently-shaped, so now you're comparing two things that aren't comparable, on a task where you never defined the target.

Four things that make an output checkable

Shape. Fields, sections, rows, or an explicit format. "A table with one row per interview and columns: participant, top pain point, quoted evidence, feature requested."

Size. A bound. "No more than 15 words per cell." "Exactly five themes." "Under 200 words total." Unbounded requests reliably drift long, because length correlates with apparent effort.

Purpose. What the output will be used to decide. This is the one people skip, and it does more work than the other three combined. "I'm using this to pick which two features to build next quarter" tells the model what to keep and what to drop far better than any list of instructions you could write.

Absence handling. What to do when the source doesn't contain what you asked for. Without this, models fill gaps. Plausibly, fluently, and wrongly. "If an interview doesn't mention pricing, write not mentioned. Do not infer."

That last one is the difference between a summary you can trust and one you have to re-verify against the source, which defeats the point of the summary.

The better version

Summarise these 12 customer interviews as a table, one row per interview.

Columns: participant role | top pain point (≤15 words) | direct quote supporting it | feature they asked for, if any.

I'm using this to choose which two features to build next quarter, so favour pain points tied to money or time lost over general dissatisfaction.

If an interview doesn't contain one of these, write "not mentioned". Never infer a quote. Every quote must appear verbatim in the source.

After the table, list any theme that appeared in 4+ interviews. Nothing else.

Now check it against the test: two people looking at the output would agree on whether it complied. Wrong column count is wrong. An invented quote is wrong. A concluding paragraph of reflections is wrong. You said "nothing else."

You have made failure visible, which is the entire goal. You cannot iterate on something you can't grade.

The sharpest version of this technique

Ask for a format a machine can validate. JSON with a fixed schema, CSV with fixed headers, a numbered list with a known count. Not because you need the structure downstream, but because you get to run the check automatically instead of eyeballing it.

If you're calling the API rather than chatting, structured outputs enforce this at the decoding level. The response is constrained to your schema and cannot come back malformed. That converts "usually the right shape" into "always the right shape", which matters enormously the moment anything runs unattended.

Where this doesn't help

Over-specifying kills genuinely open-ended work. If you're brainstorming, or you want to be surprised, or you don't yet know what good looks like, a rigid schema will get you exactly what you asked for and nothing better. Exploration and production are different modes: explore loosely, then specify tightly once you know the shape you want.

The failure mode to avoid is specifying tightly while still exploring. You lock in a bad structure before you've learned what structure you need.

The move

Before sending: write the sentence "I will know this worked if ___." If you can't finish it, you're not ready to send. Finish that sentence first, then put it in the prompt.

Exercise

Take a vague request, "summarise these customer interviews", and rewrite it so the result is checkable. Someone else should be able to look at the output and say yes or no without asking your opinion.

How this gets marked

  • 30%Says what shape the output takes. Fields, sections, rows, or a format.
  • 20%Bounds the size in words, items, or rows.
  • 30%Says what the summary will be used to decide, so the model knows what to keep.
  • 20%Says what to do when the source does not contain something. This is where summaries invent.