Applied · Lesson 2

Debug the response, do not reroll it

You asked for 200 words. You got 800. You listed three required points and two are missing. The instinctive move is to hit regenerate.

Rerolling is sampling from the same distribution and hoping for a better draw. Sometimes you get one. You have learned nothing, the same failure will recur tomorrow, and if this request runs unattended you have shipped a defect you cannot reproduce.

Treat it as a bug. Bugs have causes.

The five common causes

Nearly every instruction-following failure traces to one of these. Learning to tell them apart is most of the skill.

1. Buried instruction. The requirement is in the middle of a long paragraph, after the interesting part. Content in the middle of long prompts gets less weight than content at the start or end. Symptom: one specific requirement ignored while others are honoured.

2. Competing instructions. You asked for "comprehensive" and "under 200 words". Both are instructions; they cannot both be satisfied; one wins. Symptom: the output looks like a reasonable compromise you didn't ask for.

3. Instruction contradicted by example. Your rule says 200 words, your example is 600. Demonstrated behaviour beats stated rules. Symptom: output matches the example's properties, not the stated rule's.

4. Unmeasurable requirement. "Concise", "professional", "thorough". Nothing was violated because nothing was specified. Symptom: you're dissatisfied but cannot point at a rule that was broken.

5. Wrong task shape. You asked for one thing that's really three, and it did the first one well. Symptom: partial output that is good as far as it goes.

Diagnosis on the worked example

800 words instead of 200, two of three points ignored. Working through the list:

Is it buried? Check where the three points appear. If they were in a paragraph after two paragraphs of background, that's cause 1 and it's the likeliest single explanation for "two of three ignored" specifically. The first point survived because it was closest to the surrounding structure.

Is it competing? "Product update" plus an implied audience of customers pulls toward marketing register and length; "200 words" pulls the other way. If the prompt also said anything like "make it compelling", that's cause 2 and it explains the fluff directly.

Is there a contradicting example? If you pasted last month's update as a template and it ran long, cause 3 explains the length on its own, and no amount of restating "200 words" will fix it while the example is still there.

Measurable? "200 words" is measurable. "Product update" is not. It's a genre, and the model picked the most common member of that genre. That's why you got marketing fluff rather than a status report.

The most likely story: the three points were buried, and "product update" was read as a marketing genre. Two causes, and they're independent.

Fix one thing at a time

This is the part people skip. If you rewrite the whole prompt and it works, you don't know which change fixed it, so you don't know what to do next time, and you may be carrying three superstitions forward.

Change the highest-probability cause first, alone. Here: pull the three required points out of the prose and put them at the end as an explicit numbered list. Nothing else. Re-run.

If the points are now covered but it's still 800 words, you've isolated a separate length problem and you know the burial theory was right. That's real information.

Make the requirement checkable, not just louder

The weak fix is to restate the rule with more emphasis: "IMPORTANT: you MUST cover all three points." This sometimes works and teaches you nothing about why it failed.

The strong fix changes the structure so compliance is visible:

Cover exactly these three points, each as its own paragraph with the point as a bold heading:

  1. Migration completed ahead of schedule
  2. Search latency down 40%
  3. Pricing changes effective 1 September

Then state the total word count on the final line.

Hard limit: 200 words excluding headings. Status-report register. Flat, factual, no adjectives that could appear in an advertisement.

Now the failure is visible without careful reading: three headings or not, word count stated or not. You can check it in two seconds, and so can a script.

Note "status-report register. No advertising adjectives" replacing "product update". That's a measurable constraint replacing a genre label. The fix for cause 4.

Escalation order

When you're not sure which cause it is, try in this order. Cheapest and most often correct first:

  1. Move the ignored requirement to the end of the prompt, alone.
  2. Convert it into a checkable form (count, list, explicit format).
  3. Look for and remove a contradicting example or competing instruction.
  4. Split the task into two requests.
  5. Only then reach for a longer, more elaborate prompt.

Most people start at step 5. Steps 1 and 2 fix the large majority of cases and take thirty seconds.

Where this doesn't help

Some failures are genuinely stochastic. You'll get a bad draw occasionally on a prompt that's fine. The tell is inconsistency: run the same prompt five times, and if four are good, you have variance, not a bug, and the fix is a validity check on the output rather than a prompt change.

If all five fail the same way, it's a bug, and the causes above apply.

The move

Before rerolling: name the cause from the list of five, make one change, re-run. If you can't name a cause, run the prompt five times first. You need to know whether you're debugging a defect or a coin flip.

Exercise

You asked for a 200 word product update and got 800 words of marketing fluff that ignored two of your three required points. Write the diagnosis. What most likely went wrong, what one change would you make first, and why that one first.

How this gets marked

  • 30%Identifies a specific likely cause in the request, not just that it ignored instructions.
  • 25%Proposes a single first change so the effect can be attributed.
  • 25%Says why that change should fix it, showing a model of the failure.
  • 20%Makes the requirement checkable so the same failure is visible next time.