Iterative Prompt Refinement: The Loop
Prompting is not 'set and forget'. Learn the iterative loop of testing, analyzing failure cases, and refining your instructions (Chain of Thought).
Iterative Prompt Refinement
Nobody writes perfect code on the first try. Nobody writes a perfect prompt on the first try either.
The Loop
- Draft: Write your best guess prompt.
- Test: Run it against 10 diverse inputs.
- Analyze: Look at the 2 cases where it failed.
- Refine: Add a constraint or example to fix just those failures without breaking the successes.
- Repeat.
Chain of Thought (CoT) prompting
One of the best refinement techniques for logic failures is Chain of Thought.
- Standard: "Which is larger, a toaster or a shoebox?" -> "Shoebox." (Maybe wrong).
- CoT: "Which is larger? Think step by step."
- Model: "A standard toaster is about 10x8x6 inches. A shoebox is about 12x8x4 inches. Volume of toaster = 480. Volume of Shoebox = 384. Therefore, the Toaster is larger."
By forcing the reasoning steps, you allow the model to error-correct itself.
Negative Constraints
Sometimes it is easier to say what not to do.
- Refinement: "Do not use flowery language. Do not apologize. Do not say 'As an AI'."
Summary
Prompt Engineering is debugging. Treat bad outputs as bugs in your prompt "code." Trace why the model got confused and patch it with an instruction or example.
In the final lesson of this module, we look at Prompt Templates.