Prompting as an engineering discipline

Most people treat a prompt like a sticky note. Write it once, use it, forget about it. That works fine until the prompt is running a thousand times a day, and one day it quietly gets worse, and nobody notices for a week because nothing threw an error. The output just got a little less useful, a little at a time.

That's the difference this post is about. A prompt you type into a chat window once is a question. A prompt baked into a system that runs unattended is a piece of code, and it deserves the same discipline: written on purpose, tested before it ships, and versioned so you can tell what changed when something breaks.

Zero-shot vs few-shot

Zero-shot means you just ask, no examples, only instructions. It works for simple, well-defined tasks. The moment the output needs a specific shape, a specific tone, or a specific edge case handled a particular way, zero-shot starts drifting.

Few-shot fixes that by showing the model two or three real input-output pairs before asking it to do the actual task. This is usually the single highest-leverage change you can make to a prompt. Two good examples beat five paragraphs of instructions, because the model is pattern-matching on the shape of your examples, not parsing your prose for rules.

Chain-of-thought, and when it actually helps

Chain-of-thought means asking the model to reason step by step before it gives a final answer, instead of jumping straight to the answer. On anything with multiple steps, like a math problem, a multi-part decision, or a plan with dependencies, this measurably improves accuracy. The model isn't smarter for it. It's just spending more of its own output tokens working the problem, the same way writing out a proof beats trying to do it in your head.

On simple, single-step questions, chain-of-thought mostly just adds tokens and latency for no gain. It's not a default you turn on everywhere. It's a lever for problems that actually have steps.

Structured output, so your code doesn't have to guess

If something downstream is going to parse the model's answer, plain prose is the wrong output format. Structured output means constraining the response to a schema, usually JSON, so your code gets a predictable shape back instead of a paragraph it has to scrape for the answer.

This matters more than it sounds like it should. A model asked to "return the price" might answer "The price is ₹499" one time and "₹499 is the price" the next. A model constrained to {"price": 499} returns the same shape every time, and your code can trust it without writing a parser that guesses at sentence structure.

Treat the prompt like code

Once a prompt is running in production, three things follow from that:

• Keep it in version control, not buried in a string inside a function somewhere
• Test it against a fixed set of inputs before changing it, the same set every time, so you can tell if a change actually helped
• Write down why a specific instruction is there. A year from now, you or someone else will want to delete the weird-looking line that says "never mention competitors," and if nobody remembers why it's there, someone will delete it and the bug it was preventing will come back

None of this is exotic. It's the same discipline you'd apply to any code that runs unattended. The only reason it feels different is that a prompt is plain English, so it doesn't look like code. It behaves like code the moment it's load-bearing.

What prompting can't fix

Here's the wall you hit eventually. A prompt can only work with what's inside the model's context window, whatever text you actually sent it. If the answer depends on information the model was never trained on and you never included in the prompt, no amount of instruction-tuning gets you there. You can ask as clearly as you want. The model will still answer confidently and still be wrong, because it's filling the gap with something plausible instead of something true.

That's the actual problem RAG exists to solve, and it's next.