Prompt Engineering

Prompt engineering is the deliberate design of the inputs given to a language model so that it produces accurate, relevant, and well-structured outputs. Because it works entirely through the prompt — the instructions, context, and examples you supply — it changes the model's behaviour without modifying any internal parameters. It is the cheapest and fastest lever for improving results, and the first one a self-hoster should reach for before considering fine-tuning.

Core techniques

Several patterns recur. Zero-shot prompting gives the model only instructions and context, relying on its pretrained knowledge with no examples. Few-shot prompting includes a handful of input-output examples (typically three to five) to demonstrate the desired task and format. Chain-of-thought prompting asks the model to reason step by step, improving accuracy on multi-step problems. Clear, precise, unambiguous instructions consistently outperform vague ones.

Why it matters

For sovereign, locally-run AI, prompt engineering is often the difference between an unreliable toy and a dependable tool. Good prompts let a modest open-weight model punch well above its size on focused tasks — summarising a spec sheet, extracting an error code, drafting a diagnostic checklist — all on hardware you control.

Prompt engineering pairs naturally with a strong system prompt to set persistent behaviour, and with retrieval-augmented generation when answers need to be grounded in your own trusted data.

Structure That Actually Works

Reliable prompts read like specifications, not conversations. The recurring winning patterns: separate instructions from data with clear delimiters (XML-style tags or triple backticks) so the model never confuses your content with your commands; state the output format explicitly — down to showing the exact JSON skeleton or table you expect; put role and standing rules in the system prompt and per-task material in the user turn; and prefer positive instructions (“answer in two sentences”) over negative ones (“don't be verbose”), which models follow less reliably. When output must be machine-parsed, showing one complete worked example of the format outperforms paragraphs of description.

Failure Modes to Engineer Against

Prompts fail in characteristic ways. Instructions buried mid-prompt get overridden by later text — order and placement matter, with the start and end of the context window carrying the most weight. Ambiguity gets resolved silently instead of flagged, so a prompt should tell the model what to do when the input is malformed or the answer is unknown (“say so” beats a confident guess). And any prompt that processes untrusted content — web pages, emails, user uploads — is exposed to prompt injection, where the content itself carries instructions the model may obey. Treat retrieved text as data to be quoted, never as commands, and say so explicitly in the prompt.

Prompting Small Local Models

Techniques that are optional with frontier models are mandatory with the 7–14B-parameter class a home GPU serves. Small models need shorter, more literal instructions, benefit disproportionately from few-shot examples, and drift more over long contexts — so keep prompts tight and repeat critical constraints near the end. Local runtimes add a lever cloud APIs rarely expose: grammar-constrained generation, which forces output to match a schema at the sampler level, making even a small model's JSON reliable. Iterating costs nothing when inference is free on your own hardware, so build a small test set of representative inputs and treat prompt changes like code changes: run the suite, compare, keep what measurably wins. Version your prompts alongside your configuration, too — six months later, “which wording produced the good output” is a question you want answered by a file, not a memory.

Estimate token usage in the inference cost calculator.