Prompting & Context

We've spent eight posts inside the machine. Now we cross over to the side most people actually touch: how you talk to a model. The prompt is the only lever a user has over a frozen set of weights — and it's a far more powerful lever than it first appears.

The big idea of this post:

The prompt isn't just a question. It's a short program you write in plain language, and the model's context window is its working memory.

In-context learning: teaching without training

Here's something genuinely strange about LLMs. You can teach them a brand-new task at the moment you ask, just by showing examples in the prompt — no retraining, no weight changes. This is in-context learning, and it was one of the most surprising discoveries about large models.

   Translate to French:
       sea otter   →  loutre de mer
       cheese      →  fromage
       hello       →  ?

   The model answers "bonjour" — it inferred the task purely from the examples.

Nothing about the model changed. It recognized the pattern of the prompt and continued it (post #2's next-token prediction, doing exactly its job). This is why showing a couple of examples — few-shot prompting — is often the single highest-leverage thing you can do to improve output quality and lock in a format.

Why does this work at all? During pretraining the model saw countless instances of "pattern established, then continued." Following an in-prompt pattern is just more of the same skill, now pointed at your task.

The anatomy of a good prompt

Most production prompts have recognizable parts, even if they blur together:

You don't always need every section, but the failures usually come from a missing one: vague rules, no examples, or an unspecified output format that leaves the model guessing.

A handful of techniques carry most of the weight:

• Be specific about the output. "Return JSON with keys title and summary" beats "summarize this" by a mile. Models are eager to please but bad at reading your mind.
• Show, don't just tell. One good example often does more than a paragraph of instructions, because it pins down format, tone, and edge cases at once.
• Give the model a role. "You are an expert epidemiologist" genuinely shifts the style and care of the response, because it conditions the model toward the relevant slice of what it learned.
• State constraints positively. "Respond in under 100 words" works better than a pile of "don't" rules. Models follow targets better than prohibitions.

"Think step by step": reasoning in the open

One technique deserves special mention because it changed how people use these models. If you ask a model to reason through a problem out loud before answering — chain-of-thought prompting — accuracy on hard problems jumps.

   Weak:    "What's 17 × 24?"                  → model blurts a guess, often wrong

   Strong:  "What's 17 × 24? Think step by step."
            → "17 × 24 = 17 × 20 + 17 × 4
                       = 340 + 68
                       = 408"                   → far more reliable

Why does this help? Remember from post #6 that the model commits to one token at a time and can't go back. Forcing it to write out intermediate steps gives it more "room to compute" — each step it writes becomes context that supports the next. Reasoning silently, it has to leap to the answer in one shot; reasoning out loud, it can build up to it.

Modern "reasoning models" bake this in: they're trained (post #5) to generate a long internal chain of thought before their final answer, often hidden from you. But the principle is the same one you can invoke yourself in any model.

Context engineering: managing the working memory

Here's where this post connects back to the machine. The context window (posts #3, #6) is the model's entire working memory. It has no other state. So a deeper discipline has grown up around prompting: context engineering — deciding what goes into that limited window, and in what order.

The hard truths of context engineering:

• More context isn't always better. Stuffing the window with marginally relevant material can hurt — the model gets distracted, and the signal you care about gets diluted. Curate, don't dump.
• Position matters. Models tend to pay most attention to the beginning and end of a long context and can overlook things buried in the middle (the "lost in the middle" effect). Put the most important material where it'll be seen.
• Stable content first. Putting the unchanging parts of your prompt (system instructions, fixed examples) at the front lets the inference engine's prefix cache (post #7) reuse them — saving latency and money on every call.
• Tokens are budget. Every token of prompt is a token you pay for and a token of room you don't have for the answer (post #3). Concise, well-organized context beats sprawling context.

This is the bridge to everything that follows. When we get to agents (post #12), their single hardest engineering problem will turn out to be exactly this: managing what's in the context window as a task drags on across many steps and the relevant information piles up far past what fits.

A note on prompt injection

The takeaway

A prompt is a program written in language, and in-context learning lets you teach new tasks on the fly just by showing examples. Be specific, show examples, assign a role, and let the model reason out loud for hard problems. Underneath it all is context engineering: the model's only memory is its context window, so curating what goes in it — and where — is the real skill.

So far the model only ever produces text. To make it genuinely useful, we wrap it in code: a loop, a set of rules, and machinery to parse and act on its output. That wrapper is the harness.