Fine-tune or Prompt? Choosing the Right Tool

The single most important thing to understand is that prompting leaves the model unchanged. Every instruction, example, and constraint you place in a prompt operates entirely within the context window at inference time. The model's underlying parameters, its weights, are untouched, and when the conversation ends nothing is retained. Fine-tuning is the opposite: it adjusts those weights directly by running a training process on a curated dataset, and the model that emerges is a different artefact, its parameters shifted to favour certain patterns, formats, tones, or domain vocabularies. A useful analogy: prompting is briefing a colleague before each meeting, while fine-tuning is sending them on a specialist training programme. The briefing is flexible and cheap; the training programme produces durable change but takes time, costs money, and cannot be undone easily. Prompting can do a great deal. Few-shot prompting, providing a small number of examples within the prompt, chain-of-thought prompting, asking the model to reason step by step, and structured output formats can all dramatically improve output quality, and frontier models in 2026 are genuinely responsive to well-engineered prompts. But instructions alone cannot conjure knowledge the model does not have. If your model does not know your company's internal taxonomy, your proprietary product names, or a regulatory framework that post-dates its training cut-off, no amount of instruction will remedy that: the missing knowledge has to be supplied from somewhere. That is a knowledge gap, and it usually calls for retrieval-augmented generation, a technique that supplies relevant documents to the model at inference time rather than embedding knowledge in its weights; fine-tuning can help where domain vocabulary and framing are pervasive, but it is an unreliable way to teach a model new facts. It is equally important to understand what fine-tuning cannot do. It excels at shaping behaviour, style, format, tone, and domain-specific vocabulary, but it is a poor substitute for reasoning ability. Fine-tuning does not make a model smarter; it makes a model behave more consistently in a particular way, so if your problem is that the model reasons incorrectly, fine-tuning is unlikely to fix it.