Technology

Fine-tuning.
When, and when not.

Fine-tuning is the most frequently proposed and least frequently correct answer to a model quality problem. It teaches behaviour, not facts — and most quality complaints are about facts.

THE DISTINCTION THAT DECIDES IT

Fine-tuning adjusts a model's weights so it behaves differently: a consistent format, a house tone, a domain idiom, a task convention. Retrieval puts information in front of a model at inference time so it can answer from current, citable sources. These solve different problems and are constantly confused.

If the complaint is “it does not know our products”, that is retrieval. Fine-tuning facts into weights makes them uncitable, un-updatable and prone to confident staleness — you cannot correct a fact in a weight without retraining, and you cannot show a user where it came from.

If the complaint is “it will not stay in our format” or “it does not write like us” or “it keeps hedging when our users need directness”, and prompting has genuinely been exhausted, that is fine-tuning. It is also frequently a cost play: a small fine-tuned model matching a large general one on a narrow task at a fraction of the price.

THE THREE APPROACHES

LoRA and adapters

Train a small set of additional parameters rather than the whole model. Cheap, fast, and swappable — you can keep several adapters for one base model and load per task. This is the default and covers most legitimate cases.

Full fine-tuning

Update all weights. Expensive, needs substantial data, and risks degrading general capability. Justified for a genuinely distinct domain with a lot of data — and rarely justified otherwise.

Distillation

Use a large model's outputs to train a small one on your specific task. Frequently the highest-value option: it converts capability you are renting into capability you own, at a fraction of the serving cost. Check the licence permits it: most hosted-API terms prohibit training on their outputs, so the lawful path is usually from an open-weight model whose licence explicitly allows it — several do.

THE COST NOBODY BUDGETS

A fine-tuned model is a maintained asset. The base model gets deprecated or superseded and yours does not automatically follow. Your data distribution shifts and the tuning goes stale. You need an evaluation set to know when either has happened, and a pipeline to redo it when they have.

That is a standing commitment, and it is the reason we push teams to exhaust prompting and retrieval first — not because fine-tuning is hard, but because a fine-tune is something you now own for the life of the system. On a self-hosted deployment the calculation shifts favourably, since you control the base model and nobody can deprecate it out from under you.

The data requirement is also usually underestimated. Hundreds of high-quality, consistent examples is a realistic floor for behavioural tuning; a few dozen inconsistent ones will teach the model your inconsistency faithfully.

QUESTIONS

QUESTIONS — 3

Not reliably, and this is a common and expensive misunderstanding. You can tune a model toward hedging or toward refusing when uncertain, which changes the style of failure. Making outputs verifiable is a retrieval and architecture problem — the model needs the information and the obligation to cite it.

For behavioural tuning with LoRA, hundreds of consistent examples is a realistic floor and thousands is comfortable. Consistency matters more than volume: five hundred examples that agree with each other beat five thousand that do not.

Open, in most cases, because you keep the artefact. A fine-tune on a hosted platform is tied to that provider and to a base version they control. If the point of tuning was to own a capability, tuning it inside somebody else's platform undercuts the reason for doing it.

Keep reading

Small Language Models

→

Technology

Evaluating AI Systems

→

30-minute technical call · no deck