Model families

Llama on
your own hardware.

Llama is the default answer to “we need to run this ourselves” — not because it always wins on capability, but because it has the deepest tooling ecosystem of any open-weight family, and on a self-hosted deployment the tooling is most of the work.

SPECIFICATION

Provider: Meta, United States
Weights: Published and downloadable
Licence: Meta's community licence, not OSI open source. Commercial use is permitted with conditions, including an attribution requirement and a threshold on monthly active users above which separate terms apply. Read it; it is short, and it is not Apache-2.0. As of the version current at the time of writing; confirm against the provider's current terms. Not legal advice.
Sizes: A range from small models that run on a single accelerator to large ones needing a multi-GPU node
Ecosystem: The broadest of any open family — vLLM, Ollama, llama.cpp, quantisation formats, fine-tuning toolchains, cloud model catalogues
Where we use it: On-premise and private-cloud deployments, and as the baseline any other open model has to beat

Provider-published specifications as of 1 September 2026, not figures we measured. Model families move quickly — check the provider's own documentation before committing to a number.

WHY THE ECOSYSTEM IS THE ARGUMENT

On a hosted API the model is the product. On a self-hosted deployment the model is one component of a system that also has to do batching, KV caching, quantisation, tensor parallelism, health checks, GPU scheduling, upgrades and rollback. Llama's advantage is that every one of those has a well-trodden path, several implementations and a large body of people who have hit the failure mode before you.

That matters more than a benchmark point. A model that scores marginally higher but has one immature serving implementation will cost you more in engineering time than the difference is worth, and it will cost you more again the first time you need to debug throughput at two in the morning. When we recommend against Llama it is usually because a specific capability requirement points elsewhere, not because the deployment story is better anywhere else.

The licence deserves ten minutes. It permits commercial use and it is not restrictive for the overwhelming majority of deployments, but it is a bespoke licence rather than a standard one, it carries an attribution obligation, and it has a scale threshold. If you are building a product on top of it, have someone read it who is qualified to. If a permissive standard licence is a hard requirement — and for some procurement processes it is — Mistral publishes Apache-2.0 options.

THE PRACTICAL LEVERS

Pick the size from the task, not from the headline. The mid sizes handle the overwhelming majority of production work, and the difference in hardware between a mid-size model and the largest is the difference between one node and several. Test the smaller one first; the result is frequently that it was enough.

Quantisation is the cheapest capacity you will buy. Running at eight or four bits cuts the memory requirement proportionally, and on many tasks the quality difference does not register on a real evaluation set. It has to be measured rather than assumed, and measuring it takes an afternoon.

Serving configuration outweighs model choice. Continuous batching, paged attention and prefix sharing decide how many concurrent users a node supports, and the gap between a naive loop and a properly configured vLLM deployment is a multiple rather than a percentage. See inference optimisation.

Distil where the task is narrow. Using a large model's outputs to train a small one converts capability you are renting into capability you own. Check the licence permits it for the specific model you deploy.

WHERE WE WOULD CHOOSE SOMETHING ELSE

Where a standard permissive licence is a procurement requirement. Llama's community licence permits commercial use but is bespoke, carries attribution obligations and has a scale threshold. Mistral publishes Apache-2.0 options and typically clears a standard open-source policy far faster.

Where reasoning-heavy work dominates and inference economics matter, DeepSeek's mixture-of-experts design frequently wins on throughput per unit of compute. Where the deployment has to span an edge device and a server with one tokeniser, Qwen's size ladder is the better fit.

And where nobody wants to operate inference infrastructure at all, a hosted API is cheaper once you price the people honestly. Self-hosting is a standing commitment, and we would rather say that in the first meeting than in the retrospective.

QUESTIONS

QUESTIONS — 3

Not by the OSI definition, and the distinction is worth keeping straight because procurement sometimes cares. The weights are published and commercial use is permitted, which is what most people mean when they say open. But the licence carries conditions a standard open-source licence does not, including attribution and a user-count threshold. “Open weights” is the accurate term.

Dominated by GPU capacity and the people who look after it, not by anything per-token. That flips the economics against hosted APIs only above a certain sustained volume — below it, you are paying for idle hardware. We work the crossover out explicitly rather than assuming it; see GPU sizing and cost.

DeepSeek tends to lead on reasoning-heavy work and has favourable inference economics from its mixture-of-experts design. Llama has the deeper tooling ecosystem and a US-domiciled provider, which some organisations' governance policies prefer. Both are credible; we benchmark both when the decision is open.

Keep reading

Inference Optimisation

→

30-minute technical call · no deck