Comparison Reviewed September 2026

On-premise or cloud?

Framed as a binary this is a bad question, because most mature deployments end up using both. Framed as “which requests go where”, it has a clean answer that falls out of four constraints.

SIDE BY SIDE

	Cloud API	On-premise
Best available model capability	✓	—
Data stays inside your perimeter	—	✓
Time to a working prototype	Days	Weeks
Cost at low volume	Lower	Higher
Cost at high sustained volume	Higher, and linear	Lower, and flat
Marginal cost of one more request	Per token	Near zero
Survives vendor deprecation	—	✓
Works without connectivity	—	✓
Requires people to operate	—	✓
New sub-processor for your customers	Usually yes	No

Structural properties, not vendor claims. Cost positions depend on volume, which is why the crossover is arithmetic rather than a preference.

THE FOUR CONSTRAINTS THAT DECIDE IT

Sensitivity. If the data cannot leave, the decision is made and nothing else in the comparison matters. This is the constraint that most often settles it, and it settles it absolutely.

Volume. Per-token pricing is linear forever; hardware is a fixed cost. Above a sustained threshold, ownership wins on cost and keeps winning. Below it, hardware sits idle and the API is cheaper. The threshold is calculable from your numbers.

Capability ceiling. Some workloads genuinely need the frontier. Most do not, and the honest way to find out is to test rather than to assume — teams are often surprised by how much moves down.

Operational capacity. The one that quietly decides more of these than the other three combined. Self-hosting is a standing commitment. If there is no team and no budget for managed operation, choose the cloud and revisit when that changes.

WHY IT IS USUALLY BOTH

Request difficulty and sensitivity both have long tails. Most requests are routine and non-sensitive; a minority are one or the other. Routing on those two axes lets the bulk run locally at near-zero marginal cost while a small, non-sensitive remainder escalates to a frontier API — and confidential work is structurally prevented from escalating at all.

That arrangement gets you most of the cost profile of self-hosting and most of the capability ceiling of the cloud, at the price of building and maintaining a routing gate. For most organisations at meaningful scale it is, in our view, the right answer, and the reason the either-or framing is worth abandoning early.

QUESTIONS

QUESTIONS — 3

Consuming a frontier model inside your own cloud tenancy and region resolves data residency and is often the fastest path through procurement, because the agreement already exists. It does not give you fixed costs, hardware control or independence from deprecation. It is a genuine middle option on one axis only, and it is worth being precise about which.

Take projected monthly tokens at the API rate you would actually pay, against hardware amortised over three years plus power plus a realistic fraction of an engineer. The last term is the one usually omitted and frequently the one that decides it.

Yes, and it is often the right sequence — provided the system is built for it: model behind an interface, no provider-specific dependencies, prompts and evaluation sets portable. Retrofitting that onto a system welded to one provider is where the cost actually lands.

Keep reading

Infrastructure

Deployment Options Compared

30-minute technical call · no deck