Resource

What an AI system costs.
The lines left out.

Initial AI cost estimates are usually wrong in the same direction and for the same reasons. Here is the full list, so yours can be wrong for more interesting ones.

THE OMITTED LINES

Retries and failures

Requests that error, time out, fail validation or get retried are billed and never reach a user. Depending on how strict your output validation is, this can be a meaningful fraction of total spend and it appears nowhere in a naive estimate.

Development and evaluation traffic

Building an evaluation set and running it repeatedly consumes real tokens, and running it continuously after launch — which you should — is a standing cost. Teams routinely forget the eval loop entirely.

Conversational growth

Each turn re-sends the history, so cost per conversation grows faster than linearly with length. An estimate built from single-request cost understates a chat product substantially.

The people

Someone maintains the prompts, watches quality, handles provider changes, reviews escalations and owns the incident when it misbehaves. This is the largest omitted line by a wide margin, and it does not disappear on either deployment model.

Reindexing

Embedding a corpus is a one-off; keeping it current is not, and changing embedding model means doing the whole thing again. Budget for at least one full reindex.

Migration on deprecation

Hosted models get retired. Each migration is an evaluation run, a prompt review, and a period of reduced confidence. Assume roughly one per year on a hosted stack, and none on a self-hosted one.

BUILDING THE MODEL

Start from a unit of business meaning rather than from tokens. Cost per resolved ticket, per document processed, per user per month. That number can be compared against the value of the thing it replaces, and it survives a change of model or provider in a way that a token estimate does not.

Model three volumes: the pilot, the realistic first year, and the scenario where it works better than expected. The third is the one that matters, because AI systems that deliver get used more, and a cost model that only works at pilot volume is a problem waiting for success to trigger it.

Put the sensitivities in explicitly. Which assumption, if wrong, changes the answer? Usually it is request volume or the proportion needing an expensive model, and stating that lets a finance function interrogate the model properly instead of accepting or rejecting a single number.

Then compare deployment models on the same footing. A hosted estimate that omits migration and governance overhead against a self-hosted estimate that omits operator time is not a comparison. The business case page sets out both sides in full.

A WORKED MONTHLY COST

The lines above are easier to take seriously with figures against them, so here is a retrieval assistant over an internal document set, costed for one month. Every figure is an illustrative assumption — the shape is what transfers, not the numbers.

	Monthly	Why it is usually missed
Answer generation	€310	This is the line everyone estimates, and it is rarely the largest
Retrieval and reranking	€95	Every query hits the index before it hits the model
Failed and retried calls	€40	Timeouts and malformed output still bill
Evaluation traffic	€120	The regression set runs on every change, and it is not free
Reindexing as documents change	€85	Embedding is a running cost, not a one-off
Logging, tracing and storage	€60	Traces are the thing that makes failures diagnosable
Engineer time to keep it honest	€960	Two days a month. Almost always the biggest line

Illustrative monthly cost for a retrieval assistant. These are assumptions for shape, not measured client outcomes.

The total is about €1,670 a month, of which the model call — the line most estimates consist entirely of — is under a fifth. The human line is more than half. That ratio is the single most useful thing on this page: a cost model that omits the engineer is not out by a rounding error, it is out by a factor of two, and it is the reason projects that looked cheap at pilot become arguments at renewal.

Two things move this materially. Routing routine queries to a smaller model changes the first line by an order of magnitude on the affected traffic and leaves the others alone. Scale changes it the other way: at ten times the query volume the first three lines grow, evaluation and engineering barely do, and the cost per answer falls sharply. Model both ends before committing to either.

QUESTIONS

QUESTIONS — 3

Routing more requests to a smaller model. It is typically an order-of-magnitude difference on the affected traffic and, when the routing is measured rather than guessed, frequently costs nothing in quality. Context hygiene — not sending tokens nobody needed — is a close second.

Within a reasonable margin on per-request cost, if you tokenise real content rather than estimating from word counts. Volume is the uncertain term, which is why it belongs in the sensitivity analysis rather than buried as a point assumption.

Yes, at several levels — per user, per tenant and globally, with alerting well before the cap. A runaway loop in an agentic system can generate a startling bill quickly, and the cap is a cheaper control than the retrospective.

Keep reading

Tokens & Context Windows

→

30-minute technical call · no deck