The all-or-nothing framing of self-hosting is what makes it look like a bad trade. Most mature deployments run a local model for the bulk of requests and escalate a minority — and the interesting engineering is entirely in the gate that decides.
All AI Hosting & Infrastructure →
Request difficulty in a production system is not normally distributed; it has a long thin tail. Most requests are routine — a classification, a lookup, a summary of a short document, a rewrite. A small proportion are genuinely hard, and a smaller proportion still are hard in ways that need the best available model.
Sending the whole distribution to a frontier API means paying tail prices for the body. Sending it all to a local model means the tail fails, visibly, and users generalise from those failures to the whole system. Routing lets each part go where it belongs, and the cost curve that results looks nothing like either extreme.
The same logic applies to sensitivity, and this is the axis that matters more for a regulated client. Some requests touch material that must not leave the building; most do not. A gate that routes on sensitivity as well as difficulty means the confidential subset is structurally incapable of escalating, while everything else keeps access to the best model available.
For requests eligible for a hosted decision service, our Jev model-router design explores fast task assessment and verification before escalation. Jev itself must pass the same data-handling constraints as any other external provider.
from your application
sensitivity + difficulty
handles the majority
did it actually answer?
with route recorded
non-sensitive only, redacted
Escalation rate, first: the proportion of requests that leave the local model. As a working heuristic, if it climbs above roughly a fifth the local model is probably too small or the gate is too eager, and the economics of the whole arrangement erode quietly.
Then escalation precision, which is the number people skip. Of the requests that escalated, how many did the frontier model actually answer better? If the answer is not most of them, the gate is firing on the wrong signal and you are paying for nothing. This needs a periodic sample scored against both, which is a small standing cost with a large return.
And unnecessary local failures: requests the local model got wrong that did not escalate. These are the expensive ones, because they reach a user as a confidently wrong answer. They are found by sampling, not by monitoring, which is why the evaluation harness has to keep running after launch rather than being a launch gate.
On the escalated path, yes — you have paid for a local attempt before making a remote call. On the majority path it is faster, because the local model has no network round trip. Whether the average improves depends on your escalation rate, which is the number to watch.
Usually from metadata you already have rather than from the text: which system the document came from, which folder, which user, which classification label. Inferring sensitivity from content with a model is a fallback, not a foundation — it is probabilistic, and this gate should not be.
Yes, and sometimes it should: frontier first, with a local model absorbing repeated or cacheable requests. That suits workloads where quality dominates and volume is moderate. The routing machinery is the same; only the default changes.
It can be. If you have one model, low volume and no sensitivity constraint, route everything to it and revisit when one of those three changes. What we would still build on day one is the recorded-route field, because retrofitting attribution across a live system is far more work than adding it at the start.
We build the gate, instrument it, and show you the escalation rate before you commit to hardware.
30-minute technical call · no deck
What we built, what stage it is at, and a link to the live thing wherever there is one.
yourOS is an operating system with an AI at its heart. You talk to it like a person, out loud or typed. It makes your calls, plans your days and works on your projects with a crew of agents, on your own private server. Our own product, built from the first commit; the waitlist is open.
Website, backend and app assets for Sonora, a free AI sound-therapy app on iOS and Android published by Cognitive AI.
Brand, website, prototype, apps on five platforms, the AI backend and the algorithm-intelligence pipeline for Podswap, a social growth engine that runs without a human in the loop.
In closed beta: Roxy, a private AI assistant with permanent memory running on infrastructure the user owns. Brand, site, product, backend and conversational design.
Live at coherence.com: brand identity generated with AI agents we directed, the motion system and website for the group behind Sonora, AboutAI, WakeUp and MudShadows. Agent work remains in build.
Live at domainbooth.com for DomainBooth, a premium domain brokerage: a new animated mark, every page redesigned and rebuilt with designed mobile layouts, and a custom CMS for its sales record, featured names, press and enquiries.
Apps, platforms and campaigns for these teams, long before the AI work. Every one is written up on the work page — what we did, and where our part ended.
