The word covers three quite different things: filtering what goes in, checking what comes out, and constraining what the system is able to do. Only the third is deterministic, and it is the one teams reach for last.
Ask three teams what guardrails they have and you get three unrelated answers: a paragraph in the system prompt, a toxicity classifier, a human approval step before anything is sent. All three are guardrails. Only one of them survives a determined adversary, and it is not the paragraph.
It helps to drop the word and describe the control precisely: what it inspects, what it does when it fires, whether it fails open or closed, and who finds out. A control described that way can be tested and audited. “We have guardrails” cannot, which is why it appears so often in vendor material and so rarely in an architecture review.
The underlying position on this page is simple. Content controls reduce the rate of bad outputs; they do not bound the damage of one. Only the action layer bounds damage, because it is the only layer where the answer to “what is the worst this system can do?” is enforced by something other than a model's judgement.
user, document, tool result
policy · PII · heuristics
reasoning + tool proposal
grounding · schema · leaks
typed refusal or a human
scope · confirm · caps
logged with its arguments
Everything above is probabilistic. The action layer is not, and that asymmetry should decide where the effort goes. A content filter reduces how often the system says something it should not. A permission boundary determines what the system can do when every content filter has already been defeated — which is the case you are actually designing for.
Least privilege on tools. Read-only by default. A separate, narrower credential per tool rather than one service account with broad rights. Scopes that match the task and not the convenience of whoever wired it up. An agent that can only read cannot be talked into writing, no matter what appears in a retrieved document.
Act as the user, not as the service. Authorisation should be evaluated against the permissions of the person whose request triggered the action, so an agent working on behalf of a junior employee cannot reach what that employee cannot reach. This is the single control that most reliably contains an injection, and it is architectural work rather than model work.
Confirmation for consequential actions. Not for everything, which trains people to click through; for the small set that is irreversible or externally visible. Sending an email, moving money, deleting a record, publishing anything. The confirmation must show the actual arguments — a dialogue that says “approve this action?” without showing the payload is theatre.
Caps, budgets and blast radius. A maximum number of tool calls per conversation, a spend ceiling, a limit on how many records a single run may touch, a rate limit on outbound messages. These bound the worst case arithmetically, which is a different kind of assurance from a classifier's accuracy figure, and they are what makes an incident recoverable rather than large.
Reversibility and a real audit trail. Prefer designs where an action can be undone: a draft rather than a send, a proposed change rather than an applied one, a soft delete. And log the arguments, the identity, the retrieved context and the decision path — not only the outcome. Reconstructing why a system did something is the question every post-incident review asks, and a log of outputs alone cannot answer it.
| Criterion | In the prompt | In a classifier | In the architecture |
|---|---|---|---|
| Holds against a determined adversary | No | Sometimes | Yes, within its scope |
| Deterministic | No | No | Yes |
| Added latency | Tokens only | A model call, or a parallel one | Negligible |
| Added cost per request | Low | Real, and per request | Near zero |
| Testable | Against an eval set | Precision and recall on labelled data | As ordinary code |
| Evidence for an auditor | The prompt text | A score distribution | A permission model and a log |
| Failure mode | Silently ignored | False positives and negatives | Blocks legitimate work |
| What it is genuinely good at | Tone, format, default behaviour | Volume triage and routing | Bounding the worst case |
This is the point most guardrail discussions skip. A safety classifier has a false positive rate and a false negative rate, it can itself be manipulated by the text it is reading, and it has to be evaluated and monitored exactly like the system it protects. Adding one does not remove uncertainty from the system; it moves some of it and adds a component.
Base rates decide whether it is worth it. If one request in ten thousand is genuinely a problem, a classifier at 95% precision still floods the review queue with false positives, and the people staffing that queue stop reading carefully within a fortnight. The useful question is never the accuracy figure on its own; it is how many alerts a day the deployment produces and whether anyone can act on that number.
Pick the direction deliberately. Every threshold is a choice between blocking legitimate work and letting something through, and the right side of it differs by application. A public-facing assistant should probably fail closed; an internal research tool that refuses a fifth of its queries will be routed around by its users, which is a worse security outcome than the one you were preventing.
They can be attacked too. Input that reads innocuously to a classifier and carries an instruction to the main model is a known technique, and so is padding an input past the classifier's own context limit. Treat a guardrail model as part of the attack surface, keep it updated, and never let its verdict be the only thing standing between a request and a consequential action.
None of this argues against classifiers. It argues for measuring them: precision and recall on labelled data drawn from your own traffic, tracked over time, re-checked whenever either model changes. A guardrail nobody has measured is a belief, and beliefs are what incidents are made of.
What happens when a control fires is designed far less often than the control itself, and it is where users form their opinion of the system. A blank refusal with no reason teaches people that the tool is unreliable, and they take their question somewhere with no controls at all — a personal account, a consumer chatbot, a colleague. Shadow usage is the direct product of a badly designed refusal.
A refusal worth shipping does four things: it says what it will not do and, where it can, why; it offers the route that does work, including a human one; it preserves what the person already typed so they are not starting again; and it records the event with enough context that someone can review whether the boundary is in the right place. The fourth is what converts refusals from an irritation into evidence — a weekly look at what fired, and how much of it was wrong, is how a policy gets tuned rather than argued about.
Escalation is the other half. A control that fires should have a defined destination: a typed error the model can recover from, a queue a human owns, a support handover carrying the full context, or a hard stop with a reference the user can quote. “The request was blocked” with no destination is not a control, it is a dead end, and it is the version most systems ship.
For content moderation at volume, a hosted classifier is a reasonable buy and cheaper than building one. What cannot be bought is the part that matters most: the permission model, the tool scopes, the confirmation steps and the audit trail are specific to your systems and your authorisation logic. A product can tell you a response looks unsafe; it cannot tell you this user is not allowed to refund that order. Buy the commodity layer if it helps and build the action layer regardless.
Input classification and output checks each add a model call, so a naive implementation can double perceived latency. Most of it is recoverable: run independent checks in parallel rather than in series, use a small fast model for triage and reserve the larger one for the ambiguous middle, stream the response while the output check runs and retract if it fires, and skip checks that cannot apply to the request type. Architectural controls add essentially nothing, which is another argument for putting the weight there.
Partly, and it is worth doing because it is nearly free — asking a model to check its own output against explicit criteria catches real errors. It is not independent, though: a model that was talked into a bad answer is quite likely to approve it, and self-review shares the failure modes of the generation it is reviewing. Use it as a cheap first pass, not as the control you would describe to a regulator.
Describe the controls concretely rather than by category: the tool scopes and what each credential can reach, which actions require confirmation and who confirmed them, the caps in force, what is logged and for how long, the eval set that gates deploys, and the review process for what the controls block. That is a stronger position than a list of filters, and it is the version that holds up when someone asks what happens if a filter is wrong. We hold no security certifications, and we say so plainly on the trust pages; what we bring is the design and the evidence trail your own audit can use.
They reduce the rate of the obvious attempts and do not close the problem. Injection is not solved, and detection is the weakest of the available responses because there is no reliable signal separating an instruction from text describing one. What genuinely limits the damage is the action layer: least privilege, per-user authorisation, confirmation on consequential steps, and a blast radius small enough that a successful injection is a nuisance rather than an incident.
If the answer depends on a classifier being right, the control is in the wrong layer. That is a good hour of conversation.
30-minute technical call · no deck
What we built, what stage it is at, and a link to the live thing wherever there is one.
yourOS is an operating system with an AI at its heart. You talk to it like a person, out loud or typed. It makes your calls, plans your days and works on your projects with a crew of agents, on your own private server. Our own product, built from the first commit; the waitlist is open.
Website, backend and app assets for Sonora, a free AI sound-therapy app on iOS and Android published by Cognitive AI.
Brand, website, prototype, apps on five platforms, the AI backend and the algorithm-intelligence pipeline for Podswap, a social growth engine that runs without a human in the loop.
In closed beta: Roxy, a private AI assistant with permanent memory running on infrastructure the user owns. Brand, site, product, backend and conversational design.
Live at coherence.com: brand identity generated with AI agents we directed, the motion system and website for the group behind Sonora, AboutAI, WakeUp and MudShadows. Agent work remains in build.
Live at domainbooth.com for DomainBooth, a premium domain brokerage: a new animated mark, every page redesigned and rebuilt with designed mobile layouts, and a custom CMS for its sales record, featured names, press and enquiries.
Apps, platforms and campaigns for these teams, long before the AI work. Every one is written up on the work page — what we did, and where our part ended.
