AI safeguarding means designing an automated system so that any sign someone may be at risk takes them out of the automation and to a person, immediately. This is that design: the screen, the handover, what is shown to the person, and how it is tested.
All Mental Health & Wellbeing →
A chatbot that tries to handle a crisis is the wrong design, however good its responses. The system's job when it sees a sign of risk is not to help, reassure or assess. It is to stop, show the person where to get help now, and alert a human who can act. Everything else on this page follows from that.
Signs of risk are rarely phrased clearly. They arrive in passing, in a booking request, in a reply to a reminder, as a joke. So the screen runs on every message, before any other processing, and the threshold is set low on purpose. We accept a high rate of unnecessary handovers, because a missed one is not a cost to be balanced. It is a failure that stops the system.
The design applies anywhere automated systems meet people who may be vulnerable: therapy practices, student services, helplines' administrative channels, HR and employee-assistance tools. In the UK, schools and colleges add statutory safeguarding duties such as Keeping Children Safe in Education, which name a designated safeguarding lead; the system routes to that person, not around them.
any channel, any topic
before anything else
no further questions
for their country, at once
with the conversation attached
only if the screen is clear
The evaluation set is built mainly from the hard cases: risk mentioned in passing, indirect phrasing, slang, other languages, messages that look administrative. Clinicians or safeguarding leads write and review it. Passing it is the gate for release, and it is re-run whenever the model, the prompts or the policy change. A missed escalation in testing or in production stops the system until the cause is understood. Every handover is logged with its reason and sampled by a person.
AI safeguarding is the set of design choices that protect people who may be at risk when they interact with an automated system. In practice it means screening every message for signs of risk, stopping the automation when one appears, showing the person where to get help immediately, and alerting a responsible human. It also means testing that behaviour on hard cases before release and monitoring it afterwards.
It should not try. Professional bodies and researchers have raised concerns about chatbots responding to crisis cues, and the stakes are too high to rely on a model getting it right. A well-designed system recognises possible risk, stops, shows crisis contacts and alerts a person. That is the whole of its role in a crisis.
Yes. We test an existing chatbot, intake form or messaging flow against an evaluation set of hard cases written with your clinical or safeguarding lead, and report where it fails to hand over, what it says when it does, and what happens out of hours. Fixing what we find is a separate decision, and you can take the findings to whoever built the system.
What we built, what stage it is at, and a link to the live thing wherever there is one.
yourOS is an operating system with an AI at its heart. You talk to it like a person, out loud or typed. It makes your calls, plans your days and works on your projects with a crew of agents, on your own private server. Our own product, built from the first commit; the waitlist is open.
Website, backend and app assets for Sonora, a free AI sound-therapy app on iOS and Android published by Cognitive AI.
Brand, website, prototype, apps on five platforms, the AI backend and the algorithm-intelligence pipeline for Podswap, a social growth engine that runs without a human in the loop.
In closed beta: Roxy, a private AI assistant with permanent memory running on infrastructure the user owns. Brand, site, product, backend and conversational design.
Live at coherence.com: brand identity generated with AI agents we directed, the motion system and website for the group behind Sonora, AboutAI, WakeUp and MudShadows. Agent work remains in build.
Live at domainbooth.com for DomainBooth, a premium domain brokerage: a new animated mark, every page redesigned and rebuilt with designed mobile layouts, and a custom CMS for its sales record, featured names, press and enquiries.
Start with your escalation policy, or with a system you already run that we can test.
30-minute technical call · no deck