Evaluation is the difference between an AI system you operate and one you hope about. It is also the least glamorous part of the work, which is why most projects skip it and then cannot explain why quality moved.
All Technology & Model Stack →
Traditional software is deterministic: a test passes or fails, and a regression is obvious. LLM systems are not. The same input can produce different output; a prompt change that fixes one case can break four you were not looking at; a model version bump can shift behaviour with no code change at all.
Without evaluation, none of that is visible. Quality becomes anecdote — someone shows a bad output in a meeting and the room concludes the system is bad, or nobody notices a slow degradation until customers do. Both are common, and both are largely avoidable with a regression gate in place.
An evaluation set is a hundred or so examples with known-good answers, drawn from real usage, scored automatically. A first version is typically a day or two of work, and it converts every subsequent argument from opinion into measurement. We build it in the first week of an engagement, before the architecture is finalised, because it also forces the question nobody has answered yet: what is a correct output here?
Using a model to grade another model's output is enormously convenient and genuinely useful — provided you know its failure modes, because they are systematic rather than random.
Position bias. In pairwise comparison, published studies report that judges favour the first option presented at rates well above chance. Randomise order and evaluate both orderings.
Verbosity bias. Longer answers score higher, independent of correctness. If your prompt change made outputs longer, expect a score improvement that is not a quality improvement.
Self-preference. A model tends to rate its own family's output more favourably. Judging with the same model you are evaluating is a rigged comparison.
Sycophancy toward the rubric. A judge asked "is this a good answer?" says yes far too often. Concrete criteria — "does every claim appear in the provided source?" — produce far more reliable grading than holistic quality judgements.
The practical rule: use LLM-as-judge for scale, calibrate it against human labels on a subset, and re-check that calibration whenever the judge model changes. An uncalibrated judge is a number that feels rigorous and is not.
| Task | n | Baseline | Target |
|---|---|---|---|
| Retrieval recall@10 | 250 queries | keyword search | ≥ 0.88 |
| Answer supported by retrieved text | 250 | unmeasured | ≥ 0.95 |
| Refuses when nothing relevant retrieved | 80 adversarial | unmeasured | ≥ 0.90 |
| Survives prompt injection in source docs | 60 adversarial | unmeasured | 1.00 |
Method: An illustrative gate of the shape we build, not results from a specific deployment. Retrieval and generation are scored separately so a generation drop caused by a retrieval regression is not misread. Adversarial rows are graded by a human against a written rubric; the remainder use LLM-as-judge calibrated against 50 human labels and re-calibrated whenever the judge model changes. Thresholds are set per engagement against the status quo, never against 100%.
A hundred well-chosen examples beat a thousand random ones. Cover the common cases, the known-hard cases and the edge cases that have actually caused problems. Then grow it: every production failure becomes a new test case, which is how the set gets genuinely valuable over time.
The people whose judgement the business accepts — the support lead, the underwriter, the clinician. Not the engineers, and not a contractor unfamiliar with the domain. If the person who will veto the system in a steering meeting did not define what correct means, you have measured the wrong thing very precisely.
There is no universal threshold; it depends on the cost of an error and what the current process achieves. A support agent at 90% where humans reach 92% may be excellent given cost and latency. A system making credit decisions at 99% may be unacceptable, and regulatory requirements apply to that kind of decision. The right comparison is against the status quo, not against 100%.
The golden set on every change, in CI, as a merge gate. The full suite including adversarial cases nightly. Online evaluation continuously on a sample. And always after a model version change, which is the highest-risk event that involves no code change at all.
If the answer is no, this is the thing to fix before anything else.
30-minute technical call · no deck
What we built, what stage it is at, and a link to the live thing wherever there is one.
yourOS is an operating system with an AI at its heart. You talk to it like a person, out loud or typed. It makes your calls, plans your days and works on your projects with a crew of agents, on your own private server. Our own product, built from the first commit; the waitlist is open.
Website, backend and app assets for Sonora, a free AI sound-therapy app on iOS and Android published by Cognitive AI.
Brand, website, prototype, apps on five platforms, the AI backend and the algorithm-intelligence pipeline for Podswap, a social growth engine that runs without a human in the loop.
In closed beta: Roxy, a private AI assistant with permanent memory running on infrastructure the user owns. Brand, site, product, backend and conversational design.
Live at coherence.com: brand identity generated with AI agents we directed, the motion system and website for the group behind Sonora, AboutAI, WakeUp and MudShadows. Agent work remains in build.
Live at domainbooth.com for DomainBooth, a premium domain brokerage: a new animated mark, every page redesigned and rebuilt with designed mobile layouts, and a custom CMS for its sales record, featured names, press and enquiries.
Apps, platforms and campaigns for these teams, long before the AI work. Every one is written up on the work page — what we did, and where our part ended.
