Reference architecture 03 Immutable traces Deterministic tools Four-eyes review

Risk agents with an audit trail.

In regulated decisioning the output is not the deliverable. The reconstructable reasoning behind it is — because that is what gets examined when a decision is challenged.

THE PROBLEM

A decision you cannot reconstruct is a decision you cannot defend.

Risk, underwriting and compliance decisions get challenged — by a customer, an auditor or a regulator — often months after they were made. The question is never "what did the model output". It is "on what basis", and "would the same inputs produce the same result today".

That second question is the one that eliminates most naive agent designs. A model behind an API changes underneath you. Temperature above zero makes runs non-reproducible. A tool that reads live data returns something different next Tuesday. None of that is acceptable when the answer has to be defensible in a year.

So the architecture optimises for reconstructability first and cleverness second, which inverts the usual agent design priorities.

THE DESIGN

DECISION PATH

Case intake

snapshot inputs
Policy fetch

versioned rules
Reason

pinned model + seed
Tool calls

deterministic, logged
Decide

with rationale
Four-eyes

above threshold
Commit

immutable trace

Lower box = mandatory human review Marker = one case through the system

Every arrow persists its input and output. Replaying a decision means replaying the trace, not re-running the model and hoping.

Inputs are snapshotted, not referenced. The case record stores the values used at decision time, not pointers to rows that will change. Reconstruction reads the snapshot.

The policy is versioned like code. Each decision records the policy version it applied. "The rule changed in March" then becomes checkable rather than an argument.

The model is pinned. An explicit model version, temperature zero, a recorded seed where the provider supports it. Note that hosted inference is not bit-for-bit deterministic even so — batching and floating-point non-determinism see to that — which is why the stored trace, not a re-run, is the record of what happened. You accept losing access to whatever the newest model does better, in exchange for a system whose behaviour does not drift without a deliberate, logged upgrade.

Tools are pure functions over snapshotted inputs. A tool that hits a live endpoint makes the decision unreproducible. Tools read from the snapshot; anything genuinely live is captured into the snapshot at intake.

Confidence never bypasses four-eyes. Above a materiality threshold, a human reviews — regardless of how certain the agent is. The agent's role there is to assemble the evidence and draft the rationale, which is where it saves real time.

trace/decision-8842.json JSON

{
                  "decision_id": "8842",
                  "policy_version": "credit-risk-v14",
                  "model": "<provider-model-id-pinned>",
                  "temperature": 0,
                  "inputs_snapshot_sha": "a3f9c1...",
                  "tool_calls": [
                    { "tool": "affordability_ratio", "args": { "income": 84000, "obligations": 21400 },
                      "result": 0.2547, "duration_ms": 3 }
                  ],
                  "retrieved_policy_clauses": ["4.2.1", "4.2.7"],
                  "decision": "refer",
                  "rationale": "Ratio within band but clause 4.2.7 requires referral for
                                self-employed applicants under 24 months trading.",
                  "reviewed_by": "analyst-114",
                  "committed_at": "2026-03-11T09:22:41Z"
                }

The stored trace. Everything needed to reconstruct the decision, with no reference to mutable state.

WHERE IT BREAKS

Model deprecation

Pinned versions get retired by the provider, and your reproducibility guarantee expires with them. Mitigation: know the deprecation window before you commit, keep an open-weight fallback you can host, and re-validate against the archived trace set on every forced migration.

Rationales that read well and explain nothing

Models are fluent, so a plausible-sounding rationale is cheap and a genuine one is not. Mitigation: require the rationale to cite specific policy clause IDs, and reject any that cites none — fluency without a citation is not an explanation.

Reviewers who rubber-stamp

Four-eyes review degrades into clicking approve when the agent is usually right. Mitigation: track reviewer override rate; a rate at or near zero means the control is theatre, and salting the queue with known-bad cases measures whether it is.

Snapshot drift

If any tool quietly reads live state, reproducibility is gone and nothing surfaces it. Mitigation: run tools in a sandbox with no network, so a live read fails loudly in development rather than silently in production.

WHAT TO MEASURE

Replay fidelity

Re-run a sample of archived traces. The share producing an identical decision is your reproducibility rate — and it should be 100%, with any gap investigated as a defect.

Reviewer override rate

How often four-eyes changes the outcome. Too high means the agent is not useful; near zero means the control is not real.

Rationale citation coverage

The share of decisions whose rationale cites at least one specific policy clause. Below 100% means some decisions are unexplained.

These are the targets we design against and instrument for; they are not results from a specific deployment. This page describes an engineering pattern, not legal advice. Whether a given design satisfies a particular regulator — including EU AI Act obligations for credit decisioning and other Annex III uses — is a question for your compliance function and your counsel.

Keep reading

Industry

AI for Financial Services

30-minute technical call · no deck