Reference architecture 01 Self-hosted Open-weight models Span-level citation

Document intelligence over regulated archives.

Turning tens of thousands of long, unstructured PDFs into a queryable table — without a page leaving your infrastructure, and with every extracted value traceable to the sentence it came from.

THE PROBLEM

The data exists. It is just locked inside 200-page PDFs.

Regulated industries accumulate decades of long-form documents — clinical study reports, policy wordings, contracts, filings — that contain structured facts trapped in unstructured prose. The facts are needed as a table. Reading them out by hand is the bottleneck.

Two constraints make the obvious approach unavailable. First, the documents frequently contain personal or privileged data, so routing them through a public API is either contractually prohibited or a regulatory problem. Second, the output feeds decisions that get audited — a number that cannot be traced back to its source is worse than no number, because it looks authoritative and is unfalsifiable.

So the design constraint is not "extract accurately". It is extract in a way that a human reviewer can check in seconds, on infrastructure you control.

THE DESIGN

EXTRACTION PATH

Ingest

OCR · layout parse
Segment

structure-aware chunks
Extract

constrained decoding
Ground

span match to source
Validate

schema + range checks
Review

human queue
Store

row + citation

Lower box = exception path Marker = one document through the pipeline

Documents that fail validation route to human review rather than into the dataset — the exception path is the design, not an afterthought.

Four decisions carry the architecture:

Inference stays inside the perimeter. Open-weight models on your own GPUs. This costs accuracy relative to a frontier API, and the honest trade is that for narrow extraction against a fixed schema, a well-prompted mid-size open model is close enough that the compliance benefit dominates.

Chunking respects document structure. Splitting at a fixed character count severs tables from headers and clauses from their definitions. Segmenting on structural boundaries — sections, tables, numbered clauses — keeps a retrieved passage self-contained.

Decoding is constrained to a schema. The model does not write prose that a parser then guesses at. Grammar-constrained decoding makes malformed output structurally impossible, which removes an entire class of silent failure.

Every value carries a span. The model returns the verbatim sentence and page it extracted from; a post-step verifies that span actually exists in the source. An extraction whose span does not match is rejected — which is what converts "trust the model" into "check the model in two seconds".

extract/schema.py PYTHON

from pydantic import BaseModel, Field

                class Extraction(BaseModel):
                    class="text-dim"># Every field carries the span it came from, so a
                    class="text-dim"># reviewer can jump to the source in one click.
                    value: str
                    page: int = Field(ge=class="text-plasma">1)
                    quote: str  class="text-dim"># verbatim, verified against source
                    confidence: float = Field(ge=class="text-plasma">0.0, le=class="text-plasma">1.0)

                class Record(BaseModel):
                    cohort_size: Extraction
                    dosage_mg: Extraction
                    adverse_event_rate: Extraction

The schema is the contract. Constrained decoding makes it unbreakable rather than merely requested.

extract/ground.py PYTHON

def ground(record, source_pages):
                    class="text-dim"># Reject anything we cannot locate verbatim.
                    for field, ex in record:
                        page = source_pages[ex.page - class="text-plasma">1]
                        if ex.quote not in page:
                            yield Rejected(field, reason=class="text-emerald-data">"span not found")
                        elif ex.confidence < THRESHOLD:
                            yield NeedsReview(field, ex)
                        else:
                            yield Accepted(field, ex)

The grounding check. An extraction whose quote is not in the source document never reaches the dataset.

WHERE IT BREAKS

Scanned documents older than the OCR

Pre-2000 archives are frequently photocopies of faxes. OCR confidence collapses, and the model then extracts confidently from garbage. Mitigation: gate on OCR confidence per page before extraction runs at all, and route low-confidence pages straight to review.

Tables that span pages

A table split across a page break loses its header, and the model silently misattributes columns. Mitigation: table-aware segmentation that reattaches headers, plus a range check on every numeric field — a dosage of 40,000mg fails validation regardless of how confident the model was.

Verbatim spans that are nearly right

Models paraphrase under pressure — a quote off by one word fails an exact-match check, which is correct but produces review volume. Mitigation: normalised matching for whitespace and ligatures only, never for content words.

The reviewer becomes the bottleneck

If the confidence threshold is set conservatively, everything routes to a human and you have automated nothing. Mitigation: tune the threshold against a labelled set, and treat review volume as a headline metric rather than an implementation detail.

WHAT TO MEASURE

Field-level accuracy against a human-labelled holdout is the only number that means anything, and it has to be reported per field — an aggregate hides the one field that fails 30% of the time. Alongside it, track the two numbers that determine whether the system is economically worth running:

Field-level accuracy

Per field, against a holdout labelled by two independent annotators with disagreements adjudicated. Report the per-field table, not the mean.

Review rate

The share of documents routed to a human. This is the automation rate inverted, and it is the number that decides the business case.

Cost per document

Fully loaded — GPU time, storage, and the human minutes on the review queue. A pipeline that is cheaper per document than the status quo only on paper is not cheaper.

These are the targets we design against and instrument for; they are not results from a specific deployment. The schema above is illustrative. Nothing on this page is medical or regulatory advice, and this design has not been validated against any clinical dataset — validation is part of the engagement, against your data and your regulatory pathway.

Keep reading

Architecture

Secure Data Pipelines

Self-Hosted & Open Models

→

30-minute technical call · no deck