Reference
How it works
Two paths run through this system: documents become structured facts, and questions become guarded answers. Both are built so a clinician can ask “why does it say that” and get an answer that is not “the model decided so”.
1. Ingestion
A document arrives as a raw text layer: capitalised headings, columns held together by runs of spaces, page furniture repeating on every page. The parse stage segments it deterministically into prose, key-value and tabular blocks.
The normalize stage is the interesting one. The first version handed the whole document to a model and asked for Markdown. It obeyed for tables and quietly summarised the prose — a two-page clinic note came back at 13% of its original length, taking a documented anomaly with it. That failure is invisible downstream, because every fact extracted afterwards still verifies against the shortened text.
So the model no longer sees prose. Paragraphs pass through verbatim by code, key-value blocks become two-column tables mechanically, and the model is scoped to the one transformation that genuinely needs judgement: aligning multi-column blocks where blank cells make naive splitting wrong. Its output is row-count checked against the source, with a deterministic fallback. Retention went from 13–28% to 102–122%.
The extract stage turns that Markdown into FHIR-lite entities, each carrying a verbatim quote from the document. Every quote is then checked back against the source. Anything unverifiable is penalised rather than stored as fact.
Finally resolve collapses the same fact restated by several documents into one row. Without it, a problem documented three times is deducted for three times and a five-drug regimen counts as nine.
2. The score
Five weighted domains — cardiometabolic, renal, medication safety, respiratory and care gaps — computed by a pure function over the resolved rows. No model is involved anywhere in that file.
Every deduction names its evidence, and some of that evidence is an absence: a lipid panel fourteen months old on a post-infarction patient is scored on the document that is not there. The reference date is the most recent record in the set rather than today, so re-running next week does not silently change last week’s numbers.
The findings worth having are the ones no single document states. Metformin stays active while eGFR falls from 52 to 38 across two panels eleven months apart; an NSAID is prescribed by a service that could not see the cardiology record. Both are cross-document facts, and both are found by named rules rather than by inference.
3. The guardrail harness
Nine rules. The deterministic ones run first and can short-circuit, so no identifier reaches a provider even if the model stages fail. Every verdict and its measured latency is written to the database and rendered beside the answer.
| Rule | Phase | Kind | Behaviour |
|---|---|---|---|
| crisis.detect | input | deterministic | A fixed term list for self-harm language. Runs first and short-circuits everything, so this response never depends on a model being reachable or well-behaved. |
| phi.redact | input | deterministic | Identifiers are replaced with placeholders before any text leaves the process. The ledger stores a SHA-256 of each detected value and never the value itself. |
| injection.document | input | deterministic | Scans the retrieved chart, not the question. Text in a record addressed to an automated reader is quarantined before the model call and reported to the clinician by code, not by the model. |
| scope.clinical | input | model | Is this a question about the open patient's record? Fails closed — if the classifier is unavailable the assistant answers less, not more. |
| role.patient_directed | input | model | Refuses requests written in a patient's voice. A clinician asking about dosing is not this; that is an ordinary clinical question, caught later if the answer oversteps. |
| output.no_dosing | output | deterministic | Describing the regimen already in the chart passes. Directing one — take, start, titrate, switch, stop — blocks the response before it is shown. |
| output.provenance | output | deterministic | Every clinical claim must cite a document marker. Uncited claims and citations to documents that do not exist are both flagged. |
| output.unsupported_condition | output | deterministic | Names a condition appearing nowhere in the supplied chart? Flagged. Negated mentions are ignored — flagging a correct denial teaches the reader to ignore the flag. |
| output.disclaimer | output | deterministic | Appends the clinician-review banner. Trivial, and it runs on every turn rather than being left to the prompt. |
The turn is deliberately not streamed. Output rules have to inspect a complete response before any of it reaches the screen; a harness that inspects text the clinician has already read is decoration. That costs perceived latency and it is the right trade.
4. Model portability
Every model call goes through one complete() interface with three providers behind it, selectable live from the console header. Swapping vendors touches one file; the guardrails, the scorer and the pipeline do not move.
That matters more than it looks. It turns “could we self-host an open-weight model?” from an architecture question into a scheduling question — and it is why the rules that must not change behaviour when the model changes are precisely the deterministic ones.