Different domains, same instinct: when the data cannot be trusted —
garbled OCR in one case, lying file dates in the other — design the
system to fail toward the cheap error, and make every uncertain case
loudly visible instead of quietly wrong.
Centennial Labs
High-volume lab result triage under COVID surge · 2020–21
What
Lab volume spiked during the COVID influx. Results arrived as scanned PDFs whose OCR was badly degraded, staff were filing by hand, and the backlog was the bottleneck. The matcher recovers the intended patient name and clinic from garbled text and files each document into a deterministic folder tree, creating missing folders as needed.
Why it was hard
A patient name could come through as “C1a to schn1cr.” Naive string equality fails completely against the insertions, drops, and substitutions OCR produces — and the failure modes are not symmetric: an unfiled result costs someone ten minutes, a misfiled one attaches a patient's bloodwork to the wrong chart.
What I decided
Score candidates character-by-character with each character allowed to land within a ±n positional window, normalise to [0,1], accept above 0.85 — and fail toward the cheap error: anything the system cannot confidently resolve is routed to a review folder and a review log with whatever fields did parse. Nothing is ever silently misfiled.
What happened
Shipped as a packaged Windows executable with a GUI, because the users were lab techs, not engineers.
the fork is the design: an unfiled result costs ten minutes, a misfiled one attaches bloodwork to the wrong chart
The matcher
The matcher scores a garbled candidate against each canonical entry
character-by-character, allowing each character to land within a
±n positional window rather than demanding an exact index —
which is what makes it robust to the insertions, drops, and
substitutions OCR produces, where naive equality fails completely.
Scores normalise to [0, 1] and are accepted above 0.85. Complexity
is O(m·n) over the candidate list — m entries × n
characters, a straight scan with no sort or search structure — which at
the scale of a clinic list is instant, and at any scale is predictable.
Accepted matches file into a deterministic tree —
2020/November2020/11.20.2020/Centennial State Lab/John_Doe_2020-11-20.pdf
— so a human can always find a document by date and clinic without the
system's help.
The decision that mattered
A document the system cannot confidently resolve is never forced into
the tree: it is routed to a dedicated review location and written to a
review log with whatever fields did parse, for human eyes.
Extracted fields are cross-validated against a second occurrence in the
document — the clinic name has to match between the collection-site line
and the account-name line — and disagreement raises the review flag
rather than picking a winner. The system fails toward the cheap error by
design.
Woodford House
Corpus analysis & module productisation · Shine programme
What
Woodford House runs Shine, a mentoring programme built around 12 “Shine Strengths” — 18 workshops a year, 3,400 student sessions, co-designed with a Girls' Advisory Team and 250+ mentors. The school wanted to package its workshop modules for distribution to other schools. The obstacle: nobody could say which version of anything was current.
Why it was hard
18.7 GB and 3,712 files of accreted institutional memory, where the usual tools for establishing currency — dates, filenames, folder names — all actively mislead. And the corpus contains data on named minors, so the analysis itself had to be built not to leak what it was reading.
What I decided
A strictly read-only pipeline: full walk with sha256 hashing, text extraction across pptx/docx/pdf/xlsx/msg, and a search index over 8.2M characters — so the corpus could be queried without reopening (or risking) a single file. Findings reported in aggregate; people referred to by role, never by name.
What happened
Surfaced the mechanism behind the version drift staff had noticed but couldn't explain, reframed the engagement around access scoping, and established that the module template had to be defined, not derived.
18.7 GB
corpus · 3,712 files
8.2M
characters indexed
276
docs w/ personal-data signals
1 of 13
modules with a feedback form
From the audit's own manifests and reports; corpus contents reproduced nowhere, by rule.
What it surfaced
The folder named “Current Art Files” holds the oldest
copies. Staff doing the right thing — going to Marketing for
the approved asset — reliably got stale files. This was the mechanism
behind the version drift staff had noticed but couldn't explain.
Dates and filenames both lie. 688 files share one
modified date; only 0.1% have created/modified dates that agree;
byte-identical videos carry names for different speakers. Sorting by
date — the instinctive fix — is wrong here in a way that looks right.
276 documents carry personal-data signals inside a folder
named “All Staff.” That reframed the engagement: access
scoping became urgent and independent of any restructure.
There is no module anatomy to extract. Not one
artefact type appears in all 13 modules; a feedback instrument exists
in exactly 1 of 13. The module template had to be defined,
not derived — the core design decision of the packaging work.
40+ decks descend from a single 2021 ancestor by
Save-As, with revision counters that branch and run backwards.
The corpus contains data on named minors. This page describes findings
and method in aggregate only — no student names, no file listings, no
excerpts, no screenshots of corpus contents; staff and students by role,
never by name.