Reader paradigm
Whether a clinician sees an algorithm's output before, during or after forming their own impression is not a preference. It is the condition under which the device's performance was measured — and changing it in the interface silently invalidates the evidence the product was cleared on.
Overview
A device that marks findings is evaluated by having readers read cases with it and without it. How they read with it is part of the experiment. Show the marks from the first second and you measure one thing; show them after the reader has committed an impression and you measure something else. Both are legitimate designs. They produce different sensitivity, different specificity, and different failure modes.
The interface is what decides which one actually happens in the field. So this page exists to make a single point: the reading paradigm is a design decision with regulatory consequences, and it belongs in the product on purpose rather than by accident.
ForeCall is triage-only: its output never reaches the reading room at all, which is the strictest paradigm there is and the reason its evidence is about queue position rather than about reading accuracy. It anchors this page because it shows the boundary from outside — the moment its output appeared on the viewer, it would be a different device with different obligations. See Reprioritisation.
Read the way it was validated
The interface enforces the paradigm the evidence was gathered under. Anything else is a performance claim about a workflow nobody tested.
The order is the evidence
In a second-reader design, the reader's own impression must exist before the marks do — which means it must be recorded, not remembered.
Deviation is visible
Reading outside the paradigm is sometimes reasonable. It is never invisible, because it changes what the resulting data means.
The three paradigms
| Paradigm | When output appears | What it changes | Principal risk |
|---|---|---|---|
| Triage only ForeCall |
Never to the reader. Only the queue order changes. | When a study is read, not how. | Delay to displaced studies — see Reprioritisation. |
| Concurrent | From the moment the study opens. | Reading behaviour throughout. Attention goes to the marks first. | Anchoring, and satisfaction of search — findings outside the marks get looked for less. Dependency grows quietly over months. |
| Second reader sequential |
After the reader records an initial impression. | Only the revision step. The first read is unassisted. | Reading time, and a bias toward accepting the prompt at the revision stage. |
Performance claims are only valid under the paradigm they were measured in. A device evaluated as a second reader, deployed so that its marks are visible from the moment the study opens, is operating outside its evidence — its measured specificity was obtained from readers who had already committed an opinion, and those readers no longer exist. The interface is the only place this can be prevented, because nothing in the file or the clearance stops a configuration flag.
Enforcing the order
A second-reader design asks the clinician to form an impression first. Asking is not enough: under time pressure, with the marks one click away, an unrecorded "first impression" becomes a formality. The impression has to be committed before the marks exist.
Recorded and retained. It is not overwritten by anything you decide after the marks appear.
The second button exists deliberately. Removing it would produce compliance by obstruction, and a radiologist who genuinely needs the marks early would work around the product instead. Pressing it is a recorded deviation.
- Recorded, not remembered. An unrecorded first impression cannot be distinguished later from one formed after the marks appeared.
- The first impression is never overwritten. Both readings are kept, with their times, so the effect of the assistance is reconstructable — the same instinct as Clinician override.
- Marks are dismissible after they appear. A reader who wants to re-examine without them must be able to, and their turning them off is not a deviation.
- No score at the impression stage. A probability shown alongside the marks does not aid revision; it substitutes for it.
- Never pre-fill the report from the marks. That converts a second reader into a first draft, which is a different device again.
Why the order does the work
The mechanism is the same one that puts model annotation behind a deliberate action on ECG review: an impression formed after seeing a machine's opinion is not an independent impression, and a reader cannot introspect their way out of it. Two consequences specific to spatial marks make it sharper here.
- A mark cannot be unseen. Dismissing an overlay removes the pixels, not the knowledge of where the software was looking. This is why the order matters more than the toggle.
- Satisfaction of search. Finding what the marks pointed at reduces the search for anything else, including findings the device was never trained on and does not claim to detect.
- A false mark is not free. It costs the reader time, and it redirects the eye away from regions they would otherwise have examined.
- Dependency is slow. It does not appear in an evaluation lasting a week. It appears after a year, and it is a post-market surveillance question rather than a pre-market one.
When a reader deviates
Deviation is not misconduct and must not be designed as though it were. A radiologist calling a stroke pathway at two in the morning may have excellent reasons to look at everything at once. The requirement is only that the resulting data is not silently pooled with data from readings that followed the paradigm.
-
02:14:08Marks shown before impression recorded — read performed outside the second-reader paradigm.Dr S. Iyer · no reason required · excluded from paradigm performance data
-
02:14:02Study opened · assistance available, not yet shownAcc. 88-4412 · ForeCall v2.4
No justification is demanded. Requiring one would produce a free-text field full of "n/a" and a workflow people avoid — the same asymmetry argument as Clinician override.
- Never blocked. A hard stop on deviation is a hard stop on clinical judgement, and it will be defeated by reading the study somewhere else.
- No mandatory reason. Recording that it happened is the requirement; interrogating the reader is not.
- Excluded from paradigm performance data, and that exclusion is stated rather than silently applied — an evaluation whose denominator quietly excludes the hard cases is worse than no evaluation.
- Deviation rate is a signal about the design. If most reads deviate, the paradigm does not fit the work, and the correct response is to re-evaluate the device, not to discipline the readers.
States
| State | Rendering |
|---|---|
| Paradigm declared | Named on the reading surface — "second reader" — not buried in a settings page. A reader must know which mode they are in without asking. |
| Unassisted phase | Marks demonstrably absent, and stated as not yet shown. Silence is ambiguous: a reader cannot tell "no marks" from "marks not yet available". |
| Assistance unavailable | Stated. Distinct from "analysed, nothing marked" — the distinction Key–value pair exists to protect. |
| Assisted phase | Marks shown with their version and the time of analysis; the recorded first impression remains visible. |
| Deviated | Marked on the read, logged, excluded from paradigm data. No obstruction. |
| Paradigm changed by configuration | Announced to readers with author and time — a change of this kind alters what the product's performance claims mean. See Safety configuration. |
Who holds each rule
The reader paradigm is about when a model’s marks appear relative to a human’s first look. That is sequencing in a viewer the package does not ship. What it holds is the record of the sequence.
| Rule | Held by | Where |
|---|---|---|
| Recorded, not remembered; the first impression is never overwritten | Event log | The first impression is an entry. A later one supersedes it and both stand. Asserted: EventLog: no entry can be edited, removed or cleared |
| The sequence of impression and marks is a real order | Event log | The log states its order and is ordered markup, so “impression before marks” is checkable after the fact. Asserted: EventLog: the order is stated and the list is ordered markup |
| Marks are dismissible after they appear, in one action | Switch | A switch acts on contact. That the marks are on a switch, and off until the impression is recorded, is your sequencing. Asserted: Switch: effect has exactly one permitted value |
| No score at the impression stage; never pre-fill the report from the marks | The implementer | What the screen shows before the impression is committed is your viewer’s state. |
| A mark cannot be unseen; a false mark is not free | The implementer | Facts about attention, which the page argues from and no control can enforce. |
| Satisfaction of search; dependency is slow | The implementer | Same: the hazards the sequencing exists to mitigate. |
| Never blocked; no mandatory reason | The implementer | Two things not to build. The Dialog requires a way out, which removes the easiest way to block; it cannot remove the others. |
A pattern page has no component underneath it, so a rule stated here is held by whoever builds against it unless a component already holds it. Of these 7 rules, 3 are held by a component this package ships, and 4 are the implementer’s. Three held, four not. The paradigm is a viewer behaviour and the viewer is yours; the record that proves you followed it is the package’s.
Do's and don'ts
Unassisted read · marks not yet shown
The reader knows which phase they are in, and that assistance exists but has been withheld. Nothing about the state has to be inferred.
(no marks visible)
Ambiguous. Nothing marked, not yet shown and analysis failed are three different facts sharing one rendering.
The paradigm is the default path, and departing from it costs one click and a log entry. Neither obstruction nor pretence.
A second-reader device with no first read. The evidence it was cleared on describes readers who no longer exist.
Impression before assistance: no acute
haemorrhage · 02:11
After assistance: small left parietal haemorrhage · 02:15
Both readings kept with their times. The effect of the assistance is reconstructable, which is the only way it can ever be evaluated.
Impression: small left parietal haemorrhage
The first read overwritten by the revision. The device now appears to have changed nothing, on every case it changed.
Read performed outside the paradigm · logged · excluded from paradigm performance data
The exclusion is stated. Anyone reading the performance figures knows what the denominator does and does not contain.
Deviation requires a documented clinical justification.
A free-text field that will fill with "n/a", plus a workflow readers avoid — so the deviations stop being recorded rather than stopping.
Accessibility
- The current phase is text in the reading surface's accessible name — "unassisted read, marks not yet shown". Absence of an overlay conveys nothing non-visually.
- Marks are described, not only drawn. Each has a text description of location and extent, so the assistance is available to a reader using magnification or assistive technology.
- Marks appearing is announced politely, since the reader requested them.
- The first impression field is a real labelled control with
aria-describedbycarrying the retention statement — see Text field. - The deviation control is a plain button with its consequence in the label, never a hidden shortcut or a long-press.
- Overlay toggling never traps focus and returns focus to the image region it was invoked from.
- No colour-only marks. Shape and a text description carry the meaning; colour is redundant reinforcement — see Accessibility.
- Targets ≥ 24 px (SC 2.5.8) and no flashing when overlays appear (SC 2.3.1).
Outcomes of use
What this contributes to, in the terms of Usability & context of use. These are attributes believed to contribute to an outcome; the outcome itself is settled by observing real use in a specified context, not by this page.
- Effectiveness — reading accuracy in the field matches the accuracy the device was evaluated at, because the field workflow is the evaluated workflow.
- Efficiency — the second-reader design costs reading time and that cost is real. It buys an independent first impression, and whether the trade is worth making is a decision for the evidence, not for the interface.
- Satisfaction — readers who feel assisted rather than audited. A paradigm enforced by obstruction produces resentment and workarounds; one enforced by default path and honest logging does not.
Clinical safety notes
Trace these in your risk file (ISO 14971) and usability engineering file (IEC 62366-1). Each bullet says what checks it: a component that holds the rule, a test in your verification file, deployment configuration, post-market surveillance, or a design review where no test can see it. A control checked by nothing is a sentence.
- Interface enforces the validated paradigm by default. (a test in your verification file) Mitigates: field performance diverging from cleared performance with nobody able to detect it.
- First impression committed before marks appear. (held by Event log) Mitigates: an assisted read recorded as an independent one.
- Both readings retained with times. (held by Event log) Mitigates: the effect of assistance becoming unmeasurable.
- Deviation permitted, logged and excluded. (a test in your verification file) Mitigates: pooled performance data mixing two different reading conditions.
- Deviation never blocked or interrogated. (a test in your verification file) Mitigates: readers defeating the control by working outside the product, which loses the record entirely.
- Assistance-unavailable distinguished from nothing-marked. (a test in your verification file) Mitigates: a failed analysis read as a negative one.
- No report pre-population from marks. (a test in your verification file) Mitigates: a second reader operating as an unvalidated first draft.
- Paradigm named on the reading surface. (a test in your verification file) Mitigates: a reader unaware which mode they are in, and therefore what their impression will be used for.
- Configuration changes to the paradigm announced. (a test in your verification file) Mitigates: performance claims silently ceasing to apply after a deployment change.
- Deviation rate monitored. (post-market surveillance) Mitigates: a mismatch between the validated workflow and the real one persisting unexamined.
Related
- Reprioritisation & notification — the triage-only paradigm, and why its output stays out of the reading room.
- ECG review — the same anti-anchoring argument for a waveform, where this system first establishes it.
- Confidence disclosure — designing against automation bias once the output is visible.
- Clinician override — recording disagreement without interrogating it, and feeding surveillance honestly.
- Safety configuration — where a paradigm change would be made, and what must happen when it is.
- ForeCall — the reference application.