Confidence disclosure
How an algorithmic output tells a clinician how much to trust it — and, just as importantly, what it was never designed to see.
Overview
Every AcuteLine output is a probability wearing the clothes of a statement. "STEMI pattern detected" is really "a model trained on a particular population assigns 0.94 to this pattern, given inputs of a particular quality." The gap between those two sentences is where patient harm lives, in both directions: a clinician who over-trusts a confident wrong answer, and one who dismisses a hedged right one.
This pattern exists to keep that gap visible without drowning the clinician in statistics.
Always present
Confidence travels with the finding, on the same surface, at the same moment. Never behind a hover, a tooltip, an expander or a details page.
Calibrated, not flattering
The number shown means what it says across the deployed population. A score tuned to look reassuring is a fabrication with a decimal point.
Bounded by scope
A high score inside the model's competence and a high score outside it look identical unless the interface says otherwise. Scope is disclosed alongside confidence.
Anatomy
| Element | Rule |
|---|---|
| Numeric score | Two decimals, 0–1. The primary representation. Never a word, never a percentage rounded to the nearest ten. |
| Bar | Secondary, for scanning a list. Neutral --primary fill — never coloured
by value, which would smuggle in a judgement the number does not make. |
| Model version | Always. A score from v4.1 and v4.2 are not comparable. |
| Training recency | The month training data ends. Lets a clinician reason about drift. |
| Scope statement | The population and input conditions the model was validated on. |
What confidence must never become
A number the clinician can reason with, compare across cases, and discount if they disagree.
"High" is not calibrated, cannot be compared between cases, and quietly converts a probability into an endorsement.
A low score is shown exactly like a high one. Same treatment, same prominence, no apology.
Shrinking and fading low confidence hides the cases most needing human judgement. Uncertainty is not a defect to be minimised.
- Never a colour scale. Green-to-red confidence competes directly with the alarm palette and reads as clinical priority. The bar stays one neutral colour.
- Never a threshold badge alone. "Above threshold" hides where the threshold sits and who set it.
- Never suppressed below a cut-off. If a finding is shown at all, its confidence is shown with it.
- Never aggregated across findings. One score per finding. A single "case confidence" is not a meaningful quantity.
Out-of-scope disclosure
The most dangerous output is a confident score on an input the model was never validated for. Nothing in the number itself reveals this — so the interface must.
This section is the post-analysis half of withholding: the model ran, and the interface declines to extrapolate beyond what it was validated for. The pre-analysis half belongs to Capture & quality gate, where an input that cannot support a finding is refused before anything is computed. Where there is no qualified reader to weigh a caveat, that earlier gate is the only one that works.
Where an input falls outside the validated population — paced rhythm, paediatric,
reduced lead set, unacceptable signal quality — the system reports no score at all
and says why. It does not report a lower score. A reduced number invites the clinician to
discount it slightly; — plus a reason tells them the tool has nothing to offer here,
which is the truth.
Scope conditions AcuteLine discloses:
| Condition | Behaviour | Priority |
|---|---|---|
| Ventricular pacing | No score; explain | Advisory |
| Left bundle branch block | Score reported with an explicit caveat | Advisory |
| Fewer than 12 leads | No score; explain | Advisory |
| Signal quality below threshold | No score; name the affected leads | Advisory |
| Age under 18 | No score; outside validated population | Advisory |
| No prior ECG on file | Score reported; serial comparison unavailable | Advisory |
Negative results
A confident negative is the output most likely to change a clinician's behaviour and least likely to be scrutinised. It carries its limits in the same breath, every time — not in a footnote, not once per session.
"A negative result does not exclude ACS" is fixed copy. It is not shortened for space, not moved to a tooltip, not shown only on first use, and not translated freely — the translated string is reviewed for the same meaning. It is a risk control expressed as a sentence.
Designing against automation bias
Automation bias is the tendency to accept a machine's answer over one's own judgement, and it grows with how authoritative the interface looks. Four counter-measures are built in:
- Evidence precedes conclusion. The primary action opens the ECG, not a summary. The clinician sees the waveform before they see agreement.
- The model never appears to act. No output triggers an order, a page or a pathway. Every downstream step is a human pressing something, attributably.
- Disagreement is one click and is recorded. Marking a finding incorrect is as easy as accepting it, and feeds monitoring. If disagreement is harder than agreement, the measured agreement rate is meaningless.
- Language stays sub-diagnostic. "Pattern detected", "cannot be excluded", "consistent with" — never "diagnosed", "confirmed", "ruled out".
Three options, identical weight. No primary styling on "Agree" — that would be a thumb on the scale of your own post-market data.
Do's and don'ts
A low score shown at full prominence, exactly like a high one. Uncertainty is not a defect to minimise.
Shrunk and faded. The cases most needing human judgement are the ones hardest to see.
Accessibility
- The number is the accessible value. The bar is
aria-hidden; the score is read as text. Never encode confidence only graphically. - Announce as a labelled value — "confidence 0.94" — not "94 percent full" or a bare progressbar role.
- Scope and caveat text is real text, in the reading order, never a title attribute.
- No colour encoding, so nothing is lost in greyscale or with a colour-vision deficiency.
Outcomes of use
What this contributes to, in the terms of Usability & context of use. These are attributes believed to contribute to an outcome; the outcome itself is settled by observing real use in a specified context, not by this page.
- Effectiveness — The clinician's reliance on an output matches that output's actual reliability. Over-reliance and dismissal are both effectiveness failures.
- Efficiency — The cognitive effort of integrating a probability into a decision under time pressure — the reason the score is numeric, prominent and always in the same place.
- Satisfaction — This pattern is almost entirely a satisfaction intervention. Trust, perceived safety and perceived competence of the tool are what it manages; scope disclosure is how.
Clinical safety notes
Trace these in your risk file (ISO 14971) and usability engineering file (IEC 62366-1).
- Confidence always co-located with the finding. Mitigates: a probabilistic output being read as a determination.
- Numeric, never categorical. Mitigates: "high confidence" being read as clinical endorsement.
- Low scores shown at full prominence. Mitigates: uncertainty being overlooked precisely where human review matters most.
- No score outside validated scope. Mitigates: confident output on an input the model cannot assess.
- Model version and training recency always shown. Mitigates: comparing incomparable scores; undetected model drift.
- Fixed negative-result caveat. Mitigates: false reassurance and premature discharge.
- Symmetric agree/disagree. Mitigates: automation bias, and biased post-market performance data.
- No colour scale on confidence. Mitigates: confusion with the IEC alarm palette.
Related
- Triage worklist — the confidence column, and why it is never the default sort.
- Alert Banner — the provenance line.
- Colour — why confidence gets no hue.
- Clinician override — what happens after "Disagree".