How it works: bias at clinical decision limits

The flow

How a report is made

Step 1: Bring your paired results
The same samples measured on the old and the new analyser, from the method comparison you already ran, with your decision limits.
Step 2: Run it locally
The program runs offline in your laboratory. Settings are agreed and fixed before the data are opened.
Step 3: Size the change at your limits
Bias at each decision limit, patients crossing each limit set against repeat testing, and a re-test band for the transition.
Step 4: Release and act
You sign off the aggregate report that leaves. Clinicians get a one-page note on what changed and what the laboratory is doing.

What you get

What every report contains

The comparison line

A Passing-Bablok line of new results on old, with its interval, so you see whether the difference is constant, proportional or both.

Bias with limits of agreement

The Bland-Altman average difference and the range most differences fall in.

Bias at each decision limit

The difference at the concentration where a clinical decision changes, with a bootstrap interval from your own pairs.

Patients across each limit

Crossings in each direction, set against what repeat testing alone would move, so the count reflects the change and not ordinary noise.

A re-test band

A narrow range around each limit where a new result is repeated, sized to a budget you choose and scored on pairs it was not fitted to.

A note for clinicians

One page on what changed, at which limits, and what the laboratory is doing about it.

Site-to-site comparability

For networks, the same analysis across sites at your limits, showing which site moves which patients.

In practice

What it looks like on a real platform change

Two figures from the sample report: 1,885 paired CD4 counts from a published platform replacement. The new platform reads about 14% lower, and 535 results change category against about 238 expected from repeat testing alone.

Sample

Fig. 1. Where the new platform reads at each CD4 clinical threshold. The comparison line (slope 0.87) sits below the line of no change, so the new platform reads lower, by about 14% on average across the paired counts.

Source Coetzee LM, Glencross DK (2017). PLoS One. doi:10.1371/journal.pone.0187456 (opens in a new tab); PMC5669480 (opens in a new tab). Licence: CC BY 4.0 (opens in a new tab). Re-analysed by NakedSignal.

Show the numbers

CD4 sample comparison line
Measure	Value
Comparison line slope (Passing-Bablok)	0.867
Slope, 95% interval	0.856 to 0.877
Comparison line intercept (cells/µL)	-1.2
Mean difference, new minus old, as % of old	-14.284
Clinical thresholds (cells/µL)	100, 200, 350, 500

Sample

All four clinical thresholds (100, 200, 350, 500 cells/µL)

Crossed after the change: 535

Expected from repeat testing alone: about 238

Fig. 2. Sample built from public data: 1,885 paired CD4 counts from an open-access deposit, one platform replaced by another. Reported without adjustment, 535 results change category against about 238 expected from repeat testing alone. Flagging the results that land in a narrow band around each threshold for re-test, and converting the rest with the comparison line, catches 66% of that excess on data the band was not fitted to, while re-testing 15% of results; the comparison line alone catches 46%.

Show the numbers

CD4 sample: results crossing a clinical threshold
Measure	Value
Paired CD4 counts	1,885
Clinical thresholds (cells/µL)	100, 200, 350, 500
Results that changed CD4 category, unadjusted	535
Expected to change from repeat testing alone	238.49
Excess caught by comparison line and re-test band, unseen data (%)	66
Results re-tested, unseen data, median (%)	15
Repeat-test imprecision	Default value

Tested

Run end to end on published method comparisons

Public data

34 method comparisons at 14 study sites, from 12 open-access studies, each data set analysed the way one laboratory's would be.

paired results analysed: 10,579
method comparisons: 34
with a full re-test band: 21
published studies: 12

Validation

All passed

Every check is set in advance and shown with its result.

Reproduces reference results (41 of 41): passed
The program reproduces its reference comparison lines and crossing counts exactly.
Re-test budget held on the fitted data: passed
On each comparison's own pairs, the share of results in the re-test band stays within the budget fixed in advance.
Re-test band holds on unseen data: passed
For every comparison large enough for a full band, the re-test share on held-out data stays within the limit fixed in advance.
Small-study safeguard: passed
A comparison too small for a full band gets an indicative grade and keeps its re-test share within the limit on held-out data.
Handles messy exports (15 of 15): passed
Awkward export files with missing columns, decimal commas, text results, duplicate IDs and similar problems each give the expected outcome.

Source 12 open-access method-comparison studies, each cited with its authors and licence on the public data page.

Data handling

Your data stays in your laboratory

The program runs on your own computer and opens no network connection. Only aggregate files leave, and you sign off each one. The list of samples that crossed a limit stays with you, keyed by your own sample reference.

Security and data handling

Try the change explorer Read the sample report