Central lab, local lab, new device: measurement change in trials and in medical AI

Where a lab value decides

Many protocols turn on a laboratory number: an HbA1c range for eligibility, a haemoglobin floor for safety, a creatinine limit for dosing, a change in a marker as the endpoint. Each is a threshold, and a threshold converts a small measurement difference into a different decision.

Whether to measure locally or centrally is an old question with a measured answer. In a multicentre trial of children with newly diagnosed type 1 diabetes, HbA1c measured at the local centres was compared with the same children’s samples measured centrally: almost six hundred pairs from fifteen centres. There was no overall bias, yet only four in five local results were within ten percent of the central one, discrepancies were present at every centre, and the authors recommended central measurement for multicentre trials [Arch et al., 2016].

Eligibility is where it bites first. In one study of point-of-care haemoglobin meters, the authors note that anaemia is a frequent reason for exclusion from clinical studies, and that their evaluation began after they saw discrepant results between a meter and the laboratory during screening for a clinical study [Jaggernath et al., 2016]. The current good clinical practice guideline expects investigators to review data from external sources that can affect eligibility, treatment or safety, and names central laboratory data as an example [ICH E6(R3), 2025].

What changes between sites and periods

• Between sites: different analysers or methods for the same analyte, point-of-care devices against a central laboratory, capillary against venous blood, different operators.
• Within a site, over time: an analyser replaced mid-study, a new reagent or calibrator lot, a device firmware update, a move from local to central testing or back.
• Between a model’s development and its use: any of the above, behind the values the model reads.

Any of these can matter at a threshold without showing up as an average bias. The HbA1c study above found no overall bias and still found individual differences wide enough to matter.

Sizing crossings at the protocol threshold

The trial team’s question is how many participants would be on the other side of the protocol’s threshold had their sample been measured on the other system. With paired results, that is three numbers per threshold: the bias at the threshold, read off the comparison line; the number of participants whose category differs between the systems; and the number that repeat testing on a single system would move anyway, from its imprecision. The excess over that baseline estimates what the change did; only an exact test at a measured imprecision says whether it is more than chance. The guide to bias at clinical decision limits sets out the method.

Fix the check before unblinding

A comparability check that is designed after the outcome is known invites the suspicion that it was designed to find, or not find, something. The good clinical practice guideline expects a statistical analysis plan consistent with the protocol, criteria for including participants in an analysis set defined in advance, and any change made after unblinding documented, justified and reported; it also asks sponsors to set pre-specified acceptable ranges for risks to what is critical to quality [ICH E6(R3), 2025].

NakedSignal’s check is built for the same discipline: the thresholds, the analysis and the pass criteria are written down and sealed before the data are opened, and any later change is a dated amendment. It is designed to run inside the sponsor’s environment with only aggregate results leaving it; no sponsor has run it yet. Our public comparisons follow the same rule for the checks, on published data we had already analysed: the five checks sealed before our first public run passed it; a sixth was added afterwards by a dated amendment, and all six passed on the run shown on the evidence page.

The same logic for models whose inputs are lab values

A model inherits every instrument that feeds it. Clinicians and developers have been warned that dataset shift can come from changes in technology as well as in populations and behaviour; one of the examples given is that adopting high-sensitivity troponin assays changes the clinical interpretation of detectable troponin levels [Finlayson et al., 2021]. The TRIPOD+AI reporting guideline asks model reports to define every predictor, including how and when it was measured, and to identify any differences between development and evaluation data in healthcare setting, eligibility, outcome and predictors [Collins et al., 2024].

When the analyser, device or site behind a model’s inputs changes, the method above sizes how far those inputs moved at the thresholds the model relies on, against measurement noise, before anyone looks at the model’s output.

Worked examples on public data

Public data

Both examples are public method comparisons of the kind a trial faces when one site uses a point-of-care device and another a central laboratory. The decision lines are common clinical lines in place of a protocol threshold, with a default repeat-test imprecision for both.

The first compares a point-of-care haematology analyser with a central laboratory analyser on 201 samples from hospital and outpatient settings [Shean and Bennett, 2026], with haemoglobin lines at 8, 12 and 13 g/dL.