Clinical trials and medical AI
Site, lab and device differences, sized at your thresholds.
When eligibility, an endpoint or a model input rests on a measured value, we show how many participants a platform change moves across the threshold, with the check fixed before the data are opened.
What you get
Measurement change, sized before it reaches your analysis
Multi-site lab values
Local and central laboratories read the same sample differently.
Bias and crossings at the protocol's eligibility and endpoint thresholds, against what repeat testing alone would move.
Device changes mid-study
A new device generation or firmware update can move an endpoint for reasons unrelated to the participant.
A bridging protocol written before the change, the shift with its uncertainty, and how the analysis handles it fixed before unblinding. A bridging correction is used only where the bridging data show it helps; on noisy pairs it can cost power.
Pre-specified and auditable
Analyses chosen after the data are seen invite questions.
Check, thresholds and pass criteria fixed and hashed before the data are opened, written with the data-reliability expectations of ICH E6(R3) in mind.
Model inputs and evaluation
A model inherits every instrument that feeds it.
How far inputs moved at the thresholds the model relies on, and held-out evaluation in the style of MedEval-1.
See it on real data
Central laboratory against point of care
Whether to test locally or centrally is a known question in multicentre trials. Here is what it looks like at the HbA1c lines.
- paired HbA1c results in this comparison
- 352
- paired results analysed across all public comparisons
- 10,579
- published studies
- 12
HbA1c at 5.7, 6.5, 7 %
Crossed between the two systems: 236
Expected from repeat testing alone: about 59
Fig. 1. The same people measured on a central lab chemistry analyser and on a point of care device A. At the HbA1c lines of 5.7, 6.5, 7%, 236 results fall in a different category, against about 59 expected from repeat testing alone.
Source Giachino M, Vetter B, Perone SA, Correia JC, Erkosar B, Heller O, Khanal VK, Lab B, Pataky Z, Poudel S, Rai M, Sharma SK (2024). Performance and usability of cardiometabolic point of care devices in Nepal: A prospective, quantitative, accuracy study. PLOS Glob Public Health. doi:10.1371/journal.pgph.0003760 (opens in a new tab); PMC11449279 (opens in a new tab). Licence: CC BY 4.0 (opens in a new tab). Re-analysed by NakedSignal. Open this comparison in the explorer.
Show the numbers
| Measure | Value |
|---|---|
| Paired results | 352 |
| Decision limits (%) | 5.7, 6.5, 7 |
| Results that fall in a different category on the second system | 236 |
| Expected to change from repeat testing alone | 59.17 |
| Mean difference, new minus old, as % of old | 18.49 |
| Repeat-test imprecision | Default value |
Further reading: Arch BN, Blair J, McKay A, Gregory JW, Newland P, Gamble C (2016). Measurement of HbA1c in multicentre diabetes trials - should blood samples be tested locally or sent to a central laboratory: an agreement analysis. Trials. doi:10.1186/s13063-016-1640-6 (opens in a new tab)
How it works
A pre-specified check in three steps
Pre-specify
We fix the check, its thresholds and its pass criteria with you before any data are opened.
Run where the data live
The analysis runs in your environment. What leaves it is designed to carry only aggregates.
Report against the plan
Shift, crossings and any bridging correction, judged against the criteria fixed up front.
Guides
Go deeper
- GuideMeasurement change in trials and medical AIWhen a lab value sets trial eligibility, an endpoint or a model input, a change of laboratory, device or site can move people across the threshold. How to size that against measurement noise, fixed before unblinding.
- GuidePoint-of-care HbA1c against the laboratoryHow far point-of-care HbA1c agrees with the laboratory at the diagnostic and target lines, and how many results change category against repeat testing alone, on five public comparisons.
- GuideBias at clinical decision limitsWhy average bias hides what matters in a method comparison, and how to count patients who cross a decision limit, with a worked example on public data.
A threshold in your protocol or your model?
Tell us which value it rests on and where it is measured. We will scope the check with you.