CLIA acceptance limits as allowable bias: what they say at your decision limits

What they grade, and what they do not

A proficiency testing criterion grades one result on one survey sample against that sample’s target value [§ 493.931]. It is a limit on total error for a single measurement: bias and imprecision together. Read as an allowable bias for a method change, it is generous, because the whole limit goes to the systematic difference and none is left for the scatter each result also carries. A bias that uses most of the criterion leaves a single result little room before it fails a survey.

Nor does the verification standard require these limits. § 493.1253 asks the laboratory to show performance comparable to the manufacturer’s for accuracy, precision and reportable range, and names no acceptance limit [§ 493.1253]. Choosing one is the laboratory director’s decision; the guide to CLIA verification for a new analyser sets out the rest of that standard.

Most important, a criterion is a statement about results, not about patients. It says nothing about how many results sit close to a decision limit, which is what decides how many patients a given bias moves across it. The way to see that is to read both on the same comparisons.

The limits against public comparisons

Public data

We took every public comparison in the change explorer whose analyte has a CLIA criterion and which has decision lines set: 25 comparisons across 6 analytes. For each, we read the Passing-Bablok line at every decision line, expressed the bias there as a share of the CLIA criterion at that concentration, and set it beside the number of results that changed category at those lines.

What the table shows

Of the 25 comparisons, 12 have a bias inside the CLIA criterion at every decision line and 13 exceed it at one line or more. Inside the criterion is not the same as no patients moved: 11 of the 12 inside it still changed the category of at least one result, between 6 and 89 results each.

Part of every such count is repeat-testing noise, which would move some results across a line even with no bias at all, so a count alone is not evidence of harm. The useful comparison is the count against what repeat testing alone would move at the method’s measured imprecision, a published procedure [Schmidt et al., 2015, Clin Chim Acta]. For the outpatient creatinine study, where that imprecision was measured [Schmidt et al., 2015, PLoS ONE], the bias uses 19% of the criterion at most, yet 23 results changed category against about 12 expected from repeat testing alone. That expected count uses the 1.8% imprecision the study itself measured; at that imprecision the count from repeat testing alone falls between 6 and 17 with 95% probability, so the 23 crossings are about twice the expected count, and by the exact Poisson-binomial test 23 or more from repeat testing alone has probability 0.00017. That 1.8% is also our program's laboratory-analyser default (1.79%, this study's own reported precision), so the default gives about the same count; the program's cautious point-of-care default of 6%, which it uses when a point-of-care device is involved and no precision data are supplied, would give about 41, which overstates repeat-testing noise for these laboratory analysers. At the upper end of the study's own precision estimate, 2.34%, about 15 would be expected and 23 or more has probability 0.014. The CLIA verification guide works through that example.

Choosing a limit for verification

• Decide the limit before the data come in, and say where it comes from: the maker’s claim, a CLIA criterion, biological variation or a clinical requirement.
• If you use a CLIA criterion, use part of it for bias, so that imprecision fits in the rest.
• Read the bias at each decision limit, not as one figure for the whole range. The guide to bias at decision limits shows how.
• Count the patients as well, against the repeat-testing expectation, so that a change inside the limit that still moves more results than noise would is seen before go-live.

Nothing on this page is regulatory advice, and it does not replace the laboratory’s own verification.

Run it on your own pairs

The change check reads your paired results in the browser. For one analyte, free, it gives the bias at each decision line you set and the results that crossed each line against those repeat testing alone would move. The public example runs the full report free. Your file is not uploaded.

Sources

1. US Code of Federal Regulations, Title 42, Part 493. § 493.931 Routine chemistry (proficiency testing criteria for acceptable performance). Electronic Code of Federal Regulations, as amended in 2022. www.ecfr.gov/current/title-42/section-493.931
2. US Code of Federal Regulations, Title 42, Part 493. § 493.941 Hematology (including routine hematology and coagulation). Electronic Code of Federal Regulations. www.ecfr.gov/current/title-42/section-493.941
3. US Code of Federal Regulations, Title 42, Part 493. § 493.1253 Standard: Establishment and verification of performance specifications. Electronic Code of Federal Regulations. www.ecfr.gov/current/title-42/section-493.1253
4. Schmidt RL, Kordy MA, Genzen JR, Straseski JA, Greene DN, Lehman CM (2015). A mathematical procedure to estimate the impact of a change in method on discordance or misclassification at a decision limit in laboratory method comparison studies. Clinica Chimica Acta. doi.org/10.1016/j.cca.2014.10.043
5. Schmidt RL, Straseski JA, Raphael KL, Adams AH, Lehman CM (2015). A Risk Assessment of the Jaffe vs Enzymatic Method for Creatinine Measurement in an Outpatient Population. PLoS ONE. doi.org/10.1371/journal.pone.0143205