CLIA method verification for a new analyser: what § 493.1253 requires

What the rule says

The standard is § 493.1253 of the CLIA regulations. A laboratory that introduces an unmodified, FDA-cleared or approved test system must, before reporting patient test results, demonstrate that it can obtain performance specifications comparable to the manufacturer’s for accuracy, precision and the reportable range, and verify that the manufacturer’s reference intervals are appropriate for its patient population [§ 493.1253].

A laboratory that modifies a cleared system, runs a test not subject to FDA clearance (including methods developed in-house), or uses a system whose maker gives no performance specifications must instead establish them: accuracy, precision, analytical sensitivity, analytical specificity including interfering substances, reportable range, reference intervals and any other characteristic the test needs. In both cases the laboratory then sets its calibration and control procedures from those specifications, and documents all of it [§ 493.1253].

What it leaves to the laboratory

The section names characteristics, not a protocol. It gives no number of samples, no number of days and no acceptance limit for any of the four [§ 493.1253]. Those choices are the laboratory director’s, and the documented record should say how each was made. Published protocols fill the gap: CLSI EP15 for precision and an estimate of bias [CLSI EP15], CLSI EP09 for a comparison on patient samples [CLSI EP09], and CLSI EP28 for reference intervals [CLSI EP28]. The acceptance limit is the choice that matters most, and the guide to CLIA acceptance limits as allowable bias looks at one ready-made source for it.

The four characteristics, in practice

Accuracy. Run patient samples on the current and the new system and estimate the bias between them across the reporting range, with extra samples near each decision limit [CLSI EP09]. The guide to method comparison regression covers which line to fit and the checks that matter more than the choice.
Precision. Repeat measurements of control or patient material over several days, and compare the imprecision with the manufacturer’s claim [CLSI EP15]. Keep the figure: it is also what tells you how many results would cross a limit from repeat testing alone.
Reportable range. Show that results are reliable from the lowest to the highest value you will report. The calibration verification standard asks for at least a minimal (or zero) value, a mid-point value and a maximum value near the upper limit of the range [§ 493.1255].
Reference intervals. Verify that the maker’s intervals suit your population. The usual approach tests a minimum of 20 samples from healthy local subjects [Ozarda et al., 2018], [CLSI EP28].

After go-live

Verification is not the last time the regulation asks about the change. Calibration verification is due at least every six months and whenever a complete change of reagents is introduced, after major preventive maintenance or replacement of critical parts, and when controls show an unusual trend or shift that other means do not correct [§ 493.1255]. If the laboratory runs the same test on more than one instrument or method, or at more than one site, it must have a system that twice a year evaluates and defines the relationship between their results [§ 493.1281]. A laboratory that keeps the old analyser running during a transition is in that position from day one; the guide to comparability across analysers and sites covers how to report it.

A worked example on public data

Public data

An outpatient study ran enzymatic and Jaffe creatinine on the same samples on the same analyser [Schmidt et al., 2015, PLoS ONE]. Treat the enzymatic method as the current one and the Jaffe method as the new one, and read the result as the accuracy step of a verification. Across 529 pairs the Passing-Bablok line has a slope of 0.939 (95% interval 0.914 to 0.967) and an intercept of 0.053 mg/dL. The slope interval does not include one, but a Deming or least-squares fit to the same pairs gives a slope close to one: the lower Passing-Bablok slope reflects curvature at low creatinine, not a proportional difference across the range. As an acceptance limit, take the CLIA proficiency testing criterion for creatinine, target value ±10% or ±0.2 mg/dL, whichever is greater [§ 493.931]. Reading the line at the two decision lines the comparison was run at:

Bias read from our Passing-Bablok line, fitted to the study’s published pairs, at each creatinine decision limit, the CLIA criterion at that limit, and the share of the criterion the bias uses
Decision limit, mg/dL	Bias, mg/dL	Bias, %	CLIA criterion, ± mg/dL	Share of criterion used
1.2	−0.020	−1.6%	0.20	10%
1.5	−0.038	−2.5%	0.20	19%

Source: Schmidt RL, Straseski JA, Raphael KL, Adams AH, Lehman CM (2015). A Risk Assessment of the Jaffe vs Enzymatic Method for Creatinine Measurement in an Outpatient Population. PLoS One. doi:10.1371/journal.pone.0143205 (opens in a new tab); PMC4657986 (opens in a new tab). Licence: CC BY 4.0 (opens in a new tab). Re-analysed by NakedSignal.

The bias sits well inside the limit chosen: it uses at most 19% of the criterion at any of the lines. Now count patients instead. Across the 529 pairs, 23 results changed category at the two lines, against about 12 expected from repeat testing alone. That expected count uses the 1.8% imprecision the study itself measured; at that imprecision the count from repeat testing alone falls between 6 and 17 with 95% probability, so the 23 crossings are about twice the expected count, and by the exact Poisson-binomial test 23 or more from repeat testing alone has probability 0.00017. That 1.8% is also our program's laboratory-analyser default (1.79%, this study's own reported precision), so the default gives about the same count; the program's cautious point-of-care default of 6%, which it uses when a point-of-care device is involved and no precision data are supplied, would give about 41, which overstates repeat-testing noise for these laboratory analysers. At the upper end of the study's own precision estimate, 2.34%, about 15 would be expected and 23 or more has probability 0.014.

Both readings are correct. The first asks whether the new method agrees with the old one within a stated limit; the second asks how many patients land on the other side of a line because of the change. A bias that is small against a proficiency testing criterion can still move a measurable number of patients near a limit, because that is where results crowd. Separating the crossings seen at a decision limit into the part imprecision alone would give and the part due to the change is a published procedure (Schmidt et al., 2015, Clin Chim Acta), which the creatinine study we use applies. What we add is a program that runs it on a laboratory’s own comparison and precision data at its own decision limits, keeps the re-test list in the laboratory and releases only aggregates.

What to add to the verification file

Bias at each decision limit, with an interval, not only the slope and intercept for the whole range. The guide to bias at decision limits shows how.
The results that crossed each limit, against those repeat testing alone would move, at the imprecision you measured in the precision step. The guide to reclassification from imprecision and bias sets out the expected count.
A note for clinicians where the count is above that expectation: which tests, which lines, from which date, and whether earlier results should be read differently. The analyser change checklist puts this step in order.

None of this replaces the laboratory’s own verification or its director’s judgement, and nothing on this page is regulatory advice.

Run it on your own pairs

The change check reads the paired results from your accuracy comparison in the browser. For one analyte, free, it gives the bias at each decision line you set and the results that crossed each line against those repeat testing alone would move. The public example runs the full report free. Your file is not uploaded.

Sources

US Code of Federal Regulations, Title 42, Part 493. § 493.1253 Standard: Establishment and verification of performance specifications. Electronic Code of Federal Regulations. www.ecfr.gov/current/title-42/section-493.1253
US Code of Federal Regulations, Title 42, Part 493. § 493.1255 Standard: Calibration and calibration verification procedures. Electronic Code of Federal Regulations. www.ecfr.gov/current/title-42/section-493.1255
US Code of Federal Regulations, Title 42, Part 493. § 493.1281 Standard: Comparison of test results. Electronic Code of Federal Regulations. www.ecfr.gov/current/title-42/section-493.1281
US Code of Federal Regulations, Title 42, Part 493. § 493.931 Routine chemistry (proficiency testing criteria for acceptable performance). Electronic Code of Federal Regulations, as amended in 2022. www.ecfr.gov/current/title-42/section-493.931
Clinical and Laboratory Standards Institute (2014). EP15: User Verification of Precision and Estimation of Bias. CLSI guideline, third edition. clsi.org/shop/standards/ep15/
Clinical and Laboratory Standards Institute (2018). EP09: Measurement Procedure Comparison and Bias Estimation Using Patient Samples. CLSI guideline, third edition. clsi.org/standards/products/method-evaluation/documents/ep09/
Clinical and Laboratory Standards Institute (2010). EP28: Defining, Establishing, and Verifying Reference Intervals in the Clinical Laboratory. CLSI guideline, third edition. clsi.org/shop/standards/ep28/
Ozarda Y, Higgins V, Adeli K (2018). Verification of reference intervals in routine clinical laboratories: practical challenges and recommendations. Clinical Chemistry and Laboratory Medicine. doi.org/10.1515/cclm-2018-0059
Schmidt RL, Straseski JA, Raphael KL, Adams AH, Lehman CM (2015). A Risk Assessment of the Jaffe vs Enzymatic Method for Creatinine Measurement in an Outpatient Population. PLoS ONE. doi.org/10.1371/journal.pone.0143205
Schmidt RL, Kordy MA, Genzen JR, Straseski JA, Greene DN, Lehman CM (2015). A mathematical procedure to estimate the impact of a change in method on discordance or misclassification at a decision limit in laboratory method comparison studies. Clinica Chimica Acta. doi.org/10.1016/j.cca.2014.10.043

Check an analyser change CLIA acceptance limits as allowable bias