Risk analysis and assessment based on Sigma metrics and intended use

Introduction In order to ensure the quality in clinical laboratories and meet the low risk requirements of patients and clinicians, a risk analysis and assessment model based on Sigma metrics and intended use was constructed, based on which differential sigma performance (σ) expectations of 42 analytes were developed. Materials and methods Failure mode and effects analysis was applied to produce an analytic risk rating based on three factors, each test of which was graded as follows: 1) Sigma metrics; 2) the severity of harm; 3) intended use. By multiplying the score of Sigma metrics by the score of severity of harm by the score of intended use, each was assigned a typical risk priority number (RPN), with RPN ≤ 25 rated as low risk. Low risk was defined as acceptable standards; the sigma performance expectations were calculated. Results Among the 42 analytes, tests with σ ≥ 6, 5 ≤ σ < 6, 4 ≤ σ < 5, 3 ≤ σ < 4, σ < 3 were 21, 5, 5, 6, and 5, respectively; there were 7 high-risk tests, 8 of them medium risk tests. According to the risk assessment conclusion, 13 tests had sigma performance expectations ≥ 6; 15 test items had sigma performance expectations ≥ 5, while 3 test items had sigma performance expectations ≥ 4; 11 test items had sigma performance expectations ≥ 3. Conclusions Constructing the risk analysis and assessment model based on Sigma metrics and intended use will help clinical laboratories to identify the high-risk tests more objectively and comprehensively. Such model can also be used to establish the sigma performance expectations and meet the low risk requirements of patients and clinicians.


Introduction
Laboratories have a major impact on patient safety, as 80 -90% of all diagnoses are made on the basis of laboratory tests (1). Laboratory errors have a frequency of 0.012 -0.6% for all test results (2). A series of regulatory requirements and practice guidelines have been introduced to guide the establishment and continuous improvement of the quality management system to reduce the risk of the total testing process (3)(4)(5).
Failure mode and effects analysis (FMEA), one of the most proactive methods of risk management, has been accepted as the method of choice in the identification of potential points of failure within a process, their effects being determined and action identified for mitigating failures (6). The first step in FMEA is to identify all potential possible failure modes of the product or system. After that, critical analysis is performed on these failure modes and the risk priority number (RPN) is calculated by the multiplication of the occurrence (O), severity (S) and detection (D). Finally, the failure modes can be ranked and then proper actions will be preferen-Xia Y. et al.
Risk analysis based on Sigma metrics intended use tially taken on the high-risk failure modes (7). The application of the FMEA tool is consistent with the risk-based thinking required by ISO 9001 in the critical decisions, and plays an important role in ensuring the reliability of the product system. In contrast, Shebl et al. conducted numerous interviews with hospital staff in the United Kingdom and concluded: "FMEA in health care is associated with a lack of standardization in how the scoring scales are used and how failures are prioritized." (8). Different technicians and different scoring methods yielded dissimilar results; it is a tool for which there is a lack of evidence (9). The Clinical and Laboratory Standards Institute (CLSI) EP23A guideline: Laboratory Quality Control Based on Risk Management provides an introduction to risk management techniques and guidance on developing a risk-based quality control plan (QCP) (3). The 2-factor model that includes only the probability of occurrence of harm and the severity of harm, does not consider the detection capability, is not conducive to the development of a robust laboratory QCP (10). Six Sigma is a technique that allows objective assessment of process performance. The resulting RPN on a sigma-scale is more objective, because it is less reliant on subjective rankings and more reliant on observed performance (11). Six sigma quality control (QC) design tools can enhance FMEA, the risk assessment process and design of QC plans (11).
In this study, a risk analysis and assessment model based on Sigma metrics and intended use was constructed to ensure the quality in clinical laboratories and meet the low risk requirements of patients and clinicians.

Risk assessment based on Sigma metrics and intended use
The severity of harm due to exceeding TEa was investigated via a questionnaire survey through the Internet (www.wenjuan.com). Construction of questionnaire survey referred the manuscript of "Guidelines for constructing a survey" (13 A modified FMEA was applied to produce an analytic risk rating based on three novel factors, each test of which was graded as follows: 1) Sigma metrics; 2) the severity of harm; 3) intended use (diagnosis, screening, patient management decision).

Xia Y. et al. Risk analysis based on Sigma metrics intended use
Three novel factors were in accordance with the 5 point system, as shown in Table 2. By multiplying the score of Sigma metrics by the score of severity of harm by the score of intended use, each was assigned a typical RPN. When a test had a different intended use in different clinical applications, it was classified according to the use with the highest risk score. RPN > 50 was considered high risk, the degree of risk was unacceptable; 25 < RPN ≤ 50 for the medium risk, laboratory personnel needed to pay attention to the test; and RPN ≤ 25 for low risk was here considered acceptable. According to the intended use and the accumulated score of the severity of harm, the sigma performance expectations were calculated.

Sigma metrics at the critical decision levels
The 42 clinical chemical analytes were performed on five instruments. The results of the Passing-Bablok regression and 95% confidence intervals (CI) for slope and intercept are listed in Table 1. The tests whose 95% CI for slope do not include 1 were as follows: ALT, AST, ALP, BT, Alb, AMY, TG, HDL, Ca, Mg, C3, CRP, cTn-T, pH, pO2. Most tests also had the 95% confidence interval of the y-axis intercept including zero; except for ALB, BT, HDL-C, Cl, Ca, Mg, Fe, PA, pO2, HbA1c.
The Sigma metrics for the critical decision levelmaking was calculated and listed in Table 1 and the normalized method decision chart demonstrating the sigma values was showed in Figure 1. There were 21 analytes with world class performance (σ ≥ 6). The analytes with excellent performance (5 ≤ σ < 6) were urea, IgG, IgM, cTn-I, pH; the analytes with good performance (4 ≤ σ < 5) were Glc, CHOL, Cl, Ca, PA; the analytes with marginal performance were Na, IgA, C 4 , Cys-C, cTn-T, PCO 2 and the analytes with poor or unacceptable performance (σ ≤ 3) were Alb, C3, BMG, pO2, HbA1c.

Risk analysis and assessment
A total of 52 professional personal participated in the questionnaire survey, which included 32 doctors, 12 laboratory technicians and 8 clinical biochemistry laboratory supervisors. The number of negligible, minor, serious, critical, and catastrophic was 3, 13, 14, 9 and 3, respectively; the number of diagnostic, screening, and patient management decisions tests was 14, 11, and 17, respectively. There were 7 tests including Glc, Na, Ca, BMG, cTn-T, PCO 2 and PO 2 with high-risk of RPN > 50; 8 medium risk items with 25 < RPN ≤ 50. The 5 tests with σ < 3 were evaluated as high risk or medium risk items. All of these results were shown in Table  1.

Establishing a differential sigma performance expectations
Here, 13 tests had sigma performance expectations ≥ 6; 15 tests had sigma performance expectations ≥ 5; 3 tests had sigma performance expectations ≥ 4; 11 tests had sigma performance expectations ≥ 3. The results were shown in detail in Table 3.

Discussion
When assessing quality on the σ scale, the higher the σ metric, the better the quality. Here, quality   was assessed on the σ scale with a benchmark for minimum process performance of 3σ and a goal for world-class quality of 6σ (14). There were 21 tests with σ ≥ 6 and 5 tests with σ ≤ 3 of the 42 tests in this study. When calculating Sigma metrics, the selection of appropriate TEa and analyte concentration is crucial. A study in Belgium showed the Sigma metrics of Alb ranged from 1.3 to 32 varied with analyte concentration and the TEa target (15). It is desirable that TEa is defined by the highest possible hierarchical model, and then, simple point estimates of sigma at medical decision concentrations are sufficient for laboratory applications (16)(17). However, outcome-based approaches for goal setting may not be possible to set for all analytes (18). In this study, the TEa specifications were obtained from the SVP and the EQA criteria from NCCL of China; the CV values and bias were estimated at the critical decision level and the Sigma metrics at that level was calculated.
The integration of RPN of this study is based on three novel factors of Sigma metrics, the severity and intended use. Sigma metrics are directly related to the probability of risk and they can also be indirectly associated with the detection capability of 6 sigma QC rules. Thus, the use of Sigma metrics directly determined the probability of occurrence, simplifying the process of risk assessment. The evaluation of the severity is usually highly subjective and ultimately depends on the team's experience and competence. So, the summarized data of the survey collected from clinicians and technicians in this study is benefit to making a relatively objective evaluation. Accounting for the intended use of test will also help design a comprehensive risk assessment model. For example, when HbA1c σ = 2.8, HbA1c is mainly used as patient management decisions in China, so RPN score of 45 is moderate risk. However, HbA1c was approved by the American Diabetes Association for use as a di- HbA1c agnostic indicator of diabetes, and the RPN score would therefore be adjusted to 75, which is high risk.
In this study, bias was estimated by the EQA data. However, it is several limited, such as the acceptance criteria and peer group comparison, compared to the primary method using a reference standard material (19). The intended use of the test is mainly based on the expert advisor, application guide, or reagent manual. Some of them may lack clear criteria. These problems and their solutions still need to be explored and further standardized.
At present, clinical laboratories can't achieve world class quality (σ ≥ 6) for all tests. The results of the risk assessment also showed that tests that posed negligible risk to the patient could be allowed to reach lower Sigma metrics. Identifying the differentiated sigma performance expectations can avoid repeated residual risk evaluation, which is regarded as a time-consuming task (8)(9). If one test can't achieve the sigma quality performance, it should be adjusted or changed. If intended use lowers the PRN so that the "Sigma performance expectation" isn't 6 but is only 3 or 4, that still needs to be aligned with the QC procedures implemented. Currently, a test with 3 sigma or below will need more sensitive QC rules, testing multiple QC samples at each QC event, and more frequent QC events to reducing patient risk (5,(20)(21)(22).
In conclusion, this study demonstrates that the implications of Sigma metrics can be extended beyond the QC design and method acceptability. A new RPN based on Sigma metrics and intended use have been explored, which can make a more comprehensive and objective assessment of the risk of tests. Such model can also be used to estab-   lish the Sigma performance expectations and meet the low risk requirements of patients and clinicians.

Potential conflict of interest
None declared.