Contact

Daria Pašalić
Editor-in-Chief
Department of Medical Chemistry, Biochemistry and Clinical Chemistry
Zagreb University School of Medicine
Šalata ul 2.
10 000 Zagreb, Croatia
Phone +385 (1) 4590 205; +385 (1) 4566 940
E-mail: dariapasalic [at] gmail [dot] com

Useful links

Special issue: Quality in laboratory diagnostics: from theory to practice

Laura Sciacovelli, Sandra Secchiero, Lorena Zardo, Mario Plebani*. The role of the External Quality Assessment. Biochemia Medica 2010;20(2):160-4. http://dx.doi.org/10.11613/BM.2010.019
 
Department of Laboratory Medicine, University Hospital, Padova, Italy; Center for Biomedical Research, Castelfranco Veneto (TV) and Leonardo Foundation, Abano Terme Hospital, Abano Terme, Padova, Italy
*Corresponding author: mario [dot] plebani [at] unipd [dot] it
 
Abstract
 
External Quality Assessment (EQA) and Proficiency Testing (PT) are valuable tools in the quality improvement process. They provide objective evidence of laboratory competence for customers, accrediting bodies, and regulatory agencies, and serve as a unique source of information that is not obtainable in other ways. Traditional EQA/PT schemes tend to address only the analytical process (examination procedures), but some innovative schemes have been recently introduced to evaluate both pre- and post-analytical activities of the medical laboratory. The most important considerations in selecting an EQA/PT program include the evidence that: (i) samples are commutable (i.e. they behave like patient samples), (ii) reference values are traceable, (iii) method comparability with available peer groups is allowable, (iv) the size of peer group is statistically appropriate, and (v) both the timeliness and the usefulness of reports is granted. Careful evaluation of a single EQA/PT result should be coupled with active tracking of all results over time, and monitoring performances represent a challenging issue for medical laboratories. It is also important to consider that every EQA/PT scheme presents some limitations and that it is not appropriate to use EQA/PT as the sole means for evaluating laboratory quality. Therefore, there is the need to underline that internal quality control (IQC), EQA/PT and other tools have to be implemented and used to monitor and improve the quality in laboratory diagnostics.
Key words: External Quality Assessment; proficiency testing; laboratory performance; analytical quality; analytical quality control
Received: March 23, 2010
Accepted: May 1, 2010
 
Introduction
 
Quality management guidelines and practices keep evolving in the medical laboratory (1). The analytical quality still remains however the primary issue, because none of the other laboratory quality characteristics matter unless analytical quality is achieved. The term “quality control” (QC) has been introduced in the clinical laboratory setting many decades ago, and refers to the “statistical quality control that is commonly used in laboratories to monitor the routine performance of testing processes, detect possible errors, and correct problems before test results are reported” (2). In particular, internal quality control (IQC) and external quality assessment (EQA) programs are used to evaluate and continuously improve analytical quality. Currently, the terms External Quality Assessment (EQA) and Proficiency Testing (PT) are used interchangeably as valuable tools in the quality improvement process of clinical laboratory services (3). However, the primary objectives of EQA are educational, and may be supported by additional elements such as specific schemes aimed to extend the evaluation throughout all phases of the testing cycle, including interpretation of results. According to a widely-accepted definition, PT is “a program in which multiple samples are periodically sent to members of a group of laboratories participating for analysis and/or identification, in which each laboratory results are compared with those of other laboratories in the group and/or with assigned value, and reported to the participating laboratory and others” (3). On the other hand, the best definition of EQA is “an integrated professional quality assurance activity of medical laboratories” (4), stressing its use for self-assessment and continuous improvement of quality in medical laboratories.
PT and EQA provide objective evidence of laboratory competence for customers, accrediting bodies, and regulatory agencies, and serve as a unique source of information that is not obtainable in other ways. For medical laboratories, they have been found useful, in that they initiates a “peer-review” process towards solving technical and methodological problems to improve the quality of service for each individual laboratory as well as to achieve comparability of results among different laboratories. For accrediting bodies and regulatory agencies, they provide objective data on the quality of delivered services as EQA/PT performance has been shown to reflect the quality of patient specimen testing (5,6). The contents reported in this paper relate equally to PT and EQA, and for that reason the term EQA/PT will be used even if it is graceless.
Key elements of External Quality Assessment schemes
EQA/PT is an external assessment of a laboratory performance in comparison to its peer and/or to accuracy based reference system (7). An EQA/PT survey is carried out by sending a set of samples from an organizing body to a participating laboratory, for measurement that laboratory performs in the same manner as is usually used for clinical samples. The EQA/PT samples are intended to simulate the native clinical samples usually measured by the laboratory itself. Results for the set of samples are returned to the EQA/PT organizer for evaluation of conformance to the results expected for those samples. A report is prepared by the organizer that includes the results reported by a laboratory, the method used for the measurement, the target values expected for each measurand in each sample, the distribution of results in the survey, and an evaluation on the acceptability of each result. Considerations in selecting an EQA/PT program include several elements, but first and foremost is the evidence that samples behave like patient samples. This property is commonly termed “commutability”. A commutable sample is one that has an equivalent mathematical relationship as that observed for native clinical samples between all the different measurements procedures represented in the survey (7). Commutability and its influence on EQA/PT and other reference materials have been extensively described in some reviews (8,9). Numerous investigations have reported that surrogate patient samples typically used as EQA/PT materials are not commutable with native clinical samples due to manipulation and alteration of the sample matrix during the preparation of EQA/PT materials (10,11). Because of the non-commutability limitations, EQA/PT results from such materials must be compared to the mean of results from participants that use measurement procedures and diagnostic systems that are expected to have the same or very similar matrix bias. The categorization into “peer groups” that use the same or similar measurement conditions on the basis of instrument and method combinations provides useful information to a single participant but does not allow improvements in harmonization and inter-laboratory agreement of results. Moreover, traceable reference values and the extent to which EQA/PT samples have traceable reference values are strongly affected by the commutability of these materials. Therefore, the procedure to assign target values, by measuring an EQA/PT sample using a reference measurement procedure, can be safely adopted when the material is commutable with native clinical samples. In turn, due to the lack of commutability of EQA/PT samples, the most common procedure still adopted is to categorize participant methods into “peer groups” that represent similar technologies. A fundamental role in assuring reliability when comparing “peer groups” results is the number of data which is related to the number of participant laboratories. Today, however, all major hurdles to achieve this objective are fortunately crumbled and the time has come to definitively introduce commutable survey specimens. Indeed, convincing examples of ongoing EQA/PT schemes using commutable samples have been published (12,13).
Another fundamental issue in EQA/PT programs is the adoption of appropriate statistical procedures and evidence-based goals for assessing the reliability of laboratory performances. In particular, the design of reliable EQA/PT schemes based on valuable quality specifications is a pre-requisite for assuring efficiency and effectiveness to these schemes (14), and the hierarchy of models consensually defined in the 1999 Stockholm Conference represents a valuable standpoint for setting quality specifications (15).
Sample handling procedures
EQA/PT samples should be tested in the same manner as patient samples, to the possible extent. In fact, these samples often require some additional preparatory manipulations that, unfortunately, may distinguish them from routine clinical samples. Some programs provide samples that closely resemble clinical samples, thereby avoiding some of these preparatory steps. When available, this feature, particularly if commutable materials are provided, should represent a competitive advantage for participant laboratories. Where special preparatory and handling procedures are required, they should be carried out according to the specific information provided by the EQA/PT provider. It should be highlighted that repeat testing of EQA/PT samples, when patient samples are tested only once, or adopting a specific test EQA/PT samples rather than rotating EQA/PT testing among all the personnel who perform patient testing, should be considered “improper procedures” as they defeat the utility of EQA/PT testing and decrease the informative value about the quality status of the true laboratory’s procedures and processes (3).
Web data entry and review
Several efforts have been done for making EQA/PT schemes web friendly in recent years. Both data entry and result review are available via internet in many programs (16). This strongly reduces time in sending and receiving EQA/PT reports. It also reduces the risk of manual transcription of data and results. However, some programs require the delivery of EQA/PT results, at least once per year, in the same format of the report used for clinical samples and patients (17). This seems to have an additional value for verifying and assessing the effective measurement units and reference values, thus allowing some interesting post-analytical considerations. Some special schemes have also been developed to assess the quality of interpretative comments which are widely recognized tools for improving the interpretation of laboratory results (18).
Monitoring EQA/PT performance
Several studies illustrate that the value of EQA/PT as a part of a proactive, integrated approach to laboratory quality management. Laboratories can take proactive steps such as:
a) narrowing their internal quality control ranges;
b) increasing the frequency of calibration;
c) performing instrument function verification; and
d) examining EQA/PT results closely for trends and bias, even when they are deemed acceptable.
However, EQA/PT schemes, likewise any other control system, cannot improve the analytical quality by itself. They can only, at best, tell that something is going wrong. As such, only changes in the laboratory procedures and processes may improve the quality of laboratory services. The careful evaluation of single results, both satisfactory and unsatisfactory, should be coupled with active tacking of results over time. Unsatisfactory results from a single EQA/PT event are lagging indicators of potential systemic laboratory problems as well as an individual satisfactory score in only one measure of performance, from one point in time. Ongoing monitoring of EQA/PT performance allows clinical laboratories to obtain a more complete and reliable picture of their day-to-day performance. Moreover, continuous monitoring allows clinical laboratories to take preventive action before small problems become large (3). Monitoring performance over time may reveal the impact of corrective actions or can provide information that can address preventive actions.
Current achievements and future perspectives
The visionary intuition of Belk and Sunderman in 1947 (19) has really developed an effective way for assessing the quality of laboratory performances. Pioneer experiences in the field of clinical chemistry have been progressively translated to virtually all fields of laboratory medicine, contributing to an effective improvement of analytical quality. Today EQA/PT schemes aim at improving the performances of laboratories by mean of education, metrological recommendations and standardization, taking also into account the developments in the discipline, rather than being limited to an assessment of the state-of-the-art. Many EQA/PT schemes include an assessment of the ability of laboratories to validate the medical relevance of their observations. The International Standard for accreditation of medical laboratories encourages use of EQA/PT programs that consider pre- and post-analytical sources of error. Therefore, while traditional EQA/PT schemes are shown to be useful tools for assessing and improving analytical performances, specific schemes have been developed for evaluating pre- and post-analytical processes and procedures. If the overarching and ultimate goal of EQA/PT is to enhance the quality of care and patient safety by improving the quality of laboratory services, other benefits have to be emphasized, as shown in Table 1.
 
Table 1. Benefits of EQA programs (from reference 20)
 
More recently, data on EQA/PT performance have been used to evaluate the influence of the training, experience, and credentials of testing personnel. A retrospective study compared the relationship of EQA/PT performance with the credentials of the testing personnel, revealing that participants without a clinical laboratory science major and only 2 years or less of experience had almost twice the odds of producing errors (21).
 
Conclusion
 
In conclusion, the role of EQA/PT is to provide the reliable information that allows laboratories to assess and monitor the quality status of internal procedures and processes, suitability of the diagnostic systems, accountability and competence of the staff. Furthermore, it cannot forget the central role recognised to EQA/PT to define the measurement uncertainly of laboratories results (22).
The EQA/PT programs effectiveness depends on at least two different factors: design of the EQA/PT scheme (suitability of control materials, use of appropriate statistical procedures, appropriate criteria for performance evaluation, clear and unequivocal reporting of results to laboratories) and, especially, procedure used by laboratories when they receive the evaluation of its own results (analysis of report; implementation of improving and/or preventive and/or corrective actions; spreading of information to involved staff; checking of effectiveness of implemented actions).
Different studies demonstrated that only the correct use of information provided from EQA/PT produces laboratory performance improvements (23). The EQA/PT information have to highlight defects of laboratory systems and laboratory staff have to assume the responsibility to determine EQA/PT samples in the same manner as patient samples (to the extent possible), analyse the received information carefully, detect trend or bias that might not be apparent in single results, investigate the root causes that provide the performance worsening, apply and monitor the opportune actions in order to remove the cause of the problem, verify their effectiveness and, above all, determine whether the problem could have affected patient care.
The accreditation of EQA/PT programs has a fundamental role. The administrators have to validate the management of EQA/PT verifying that the laboratorychose EQA/PT schemes in agreement with the international recommendations and the national guidelines and handles the EQA/PT information correctly.
 
References
 
1. Westgard JO. Managing quality vs. measuring uncertainty in the medical laboratory. Clin Chem Lab Med 2010;48: 31-40.
2. Westagard JO. Internal quality control: planning and implementation strategies: Ann Clin Biochem 2003;40:593-611.
3. Clinical and Laboratory Standards Institute (CLSI). Using Proficiency Testing to improve the clinical laboratory; Approved Guideline-Second Edition. CLSI document GP27-A2. Clinical and Laboratory Standards Institute, 940 West Valley Road, Suite 1400, Wayne, Pennsylvania 19087-1898 USA, 2007.
4. Uldall A. Origin of EQA programmes and multidisciplinary cooperation between EQA programme organizers within laboratory medicine. EQA News 1997;8:1-27.
5. Hoeltge GA, Duckworth JK. Review of proficiency testing performance of laboratory accredited by the College of American Pathologists. Arch Pathol Lab Med 1987;111:1011-4.
6. Jenny RW, Jackson KY. Proficiency test performance as a predictor of accuracy of routine patient testing for theophylline. Clin Chem 1993;39:76-81.
7. Miller WG. The role of proficiency testing in achieving standardization and harmonization between laboratories. Clin Biochem 2008;42:232-8.
8. Miller WG. Specimen materials, target values and commutability for external quality assessment (proficiency testing) schemes. Clin Chim Acta 2003;327:25-37.
9. Miller WG, Myers GL, Rej R. Why commutability matters. Clin Chem 2006;52:553-4.
10. Bock JL, Endres DB, Elin RJ, Wang E, Rosenzweig B, Klee GG. Comparison of fresh frozen serum to traditional proficiency testing material in a College of American Pathologists survey for ferritin, folate, and vitamin B12. Arch Pathol Lab Med 2005;129:323-7.
11. Schreiber We, Endres DB, MCDowell GA, Palomacki GE, Elin RJ, Klee GG. Comparison of fresh frozen serum to proficiency testing material in College of American Pathologists surveys: a-fetoprotein, carcinoembryonic antigen, human chorionic gonadotropin, and prostate-specific antigen. Arch Pathol Lab Med 2005;129:331-7.
12. Thienpont LM, Stöckl D, Friedecky B, Kratochvila J, Budina M. Trueness verification in European external quality assessment schemes: time to care about the quality of the samples. Scand J Clin Lab Invest 2003;63:195-202.
13. Baadenhuijsen H, Kuypers A, Weykamp K, Cobbaert C, Jansen R. External Quality Assessment in The Netherlands: time to introduce commutable survey specimens. Lessons from the Dutch “Calibration 2000” project. Clin Chem Lab Med 2005;43:304-7.
14. Sciacovelli L, Zardo L, Secchiero S, Plebani M. Quality specifications in EQA schemes: from theory to practice. Clin Chim Acta 2004;346:87-97.
15. Petersen HP, Fraser CG, Kallner A, Kenny D. Strategies to set gobal analytical quality specifications in laboratory medicine. Scand J Clin Lab Invest 1999;59:475-585.
16. Bais R. EQA from an Australian perspective. Clin Biochem 2007;28:175-8.
17. Sciacovelli L, Secchiero S, Zardo L, Zaninotto M, Plebani M. External quality assessment: an effective tool for clinical governance in laboratory medicine. Clin Chem Lab Med 2006;44:740-9.
18. Sciacovelli L, Zardo L, Secchiero S, Zaninotto M, Plebani M. Interpretative comments and reference ranges in EQA programs as a tool for improving laboratory appropriateness and effectiveness. Clin Chim Acta 2003;333:209-19.
19. Belk WP, Sunderman FW. A survey of the accuracy of chemical analyses in clinical laboratories. Am J Clin Pathol 1947; 17:853-61.
20. Plebani M, Sanzari MC, Zardo L. Quality control in coagulation testing. Semin Thromb Hemost 2008;34:642-6.
21. Delost MD, Miller WG, Chang GA, Korzun W, Nadder TS. Influence of credentials of clinical laboratory professionals on proficiency testing performance. Am J Clin Pathol 2009;132:550-4.
22. Panteghini M. Application of traceability concepts to analytical quality control may reconcile total error with uncertainty of measurement. Clin Chem Lab Med 2010;48:7-10.
23. Sciacovelli L, Secchiero S, Zardo L, D’Osualdo A, Plebani M. Risk management in laboratory medicine: quality assurance
     programs and professionals competence. Clin Chem Lab Med 2007;45:756-65.
 
 
 
Sažetak
 
Uloga vanjske procjene kvalitete rada medicinsko biokemijskih laboratorija
 
Vanjska procjena kvalitete rada (engl. External Quality Assessment, EQA) i ispitivanje osposobljenosti (engl. Proficiency Testing, PT) su vrijedni alati u procesu poboljšanja kvalitete. Oni pružaju objektivne dokaze laboratorijske osposobljenosti prema korisnicima, akreditacijskim i regulativnim tijelima te služe kao jedinstven izvor informacija do kojih se inače ne bi moglo doći. EQA/PT programi tradicionalno obuhvaćaju samo analitičke procese (procedure ispitivanja), međutim nedavno su predstavljeni neki od novijih programa koji procjenjuju također prije- i poslijeanalitičku fazu laboratorijske analize unutar medicinskog laboratorija. Pri odabiru EQA/PT programa treba se voditi računa da: (i) uzorci u svim značajkama odgovaraju humanim uzorcima (odnosno da se ponašaju kao uzorci bolesnika), (ii) da su referentne vrijednosti sljedive; (iii) da je moguća usporedba metode prema dostupnoj skupini metoda; (iv) da je veličina skupine metoda statistički odgovarajuća; te (v) da je zajamčeno da izvješća s rezultatima stižu na vrijeme te da su korisna. Pažljiva procjena pojedinog EQA/PT rezultata treba se moći upariti s aktivnim praćenjem rezultata tijekom vremena. Praćenje rada laboratorija predstavlja izazov za medicinske laboratorije. Također treba uzeti u obzir činjenicu da svaki EQA/PT program ima određena ograničenja te da nije primjereno imati EQA/PT program kao jedino sredstvo procjene kvalitete laboratorija. Stoga je potrebno naglasiti kako se unutarnja kontrola kvalitete (engl. internal quality control, IQC), EQA/PT i drugi postupci moraju provoditi i koristiti u praćenju i poboljšanju kvalitete rada laboratorijske dijagnostike.
Ključne riječi: vanjska procjena kvalitete rada; ispitivanje osposobljenosti; rad laboratorija; analitička kvaliteta; kontrola analitičke kvalitete