The role of the External Quality Assessment

 

Quality management guidelines and practices keep evolving in the medical laboratory (1). The analytical quality still remains however the primary issue, because none of the other laboratory quality characteristics matter unless analytical quality is achieved. The term “quality control” (QC) has been introduced in the clinical laboratory setting many decades ago, and refers to the “statistical quality control that is commonly used in laboratories to monitor the routine performance of testing processes, detect possible errors, and correct problems before test results are reported” (2). In particular, internal quality control (IQC) and external quality assessment (EQA) programs are used to evaluate and continuously improve analytical quality. Currently, the termsExternal Quality Assessment (EQA) and Proficiency Testing (PT) are used interchangeably as valuable tools in the quality improvement process of clinical laboratory services (3). However, the primary objectives of EQA are educational, and may be supported by additional elements such as specific schemes aimed to extend the evaluation throughout all phases of the testing cycle, including interpretation of results. According to a widely-accepted definition, PT is “a program in which multiple samples are periodically sent to members of a group of laboratories participating for analysis and/or identification, in which each laboratory results are compared with those of other laboratories in the group and/or with assigned value, and reported to the participating laboratory and others” (3). On the other hand, the best definition of EQA is “an integrated professional quality assurance activity of medical laboratories” (4), stressing its use for self-assessment and continuous improvement of quality in medical laboratories.

PT and EQA provide objective evidence of laboratory competence for customers, accrediting bodies, and regulatory agencies, and serve as a unique source of information that is not obtainable in other ways. For medical laboratories, they have been found useful, in that they initiates a “peer-review” process towards solving technical and methodological problems to improve the quality of service for each individual laboratory as well as to achieve comparability of results among different laboratories. For accrediting bodies and regulatory agencies, they provide objective data on the quality of delivered services as EQA/PT performance has been shown to reflect the quality of patient specimen testing (5,6). The contents reported in this paper relate equally to PT and EQA, and for that reason the term EQA/PT will be used even if it is graceless.

EQA/PT is an external assessment of a laboratory performance in comparison to its peer and/or to accuracy based reference system (7). An EQA/PT survey is carried out by sending a set of samples from an organizing body to a participating laboratory, for measurement that laboratory performs in the same manner as is usually used for clinical samples. The EQA/PT samples are intended to simulate the native clinical samples usually measured by the laboratory itself. Results for the set of samples are returned to the EQA/PT organizer for evaluation of conformance to the results expected for those samples. A report is prepared by the organizer that includes the results reported by a laboratory, the method used for the measurement, the target values expected for each measurand in each sample, the distribution of results in the survey, and an evaluation on the acceptability of each result. Considerations in selecting an EQA/PT program include several elements, but first and foremost is the evidence that samples behave like patient samples. This property is commonly termed “commutability”. A commutable sample is one that has an equivalent mathematical relationship as that observed for native clinical samples between all the different measurements procedures represented in the survey (7). Commutability and its influence on EQA/PT and other reference materials have been extensively described in some reviews (8,9). Numerous investigations have reported that surrogate patient samples typically used as EQA/PT materials are not commutable with native clinical samples due to manipulation and alteration of the sample matrix during the preparation of EQA/PT materials (10,11). Because of the non-commutability limitations, EQA/PT results from suchmaterials must be compared to the mean of results from participants that use measurement procedures and diagnostic systems that are expected to have the same or very similar matrix bias. The categorization into “peer groups” that use the same or similar measurement conditions on the basis of instrument and method combinations provides useful information to a single participant but does not allow improvements in harmonization and inter-laboratory agreement of results. Moreover, traceable reference values and the extent to which EQA/PT samples have traceable reference values are strongly affected by the commutability of these materials. Therefore, the procedure to assign target values, by measuring an EQA/PT sample using a reference measurement procedure, can be safely adopted when the material is commutable with native clinical samples. In turn, due to the lack of commutability of EQA/PT samples, the most common procedure still adopted is to categorize participant methods into “peer groups” that represent similar technologies. A fundamental role in assuring reliability when comparing “peer groups” results is the number of data which is related to the number of participant laboratories. Today, however, all major hurdles to achieve this objective are fortunately crumbled and the time has come to definitively introduce commutable survey specimens. Indeed, convincing examples of ongoing EQA/PT schemes using commutable samples have been published (12,13).

Another fundamental issue in EQA/PT programs is the adoption of appropriate statistical procedures and evidence-based goals for assessing the reliability of laboratory performances. In particular, the design of reliable EQA/PT schemes based on valuable quality specifications is a pre-requisite for assuring efficiency and effectiveness to these schemes (14), and the hierarchy of models consensually defined in the 1999 Stockholm Conference represents a valuable standpoint for setting quality specifications (15).

EQA/PT samples should be tested in the same manner as patient samples, to the possible extent. In fact, these samples often require some additional preparatory manipulations that, unfortunately, may distinguish them from routine clinical samples. Some programs provide samples that closely resemble clinical samples, thereby avoiding some of these preparatory steps. When available, this feature, particularly if commutable materials are provided, should represent a competitive advantage for participant laboratories. Where special preparatory and handling procedures are required, they should be carried out according to the specific information provided by the EQA/PT provider. It should be highlighted that repeat testing of EQA/PT samples, when patient samples are tested only once, or adopting a specific test EQA/PT samples rather than rotating EQA/PT testing among all the personnel who perform patient testing, should be considered “improper procedures” as they defeat the utility of EQA/PT testing and decrease the informative value about the quality status of the true laboratory’s procedures and processes (3).

Several efforts have been done for making EQA/PT schemes web friendly in recent years. Both data entry and result review are available via internet in many programs (16). This strongly reduces time in sending and receiving EQA/PT reports. It also reduces the risk of manual transcription of data and results. However, some programs require the delivery of EQA/PT results, at least once per year, in the same format of the report used for clinical samples and patients (17). This seems to have an additional value for verifying and assessing the effective measurement units and reference values, thus allowing some interesting post-analytical considerations. Some special schemes have also been developed to assess the quality of interpretative comments which are widely recognized tools for improving the interpretation of laboratory results (18).

Several studies illustrate that the value of EQA/PT as a part of a proactive, integrated approach to laboratory quality management. Laboratories can take proactive steps such as:

a) narrowing their internal quality control ranges;

b) increasing the frequency of calibration;

c) performing instrument function verification; and

d) examining EQA/PT results closely for trends and bias, even when they are deemed acceptable.

However, EQA/PT schemes, likewise any other control system, cannot improve the analytical quality by itself. They can only, at best, tell that something is going wrong. As such, only changes in the laboratory procedures and processes may improve the quality of laboratory services. The careful evaluation of single results, both satisfactory and unsatisfactory, should be coupled with active tacking of results over time. Unsatisfactory results from a single EQA/PT event are lagging indicators of potential systemic laboratory problems as well as an individual satisfactory score in only one measure of performance, from one point in time. Ongoing monitoring of EQA/PT performance allows clinical laboratories to obtain a more complete and reliable picture of their day-to-day performance. Moreover, continuous monitoring allows clinical laboratories to take preventive action before small problems become large (3). Monitoring performance over time may reveal the impact of corrective actions or can provide information that can address preventive actions.

The visionary intuition of Belk and Sunderman in 1947 (19) has really developed an effective way for assessing the quality of laboratory performances. Pioneer experiences in the field of clinical chemistry have been progressively translated to virtually all fields of laboratory medicine, contributing to an effective improvement of analytical quality. Today EQA/PT schemes aim at improving the performances of laboratories by mean of education, metrological recommendations and standardization, taking also into account the developments in the discipline, rather than being limited to an assessment of the state-of-the-art. Many EQA/PT schemes include an assessment of the ability of laboratories to validate the medical relevance of their observations. The International Standard for accreditation of medical laboratories encourages use of EQA/PT programs that consider pre- and post-analytical sources of error. Therefore, while traditional EQA/PT schemes are shown to be useful tools for assessing and improving analytical performances, specific schemes have been developed for evaluating pre- and post-analytical processes and procedures. If the overarching and ultimate goal of EQA/PT is to enhance the quality of care and patient safety by improving the quality of laboratory services, other benefits have to be emphasized, as shown in Table 1.

 

 

More recently, data on EQA/PT performance have been used to evaluate the influence of the training, experience, and credentials of testing personnel. A retrospective study compared the relationship of EQA/PT performance with the credentials of the testing personnel, revealing that participants without a clinical laboratory science major and only 2 years or less of experience had almost twice the odds of producing errors (21).

 

 

In conclusion, the role of EQA/PT is to provide the reliable information that allows laboratories to assess and monitor the quality status of internal procedures and processes, suitability of the diagnostic systems, accountability and competence of the staff. Furthermore, it cannot forget the central role recognised to EQA/PT to define the measurement uncertainly of laboratories results (22).

The EQA/PT programs effectiveness depends on at least two different factors: design of the EQA/PT scheme (suitability of control materials, use of appropriate statistical procedures, appropriate criteria for performance evaluation, clear and unequivocal reporting of results to laboratories) and, especially, procedure used by laboratories when they receive the evaluation of its own results (analysis of report; implementation of improving and/or preventive and/or corrective actions; spreading of information to involved staff; checking of effectiveness of implemented actions).

Different studies demonstrated that only the correct use of information provided from EQA/PT produces laboratory performance improvements (23). The EQA/PT information have to highlight defects of laboratory systems and laboratory staff have to assume the responsibility to determine EQA/PT samples in the same manner as patient samples (to the extent possible), analyse the received information carefully, detect trend or bias that might not be apparent in single results, investigate the root causes that provide the performance worsening, apply and monitor the opportune actions in order to remove the cause of the problem, verify their effectiveness and, above all, determine whether the problem could have affected patient care.

The accreditation of EQA/PT programs has a fundamental role. The administrators have to validate the management of EQA/PT verifying that the laboratorychose EQA/PT schemes in agreement with the international recommendations and the national guidelines and handles the EQA/PT information correctly.