External Quality Assessment (EQA) or proficiency testing (PT) programmes are intended to allow medical laboratories or other testing sites to compare the closeness of their output (test results) to their peers through statistical evaluation against an expected or target value. Participation in EQA is a key part of laboratory quality management and is essential for medical laboratories seeking accreditation to the international standard ISO 15189: 2012 Medical laboratories – Requirements for quality and competence (1). Through the provision of educational feedback and the sharing of best practice, as well as the assessment of individual laboratory performance, the EQA provider contributes to quality improvement, as does active monitoring of EQA performance and early action on any adverse trend in performance by the participating laboratory (2, 3).
EQA programmes may be set up on a local, regional, national or international basis and there are advantages and disadvantages to each of these. A local or regional service is generally more responsive and gives more rapid turnaround of reports; however, it may not be as statistically robust as a national or international service and hence less able to give as good an overview of the state of the art of laboratory performance. For specialist tests or countries with relatively few testing sites, an international service may be the only option to provide a sufficient number of participants.
Accreditation of EQA providers is to the international ISO 17043:2010 Conformity assessment –General requirements for proficiency testing standard (4). An effective EQA programme will demonstrate the following key principles:
Commutable assay material, i.e. which displays the same inter-assay properties as clinical specimens when tested by different methods or instruments
Stable, homogeneous specimens, which resemble and can be handled in the same way as clinical specimens where practicable
Reliable, valid target values traceable to certified reference materials where available
Rapid feedback of performance information to participating laboratories
Structured, informative and intelligible reports
The complete blood count (CBC) is one of the most frequently requested tests in laboratory medicine, performed using robust but complex technologies that utilise impedance and light scattering techniques to differentiate and count suspended particles (cells) on the basis of their size and physical characteristics. These technologies are optimised to produce comparable results using fresh, anticoagulated whole blood tested as soon as possible after collection. The CBC parameters covered by EQA vary by provider and scheme design but may include haemoglobin concentration (Hb), red blood cell count (RBC), haematocrit (Hct) or packed cell volume (PCV), mean cell volume (MCV), mean cell haemoglobin (MCH), mean cell haemoglobin concentration (MCHC), white blood cell count (WBC) and platelet count. To this may be added the automated differential white blood cell count, either three part (granulocyte, lymphocyte and monocyte counts) or five part (neutrophil, lymphocyte, monocyte, eosinophil and basophil counts) according to the properties of the analyser, the nucleated red blood cell count and other parameters, for example the red cell distribution width (RDW) and mean platelet volume (MPV).
The availability and choice of assay material is a key factor in the provision of analytical phase EQA; without an adequate supply of good quality assay material, the frequency of distributions and the determination of a valid target value are compromised. This article reviews the choices of assay material available for use in the external quality assessment of automated cell counting in haematology and the influence that these choices may have on performance assessment and EQA scheme design.
The ideal assay material for automated cell counting external quality assessment
The EQA provider has a demanding list of requirements for assay material (5). Whole blood from which the EQA samples are prepared must be ethically sourced and comply with all legislation applicable to both the EQA provider and the recipient. It should be available in sufficient volume and at a time or frequency convenient for the EQA rounds, free of pathogens (unless detection of these is the objective of the EQA scheme), practical to handle and available at a price that is acceptable to participating laboratories. The prepared material must homogeneous, stable both in transit and for the duration of the assay cycle and commutable, where possible. The provision of specimens from more than one assay material pool at each distribution is desirable as it allows the EQA provider to test analyser performance at different analyte concentrations in the same testing round, for example allowing the assessment of very low level platelet counting without compromising the performance assessment of counts in the ‘normal’ range. Comparison of the results from more than one specimen in an EQA round also gives the EQA provider additional information on whether an out-of-consensus result reflects a true problem of analyser performance, one of sample handling by the participant (e.g. inconsistent mixing) or may have resulted from deterioration of EQA material in transit.
The participant’s requirements are similar but the participant emphasises the desire for a specimen that resembles clinical samples as closely as possible, especially in terms of how it is handled in the laboratory. Therefore, the ideal is a material of sufficient volume and composition to allow it to be entered directly into specimen processing with as little special handling as possible. CBC material should be provided in a container that allows it to be sampled in the same way as clinical material, i.e. in a vial with a pierceable cap, which can be sampled in a ‘closed’ mode, and able to be processed in the same counting mode using the same algorithms as patients’ samples. The material above all should be clinically relevant, covering clinical decision points, testing the full linearity of the machine or method and include analyte concentrations in both routine and pathological or disease states during a full series of EQA cycles.
EQA materials are excluded from the European Union directive that regulates the production of in-vitro medical diagnostic devices (6), meaning that their production does not have to meet the same production standards as diagnostic kits, instruments and other devices used in diagnostic laboratories; however, demonstration of their fitness for purpose is an essential component of the accreditation of the EQA provider to ISO17043 (4). This fitness for purpose should include the demonstration of the stability of the assay material in transit, e.g. by testing material circulated by post and returned to the EQA provider for testing, tracking the temperature of the assay material in transit and understanding how this may affect the material stability, and the statistical review of the results returned by location or transit time/conditions. If the transport conditions are likely to be incompatible with the stability of the assay material, the EQA provider will either have to take preventative action, e.g. by the use of temperature controlled distribution networks or courier delivery, or restrict participation from locations that are likely to be adversely affected. The deterioration of assay material in transit must always be considered as a possible cause of an out of consensus EQA result.
Traceability and commutability in automated cell counting
Metrological traceability is the means by which laboratory results can be made comparable even where different calibration materials or methods of measurement are used. International standards applicable to EQA require assay material to be as close to patients’ samples as possible, traceable to a certified reference material (CRM) or certified reference method (where available) and commutable in the event that the EQA scheme is to be used for inter-method comparison (4, 7, 8). A CRM is a material used for calibration, method development or traceability that is certified as meeting the requirements in terms of homogeneity, stability and characterisation of the measurand specified in the relevant ISO standards. Materials that have been assessed as meeting these criteria and their producers are listed in the catalogues published by the Joint Research Centre (JRC) of the European Union, the European Reference Material (ERM) co-operation and also the National Institute for Standards and Technology (NIST) in the United States of America for materials they produce (9-11).
The Joint Committee on Traceability in Laboratory Medicine (JCTLM) provides a searchable database of certified reference materials and measurement methods/procedures to which the values of other standards, calibrators, controls or reference materials can be traced through an unbroken chain of calibrations (12). The stability of the cellular components of blood and the rapidly changing nature of counting technology mean that higher order reference materials and methods for automated cell counting are very few, posing a challenge to the provision of traceable EQA assay material. At the time of writing, the only applicable JCTLM-listed CRM is the haemoglobincyanide material (BCR-522, haemoglobincyanide) produced by the Institute for Reference Materials and Measurements (IRMM), available from the JRC catalogue and the only JCTLM-listed certified reference method in automated cell counting is the flow cytometric method for platelet counting, published in part 5 of the German national standard DIN 58932 (9, 13). The DIN 58932 standard is published in several parts, which cover reference procedures for the determination of the concentration of other blood corpuscles, for example part 3 for the determination of the concentration of erythrocytes (14). Other recommended methods may be published by guidelines or professional organisations, for example the International Council for Standardization in Haematology (ICSH) reference method for haemoglobinometry (15), reflecting the best practice available.
The use of EDTA anticoagulated blood as EQA assay material
The sample of choice for blood counting is fresh, whole blood anticoagulated with the di-potassium, tri-potassium or di-sodium salt of ethylenediaminetetra-acetic acid (EDTA) (16, 17), which maintains the size, integrity and appearance of cellular blood components and provides material that is commutable across different automated cell counting platforms. The stability of cellular components in EDTA for automated counting is limited: Hb concentration and RBC have been shown to be stable for up to 72 hours, automated total and differential white blood count for 24 hours (or longer, depending on analyser type) and platelet count stable for the same period (18, 19). Swelling of red cells in EDTA will result in an increase in the MCV and haematocrit within 12 hours of collection. Blood anticoagulated in EDTA is of limited value for the measurement of mean platelet volume (MPV) due to swelling of platelets unless the sample is processed within 1 to 2 hours of venesection (20).
Although EDTA-anticoagulated blood is inexpensive and easily available in the laboratory either as surplus patients’ samples or from volunteer donors, its limited stability make it impractical for automated counting EQA provision. In addition, surplus patients’ material is available in relatively small individual samples of 3 to 4.5 mL volume and potential ABO blood group incompatibility precludes preparing larger volumes through pooling. The infectious status of the samples is generally unknown. Blood donation bags containing EDTA as anticoagulant (e.g. di-potassium EDTA at a concentration of 1.5 – 2.2 mg/mL of blood) can be prepared for the venesection of a volunteer donor under appropriate medical supervision; however, the rapid deterioration of blood cells in EDTA means that the usefulness of a large volume of EDTA anticoagulated blood for EQA purposes is limited, except perhaps for a specifically designed inter-instrument or inter-laboratory comparison study.
The use of fresh EDTA blood, surplus to diagnostic testing requirements as assay material in quality assessment is suitable for ‘same day’ inter-instrument or inter-laboratory testing arrangements within a laboratory network or geographically conveniently located testing sites; however, it is unsuitable for CBC EQA on a wider basis. Many EQA providers offer stained peripheral blood films, either on glass slides or as digital images, for assessment of blood film morphology and manual white blood cell differential as part of their basic haematology provision. Blood cell morphology is maintained for several hours in EDTA, allowing the preparation of good quality stained blood films for microscopic examination (21). A fresh 3 to 5 ml EDTA patient’s sample is sufficient for the preparation of several hundred blood films, which will remain stable for long periods of time once fixed and stained using a standard haematology staining method for blood cell morphology, e.g. methanol fixation followed by staining with May Grunwald-Giemsa, or a Wrights stain.
The use of citrate phosphate dextrose anticoagulated blood as assay material
Donated blood taken into citrate phosphate dextrose (CPD) blood donation bags is a practical option as bulk, raw material for the preparation of automated counting EQA material. A full blood donation of approximately 475 mL provides 150 samples of 3 mL each and larger batches of specimens can be produced relatively easily by pooling ABO compatible blood. The EQA provider has two main sources of CPD-anticoagulated, human whole blood: local volunteers (usually members of the EQA service staff) or donations obtained from a national blood transfusion service. A major ethical concern regarding staff volunteers is undue influence from the employer, i.e. some staff may feel pressurised to volunteer as they may worry about potential adverse perception or repercussion from the employer if the refuse to donate. Venesection must be done under medical supervision in a suitably equipped facility such as a blood donor centre or haematology ward. The volume of blood available from one individual is limited to a single unit donation at a time and the preparation of a larger pool by combining donations is reliant on the availability of ABO group-compatible individuals on the staff.
The use of whole blood and blood components obtained from anonymised donors through a national blood transfusion service is a sustainable source of raw material but must be approved by the transfusion service. Donors must be aware and give consent for their blood to be used for non-clinical issue purposes, such as quality assurance, rather than therapeutic donation. Blood donors are screened by the donation service for recognised blood-borne infectious agents, reducing the risk of the EQA provider unknowingly exposing a large number of testing sites and the delivery services to any hazard contained in the material. Since blood for therapeutic donation is routinely leuco-depleted and separated into its component parts during preparation, the supply of whole blood, especially if non-leuco-depleted, will require special arrangement with the blood transfusion service.
If blood is given altruistically without payment to the donor, which is the norm in many national donation services, the EQA provider must consider the terms and conditions under which any charges are made for the EQA service. Good practice is that no charge is made for the EQA assay material and that fees are charged on a not-for-profit basis for the associated preparation, distribution and data analysis services only.
CPD anticoagulant maintains red cells for 21 days or more for Hb concentration and RBC (22). Total WBC or leucocyte counts are stable for several days but the features of the cells deteriorate rapidly with the same storage artefacts seen as in EDTA, meaning that the material is not suitable for automated white cell differential unless used promptly after venesection. Platelets are also of limited stability. Blood from ABO-compatible donors may be pooled to make a large assay material pool, although the maintenance of homogeneity during preparation and bottling becomes a greater challenge the greater the volume of the pool, requiring specialist mixing and bottling equipment (23). If used without further stabilisation, CPD anticoagulated material is commutable across different analyser platforms although early studies noted variation in the response of different counting technologies (24).
CPD anticoagulated blood used without further stabilisation requires distribution within 24 hours of venesection and analysis upon receipt by the participating laboratory to ensure good comparability of results between participants. This imposes restrictions on the scope of EQA service. The geographical range of the distribution is limited by the efficiency and cost of postal or courier service and the stability of the material poses a challenge for the testing of remote sites, those with restricted opening times, point of care testing facilities or out-of-hours services. It is not possible to provide material for re-testing by participants or to replace samples that are received damaged or accidentally discarded or damaged in the laboratory. To remove the potential of adverse impact on the participant of sample deterioration, the EQA provider may ask participants to analyse the material on the same day or analyse the results returned against the testing date to review the influence of transit and storage.
Volunteer donors, whether recruited from staff or the blood donation service, must be healthy to donate. A limitation of using healthy volunteers is the potentially limited range of analyte concentration obtained, which may not test the pathological range. The Hb concentration and the counts of the cellular components can be adjusted before bottling to simulate pathological conditions but manipulation will extend the preparation time of the assay material, delaying its distribution.
An EQA provider with a comprehensive range of surveys available may aim to consolidate as many different EQA surveys types as possible in the same distribution package to reduce delivery costs. Consolidated packing arrangements may conflict with the necessity for rapid distribution.
Donated blood taken into CPD from volunteers and used without further stabilisation is suitable for smaller scale national or regional EQA schemes with limited numbers of participants and access to good postal or courier delivery services.
The use of animal blood as EQA assay material
Animal blood has the potential for use as EQA assay material. For example, donkey blood has a mean cell volume at the low end of the human physiological range and hence mimics MCV results found in microcytic anaemia (25). Sheep blood appears naturally deficient in glucose-6-phosphate dehydrogenase activity in human terms and can be used as a source of deficient assay material (26). The venesection of living animals is not without ethical considerations related to animal welfare and may require the same level of licensing as the use of animals in clinical research. Commercial supply of blood from a limited range of domestic animal species is available in sufficient volumes for use in EQA and non-human blood or blood components may form part of commercial EQA assay materials, standards and controls.
Stabilised whole blood as EQA assay material
Initial CBC EQA material did not contain intact red cells but was prepared from dilutions of haemolysate for the measurement of haemoglobin concentration with fixed avian cells (pseudo-leucocytes) and fixed platelets. This material is very stable in transit but is suitable only for use with basic or manual techniques. Although this material cannot performance assess the measurement of red cell indices or automated differential count, it is cost effective and may still have an application in resource-limited countries (27). This material is not suitable for the performance assessment of the full range of blood count parameters produced by automated cell counters.
The stability of whole blood for use as automated counting EQA assay material can be prolonged by chemical stabilisation. Partial fixation of whole blood with aldehydes is possible for the EQA provider to undertake in-house and produces an assay material stable for several weeks (28, 29). Large volumes of assay material can be prepared by pooling ABO-compatible blood components but the fixation results in changes in the cell membranes that render the blood non-commutable across different analyser platforms, especially for MCV, WBC and platelet count (30). Haemoglobin concentration, Hct and RBC can be manipulated prior to stabilisation by mixing whole blood and red cell concentrates, plasma reduction or the addition of ABO-compatible fresh frozen plasma. White blood count and platelet count can be manipulated by mixing leuco-depleted and non-leuco-depleted blood with the addition of buffy-coat residues or platelet concentrates. The use of blood components that would otherwise be suitable for therapeutic transfusion may represent a conflict of interest and the EQA organiser must engage the support of the national blood transfusion service to ensure sustainability of supply. The preparation of stabilised CBC assay material by the EQA provider requires technical skill, experience and the availability of good laboratory facilities but is a cost effective means to produce assay material for a large number of participating laboratories. Despite the lack of full commutability across all platforms, the levels of most of the analytes are within the control of the EQA organiser and allow the organiser to challenge participants at critical analyte concentrations, for example, at decision levels for platelet transfusion and can highlight state-of-the-art variability in other parameters (31-33). The same material is distributed to all platforms in a single survey and it is possible to associate the same clinical scenarios with the material and report these back to the participants for educational purposes. This stabilised material is generally unsuitable for differential white blood count as the fixation alters the cell membrane and hence how the cells react in different counting methodologies, also white cells deteriorate in the time required for virological testing by the transfusion service (up to 72 hours). Even within the same instrument group, it may be necessary to know what mode has been used for WBC and platelet counts (optical, impedance, fluorescent or immunological) or to require the participant to report results from one mode only. This precludes allowing the instrument software to select the mode according to the count, where more than one mode is available, as it would for patients’ samples. The procedures used in the preparation of assay material, e.g. pooling, mixing, centrifugation etc., may result in the activation of platelets and interfere with non-impedance platelet counting methods (34, 35).
EQA assay material for reticulocyte counting can be prepared using the same methods as used for CBC assay material, as the fixation arrests the reticulocyte maturation, producing a material suitable although not commutable for most automated counters (note that this material is not suitable for Beckman-Coulter analysers, at the time of writing). There remains a challenge to produce material with a raised reticulocyte count from whole blood obtained from healthy donors; the reticulocyte count can be boosted with the use of buffy coat residues but the levels achieved still do not reach those found in a commercial high-level reticulocyte control.
Commercially prepared, whole blood assay material for blood cell counting is available (major global suppliers are Streck, Inc (Omaha, NE) (36) and R&D Systems, Inc (Minneapolis, MN) (37) and is widely used in EQA. Commercial material has a number of advantages for the EQA provider: it is stable for long periods of time, requires no laboratory facilities other than for storage, is available in the quantities required, is sustainable and the demonstration of ethical compliance is the responsibility of the manufacturer. However, the material may not be commutable across instrument groups for all parameters of the extended CBC and may require special handling, e.g. analysis in a quality control (QC) mode. The EQA organiser is able to choose the range of analyte levels but the choice is restricted to those produced by the manufacturer of the material and may not allow the specification of particularly low or high cell concentrations. The material is supplied with a certificate of analysis but its exact formulation and production methods are a matter of commercial confidence. The materials are frequently the same as those used for routine IQC materials and are designed to fulfil a tri-level control function. Where commercial assay materials are the same as those used as IQC material, it is essential that the EQA organiser specifies that the lots provided for EQA are different from any in circulation as IQC material during the time of the EQA cycle. Despite these drawbacks, commercial material may be the only choice for the EQA provider for some automated counting parameters, e.g. automated differential leucocyte count or raised reticulocyte count, where the preparation of the material is beyond the scope of the EQA provider and may require the use of artificial particles or animal cells.
Stabilised blood, whether prepared by the EQA provider or purchased commercially, is suitable for schemes with large numbers of participating laboratories and where delivery times may be prolonged, for example where services are provided internationally. Preparation of stabilised assay material in-house gives the EQA provider greatest control over analyte concentrations and hence the ability to respond to perceived performance trends but is not suitable for the performance assessment of automated differential leucocyte counting and or reticulocyte counting at high levels and for all technologies. Commercially prepared, whole blood specimens may be the materials of choice for an international scheme or one scheme with large participant numbers. However, it is significantly more expensive than many of the other options and this may preclude its use by some providers.
Discussion and conclusions
A demonstrated lack of commutability of assay material requires grouping of instruments for performance analysis. These groupings become more complex where instrument group specific material types are supplied, e.g. as may be required for automated differential leucocyte counting, and entail grouping by material type with additional sub-grouping by instrument model in some instances. The EQA organiser must take into account information and advice from the manufacturers of both the assay material and the instruments and empirical evidence from participants’ results to decide on the correct instrument grouping for analysis.
The analytical performance for most of the consolidated parameters in the CBC is recognised as very good, therefore the distribution of EQA material that focuses solely on the normal range for Hb concentration, WBC, RBC etc. to fully automated laboratories may be of questionable clinical utility, failing to provide challenges at the levels of clinical decision making and giving a false sense of security in performance (38). The use of non-commutable material however may have the same impact, where instrument grouping that results in only comparing analysers on a ‘like with like’ basis may not reflect true variation in performance. Immediate decisions are made on the basis of the results of the CBC, especially where the instruments are placed in acute care situations, and the provision of EQA for point-of-care instruments as well as those in automated laboratories is essential (39). Whether or not the scheme design includes POCT-placed instruments will affect the choice of assay material, since these instruments may not be located to allow the immediate testing of fresh material, for example they may be in remote sites, in clinics that do not operate a daily service or in community based services or emergency vehicles. EQA material capable of demonstrating comparability between different instruments is likely to become more difficult in the future with the introduction of new cell counting technologies, for example, image analysis as used by Bloodhound™ technology and point-of-care instruments utilising cartridge based microfluidics (40-42). For POCT methodologies designed for use with fresh, fingerprick blood, the haemoglobin assay method may not convert methaemoglobin to a measurable form, making this an interfering substance in stored EQA assay material and requiring the separate grouping of instruments for performance assessment.
Where the numbers of registered participants in a group is small (i.e. fewer than the ideal 10-20 instruments for statistical analysis), it is possible to evaluate performance against the all methods mean target value, if all are using the same material, even if this is not commutable. This is not ideal as the instrument may have a bias against this target that is due solely to the nature of the assay material; however, if the instrument’s performance is consistent in that bias, it is still possible for its performance to be assessed if the EQA organiser reviews and defines individual minimum acceptable performance limits for the instrument and the variability that is allowed. This approach is not possible where instrument-specific assay material is distributed. Particular EQA statistics have been described to assess performance for scheme with small numbers of participants (43).
Other parameters of the extended CBC are finding increasing clinical application, for example the reticulocyte haemoglobin concentration in the diagnosis of iron deficiency in patients with chronic renal disease (44-46). Laboratories seeking to include these new parameters in their ISO15189 accreditation scope will have to demonstrate inter-laboratory comparability for their results and are likely to prefer EQA provision to developing alternative approaches such as sample exchange.
External quality assessment for the complete blood count is an established part of comprehensive medical EQA provision. The ideal assay material that resembles clinical material and is commutable across all analyser platforms for all parameters in automated counting does not exist and the EQA provider must select the most ‘fit for purpose’ for the objectives of the scheme and be aware of the impact that this may have on the evaluation of participating laboratories’ performance. The ‘fit for purpose’ criteria must take into account the range of analysers and types of testing sites participating in the scheme, as well as the transport conditions to which the assay material will be exposed, to ensure continued clinical utility of the EQA service.