External quality assessment in resource–limited countries

Introduction Health laboratory services are a critical component of national health systems but face major operational challenges in resource-limited (RL) settings. New funding for health systems strengthening in RL countries has increased the demand for diagnostics and provided opportunities to address these constraints. An approach to sustainably strengthen national laboratory systems in sub-Saharan African countries is the Strengthening Laboratory Management Toward Accreditation (SLMTA) programme. External Quality Assessment (EQA) is a requirement for laboratory accreditation. EQA comprises proficiency testing (PT), rechecking of samples and on-site evaluation. Materials and methods A systematic literature search was conducted to identify studies addressing laboratory EQA and quality monitoring in RL countries. Unpublished reports were also sought from national laboratory authorities and personnel. Results PT schemes in RL countries are provided by commercial companies, institutions in developed countries and national programmes. Most government-supported PT schemes address single diseases using a vertical approach. Regional approaches to delivering PT have also been implemented across RL countries. Rechecking schemes address mainly tuberculosis (TB), malaria and human immunodeficiency virus (HIV); integrated rechecking programmes have been piloted. Constraints include sample transportation, communication of results, unknown proficiency of referee staff and limited resources for corrective action. Global competency assessment standards for malaria microscopists have been established. Conclusions EQA is vital for monitoring laboratory performance and maintaining quality of laboratory services, and is a valuable tool for identifying and assessing technology in use, identifying gaps in laboratory performance and targeting training needs. Accreditation of PT providers and competency of EQA personnel must be ensured.


Introduction
Medical and public health laboratory services are a critical component of national health systems and are central to disease diagnosis, treatment, prevention, surveillance and outbreak investigations (1). When used optimally, laboratory medicine generates knowledge that facilitates patient safety, improves patient outcomes and leads to more cost-effective healthcare (2). In the United States of America, laboratory testing influences 60-70% of critical decision-making in health, with community laboratories performing at least 50% of all testing (3). In the primary healthcare setting in resourcelimited (RL) countries, laboratory testing may influence 45% of medical decision-making (4).
Until recently, allocation of resources to laboratory testing was of low priority for healthcare systems in many sub-Saharan African (SSA) countries. Unreliable and inaccurate laboratory diagnostic testing has promoted the perception that laboratory testing is unhelpful and may compromise patient care (5). Major challenges facing laboratory services in under-developed settings include weak infrastructure, human capacity shortages, and lack of laboratory policies, strategic plans and integrated national quality management systems (6,7). Funding opportunities for health systems strengthening in RL countries such as the Global Health Initiative, US President's Emergency Plan for AIDS Relief, and Carter JY. EQA in resource-limited countries Global Fund to Fight AIDS, Tuberculosis and Malaria have increased demand for diagnostics and provided opportunities to address this neglect and strengthen capacity of public health laboratory networks (8).
Microscopy remains an important laboratory diagnostic procedure in RL countries and provides the basis for managing and controlling several bacterial and parasitic infections, including tuberculosis (TB) and malaria. Absent or poor malaria microscopy has long been recognised and is attributed to multiple factors, including skills of the laboratory workforce, workload, condition of microscopes and quality of laboratory supplies (9). Reyburn (12). Misclassification of malaria species occurs from lack of skills in microscopic diagnosis (13). Improving and monitoring malaria-related test performance in peripheral laboratories on a countrywide basis is achieved through training and supervision but programmes must be sustained by national commitment (14). However, focusing on improving diagnosis for one disease may lead to over-diagnosis of another illness (15). Perceptions and attitudes of healthcare providers on quality of laboratory services also affect their correct use for patient management (16).
Malaria rapid diagnostic tests (RDT) were introduced to address challenges of poor malaria microscopy and promote greater accessibility to malaria confirmation, but errors in performing RDT are widespread (17,18). Community health workers can use RDT safely and accurately for up to 12 months post-training (19). Standardised product performance evaluations that distinguish between well and poorly performing tests are also essential (20).
An innovative approach to sustainably strengthen national laboratory systems in RL countries in Africa is the World Health Organization Regional Of-fice for Africa (WHO AFRO) Stepwise Laboratory Quality Improvement Process Towards Accreditation (SLIPTA) framework linked to the Strengthening Laboratory Management Toward Accreditation (SLMTA) training programme. SLMTA was introduced in 2009 to provide a systematic, user-friendly approach to achieving laboratory quality improvement and accreditation (21) and uses a competency-based, step-wise process addressing daily laboratory operations for immediate and measurable laboratory improvement, harmonised to International Organization for Standardization (ISO) 15189 standards (8). Accreditation is the internationally accepted framework for verifying that laboratories adhere to established quality and competence standards for accurate and reliable patient testing. In some countries, laboratory accreditation is mandatory; in others, accreditation remains voluntary and may be driven by market incentives (22).
The World Health Organization (WHO) defines External Quality Assessment (EQA) as a system for objectively checking a laboratory's performance using an external agency or facility (23). EQA participation is associated with improved laboratory performance over time and is a requirement for accreditation (24). However, many professionals in SSA countries are unable to effectively implement quality improvement programmes and many countries remain without an accredited clinical laboratory (25). Establishing, maintaining and demonstrating the accuracy of diagnostic tests is a major challenge for most laboratories in SSA, and the complexity and cost of setting up and maintaining quality assurance systems mean that very few laboratories, mainly tertiary or privately owned, can achieve these standards (26,27). The common perception that EQA is costly or unnecessary has hindered the widespread enrolment of laboratories into EQA programmes (28).
EQA can be applied in three main ways, each with advantages and disadvantages: 1) Proficiency testing (PT), where an external provider sends samples of undisclosed results to laboratories or individual testers and provides feedback on results; 2) Rechecking or retesting samples in higher level or peer laboratories (inter-laboratory comparison); 3) On-site assessment by approved evaluators (23).
Carter JY. EQA in resource-limited countries

Materials and methods
A systematic search of published literature was conducted to identify studies addressing quality monitoring and laboratory performance in RL countries. RL countries were selected as countries with low and lower middle-income economies (29). Unpublished reports were sought from national laboratory authorities and personnel for further information.

Proficiency testing schemes
Several laboratories in RL countries are enrolled in international or regional PT schemes operated by commercial companies. These offer a wide range of discipline-specific schemes suitable mainly for reference or larger laboratories. Advantages include a high level of participation with different analytical methods, thereby increasing statistical validity. However, government laboratories in RL countries, unless partner supported, may be una-ble to sustain participation due to high cost. Some institutions in developed countries operate PT schemes that also support RL countries at cost. These include the United Kingdom National EQA Scheme (UK NEQAS) that provides a wide range of PT schemes and educational support; and the African Regional External Quality Assessment Scheme implemented by the National Health Laboratory Service (NHLS), South Africa, that offers a range of PT materials including stable inactivated mycobacterial dried culture spot (DCS) material for Gen-eXpert instruments performing the Xpert MTB/RIF assay (Cepheid, Sunnyvale, CA) (30).    (31). The Quality Assessment and Standardization for Immunological Measures Relevant to HIV/AIDS (QASI), Canada, was established in 1997 to provide PT for CD4 enumeration at no or low cost to RL laboratories. An impact study demonstrated PT programmes can improve overall laboratory performance despite the diversity of technologies employed (32). An impact study of CD4 EQA provided by the NHLS, South Africa, from 2002 to 2006 demonstrated how EQA provides an opportunity for post-market surveillance, standardising protocols and switching operating systems (33). A regional PT programme for TB smear microscopy conducted between 2003 and 2010 across South Asian Association for Regional Cooperation (SAARC) member states demonstrated the on-going quality of diagnostic support to countries' TB control programmes (34). The East African Regional External Quality Assessment Scheme (EA-REQAS) established in 2008 by the East African ministries of health provides integrated PT panels for basic laboratory tests. By 2015, 16 surveys had been conducted with enrolment increasing from 195 to 559 laboratories in four countries. Materials include blood slides for malaria and other blood parasites, smears for TB and Gram stain, preserved stool parasites, blood lysate for haemoglobin measurement, blood films for peripheral blood cell morphology, and human immunodeficiency virus (HIV) and syphilis serology (35

Rechecking and mixed EQA schemes
Many RL countries operate slide-rechecking schemes as part of national malaria, TB and HIV control programmes. Malaria and TB slide rechecking is performed by supervisors on-site, at next level health facilities or central reference laboratories; rechecking of HIV RDT results is usually performed at central level. Since the blanket approach of rechecking 100% positive and 10% negative acid-fast bacilli (AFB) slides was withdrawn (39), most studies reported using standard LQAS (lot quality assurance sampling) based on annual laboratory volume of AFB smears and proportion of positive smears to achieve an overall sensitivity of 75-85%, and processes for blinded rechecking. A significantly greater percentage of errors is detected on randomly selected, blinded AFB smears than on non-randomly selected, un-blinded smears; knowledge of prior results influences re-reading of TB slides (40)(41)(42).  (50). Some studies noted re-staining during rechecking at national level significantly improved sensitivity of low-positive AFB smears (41,51).
Random monthly selection of five low density and five negative malaria slides has been proposed for routine rechecking (52). A study from Rwanda reported excellent quality of malaria slide re-reading without grading of positive slides (53  five-day courses to assess competency in parasite detection, species identification and parasite quantitation conducted by an external facilitator using reference slide panels from the Research Institute for Tropical Medicine, Manila, Philippines, regional slide bank. By 2011, 60 competency assessment exercises had been conducted in 14 countries; microscopists from 5 countries showed significant improvements in performance scores (62). This programme has translated into globally recognised standards of competency in malaria microscopy (52).

Benefits
EQA schemes are valuable for recognising laboratory errors and identifying underlying problems facing peripheral laboratories (50)

Challenges
Despite the widespread use of malaria RDT in RL countries, tools available for monitoring field performance are limited. Masanja et al. compared RDT performance with malaria reference microscopy and detection of parasite DNA by real-time quantitative polymerase chain reaction (qPCR) on DBS. Malaria RDT had a higher positivity rate (6.5%) than qPCR (4.2%) or microscopy (2.9-2.5%) with poor correlation between RDT and microscopy results. Overall, agreement among the three diag-  (38) with responses using short messaging services (SMS) (17). The NICD, South Africa, scheme used express air courier (31). Motorcycles were used to support EQA of malaria microscopy in Pakistan (54). There are obvious logistical advantages to using district-level laboratories to conduct rechecking with reduced distances and costs, and ease of feedback to laboratories (58).

Costs
Published information on costs of operating EQA programmes in RL settings is limited. Khan et al. reported a capital cost of approximately 90,000 Pakistani rupees (USD 1400) to implement a district malaria-microscopy EQA scheme, including motorcycles and training of district laboratory supervisors. This amounted to a 50% increase in direct per-slide cost, but Khan argued this is marginal to the high capital and recurrent costs of microscopy services, and its value in rationalising anti-malarial drug use (54). Mukadi et al. reported the cost of conducting one PT survey in DRC was USD 10,000 excluding salaries (USD 25 per participant) (38).

Discussion
Countries with developing healthcare systems mostly lie in tropical areas where common diseases, such as malaria and diarrhoeal diseases, require immediate diagnosis (within a few hours). Coupled with poor communication across large distances, this necessitates placing laboratory testing close to where patients seek care, resulting in large numbers of small laboratories working independently. Most small laboratories still perform mainly manual assays, which are particularly prone to errors during sample collection, labelling and registration; and many laboratory staff at this level lack skills in recognising pathology, and transcribing and delivering results. Combined, these errors can lead to significant variance in the accuracy of results, leading to incorrect diagnosis, inappropriate treatment or withholding of lifesaving therapy (22). Lack of adequate resources to support these laboratory networks has resulted in equipment breakdowns, interruption of supplies and variable performance. For many laboratories across RL countries, the quality of services is unknown.
Participation in EQA is ideally required for all testing procedures performed in a laboratory. Where an established PT scheme is not available, alternate EQA mechanisms should be considered. All EQA approaches depend for their effectiveness on following national or regional protocols, good communication and feedback, and instituting corrective measures. The benefits of EQA schemes rest on mandatory participation, timely return of results with practical suggestions for corrective action and ability of participating laboratories to address deficiencies. Nothing is gained from EQA participation unless information received is directed to laboratory improvement (23).
PT programmes may be organised at national, regional and international levels and funded through government agencies, as arms of corporations, or operated on a cost-recovery basis. Most Rechecking schemes are commonly incorporated into national disease control programmes. WHO recommends integrating malaria microscopy rechecking with other microscopically diagnosed communicable diseases, which is feasible with proper coordination at peripheral and national levels (52,56). Rechecking schemes require laboratories to follow correct procedures to avoid slide selection bias. Laboratory workers may retain slides of good quality and staining regardless of instructions; some laboratories may not submit slides for rechecking due to lack of confidence in their performance or uncertainty about implications of unsatisfactory performance (47,49). Few reports indicated the competency of staff conducting the rechecking process but relied on concordance of slide reading to determine accuracy; this can be addressed by including reference centre rechecking that assesses both peripheral laboratory technicians and their immediate supervisors; and conducting regular competency assessments of slide readers. Various standards have been proposed to assess competency in TB slide reading but only malaria microscopy has an established globally recognised competency assessment programme (52,59,60). Regular competency assessment of supervisors and reference-level staff urgently needs to be incorporated into national EQA programmes.
A process of rechecking can also be applied to rapid testing assays, such as RDT for HIV, where an alternative technique, such as enzyme immunoassay (EIA) or ELISA is used on dried blood or serum spot samples (57). Rechecking of samples by peer or higher level laboratories (inter-laboratory com-parison) is appropriate for specialised tests for which no PT schemes exist, or for single unusual results; however, no published studies were identified demonstrating use or benefits of inter-laboratory comparisons in RL countries. On-site visits to laboratories by qualified auditors using standard checklists also provide a reliable EQA mechanism by implementing practical improvements to address identified gaps.
PT schemes are limited by the availability of stable PT materials that can withstand conditions of heat and humidity often found in RL countries, but have the unique advantage of being able to address uncommon pathology for which staff need to retain competence. However, PT samples are constrained by being unable to provide challenges that mimic some patient samples, such as living organisms and cells (72). PT panels may be placed within a clinical context and involve other health worker cadres in responses (73,74). Although all EQA programmes can provide inter-laboratory comparisons (benchmarking) when adequate data management systems are in place, this aspect is particularly suited to PT programmes where data are collected and analysed centrally. PT schemes use either referee laboratories or consolidated results from participants to set target values; in RL countries use of participant results may lead to lowering of precision when new participants join a scheme, but precision is usually restored once participants become more experienced (33).
Although every laboratory should treat PT samples as routine samples, this is impossible to monitor and enforce. Most laboratories in RL settings pay special attention to PT samples, especially pathology recognition by microscopy. Therefore, PT results are likely to be the very best that a laboratory can produce; poor results may reflect a worse performance under routine conditions. Participation in PT schemes should not be punitive, but regarded as an educational tool to objectively assess laboratory performance and direct improvement efforts. Regular participation is the first step to using PT as an effective tool for laboratory improvement and benefits will accrue if laboratories review results and possible causes of errors, re-examine samples, keep records of performance and use schemes as group learning exercises. Keeping health facility managers and authorities informed of reasons for poor performance provides the justification for allocating resources to maintain quality. At central level, PT schemes can provide ongoing post-market surveillance of commercial test kits, quantify errors associated with a particular technology, identify the best technologies to use, identify training needs and indicate the need for governments to validate and standardise equipment and methodologies (33).
Many published reports present unacceptable laboratory performance in EQA in RL countries, indicating the vital importance of ongoing monitoring and corrective actions. Implementation of corrective actions is primarily the responsibility of laboratory personnel and management and is dependent on established hierarchical supervisory structures. Supervisors making on-site visits can assess pre-and post-analytical aspects of laboratory procedures, and address technical performance through mentorship and ensuring functional equipment and adequate supplies; only a field visit can convey a realistic picture of the conditions under which technicians work (63). Supervisory visits are more effective when standard checklists are used systematically (75). Visits by effective supervisors are highly motivating for laboratory workers, but are time-consuming and expensive when considered across thousands of small labo-ratories; regular visits may not be sustained unless linked with PT performance to target poorly performing laboratories. Some PT schemes offer assistance with training and corrective action or link with entities that provide this support (31,33).
Novel applications using mobile technology may enhance the reach and benefits of EQA programmes in RL countries. Mobile camera phones can capture and transmit images directly from the eyepiece of an ordinary laboratory microscope to a central review site for feedback (76)(77)(78). The rapid expansion of mobile networks and internet coverage, and decreasing operational costs, offer opportunities to develop PT programmes that provide images of rare pathology or pathology that cannot be mass produced, such as histology specimens or organisms found in spleen or bone marrow. Several PT providers in developed countries are already implementing this approach.