Vanja Radisic Biljak1*, Ivan Ozvald1, Andrea Radeljak1, Kresimir Majdenic2, Branka Lasic3, Zoran Siftar1, Marijana Vucic Lovrencic1, Zlata Flegar-Mestric1. Validation of a laboratory and hospital information system in a medical laboratory accredited according to ISO 15189. Biochemia Medica 2012;22(1):86-91. http://dx.doi.org/10.11613/BM.2012.009
1Merkur University Hospital, Institute of Clinical Chemistry and Laboratory Medicine, Zagreb, Croatia
2Labnet software engineering, Josipovac, Croatia
3Grad d.o.o. computer engineering, Pula, Croatia
*Corresponding author: Vanja [dot] Radisic [dot] Biljak [at] idb [dot] hr
Introduction: The aim of the study was to present a protocol for laboratory information system (LIS) and hospital information system (HIS) validation at the Institute of Clinical Chemistry and Laboratory Medicine of the Merkur University Hospital, Zagreb, Croatia.
Materials and methods: Validity of data traceability was checked by entering all test requests for virtual patient into HIS/LIS and printing corresponding barcoded labels that provided laboratory analyzers with the information on requested tests. The original printouts of the test results from laboratory analyzer(s) were compared with the data obtained from LIS and entered into the provided template. Transfer of data from LIS to HIS was examined by requesting all tests in HIS and creating real data in a finding generated in LIS. Data obtained from LIS and HIS were entered into a corresponding template. The main outcome measure was the accuracy of transfer obtained from laboratory analyzers and results transferred from LIS and HIS expressed as percentage (%).
Results: The accuracy of data transfer from laboratory analyzers to LIS was 99.5% and of that from LIS to HIS 100%.
Conclusion:We presented our established validation protocol for laboratory information system and demonstrated that a system meets its intended purpose.
Key words: accreditation; medical informatics; data transfer; laboratory information system; hospital information system; validation
Medical laboratories are the key partners in patient safety. Laboratory results influence 70% of medical diagnoses (1). The quality of laboratory service is the major factor that directly affects the quality of health care (2). Accreditation according to an internationally recognized standard such as ISO 15189 „Medical laboratories – Particular requirements for quality and competence“ is a recognition of a laboratory’s competence and compliance with the requirements of the standard (3). It is acknowledged as the single most effective route to comprehensive quality assurance (4).
Improving quality of the work process is one of the main goals of accredited laboratories, and development of a laboratory information system is a precondition for quality improvement in this phase of laboratory process (5,6). The Institute of Clinical Chemistry and Laboratory Medicine of the Merkur University Hospital, Zagreb, Croatia, has successfully installed the Laboratory Information System (LIS) (7,8), and the Department of Laboratory Medicine, a part of the Institute, is additionally integrated into the Hospital Information System (HIS). The ISO 15189 standard requires that the computer software be addressed in the same way as the instruments (9). One of the newest accreditation requirements includes recommendations for the protection of LIS – Annex B. Amongst numerous demands, the laboratory management must ensure maximum security of data entry and reports (B5.) (10). To respond to the particular requirement of the accreditation standard for medical laboratories, the accuracy of the entire process of data transfer between laboratory equipment and laboratory (LIS) and hospital information (HIS) systems needs to be validated and appropriately documented.
The aim of this study was to establish a protocol for retrospective validation of laboratory and hospital information systems in the Institute of Clinical Chemistry and Laboratory Medicine in response to ISO 15189 requirements, as described in Annex B, section B5.
Considering the fact that only four out of more than 200 medical-biochemistry laboratories in Croatia have been accredited to ISO 15189 in the field of medical biochemistry (11) and that, to the best of our knowledge, validation process in Croatian medical laboratories has not been previously described, this paper could be of great assistance to other laboratory management in meeting accreditation requirements in the future.
Materials and methods
The Institute of Clinical Chemistry and Laboratory Medicine comprises two departments, the Department of Clinical Chemistry consisting of four units, and the Department of Laboratory Medicine consisting of three units. The Institute performs testing in the areas of clinical chemistry, laboratory hematology and coagulation, cell immunology-and immunophenotyping, and molecular diagnostics. The accuracy of data transfer from laboratory equipment to LIS was validated in the entire Institute of Clinical Chemistry and Laboratory Medicine, both in the Department of Clinical Chemistry and the Department of Laboratory Medicine. The applications used were LIS – BioNet (Labnet software engineering, Josipovac, Croatia) and HIS – BIS (Grad d.o.o. computer engineering, Pula, Croatia). The accuracy of data transfer from LIS to HIS was validated in the Department of Laboratory Medicine according to the reference documents (12-14). The main outcome measure was transfer accuracy expressed as percentage (%). Our acceptance criteria were a complete match between numeric data obtained from laboratory analyzers and results transferred into LIS, and between data entered into LIS and then into HIS. An additional acceptance criterion regarding data transfer from LIS to HIS was a complete match of comments entered by laboratory personnel.
Validation protocol was designed in cooperation with vendors of applications used according to the available literature (10,12-14) described in detail below. Validation of data transfer into LIS and HIS was carried out from 31. 03. 2009 till 13. 05. 2009 in the Department of Clinical Chemistry and from 20. 12. 2010 till 23. 02. 2011 in the Department of Laboratory Medicine. All data were processed in Microsoft Excel provided templates.
1. Transfer of data from LIS to laboratory equipment and back to LIS
Transfer of data from laboratory equipment into LIS was checked separately at each department. The analyzers used are presented in Table 1.
Table 1. List of analyzers included in validation.
The whole process was performed by entering virtual demographic data with all test requests into LIS and printing corresponding barcoded labels that provided laboratory analyzers with the information on requested tests. A total of about 200 tests at the Department of Clinical Chemistry and 180 tests at the Department of Laboratory Medicine in the areas of clinical chemistry, laboratory hematology and coagulation, cell immunology and immunophenotyping, and molecular diagnostics were included in the validation study. Results of the following tests were calculated: HbA1c (both percentage and SI units), HDL-3 cholesterol, LDL-cholesterol, VLDL-cholesterol, functional tests (albumin to creatinine ratio, creatinine clearance), and quantitative analysis of 24-h urine samples (albumin, total protein; electrolytes: potassium, sodium, chloride, calcium, inorganic phosphorous; urea, uric acid; cortisole). Barcoded labels were printed in duplicate for further purposes. Data from several virtual patients with different demographic data to cover a broad field of gender- and age-specific reference intervals were entered. Entries were made for virtual patients of both genders: 14 patients (aged 21-105 yrs) at the Department of Clinical Chemistry and 12 patients at the Department of Laboratory Medicine (aged 24-84 yrs). Labeled samples were placed on laboratory analyzers. All tests were performed from a pool of non-infective human samples. After the analysis, data obtained by transfer to LIS were verified and validated by authorised personnel (medical biochemist) and printed in the form of a laboratory report. The original printouts of the test results from laboratory analyzer(s) were compared with the LIS-derived report. All patient demographic data as well as their test results and comments from laboratory analyses were entered into the template provided by the information application vendor. Examples of the comments were as follows: “H” for high or result above the corresponding reference interval; “L” for low or result below the corresponding reference interval. Barcoded labels were also stuck on provided templates. The accuracy of all entered data, both demographics and test results, and their matches, was verified by signatures of the validator (medical biochemist) and the manager of the company which created and installed the application. The whole procedure was also performed for laboratory analyzers that were not connected to LIS with a remark that the data were manually imported into LIS. Calculated test results from LIS were also validated using only a calculator and formulas required to calculate the results. Both data from the thus verified results and those from the analyzer outside the LIS network were entered into similar templates and their accuracy confirmed by signatures of the validator (medical biochemist) and the vendor manager. The whole validation process was finalized by a thorough description of the application protocol, servers and specifications of users’ computers with installed laboratory information system.
2. Transfer of data from HIS to LIS and back to HIS
The transfer of data was examined in the Department of Laboratory Medicine, the only department integrated into HIS. HIS and LIS are connected by a Health Level 7 (HL7)-protocol. HL7 specifies the number of flexible standards, guidelines and methodologies by which various healthcare systems can communicate with each other. Such guidelines or data standards are a set of rules that allow information to be shared and processed in a uniform and consistent manner (15). Validation was performed by entering data on four virtual patients of various ages (10-54 yrs) and both genders into HIS, requesting all tests in HIS and creating real data in a finding generated in LIS. At this point there was no real sample testing because data transfer from analyzer to LIS had been previously validated. Considering that HIS environment enables screen view of the laboratory test data, a print screen from HIS with completed laboratory test results was made upon validating virtually created data in LIS. A hard copy of the laboratory report from LIS and a print screen from HIS were compared. Demographic data and test results obtained from both LIS and HIS, together with previously described comments, were entered into a corresponding template and the accuracy of their match verified by signatures of the validator (medical biochemist) and the software company manager. The entire validation process was finalized by a description of a HL7 protocol for connecting LIS and HIS.
The validation process was performed in the entire Institute of Clinical Chemistry and Laboratory Medicine of the Merkur University Hospital. The personnel authorized to validate data obtained were medical biochemists employed at the Institute. All obtained data were validated by authorised medical biochemists without employing auto-validation. The entire validation documentation is archived at the Institute’s secretariat.
1. Transfer of data from LIS to laboratory equipment and back to LIS
There were some discrepancies observed during the process. As software of some of the analyzers cannot include all reference intervals used in every day routine work, results on such analyzers were not adequately labelled when being out of reference intervals. Such labels differed between analyzers (arrows, H or L, asterix), while in LIS they were marked as H or L (in the Department of Clinical Chemistry), aligned to the left if the result was below reference interval or to the right if the result was above reference interval (in the Department of Laboratory Medicine). All these remarks were entered into provided templates with additional explanation if necessary. Four analyzers in the Institute (two in each department) did not have printers integrated or connected to the interface, for which an additional comment was added.
Nevertheless, the described discrepancies were not among our acceptance criteria because they reflected instrumentation characteristics rather than data transfer accuracy. Discrepancies in numeric data transfer were found for two tests, in which printouts of analyzer-derived data analyzer had different number of decimal places from those of data transferred to LIS, the accuracy of data transfer thus being about 99.5 %. However, a careful analysis of these two cases showed that LIS and the analyzer expressed results differently (as percentage or a whole number, respectively), for which an additional explanation was supplied in templates. Considering the fact that calculated test results from LIS also showed no discrepancies, it was concluded that complete match of data between laboratory analyser and LIS was achieved.
2. Transfer of data from HIS to LIS and back to HIS
No discrepancies were observed in the transfer of data from LIS to HIS either in the numeric data transfer or in the written comment transfer. The accuracy of the transfer from LIS to HIS was 100%. All data with corresponding comments entered into provided templates were checked and verified by an authorised medical biochemist. There was no need for corrective actions. The whole documentation including completed templates with laboratory findings and original printouts from analyzers or print screens from HIS were sent to our LIS and HIS vendors. The owners of the company, after carefully examining the data, confirmed the accuracy of transferred data by signing each completed template.
The Institute of Clinical Chemistry and Laboratory Medicine, Merkur University Hospital, Zagreb, Croatia, implemented accreditation standards in every day routine work. New accreditation requirements include Annex B with eight requirements in the B5. section regarding safety of data entry and reports. A protocol for validation was defined and the validation process performed at the Departments of Clinical Chemistry and Laboratory Medicine of the Institute of Clinical Chemistry and Laboratory Medicine. The most important problem we encountered was the difficulty in performing retrospective validation of a computer software, since this kind of validation has not been previously described in detail (16,17). There are some “what-to” but not “how-to” articles (18), and the requirements of the ISO 155189 standard are too general, leaving it to laboratories to establish a protocol and standards for the validation process and associated documentation (19). Although initial system validations were performed by vendors of commercial computer software, a documented system validation demonstrates to all concerned that the laboratory information system manages information well, with the expected accuracy and reliability, file integrity, auditability and management control (20,21). An important issue in the process is the massiveness of documentation acquired during the process, as almost 400 different tests were used. Supporting documentation such as screen prints or laboratory findings serve to document the success or failure of the test case. Some discrepancies observed in the process, already described in the previous section, were carefully examined by the authorised medical biochemist and software vendors, and were concluded not to interfere with daily workflow nor the accuracy of final laboratory finding. Although this procedure is very comprehensive and time-consuming, it is mandatory in achieving accreditation requirements as set in the Annex B of the ISO15189 Accreditation Standard.
Limitations to this study were that there among the eight sections in Annex B of the ISO 15189 Accreditation Standard our study focused only on B5. section that is related to safety of data entry and reports, whereas there are other features of LIS that require validation. In addition, data for young children reference intervals were not taken into account. However, this protocol for validation process, developed at the Institute of Clinical Chemistry and Laboratory Medicine, ensures traceability of data transferred within and between LIS and HIS, and ensures that the equipment and laboratory information systems used meets their intended purpose, minimizing some types of medical errors in laboratory diagnostics.
The authors are thankful to Mrs. Lovorka Perkovic for professional language editing.
Potential conflict of interest
1. Forsman RW. Why is the laboratory an afterthought for managed care organizations. Clin Chem 1996;42:813-6.
2. Lippi G, Simundic AM, Mattiuzzi C. Overview on patient safety in healthcare and laboratory diagnostics. Biochem Med 2010;20:131-43.
3. Zima T. Accreditation in clinical laboratories. Biochem Med 2010;20:215-20.
4. Harper JC, SenGupta S, Vesela K, Thornhill A, Dequeker E, Coonen E, et al. Accreditation of the PGD laboratory. Hum Reprod 2010;25:1051-65.
5. Flegar-Mestric Z, Nazor A, Perkov S, Surina B, Kardum-Paro MM, Siftar Z, et al. Accreditation of medical laboratories in Croatia – experiences of the Institute of clinical chemistry, University Hospital “Merkur”, Zagreb. Coll Antropol 2010;34:181-6.
6. Pantanowitz L, Henricks WH, Beckwith BA. Medical Laboratory Informatics. Clin Lab Med 2007;27:823-43.
7. Flegar-Mestric Z, Kardum-Paro MM, Siftar Z, Perkov S, Sikirica M, Ozvald I, et al. The impact of an electronic information system on clinical laboratory efficiency and prevention of identification errors. Clin Chem Lab Med 2009;47(Special Suppl):S257.
8. Siftar Z, Flegar-Mestric Z, Skegro D, Kraljevic S, Trumbic Z. Result of laboratory informatizatin implemented into hospital informatics system in the Merkur University Hospital, Zagreb, Croatia. Farmacevtski vestnik 2004;55:331-2.
9. Yanikkaya-Demirel G. ISO 15189 accreditation: Requirements for quality and competence of medical laboratories, experience of a laboratory II. Clin Biochem 2009;42:279-83.
10. Medical laboratories – Particular requirements for quality and competence (ISO 15189:2007; EN ISO 15189:2007).
11. [Registar akreditacija]. Available at: http://www.akreditacija.hr/registar Accessed September 1st 2011. (in Croatian)
12. Pearson S, Balis UJ, Fuller J, Kowalski B, Locke AP, Tillman D, Vantu QH. AUT08-A:Vol.26, No 36: Managing and validating laboratory information systems; Approved Guideline. Available at: http://www.clsi.org/source/orders/free/AUTO8-A.pdf. Accessed September 1st 2011.
13. Hawker CD, Agrawal Y, Balis UJ, Catalasan IM, Charache P, Vantu QH, Wagar EA. AUTO12-A:Vol 31, No 7: Specimen Labels: Content and Location, Fonts, and Label Orientation; Approved Standard. Available at: http://www.clsi.org/source/orders/free/auto12-a.pdf. Accessed September 1st 2011.
14. Health Level Seven Standard Version 2.5 - An application Protocol for Electronic Data Exchange in Healthcare Environments, ANSI/HL7 V2.5-2003.
15. Health_Level_7. Available at: http://en.wikipedia.org/wiki/Health_Level_7. Accessed July 12th, 2011.
16. Hoffmann A, Kähny-Simonius J, Plattner M, Schmidli-Vckovski V, Kronseder C. Computer system validation: An overview of official requirements and standards. Pharm Acta Helv 1998;72:317-25.
17. Bund C, Heinemann GW, Jäger B, Trinkler M. Validation of a customized LIMS. Pharm Acta Helv 1998;72:349-56.
18. Friedli D, Kappeler W, Zimmermann S. Validation of computer systems: Practical testing of a standard LIMS. Pharm Acta Helv 1998,72:343-8.
19. Turner E, Bolton J. Required steps for the validation of a Laboratory Information Management System. Qual Assur 2001;9:217-24.
20. Cowan DF, Gray RZ, Campbell B. Validation of the laboratory information system. Arch Patol Lab Med 1998;122:239-44.
21. Scott C, Hengesbaugh JH. Validating your laboratory information system. Med Lab Obs 1998;30:36-40.
Validacija laboratorijskog i bolničkog informacijskog sustava u medicinskom laboratoriju akreditiranom prema ISO 15189
Uvod: Cilj našeg rada je prikazati razvoj protokola za validaciju laboratorijskog informacijskog sustava (LIS) i bolničkog informacijskog sustava (BIS) u Zavodu za kliničku kemiju i laboratorijsku medicinu Kliničke bolnice Merkur, Zagreb, RH.
Materijali i metode: Validacija sljedivosti podataka ispitana je zadavanjem svih pretraga u BIS/LIS virtualnom pacijentu pri čemu su ispisivane odgovarajuće barkodirane naljepnice. Izvorni ispisi rezultata mjerenja sa laboratorijskih analizatora uspoređivali su se sa podacima iz LIS-a i unosili u odgovarajuće predloške. Prijenos podataka iz LIS-a u BIS aplikaciju provjeravao se zadavanjem svih pretraga u BIS-u te kreiranjem realnih podataka u nalazu koji se formira u LIS-u. Dobiveni podaci iz LIS-a i BIS-a su upisivani u odgovarajuće predloške. Glavna mjera ishoda je bila točnost prijenosa podataka od laboratorijskih analizatora preko LIS-a do BIS-a iskazana kao postotak (%).
Rezultati: Točnost prijenosa podataka sa laboratorijske opreme u LIS je iznosila 99,5%, a iz LIS-a u BIS 100%.
Zaključak: Predstavili smo validaciju protokola za laboratorijski informacijski sustav i pokazali da sustav u potpunosti zadovoljava svoju prvotnu svrhu.
Ključne riječi: akreditacija; medicinska informatika; prijenos podataka; laboratorijski informacijski sustav; bolnički informacijski sustav; validacija