Implementation of criteria for automatic release of clinical chemistry test results in a laboratory at an academic public hospital

1. Hospital das Clínicas da Universidade Federal de Minas Gerais (HC/UFMG), Minas Gerais, Brazil
2. Falcudade de Medicina da UFMG, Minas Gerais, Brazil

DOI: 10.5935/1676-2444.20160026

Corresponding author

Leonardo S. Vasconcellos
Av. Professor Alfredo Balena, 190, sala 403; Santa Efigênia
CEP: 30130-100; Belo Horizonte-MG, Brasil
Phone: +55 (31) 3409-9774; cell phone: +55 (31) 99105- 6929
e-mail: leonardos_vasconcellos@yahoo.com.br

First Submission on 2/12/2016

INTRODUCTION

Autoverification is a process of automatic release of laboratory results with no need for human intervention. A software automatically assesses test results based on criteria established by the laboratory, and releases these results without the intervention of a qualified professional^(1-4).

According to the document of the Clinical and Laboratory Standards Institute (CLSI) – Autoverification of Clinical Laboratory Tests Results; Approved Guideline (AUTO10-A) -, autoverification criteria can be defined by the user and adapted to different settings and services. They may include reference values, internal quality control results, instrument flagging, delta check (comparison of the current exam results against another previous result of the same patient, if there is one), reagent lot checks, demographic information about patients, clinical information, critical values and others⁽³⁾. In some more sophisticated systems, comments may also be inserted in medical records, based on patterns of laboratory results^{(5, 6)}.

Implementing autoverification leads to standardized result verification criteria, reduction of turnaround time (TAT)⁽²⁾, optimization of laboratory staff working time, allowing them to focus on results that really need a more detailed interpretation, and discuss important cases with the ordering physicians. Communication of critical results, assurance of the quality of the released results, and, greater safety to the patients cared for^{(3, 7, 8)} are other gains of autoverification.

The rules of autoverification may be inserted in the middleware and/or in the laboratory information system (LIS). Middleware is the software that links LIS and instruments^{(2, 5)}.

In most services, processes of validation and verification of results take place in a manual form, even with the widespread use of computers in laboratories. Results are released individually, and they may reach hundreds per shift. This task demands qualified professionals, and it is ruled by law⁽⁹⁾, because it is a critical moment, in which phases prior to test conduction are reviewed. It is time consuming and highly subjective, depending on knowledge and experience of the professional releasing results. A good strategy to minimize these variables is the implementation of automatic result release^{(4, 5)}.

Autoverification in laboratory medicine services may expedite result release and ensure greater safety to patients when it permits that truly altered results be reviewed with the necessary standards of care^{(2, 8, 10, 11)}.

Unfortunately, the literature is scarce in works detailing how to implement rules and algorithms for the automatic release of laboratory results^{(8, 12-14)}, mainly in the national scenario⁽¹⁵⁾.

The objective of the current work was to describe the implementation of autoverification in biochemical tests of the emergency test menu at the core laboratory of Hospital das Clínicas da Universidade Federal de Minas Gerais (HC/UFMG).

MATERIAL AND METHODS

Core laboratory of HC/UFMG

The HC/UFMG is a public university tertiary general hospital affiliated to the Unified Health System (UHS), and it provides care in all medical specialties, except radiotherapy. It is a national reference in transplantations, oncologic treatments and chemotherapy, maternity ward and high-risk nursery, among others. It has a 509-bed installed capacity, with 144 beds distributed between the intensive care unit and the emergency department.

At the core laboratory of HC/UFMG, the clinical release of results is carried out by legally able, trained professionals, such as physicians, biochemists, or biomedical scientists⁽⁹⁾. The criteria used for release are personal and depend on the professional’s technical formation and experience. During this step, professionals use their knowledge and mental algorithms for the detection of any pre-analytical, analytical or post-analytical error that can impair final results. In the absence of result inconsistencies, reports are electronically released so that they can be consulted online or printed, and delivered to the patient or the attending physician (Figure 1).

Figure 1 – Flow of test conduction at the core laboratory of HC/UFMG
HC/UFMG: Hospital das Clínicas da Universidade Federal de Minas Gerais; LIS: laboratory information system.

Definition of tests to be autoverified

The chemistry assays of the emergency test menu that underwent autoverification were those carried out in instruments Vitros^®5.1 and 5600, interfaced with LIS, available 24 hours a day, seven days a week. Internal quality control had already become fully interfaced, and all actions were taken and records related to controls were kept in the middleware.

The 26 autoverified tests were: albumin, alanine transaminase (ALT), amylase, aspartate transaminase (AST), uric acid, bilirubin assays, calcium, total creatine phosphokinase (CPK), chlorine, total cholesterol, high-density lipoprotein (HDL)-cholesterol and low-density lipoprotein (LDL)-cholesterol, creatinine, alkaline phosphatase, phosphorus, gamma-glutamyltransferase (GGT), glucose, lactate, D-lactate dehydrogenase, magnesium, C-reactive protein (CRP), total proteins and fractions, potassium, sodium, triglycerides, and urea.

Development of algorithms of automatic release

The algorithms of automatic release were developed according to the guidelines of the document AUTO10-A⁽³⁾. A general algorithm was designed for the defined tests, and the values for each test were established, in each step. As results are released in two steps – in the middleware and in the LIS -, the algorithm must foresee the actions in each one of them.

Middleware design

In the analytical phase, release criteria regarding integrity of the biological sample were set in the middleware, as well as equipment errors, interference with the reaction, and quality control. The module of quality control in the middleware was set to not carry out (block) the test, in case the results of internal control were not within the established intervals and approved by a professional trained in the system⁽¹⁶⁾.

Since the instruments read the lipemia, turbidity and hemolysis indices of each sample, it was possible to set the flag (alert sign) in the middleware, corresponding to each interferent, if it were present in enough quantity to interfere with the test result. The interference limit of each of them for each test was set in the equipment by the manufacturer, and can be sent to the middleware. When they are present, a new sample must be ordered.

Besides sample quality and quality control flags, instrument error flags – which could release anomalous results or reports without results, so as to block the test – were also set. Specific flags for some tests were set, which demand action from releasers, and block results.

In addition, tests with values outside the analytical linearity were retained for later dilution or investigation, when applicable. CRP, in its turn, was set to permit that results below linearity were released.

Definition of delta check values

Firstly, the delta check values of each analyte were established, and then the release algorithm was built in the LIS. In order to calculate the delta check value of each analyte, the reference change value (RCV)⁽¹²⁾ was adopted, using the formula: RCV = 2^1/2* Z* [CV_A² + CV₁²]^1/2, with: Z = 1.96 (95% significance) or 2.58 (99% significance); CV_A= analytical coefficient of variation of the test; and CV_I = coefficient of intraindividual biological variation.

CV_A was calculated based on the average value of all coefficients of variation (CV), in all levels of control, during the year 2014. CV_I was obtained based on literature data⁽¹⁷⁾.

Calculation of the automatic release interval

The automatic release interval was defined as the average distance between the midpoint of the reference range and the low and high linear measurement limits of each test⁽²⁾. All results within this interval and with no previous test to be compared with were programmed to be released automatically, as follows:

Test: albumin

Reference range: 3.5-5 g/dl

Linearity range: 1-6 g/dl

• midpoint of the reference range: (3.5 + 5)/2 = 4.25 g/dl

• low limit of automatic release: [(4.25 – 1)/2] + 1 = 2.62 g/dl

• high limit of automatic release: [(6 – 4.25)/2] + 4.25 = 5.1 g/dl

When reference values were distributed by age group and sex, the automatic release interval was altered based on the reference range. When results exceeded the interval of critical values defined by the laboratory, the critical values were adopted as limits, since they must be immediately reported to physicians. In other exams, when the limit of the release interval was within the adopted reference range, it was decided to keep the limit of the reference range.

For triglycerides, the automatic release was defined to be kept up to 399 mg/dl. If the result were > 400 mg/dl, the sample would be retained for LDL to be measured. In samples with triglycerides < 400 mg/dl, total cholesterol and HDL were measured, and LDL and very low density lipoprotein (VLDL) were calculated based on these three parameters, using the classic Friedewald equation.

LIS configuration

In the LIS, values of delta check and automatic release interval were configured. All the results previously released in the middleware were submitted to the LIS criteria. Rules were firstly applied just in results of one instrument and in two analytes at a time. The automatic release of each analyte was gradually enabled, after release of the first morning routine, relative to inpatients.

Validation was done with the professionals that acted in the technical area, responsible for result release. Before configuring the system, rules were applied manually in tests already released to detect any discrepancies and possible inadequate releases. For each test, values within and outside the defined release range, as well as delta check flags, were verified.

Middleware and LIS were provided by Matrix Sistemas e Serviços^® (São Paulo-SP, Brazil), MatrixMiddleware^® 2.3.12 Release Build 7 (middleware) and MatrixDiagnosis^® 2015.1.3.17 (Laboratory Information System [LIS]).

Design of a flowchart for manual result release

After the implementation of automatic release, a flowchart was designed to standardize the manual release of tests retained by the LIS. All the professionals at the core laboratory of HC/UFMG involved in test release were trained to deal with this flow.

RESULTS

Automatic release algorithm

An algorithm was developed for implementation of autoverification, which can be used in medical chemistry tests performed in instruments Vitros^®5.1 and 5600. Some rules were mandatorily configured in the middleware; and others, in the LIS (Figure 2).

Figure 2 – Algorithm of automatic release used at the core laboratory of HC/UFMG
HC/UFMG: Hospital das Clínicas da Universidade Federal de Minas Gerais; LIS: laboratory information system.

Delta check results

Two delta check values were calculated for each analyte: RCV (95%) and RCV (99%). Results higher than 99% significance were retained. Values between 95% and 99% significance were released with a flag for the supervisor (Table 1).

Automatic release interval

The automatic release intervals calculated for configuration in the LIS can be seen in Table 2. When limits exceeded the critical values, these were adopted.

Flowchart of manual release of results

The flowchart created for the manual release of tests retained at the LIS is represented in Figure 3. Its final form was defined with those in charge of test release and the technical sector medical coordination at the core laboratory of HC/UFMG.

Figure 3 – Manual release flowchart of tests retained by autoverification at the core laboratory of HC/UFMG
HC/UFMG: Hospital das Clínicas da Universidade Federal de Minas Gerais.

DISCUSSION

This study describes the implementation of autoverification in medical chemistry tests in a laboratory that serves outpatients and inpatients.

At the core laboratory of HC/UFMG, from January to October 2015, an average of 142 thousand tests per month was released, with 42 thousand (30%) inpatients and 100 thousand (70%) outpatients. Out of this total of tests, 78 thousand (56%) were chemistry assays, which may be ordered as a matter of urgency or routine, from inpatients and outpatients.

The chosen tests are part of the emergency menu of HC/UFMG, represent a significant fraction of the total released tests (56%) and are performed in two similar instruments interfaced with the LIS. The whole process of internal quality control of these tests was already interfaced with the middleware and incorporated into routine work, conditioning the beginning of autoverification to the approval of internal controls by supervisors. Initiating the process by them, a positive immediate impact can be felt in TAT reduction and the improvement of result release, as described by other authors^{(2, 8, 13)}.

The design and implementation of the autoverification algorithm followed the guidelines described in the document AUTO10-A⁽³⁾, which advocates that the place for inclusion of autoverification algorithms must be based on the characteristics and the resources of each system, as well as on the expertise and the preference of the involved professionals.

In the literature, a good part of the services also configures the rules of autoverification in the middleware and the LIS⁽¹³⁾. However, there are authors that opted to configure such rules preferably in the middleware⁽⁸⁾; and others, just in the LIS⁽²⁾.

In our service, a general algorithm was designed to be applied to all tests, and the parameterization of autoverification algorithms occurred in both the middleware and the LIS.

Flowchart programming for result autoverification is available in a good part of laboratory softwares, whose rules follow mathematical principles, such as Boolean logic (in the form of if… then statements), in a specific order^{(15, 18)}. At the core laboratory of HC/UFMG, the enforcement of such rules, in the middleware (MatrixMiddleware^®) and the LIS (MatrixDiagnosis^®) was not complex. Both softwares already present specific fields for the inclusion of delta check values and automatic release intervals, which can be differentiated as to patients’ sex and age. As to test linearity, values provided by the manufacturer, validated in the service, were adopted. On the other hand, the system proved limited for programming of Boolean rules that are more complex for the definition of automatic release intervals, with different determining factors in relation to patients’ origin or clinical profile, for example.

Aiming at patient safety, it was fundamental that the autoverification process was associated with quality internal controls^{(6, 16)}. At the core laboratory of HC/UFMG, there was no automatic result release without previous approval of internal controls recorded in the middleware, MatrixConnect^®.

Sample alerts or flags were set in the equipment itself and sent to the middleware. In the current service, Vitros^® equipment routinely read icterus, hemolysis and turbidity indices in biological samples. The presence of these interferents in specific concentrations that acknowledgely alter the analyte measure resulted in sample recollection, as demonstrated in the autoverification algorithm.

Equipment flags were also set according to the material provided by the equipment manufacturer. Some tests have specific reading flags, principally when there is a mechanical problem and error in the conduction of a test or final reading of the reaction, what causes result retention.

In the automation instruments at the core laboratory of HC/UFMG, the dilution protocols of each test were already developed. In them, the sample linearity flag was also important, because it informed the operators about the necessity to return with it to the equipment for dilution. All these conducts were foreseen in the autoverification algorithm. The tests that did not have any alert were automatically released to the LIS, MatrixDiagnosis^®.

In the literature, delta checks are also frequently used in autoverification algorithms^{(2, 3, 8, 12, 19)}. They can be produced from studies of pair result exams in a population similar to that in which it will be applied. Differences between the tests are plotted in a frequency distribution histogram. The values to be used are chosen in 1% or 5% of the encountered values. In the present work, a 99% (p < 0.01) confidence interval (CI) was used to calculate the RCV to be inserted as delta check in each test.

Delta check values can also be established according to the service experience, and adjusted in an empirical form. RCV, based on probability, analytical precision and/or intraindividual variation, also proved an objective measure for evaluation of seried results⁽¹²⁾. Fraser et al. (2002) obtained around 60% of autoverified tests, using delta check and the clinical decision limit as criteria for automatic release⁽¹²⁾. However, the theoretical disadvantage of using just this formula as a criterion is the occurrence of false positives in sick individuals, as CV_I was established in healthy individuals⁽¹⁷⁾.

In some autoverification systems, information on patient’s origin (outpatients or inpatients), besides clinical information, can also be used as criteria for result release^{(3, 20)}. Although HC/UFMG serves inpatients and outpatients of several departments, patients’ origin (outpatient or inpatient) was not considered in algorithms for result autoverification in the present work, because the used LIS neither presented this functionality nor enabled change by the laboratory. The choice was, then, extending the reference range for automatic release, since delta check values were adopted in all tests, as well as other barriers that ensured safety of the released results⁽⁸⁾. In the exams in which the interval exceeded the critical values, the decision was to establish the interval in critical values to assure immediate communication of medical results⁽²⁾.

In the literature, there is no consensus over the interval to be considered for results to be released automatically. There are laboratories that adopted the linearity range as the release limit⁽¹³⁾. Shih et al. (2011) used the limits of the linearity range, but in the exams with a large interval, they conducted a statistical study of prevalence using values between 2%-98%⁽⁸⁾.

One of the benefits of result autoverification is to ensure that all results go through the same process, and are governed by the same assessment rules, before being released, thus increasing process consistency and patient safety^{(2, 3, 8, 20)}.

The flowchart for release of results retained at the LIS was designed to assure consistency in the manual release of these exams. The actions to be taken were decided based on the cause for retention of the exam at the LIS. Those responsible for the release of these exams also participated in the development of the flow and were enabled for this function.

Although it was not the object of study of the current article, another advantage of the automatic release described in the literature is the reduction of TAT. Torke et al. (2005) reported reduction in 22% of test TAT after implementation of autoverification⁽²⁾. At the core laboratory of HC/UFMG, the percentage of released exams in up to 120 minutes changed from around 50% in the pre-autoverification period to around 83% post-implementation. Outpatient tests, released in up to two hours, changed from around 4% to around 60%. The service also had reduction of available manforce, principally the involved professional enabled for result release.

CONCLUSION

The criteria and algorithms for automatic result release were defined and implemented for clinical chemistry tests of the emergency menu at HC/UFMG. The processes of manual release of the exams retained by the LIS were also standardized and implemented with the creation of a flowchart and the training of the involved personnel. The employment of autoverification in emergency chemistry assays at the core laboratory of HC/UFMG enabled standardization and safety in result release.

REFERENCES

1. Duca DJ. Autoverification in a laboratory information system. Lab Med. 2002; 33: 21-5.

2. Torke N, Boral L, Nguyen T, Perri A, Chakrin A. Process improvement and operational efficiency through test result autoverification. Clin Chem. 2005; 51: 2406-8. PubMed PMID: 16306113.

3. CLSI. Autoverification of clinical laboratory tests results. Approved Guideline (AUTO10-A). Wayne, PA, USA; 2006.

4. Johnstone LM. A practical approach to the implementation of autoverify. Accred Qual Assur. 2004; 9: 155-8.

5. Guidi GC, Poli G, Bassi A, Giobelli L, Benetollo PP, Lippi G. Development and implementation of an automatic system for verification, validation and delivery of laboratory test results. Clin Chem Lab Med. 2009; 47: 1355-60. PubMed PMID: 19817645.

6. Implementation of criteria for automatic release of clinical chemistry test results in a laboratory at an academic public hospital Jones JB. A strategic informatics approach to autoverification. Clin Lab Med. 2013; 33: 161-81. PubMed PMID: 23331736.

7. Pearlman ES, Billelo L, Stauffer J, Kamarinos A, Miele R, Wolfert MS. Implications of autoverification for the clinical laboratory. Clin Leadersh Manag Rev. 2002; 16: 237-9. PubMed PMID: 1268427.

8. Shih MC, Chang HM, Tien N, Hsiao CT, Peng CT. Building and validating an autoverification system in the clinical chemistry laboratory. Lab Med. 2011; 42: 668-73.

9. ANVISA. Resolução da diretoria colegiada RDC nº 302, de 13 de outubro de 2005.

10. Davis GM. Autoverification of the peripheral blood count. Lab Med. 1994; 25: 528-31.

11. Davis GM. Autoverification of macroscopic urinalysis. Lab Med. 1999;30: 56-60.

12. Fraser CG, Stevenson MP, Kennedy IMG. Biological variation data are necessary prerequisites for objective autoverification of clinical laboratory data. Accred Qual Assur. 2002; 7: 455-60.

13. Krasowski MD, Davis SR, Drees D, et al. Autoverification in a core clinical chemistry laboratory at an academic medical center. J Pathol Inform. 2014; 5: 13. PubMed PMID: 24843824.

14. Martinez-Nieto O, Lozano-Gaitán A, Beltran-Diaz P, Mojica-Figueroa IL, Morales-Reyes OL, Isaza-Ruget MA. Autoverification of the automated blood cell counter (CBC) in a reference laboratory in Bogota, Colombia. J Bras Patol Med Lab. 2015; 51: 369-75.

15. Melo MR, Melo KC, Faria CDC. Utilidade de regras booleanas aplicadas à liberação de resultados de exames hormonais e marcadores tumorais. J Bras Patol Med Lab. 2006; 42: 252-64.

16. Westgard JO. Taking autoverification to the next level – up or down? In: Westgard JO, editor. Nothing but the truth about quality. US: Westgard QC; 2004. p. 103-10.

17. Minchinela J, Ricós C, Perich C, et al. Biological variation database, and quality specifications for imprecision, bias and total error (desirable and minimum). The 2014 update [Internet]. Available at: http://www.westgard.com/biodatabase-2014-update.htm. Accessed on: 12 fev 2016.

18. Dorizzi RM, Caruso B, Meneghelli S, Rizzo TTP. The DNSev^TM expert system in the autoverification of tumor markers and hormones results. Accred Qual Assur. 2006; 11: 303-7.

19. Straseski JA, Strathmann FG. Patient data algorithms. Clin Lab Med. 2013; 33: 147-60. PubMed PMID: 23331735.

20. Valdiguié PM, Rogari E, Philippe H. VALAB: expert system for validation of biochemical data. Clin Chem. 1992; 38: 83-7. PubMed PMID: 1733611.