Prevalência de exames laboratoriais desnecessários e custos evitáveis associados em unidade de terapia intensiva
Anderson Magalhães Oliveira, Marcio Vasconcelos Oliveira, Claudio Lima Souza
DOI: 10.5935/1676-2444.20140049
ABSTRACT
Introduction: Because of the increase in diagnostic resources, laboratory tests have become an essential tool in diagnostic elucidation. Therefore, we observe an increase in the number of tests request. Objectives: To determine the prevalence of laboratory tests requests by the intensive care unit (ICU) of the General Hospital at southwest Bahia, and to anticipate the expenses with unnecessary laboratory tests, indicating avoidable costs, and aiming at rationalization of laboratory tests use. Methods: This is a cross-sectional retrospective study, in which a survey on medical records and laboratory tests results of patients admitted to the ICU was conducted, in the period from August to September 2013. Results: The sample consisted of 105 patients, 58.1% males, 47.6% aged between 18-59 years. During the period, 12.217 laboratory tests were ordered, of which 49.4% was within the normal limits. The more requested exams were: complete blood count, sodium, and potassium. A number of 1.750 laboratory tests could be evaluated according to criteria established in the literature, among them, 719 (41%) were considered unnecessary. A greater number of requests (29%) were observed on Mondays, compared with the average of the rest of the week. Conclusion: Significant number of unnecessary testing was found, with occurrence pattern on Monday and not dependent on age or length of stay. The data indicate the need to implement guidelines or protocols for ordering laboratory tests in the ICU, which have proven effective in helping the prescribing professional in their clinical practice, and contribute to optimize for health care spending.
Keywords: laboratory tests, intensive care unit, unnecessary testing, cost-cutting
RESUMO
Introdução: Com o aumento dos recursos diagnósticos, os exames laboratoriais têm se tornado ferramenta indispensável nas elucidações diagnósticas. Observa-se, portanto, um crescimento nas solicitações desses exames. Objetivos: Determinar a prevalência das solicitações de exames laboratoriais da unidade de terapia intensiva (UTI) de um hospital geral e estimar os valores gastos com exames desnecessários, apontando custos evitáveis e visando à racionalização do uso deles. Métodos: Trata-se de um estudo de corte transversal, no qual foi realizado levantamento de dados de prontuário e resultados de exames laboratoriais de internados em UTI, no período de agosto a setembro de 2013. Resultados: A amostra foi composta por 105 pacientes, sendo 58,1% do sexo masculino e 47,6% com faixa etária entre 18-59 anos. Durante o período, foram solicitados 12.217 exames laboratoriais, sendo 49,4% dentro da faixa de normalidade. Os exames mais solicitados foram hemograma, sódio e potássio. Puderam ser avaliados 1.750 exames laboratoriais, segundo critérios já estabelecidos na literatura, sendo 719 (41%) considerados desnecessários. Foi observado um maior número de solicitação (29%) nas segundas-feiras, em relação à média dos demais dias da semana. Conclusão: Constatou-se importante montante de exames desnecessários, com padrão de ocorrência na segunda-feira e não dependente de faixa etária ou tempo de internação. Os dados sinalizam para a necessidade de implementação de diretrizes ou protocolos para solicitação de exames laboratoriais em UTI, que têm se mostrado eficientes em auxiliar o profissional prescritor em sua prática clínica, além de contribuir para otimização de gastos com a saúde.
Palavras-chave: exames laboratoriais, unidade de terapia intensiva, testes desnecessários, redução de custo
INTRODUCTION
Laboratory tests are mainly intended to provide the information necessary to meet the fundamental principle of clarifying diagnosis or elucidate specific situations not possible to detect by other means. For this purpose, we recommend first to precede the clinical history and physical examination that should guide the request of laboratory examination. When these premisses are not met, the tests tend to be requested without appropriate diagnostic criteria, and may be classified as “inadequate”(15).
Currently, medicine has been practiced supported by considerable technological apparatus that consisted of important achievement in many areas of health care and diagnosis(6). On the other hand, it is observed an increasing demand for complementary tests and devaluation of clinical history and extensive physical examination(21). This dichotomy between technological advances and undervaluation of semiotic findings can directly affect the performance of unnecessary testing.
A major cause of the increase in health care spending is the refinement of diagnostic that presents every day new technologies incorporated and impact on the total cost of health care(26). Technological refinement also burder the cost of medical care in the public service(20). The National Health Fund, for example, provided over than US$ 16 billion in 2012 to the medium and high complexity outpatient and inpatient care(4), where the laboratory tests are allocated.
The use of many diagnostic tools and laboratory tests are particularly exploited in intensive care units (ICU) due to the nature of the assistance designed to critically ill patients that require environmental, material, scientific, and human conditions, specific and adequated to specialized care(2). However, laboratory tests of patients admitted to the ICU are overly routine request, either by greater availability and access or by the existance of cultural factors related to critical care physicians, which may contribute to unnecessary testing(23,25).
Some studies have been conducted on the subject of performing unnecessary tests. In the surgical ICU of Massachusetts General Hospital, the implementation of guidelines to order laboratory tests impacted on a reduction of 37% in performing exams, culminating in reduction of total diagnostic costs(13). Another study, conducted at San Francisco General Hospital, California, found 12% of reduction in total laboratory tests after introducing recommendations for laboratory tests request(17).
In medical requests from the ICU in a hospital in New Zealand, after the establishing a guideline, a reduction of 16.6% in test request was observed. In the evaluation of effectiveness after three years, there was no increase in laboratory test request, demonstrating effectiveness of the guideline and of the real possibility of implementatíon(18).
Wang et al. developed guidelines and recommendations for test ordering in a coronary intensive care unit, and a reduction of 40% was verified in the request of some electrolytes (clacium, magnesium, and phosphorus)(24).
In Brazil, there are few studies that address this issue. In a study conducted at the Hospital Universitário da Universidade Federal de Santa Catarina, an average of 11.5 test request/day in ICU patients was obeserved, corrobating the studies conducted at US Hospitals and one study in Belgium(14).
At the Clinical Hospital of Universidade Estadual de Campinas, a study of day-hospital and outpatients showed high frequency of non-altered results. In day-hospital 56% and in outpatients 70%, reflecting an excess of request that resulted in the performance of unnecessary tests(10).
Due to the scarcity of national studies on the topic, and the possibility that the results of this research may contribute to the discussion on the rational use of laboratory testing, as well as subsidise the implementation of protocols and guidelines in the ICU, and also the possibility to impact public health cost decision-making, we create the motivation for this study. Furthermore, to determine the prevelence of laboratory tests requests in ICU of a General Hospital at southwest Bahia allowed us to estimate the amount spent on unnecessary laboratory tests and allowed the definition of strategies for reduction of avoidable costs related to laboratory diagnosis.
METHODS
This study falls under the retrospective cross-sectional category carried out from 1st August to 30th September 2013, in a General Hospital at southwest Bahia. This hospital provides treatment for patients of medium and high complexity, such as polytrauma, involving surgery, coronary artery disease, among others. It has 18 beds in ICU (13 for adults and five pediatric). Patients admitted to the ICU for at least three days and which underwent at least five laboratory tests during the study period composed this sample.
Data was collected from medical records, and from laboratory tests results collected by access to the laboratory data management system – Complab Advanced version 6.9.6.
The laboratory tests selected for this study were the most frequently order to intensive care patients: urea, creatínine, sodium, potassium, magnesium, calcium, phosphorus, prothrombin time (PT), activated partial thromboplastin time (APTT), aspartate aminotransferase (AST), alanine aminotransferase (ALT), alkaline phosphatase (ALP), gamma-glutamyl transferase (GGT), total bilirubin and bilirubin fractions, C-reactive protein (C-RP), complete blood count (CBC), and total protein and protein fractions.
The reference ranges established by the hospital’s laboratory guided the classification of results in altered (values above or below the reference limits), and not altered or normal (values within the reference range).
The criteria used for qualifying a test as unnecessary was based on analyzes of studies previously identified in the literature (Table 1). For this type of analysis, only four parameters could be evaluated (calcium, magnesium, phosphorus, and bilirubin), since no studies that addressed the framework of other analytical determinations within this approach were found.
TABLE 1 Classification criteria for of unnecessary laboratory tests
Laboratory tests | Criterion for prescription | Reference |
---|---|---|
Sodium, potassium, urea, creatinine, and blood count | Indicated in daily requests | Wang et al.(24) |
Blood count in unstable patient, with wounds or platelet monitoring | Indicated 12/12 hours | Mehari et al.(18) |
Blood count in afebrile patient and previous normal leukocyte count | Not required differential count | Mehari et al.(18) |
Calcium, magnesium, and phosphorus | Not indicated in routine tests | Wang et al.(24) |
Bilirubin | Not indicated as routine test, because jaundice is clinically observed when serum bilirubin concentration exceeds 2-3 mg/100ml | Martinelli(16) |
It was characterized as unnecessary tests the recurrence of normal results in tests requested repeatedly. This second laboratory test requested, with normal results, was characterized as unnecessary.
The costs of tests performed in the study were determined according to the table of values paid by the Unified Health System (Sistema Único de Saúde [SUS])(3) for an estimation of expenses for laboratory diagnosis and avoidable costs.
For statistical analysis, we used the Microsoft Office Excel 2007 and Epi Info version 7.1.1.14 software. Continuous variables were analyzed by mean, median, and standard deviation; categorical variables, by frequency and percentage. For the univariate analysis between categorical variables and outcome, we considered the chi-square test for comparison of two groups. Regarding the continuous variables, the Student’s t-test and p < 0.05 for statistical significance was adopted, considering 95% confidence range in the assessment of the analysis of interest.
The study was approved by the Research Ethics Committee of the Multidisciplinary Institute for Health of Federal University of Bahia (Instituto Multidisciplinar em Saúde da Universidade Federal da Bahia) – under number 377.503 from 29 August 2013. It was not subjected to the Informed Consent Form (ICF), because the study used secondary data of the patients and due to its retrospective nature. It is must be observed that all the precepts of ethical in research were respected in carrying out this study.
RESULTS
The sample of this study was composed of 105 patients admitted to the intensive care, during the period between August and September 2013. Analyzes were performed on the total number of examinations throughout the period in which 12217 laboratory tests were requested. Of this amount, 6040 tests (49.4%) were within the normal range.
In the study, we verified a predominance of male individuals (58.1%), aged between 18-59 years (47.6%), as shown in Table 2. The mean time of hospitalization was 8.6 days, with an average of 13.4 tests requests/day for the patients.
TABLE 2 Demographic data on patients admitted to the intensive care unit of General Hospital, between August and September 2013
Variables | Number (n) | Percentage (%) |
---|---|---|
Sex | ||
Male | 61 | 58.1% |
Female | 44 | 41.9% |
Age (years) | ||
= 17 | 27 | 25.7% |
18-59 | 50 | 47.6% |
= 60 | 28 | 26.7% |
Table 3 presents the results of laboratory tests in relation to the proportion of normal and abnormal tests according to the reference range. Bilirubin was the laboratory parameter with the highest percentage of normal results (92.1%) and, on the other hand, 95.3% of the C-RP requests had abnormal results. Complete blood counts was the most requested test, it is present in 98.3% of medical requests.
TABLE 3 Laboratory parameters profile of patients admitted to the intensive care unit in the period from August to September 2013, stratified by normal and altered results
Laboratorial test | Normal result (%)1 | Altered result (%)1 | Laboratorial test | Normal result (%)1 | Altered result (%)1 |
---|---|---|---|---|---|
Indirect bilirubin | 128 (92.1%) | 11 (7.9%) | Phosphorus | 23 (50.0%) | 23 (50.0%) |
Total bilirubin | 118 (84.9%) | 21 (15.1%) | Sodium | 422 (49.8%) | 426 (50.2%) |
Magnesium | 554 (70.4%) | 233 (29.6%) | ALT | 87 (49.7%) | 88 (50.3%) |
APTT | 253 (67.1%) | 124 (32.9%) | AST | 71 (41.0%) | 102 (59.0%) |
Potassium | 557 (65.6%) | 292 (34.4%) | GGT | 16 (38.1%) | 26 (61.9%) |
INR (PT) | 333 (65.3%) | 177 (34.7%) | Neutrophils | 291 (32.6%) | 603 (67.4%) |
Creatinine | 525 (63.3%) | 305 (36.7%) | Leukocytes | 235 (26.3%) | 659 (73.7%) |
Direct bilirubin | 88 (63.3%) | 51 (36.7%) | Albumin | 18 (23.7%) | 58 (76.3%) |
Urea | 519 (62.0%) | 318 (38.0%) | Total protein | 14 (18.4%) | 62 (81.6%) |
Platelets | 545 (61.1%) | 347 (38.9%) | Hemoglobin | 119 (13.3%) | 775 (86.7%) |
Globulin | 46 (60.5%) | 30 (39.5%) | ALP | 1 (5.50%) | 17 (94.5%) |
Calcium | 444 (53.9%) | 380 (46.1%) | CRP | 38 (4.80%) | 762 (95.3%) |
Reference values defined by the Laboratory.
APTT: activated partial thromboplastin time; INR: International Normalized Ratio; PT: prothrombin time; ALT: alanine aminotransferase; AST: aspartate aminotransferase; GGT: gammaglutamyl transferase; ALP: alkaline phosphatase; CRP: C-reactive protein.
By analyzing the requirements and the proportion of normal and abnormal tests in relation to the day of the week (Table 4), we observed a greater demand of requests on Monday, totaling 156 medical orders and 2161 laboratory tests performed. The other days of the week have an average of 125 orders and 1677 tests. This difference was significant (p < 0.05).
TABLE 4 Distribution of results of laboratory tests ordered for patients admitted in intensive care unit, sorted by days of the week
Day of the week | Order (n) | Normal tests1 (%) | Alterated tests1 (%) |
---|---|---|---|
Sunday | 128 | 824 (13.3%) | 859 (14.2%) |
Monday2 | 1562 | 1.118 (18.1%) | 1.043 (17.3%) |
Tuesday | 118 | 734 (11.9%) | 812 (13.5%) |
Wednesday | 119 | 851 (13.8%) | 781 (12.9%) |
Thursday | 136 | 883 (14.3%) | 848 (14.0%) |
Friday | 130 | 825 (13.4%) | 886 (14.7%) |
Saturday | 124 | 943 (15.3%) | 808 (13.4%) |
Reference values defined by the Laboratory;
2Number of order significantly higher (p < 0.05).
A longer stay in the ICU was not related with the highest number of laboratory test requests. Patients with up to five days stay had an average of 15.5 tests requested/day and those with permanence greater than five days had a mean of 15.1 tests/day. However, when evaluating unnecessary tests, it was observed that patients with ICU stay greater than ten days had an average of 31 unnecessary tests/patient, significantly higher (p < 0.05) than 19 tests/patient from those patients whith ICU stay lower than ten days.
When the distribution of laboratory test requests was analyzed by age group, no significant difference between groups was observed. For children, there was an average of 16.2 tests/day while for adults and elderly, respectively, 14.4 and 15.6 tests/day of hospitalization. As regards the analysis of unnecessary orders, we observed an average of 7.7 tests/day for children, 6.3 for adults and 9.1 for elderly.
There were more requests on Monday, and the most requested parameters were complete blood count, sodium and potassium.
Following criteria adopted for evaluation of unnecessary tests in other studies carried out with intensive care patients(16, 18, 24), only the analysis of some parameters: calcium, magnesium, phosphorus and bilirubin, was carried out (Figure 1). In this analysis, 1750 tests were recorded throughout the study period (August-September 2013). Of these, 719 were considered unnecessary, corresponding to 41% of tests, and US$ 638.21 of potentially avoidable costs. Magnesium and bilirubin were the tests that showed higher proportion of requests that resulted in unnecessary tests (50%). In relation to the amounts spent, based on the amount paid by the Sistema Único de Saúde (SUS), we observed cost of US$ 10,721.99 for laboratory testing. Of this amount, US$ 5,490.17 represented costs for the tests within the normal range, and, using the criteria for unnecessary, we arachieved a cost of US$ 638.21, only for the two months period.
DISCÜSSION
In the population analysis, there was a predominance of the age group between 18-59 years (47.6%). This result contradicts the expectation of a higher prevalence of elderly individuals hospitalized in ICU, since they present more risk factors for various diseases associated with. Oliveira et al. (2011) observed that elderly accounted for the majority of hospitalized in ICU (53,5%) and higher mortality (63%), when compared with the adult group(22). Our study has the youngest age group due to be held in a general hospital of reference in the General Hospital at southwest Bahia, with many assistance in polytrauma and elective and urgent surgical procedures, besides working with five beds for pediatric patients.
In this study, the average daily number requests were 13.4 tests/patient and the average time of hospitalization was 8.6 days/patient. Agente (2006) found in their study an average of 11.4 tests/day/patient, and the average time of hospitalization of 6.7 days/patient(1) in a similar complexity unit.
The data presented show that the prevalence of test ordering did not change significantly in patients who spent more time in intensive care, and there was no difference among age groups. This laboratory patterns identification is an important step towards reducing the use of unnecessary laboratory tests. However, a debate is necessary to better understand if the requests were based on clinical evidence or only in compliance with an assistential practice of requesting “routine tests”(8,23).
Machado et al. and Capilheira et al. analyzed the requests for laboratory tests in medical appointments and in ICU at a public hospital. They demonstrated higher prevalence request of CBC, fasting blood glucose, capillary blood glucose levels, blood gases, and serum potassium(5,14), corroborating the results of this study, which has a higher request rate for CBC, sodium, and potassium. Capillary blood glucose was not included in this study.
The high number of abnormal C-RP results of (95.3%) is justified due to the hospital setting of intensive care, receiving critical patients with higher morbidity and risk to infection. Bilirubin were the parameter that shows the highest number of normal results (92.1%), with 25.8% from pediatric ICU requests, and the service did not have neonatalogy admissions during the study period, so that it did not interfere substantially in the interpretation of that there is a large number of avoidable requests. According to Martinelli, jaundice is clinically noticeable in patients when the serum bilirubin concentration exceeds 2-3 mg/dl(1). The high rate of normal tests results draws attention to the need for greater valorization of physical examination in order to avoid unnecessary tests, except in selected cases, when there is the need for monitoring small changes in the bilirubin levels.
It should be noted that among the 105 patients from this study were requested other tests, such as blood lactate levels, total cholesterol and fractions, uric acid blood levels, cerebrospinal fluid analysis, creatine phosphokinase, and arterial blood gases, which were not considered in this analysis. These tests are hardly ordered in the service, and could not be included in the sample. Arterial blood gas analysis is performed in the gasometer allocated in the ICU, which also made impossible the evaluation of these data, since they were not available in the laboratory system. This reflection implies a possible underestimation of the number of unnecessary tests.
Regarding the day of week, Monday was the day with the highest number of requests for laboratory tests, statistically different from the other days of the week (p < 0.05). This argument is supported by the study of Cheng et al. (2003), in which, they observed, in a university hospital in Alberta, Canada, a largest number of orders mainly on Mondays and Fridays(8).
Santos and Godoy (1999) analyzed during 30 days the requests for complementary tests in the Department of Pediatrics, School of Medicine of São José do Rio Preto analisaram. They observed a excessive ordering tests expenditure in the amount of US$ 1,305.93, based on the SUS’s table(23). Mehari et al, after the implementation of a guideline for laboratory tests in ICU, observed a reduction of 17.1% in test requests and cost savings of US$ 16,776.95 per year (19).
Naturally, the transfer of financial resources prioritizes the critically ill patients. It is up to the professional in charge the perception that the excessive number of laboratory test requests has direct impact in the cost of treatment, thus, reasonable discussions about how to reduce this practice and consequently, the expenses(7,9).
The use of secondary data from patients who were not hospitalized during the study period and the lack of statistics of hospital discharge, of the diagnosis of the underlying disease, and deaths are limitations of this study. In addition, not all laboratory parameters were evaluated, which imply potential underestimation in the prevalence of unnecessary tests.
The great technological breakthrough and the large amount of complementary tests available for diagnosis and/or monitoring patients, it was observed a devaluation of the medical history and use of hypothetical-deductive clinical reasoning in the assessment of patients(6, 11, 12). The complementarity aspect of the laboratory tests is lost, since it should support the clinical findings. It is noteworthy also that the interfering in laboratory tests can lead to inappropriately interpreted results and taken decisions incompatible with the disease or the patients evolution, leading to further investigations with more exams, potentially unnecessary.
CONCLÜSION
The results of the study indicate that the service had significant prevalence on the request of unnecessary laboratory tests in patients admitted to the ICU and, therefore, higher cost with exams that could be avoided, although there is still underestimations in this analysis. The numbers of this study indicate the need to implement protocols or guidelines for using laboratory tests in intensive care, aiming at improving assistance and optimizing health costs, beyond the rational use of resources for specialized diagnostic.
Therefore, it is necessary that the team is open to change and understand that the rational use through guidelines helps to decrease costs without implications for the clinical practice. Rather, this study emphasizes the need and recognition of physical examination and medical history, as being guidelines and essential elements on conductiong/performing orders and laboratory tests. This action is in scope to bring the intensivist physicians and their patients closer by clinical practice, requiring the true character of complementarity to the laboratory testing.
REFERENCES
1. AGENTE, J. S. Avaliação da solicitação de exames de sangue na UTI do HU-UFSC em pacientes com menos de 60 anos de idade. Repositório da UFSC, Santa Catarina, 2006.
2. BETTINELLI, L. A.; ERDMANN, A. L. Internação em unidade de terapia intensiva e a família: perspectivas de cuidado. Av Enferm Rio Grande do Sul, v. 27, n. 1, 2009.
3. BRASIL. Sistema de Gerenciamento da Tabela de Procedimentos, Medicamentos e OPM do SUS. DATASUS, SIGTAP, 2014.
4. BRASIL. Ministério da Saúde. Fundo Nacional de Saúde. Gráfico comparativo por ano, 2012.
5. CAPILHEIRA, M. F.; SANTOS, I. S. Epidemiologia da solicitação de exame complementar em consultas médicas. Rev Saúde Pública, Pelotas, v. 40, n. 2, 2006.
6. CAPILHEIRA, M. F. Prevalência e fatores associados à consulta médica e solicitação de exames complementares: um estudo de base populacional. 2004. Dissertação (Mestrado) – Departamento de Medicina Social, Universidade Federal de Pelotas, Pelotas, 2004.
7. CHALFIN, D. B. et al. The economics and cost-effectiveness of critical care medicine. Intensive Care Med, v. 21, n. 11, 1995.
8. CHENG, C. K. et al. Temporal approach to hematological test usage in a major teaching hospital. Lab Hematol, Canadá, v. 9, n. 4, 2003.
9. COOK, D. J. Physicians’ perceptions of laboratory costs in the intensive care unit. Hamilton Regional Critical Care Group. Clin Invest Med, Canadá, v. 15, n. 5, 1992.
10. FREIREA, L. M. D. et al. Controle de qualidade laboratorial pré-analítico: avaliação de solicitações médicas de exames bioquímicos no hospital de clínicas da Universidade Estadual de Campinas, RBAC, São Paulo, v. 40, n. 2, 2008.
11. KASSIRER, J. P. Our stubborn quest for diagnostic certainty. A cause of excessive testing. N Engl J Med, v. 320, n. 22, 1989.
12. KLOETZEL, K. Usos e abusos de exame complementar. Revista Diagnóstico & Tratamento, v. 6, n. 4, p. 19-27, 2001.
13. KUMWILAISAK, K. et al. Effect of laboratory testing guidelines on the utilization of tests and order entries in a surgical intensive care unit. Crit Care Med, Massachusetts, v. 36, n. 11, 2008.
14. MACHADO, F O. et al. Avaliação da necessidade da solicitação de exames complementares para pacientes internados em unidade de terapia intensiva de hospital universitário. Rev Bras Ter Intensiva, Santa Catarina, v. 18, n. 4, 2006.
15. MAKSOUD, J. G. O uso inadequado dos exames complementares. Rev Pediatria, São Paulo, v. 17, n. 1, 1995.
16. MARTINELLI, A. L. C. Icterícia. Medicina Ribeirão Preto, São Paulo, v. 37, n. 3-4, 2004.
17. MAY, T. A. et al. Reducing unnecessary inpatient laboratory testing in a teaching hospital. Am J Clin Pathol, San Francisco, v. 126, n. 2, 2006.
18. MEHARI, S. M.; HAVILL, J. H. Written guidelines for laboratory testing in intensive care-still effective after 3 years. Crit Care Resusc, Austrália, v. 3, n. 3, 2001.
19. MEHARI, S. M. et al. A written guideline implementation can lead to reductions in laboratory testing in an intensive care unit. Anaesth Intensive Care, Austrália, v. 25, n. 1, 1997.
20. NAKAMURA, D. F. Relação entre a idade do médico e os custos gerados com exames complementares durante o atendimento ambulatorial de uma operadora de plano de assistência à saúde. 2009. Dissertação (Mestrado) – Instituto Superior de Ciências do Trabalho e da Empresa, Lisboa. 2009.
21. NETO, J. A. C. et al. Confiabilidade no médico relacionada ao pedido de exame Complementar. HU Rev, Juiz de Fora, v. 33, n. 3, 2007.
22. OLIVEIRA, V. C. et al. Clinical evolution of adult, elderly and very elderly patients admitted in Intensive Care Units. Rev Latino-Am Enfermagem, São Paulo, v. 19, n. 6, 2011.
23. SANTOS, A. A. C.; GODOY, M. F. Análise crítica da solicitação de exames complementares em pediatria. Rev Pediatria, São Paulo, v. 21, n. 3, 1999.
24. WANG, T. J. et al. A utilization management intervention to reduce unnecessary testing in] the coronary care unit. Arch Intern Med, Massachusetts, v. 162, n. 16, 2002.
25. ZIMMERMAN, J. E. et al. Evaluating laboratory usage in the intensive care unit: patient and institutional characteristics that influence frequency of blood sampling. Crit Care Med, Washington, v. 25, n. 5, 1997.
26. ZUCCHI. P. et al. Gastos em saúde: os fatores que agem na demanda e na oferta dos serviços de saúde. Saúde Soc, São Paulo, v. 9, n. 1-2, 2000.