Skip to main content
Top
Published in: Implementation Science 1/2011

Open Access 01-12-2011 | Systematic review

Can computerized clinical decision support systems improve practitioners' diagnostic test ordering behavior? A decision-maker-researcher partnership systematic review

Authors: Pavel S Roshanov, John J You, Jasmine Dhaliwal, David Koff, Jean A Mackay, Lorraine Weise-Kelly, Tamara Navarro, Nancy L Wilczynski, R Brian Haynes, the CCDSS Systematic Review Team

Published in: Implementation Science | Issue 1/2011

Login to get access

Abstract

Background

Underuse and overuse of diagnostic tests have important implications for health outcomes and costs. Decision support technology purports to optimize the use of diagnostic tests in clinical practice. The objective of this review was to assess whether computerized clinical decision support systems (CCDSSs) are effective at improving ordering of tests for diagnosis, monitoring of disease, or monitoring of treatment. The outcome of interest was effect on the diagnostic test-ordering behavior of practitioners.

Methods

We conducted a decision-maker-researcher partnership systematic review. We searched MEDLINE, EMBASE, Ovid's EBM Reviews database, Inspec, and reference lists for eligible articles published up to January 2010. We included randomized controlled trials comparing the use of CCDSSs to usual practice or non-CCDSS controls in clinical care settings. Trials were eligible if at least one component of the CCDSS gave suggestions for ordering or performing a diagnostic procedure. We considered studies 'positive' if they showed a statistically significant improvement in at least 50% of test ordering outcomes.

Results

Thirty-five studies were identified, with significantly higher methodological quality in those published after the year 2000 (p = 0.002). Thirty-three trials reported evaluable data on diagnostic test ordering, and 55% (18/33) of CCDSSs improved testing behavior overall, including 83% (5/6) for diagnosis, 63% (5/8) for treatment monitoring, 35% (6/17) for disease monitoring, and 100% (3/3) for other purposes. Four of the systems explicitly attempted to reduce test ordering rates and all succeeded. Factors of particular interest to decision makers include costs, user satisfaction, and impact on workflow but were rarely investigated or reported.

Conclusions

Some CCDSSs can modify practitioner test-ordering behavior. To better inform development and implementation efforts, studies should describe in more detail potentially important factors such as system design, user interface, local context, implementation strategy, and evaluate impact on user satisfaction and workflow, costs, and unintended consequences.
Appendix
Available only for authorised users
Literature
1.
go back to reference Miller RA: Medical diagnostic decision support systems--past, present, and future: a threaded bibiiography and brief commentary. J Am Med Inform Assoc. 1994, 1 (1): 8-27. 10.1136/jamia.1994.95236141.CrossRefPubMedPubMedCentral Miller RA: Medical diagnostic decision support systems--past, present, and future: a threaded bibiiography and brief commentary. J Am Med Inform Assoc. 1994, 1 (1): 8-27. 10.1136/jamia.1994.95236141.CrossRefPubMedPubMedCentral
2.
go back to reference Solomon DH, Hashimoto H, Daltroy L, Liang MH: Techniques to improve physicians' use of diagnostic tests: a new conceptual framework. JAMA. 1998, 280 (23): 2020-2027. 10.1001/jama.280.23.2020.CrossRefPubMed Solomon DH, Hashimoto H, Daltroy L, Liang MH: Techniques to improve physicians' use of diagnostic tests: a new conceptual framework. JAMA. 1998, 280 (23): 2020-2027. 10.1001/jama.280.23.2020.CrossRefPubMed
3.
go back to reference Wennberg JE: Dealing with medical practice variations: a proposal for action. Health Aff (Millwood). 1984, 3 (2): 6-32. 10.1377/hlthaff.3.2.6.CrossRef Wennberg JE: Dealing with medical practice variations: a proposal for action. Health Aff (Millwood). 1984, 3 (2): 6-32. 10.1377/hlthaff.3.2.6.CrossRef
4.
go back to reference Daniels M, Schroeder SA: Variation among physicians in use of laboratory tests II. Relation to clinical productivity and outcomes of care. Med Care. 1977, 15 (6): 482-487. 10.1097/00005650-197706000-00004.CrossRefPubMed Daniels M, Schroeder SA: Variation among physicians in use of laboratory tests II. Relation to clinical productivity and outcomes of care. Med Care. 1977, 15 (6): 482-487. 10.1097/00005650-197706000-00004.CrossRefPubMed
5.
go back to reference Casscells W, Schoenberger A, Graboys TB: Interpretation by physicians of clinical laboratory results. N Engl J Med. 1978, 299 (18): 999-1001. 10.1056/NEJM197811022991808.CrossRefPubMed Casscells W, Schoenberger A, Graboys TB: Interpretation by physicians of clinical laboratory results. N Engl J Med. 1978, 299 (18): 999-1001. 10.1056/NEJM197811022991808.CrossRefPubMed
6.
go back to reference McDonald CJ: Protocol-based computer reminders, the quality of care and the non-perfectability of man. N Engl J Med. 1976, 295 (24): 1351-1355. 10.1056/NEJM197612092952405.CrossRefPubMed McDonald CJ: Protocol-based computer reminders, the quality of care and the non-perfectability of man. N Engl J Med. 1976, 295 (24): 1351-1355. 10.1056/NEJM197612092952405.CrossRefPubMed
7.
go back to reference Kanouse DE, Jacoby I: When does information change practitioners' behavior?. Int J Technol Assess Health Care. 2009, 4 (01): 27-33.CrossRef Kanouse DE, Jacoby I: When does information change practitioners' behavior?. Int J Technol Assess Health Care. 2009, 4 (01): 27-33.CrossRef
8.
go back to reference Davis DA, Thomson MA, Oxman AD, Haynes RB: Changing physician performance. A systematic review of the effect of continuing medical education strategies. JAMA. 1995, 274 (9): 700-705. 10.1001/jama.274.9.700.CrossRefPubMed Davis DA, Thomson MA, Oxman AD, Haynes RB: Changing physician performance. A systematic review of the effect of continuing medical education strategies. JAMA. 1995, 274 (9): 700-705. 10.1001/jama.274.9.700.CrossRefPubMed
9.
go back to reference Freemantle N: Are decisions taken by health care professionals rational? A non systematic review of experimental and quasi experimental literature. Health Policy. 1996, 38 (2): 71-81. 10.1016/0168-8510(96)00837-8.CrossRefPubMed Freemantle N: Are decisions taken by health care professionals rational? A non systematic review of experimental and quasi experimental literature. Health Policy. 1996, 38 (2): 71-81. 10.1016/0168-8510(96)00837-8.CrossRefPubMed
10.
go back to reference McKinlay JB, Potter DA, Feldman HA: Non-medical influences on medical decision-making. Soc Sci Med. 1996, 42 (5): 769-776. 10.1016/0277-9536(95)00342-8.CrossRefPubMed McKinlay JB, Potter DA, Feldman HA: Non-medical influences on medical decision-making. Soc Sci Med. 1996, 42 (5): 769-776. 10.1016/0277-9536(95)00342-8.CrossRefPubMed
11.
go back to reference Iglehart JK: Health insurers and medical-imaging policy--a work in progress. N Engl J Med. 2009, 360 (10): 1030-1037. 10.1056/NEJMhpr0808703.CrossRefPubMed Iglehart JK: Health insurers and medical-imaging policy--a work in progress. N Engl J Med. 2009, 360 (10): 1030-1037. 10.1056/NEJMhpr0808703.CrossRefPubMed
12.
go back to reference Axt-Adam P, van der Wouden JC, van der Does E: Influencing behavior of physicians ordering laboratory tests: a literature study. Med Care. 1993, 31 (9): 784-794. 10.1097/00005650-199309000-00003.CrossRefPubMed Axt-Adam P, van der Wouden JC, van der Does E: Influencing behavior of physicians ordering laboratory tests: a literature study. Med Care. 1993, 31 (9): 784-794. 10.1097/00005650-199309000-00003.CrossRefPubMed
13.
go back to reference Buntinx F, Winkens R, Grol R, Knottnerus JA: Influencing diagnostic and preventive performance in ambulatory care by feedback and reminders. A review. Fam Pract. 1993, 10 (2): 219-228. 10.1093/fampra/10.2.219.CrossRefPubMed Buntinx F, Winkens R, Grol R, Knottnerus JA: Influencing diagnostic and preventive performance in ambulatory care by feedback and reminders. A review. Fam Pract. 1993, 10 (2): 219-228. 10.1093/fampra/10.2.219.CrossRefPubMed
14.
go back to reference Gama R, Hartland AJ, Holland MR: Changing clinicians' laboratory test requesting behaviour: can the poacher turn gamekeeper?. Clin Lab. 2001, 47 (1-2): 57-66.PubMed Gama R, Hartland AJ, Holland MR: Changing clinicians' laboratory test requesting behaviour: can the poacher turn gamekeeper?. Clin Lab. 2001, 47 (1-2): 57-66.PubMed
15.
go back to reference Thomas RE, Croal BL, Ramsay C, Eccles M, Grimshaw J: Effect of enhanced feedback and brief educational reminder messages on laboratory test requesting in primary care: a cluster randomised trial. Lancet. 2006, 367 (9527): 1990-1996. 10.1016/S0140-6736(06)68888-0.CrossRefPubMed Thomas RE, Croal BL, Ramsay C, Eccles M, Grimshaw J: Effect of enhanced feedback and brief educational reminder messages on laboratory test requesting in primary care: a cluster randomised trial. Lancet. 2006, 367 (9527): 1990-1996. 10.1016/S0140-6736(06)68888-0.CrossRefPubMed
16.
go back to reference Shojania KG, Jennings A, Mayhew A, Ramsay C, Eccles M, Grimshaw J: Effect of point-of-care computer reminders on physician behaviour: a systematic review. CMAJ. 2010, 182 (5): E216-E225. 10.1503/cmaj.090578.CrossRefPubMedPubMedCentral Shojania KG, Jennings A, Mayhew A, Ramsay C, Eccles M, Grimshaw J: Effect of point-of-care computer reminders on physician behaviour: a systematic review. CMAJ. 2010, 182 (5): E216-E225. 10.1503/cmaj.090578.CrossRefPubMedPubMedCentral
17.
go back to reference Johnston ME, Langton KB, Haynes RB, Mathieu A: Effects of computer-based clinical decision support systems on clinician performance and patient outcome: a critical appraisal of research. Ann Intern Med. 1994, 120 (2): 135-142.CrossRefPubMed Johnston ME, Langton KB, Haynes RB, Mathieu A: Effects of computer-based clinical decision support systems on clinician performance and patient outcome: a critical appraisal of research. Ann Intern Med. 1994, 120 (2): 135-142.CrossRefPubMed
18.
go back to reference Hunt DL, Haynes RB, Hanna SE, Smith K: Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA. 1998, 280 (15): 1339-1346. 10.1001/jama.280.15.1339.CrossRefPubMed Hunt DL, Haynes RB, Hanna SE, Smith K: Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA. 1998, 280 (15): 1339-1346. 10.1001/jama.280.15.1339.CrossRefPubMed
19.
go back to reference Garg AX, Adhikari NK, McDonald H, Rosas-Arellano M, Devereaux PJ, Beyene J, Sam J, Haynes RB: Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005, 293 (10): 1223-1238. 10.1001/jama.293.10.1223.CrossRefPubMed Garg AX, Adhikari NK, McDonald H, Rosas-Arellano M, Devereaux PJ, Beyene J, Sam J, Haynes RB: Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005, 293 (10): 1223-1238. 10.1001/jama.293.10.1223.CrossRefPubMed
20.
go back to reference Haynes RB, Wilczynski NL, the Computerized Clinical Decision Support System (CCDSS) Systematic Review Team: Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: methods of a decision-maker-research partnership systematic review. Implement Sci. 2010, 5: 12-CrossRefPubMedPubMedCentral Haynes RB, Wilczynski NL, the Computerized Clinical Decision Support System (CCDSS) Systematic Review Team: Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: methods of a decision-maker-research partnership systematic review. Implement Sci. 2010, 5: 12-CrossRefPubMedPubMedCentral
21.
go back to reference Poels PJ, Schermer TR, Thoonen BP, Jacobs JE, Akkermans RP, de Vries Robbe PF, Quanjer PH, Bottema BJ, van Weel C: Spirometry expert support in family practice: a cluster-randomised trial. Prim Care Respir J. 2009, 18 (3): 189-197. 10.4104/pcrj.2009.00047.CrossRefPubMed Poels PJ, Schermer TR, Thoonen BP, Jacobs JE, Akkermans RP, de Vries Robbe PF, Quanjer PH, Bottema BJ, van Weel C: Spirometry expert support in family practice: a cluster-randomised trial. Prim Care Respir J. 2009, 18 (3): 189-197. 10.4104/pcrj.2009.00047.CrossRefPubMed
22.
go back to reference Bogusevicius A, Maleckas A, Pundzius J, Skaudickas D: Prospective randomised trial of computer-aided diagnosis and contrast radiography in acute small bowel obstruction. Eur J Surg. 2002, 168 (2): 78-83. 10.1080/11024150252884287.CrossRefPubMed Bogusevicius A, Maleckas A, Pundzius J, Skaudickas D: Prospective randomised trial of computer-aided diagnosis and contrast radiography in acute small bowel obstruction. Eur J Surg. 2002, 168 (2): 78-83. 10.1080/11024150252884287.CrossRefPubMed
23.
go back to reference Tierney WM, Miller ME, McDonald CJ: The Effect on Test Ordering of Informing Physicians of the Charges for Outpatient Diagnostic Tests. N Engl J Med. 1990, 322 (21): 1499-1504. 10.1056/NEJM199005243222105.CrossRefPubMed Tierney WM, Miller ME, McDonald CJ: The Effect on Test Ordering of Informing Physicians of the Charges for Outpatient Diagnostic Tests. N Engl J Med. 1990, 322 (21): 1499-1504. 10.1056/NEJM199005243222105.CrossRefPubMed
24.
go back to reference Tierney : Computerized display of past test results. Effect on outpatient testing. Ann Intern Med. 1987, 107 (4): 569-574.CrossRefPubMed Tierney : Computerized display of past test results. Effect on outpatient testing. Ann Intern Med. 1987, 107 (4): 569-574.CrossRefPubMed
25.
go back to reference Gilutz H, Novack L, Shvartzman P, Zelingher J, Bonneh DY, Henkin Y, Maislos M, Peleg R, Liss Z, Rabinowitz G, Vardy D, Zahger D, Ilia R, Leibermann N, Porath A: Computerized community cholesterol control (4C): meeting the challenge of secondary prevention. Israel Med Assoc J. 2009, 11 (1): 23-29. Gilutz H, Novack L, Shvartzman P, Zelingher J, Bonneh DY, Henkin Y, Maislos M, Peleg R, Liss Z, Rabinowitz G, Vardy D, Zahger D, Ilia R, Leibermann N, Porath A: Computerized community cholesterol control (4C): meeting the challenge of secondary prevention. Israel Med Assoc J. 2009, 11 (1): 23-29.
26.
go back to reference Holbrook A, Thabane L, Keshavjee K, Dolovich L, Bernstein B, Chan D, Troyan S, Foster G, Gerstein H: Individualized electronic decision support and reminders to improve diabetes care in the community: COMPETE II randomized trial. CMAJ. 2009, 181 (1-2): 37-44. 10.1503/cmaj.081272.CrossRefPubMedPubMedCentral Holbrook A, Thabane L, Keshavjee K, Dolovich L, Bernstein B, Chan D, Troyan S, Foster G, Gerstein H: Individualized electronic decision support and reminders to improve diabetes care in the community: COMPETE II randomized trial. CMAJ. 2009, 181 (1-2): 37-44. 10.1503/cmaj.081272.CrossRefPubMedPubMedCentral
27.
go back to reference Holbrook A, Keshavjee K, Lee H, Bernstein B, Chan D, Thabane L, Gerstein H, Troyan S, COMPETE II Investigators: Individualized electronic decision support and reminders can improve diabetes care in the community. AMIA Annu Symp Proc. 2005, 982- Holbrook A, Keshavjee K, Lee H, Bernstein B, Chan D, Thabane L, Gerstein H, Troyan S, COMPETE II Investigators: Individualized electronic decision support and reminders can improve diabetes care in the community. AMIA Annu Symp Proc. 2005, 982-
28.
go back to reference MacLean CD, Gagnon M, Callas P, Littenberg B: The Vermont Diabetes Information System: a cluster randomized trial of a population based decision support system. J Gen Intern Med. 2009, 24 (12): 1303-1310. 10.1007/s11606-009-1147-x.CrossRefPubMedPubMedCentral MacLean CD, Gagnon M, Callas P, Littenberg B: The Vermont Diabetes Information System: a cluster randomized trial of a population based decision support system. J Gen Intern Med. 2009, 24 (12): 1303-1310. 10.1007/s11606-009-1147-x.CrossRefPubMedPubMedCentral
29.
go back to reference MacLean CD, Littenberg B, Gagnon M: Diabetes decision support: initial experience with the Vermont diabetes information system. Am J Public Health. 2006, 96 (4): 593-595. 10.2105/AJPH.2005.065391.CrossRefPubMedPubMedCentral MacLean CD, Littenberg B, Gagnon M: Diabetes decision support: initial experience with the Vermont diabetes information system. Am J Public Health. 2006, 96 (4): 593-595. 10.2105/AJPH.2005.065391.CrossRefPubMedPubMedCentral
30.
go back to reference Peterson KA, Radosevich DM, O'Connor PJ, Nyman JA, Prineas RJ, Smith SA, Arneson TJ, Corbett VA, Weinhandl JC, Lange CJ, Hannan PJ: Improving diabetes care in practice: findings from the TRANSLATE trial. Diabetes Care. 2008, 31 (12): 2238-2243. 10.2337/dc08-2034.CrossRefPubMedPubMedCentral Peterson KA, Radosevich DM, O'Connor PJ, Nyman JA, Prineas RJ, Smith SA, Arneson TJ, Corbett VA, Weinhandl JC, Lange CJ, Hannan PJ: Improving diabetes care in practice: findings from the TRANSLATE trial. Diabetes Care. 2008, 31 (12): 2238-2243. 10.2337/dc08-2034.CrossRefPubMedPubMedCentral
31.
go back to reference Borbolla D, Giunta D, Figar S, Soriano M, Dawidowski A, de Quiros FG: Effectiveness of a chronic disease surveillance systems for blood pressure monitoring. Stud Health Technol Inform. 2007, 129 (Pt 1): 223-227.PubMed Borbolla D, Giunta D, Figar S, Soriano M, Dawidowski A, de Quiros FG: Effectiveness of a chronic disease surveillance systems for blood pressure monitoring. Stud Health Technol Inform. 2007, 129 (Pt 1): 223-227.PubMed
32.
go back to reference Lester WT, Grant RW, Barnett GO, Chueh HC: Randomized controlled trial of an informatics-based intervention to increase statin prescription for secondary prevention of coronary disease. J Gen Intern Med. 2006, 21 (1): 22-29. 10.1111/j.1525-1497.2005.00268.x.CrossRefPubMedPubMedCentral Lester WT, Grant RW, Barnett GO, Chueh HC: Randomized controlled trial of an informatics-based intervention to increase statin prescription for secondary prevention of coronary disease. J Gen Intern Med. 2006, 21 (1): 22-29. 10.1111/j.1525-1497.2005.00268.x.CrossRefPubMedPubMedCentral
33.
go back to reference Lester WT, Grant R, Barnett GO, Chueh H: Facilitated lipid management using interactive e-mail: preliminary results of a randomized controlled trial. Stud Health Technol Inform. 2004, 107 (Pt 1): 232-236.PubMed Lester WT, Grant R, Barnett GO, Chueh H: Facilitated lipid management using interactive e-mail: preliminary results of a randomized controlled trial. Stud Health Technol Inform. 2004, 107 (Pt 1): 232-236.PubMed
34.
go back to reference Cobos A, Vilaseca J, Asenjo C, Pedro-Botet J, Sanchez E, Val A, Torremade E, Espinosa C, Bergonon S: Cost effectiveness of a clinical decision support system based on the recommendations of the European Society of Cardiology and other societies for the management of hypercholesterolemia: Report of a cluster-randomized trial. Dis Manag Health Out. 2005, 13 (6): 421-432. 10.2165/00115677-200513060-00007.CrossRef Cobos A, Vilaseca J, Asenjo C, Pedro-Botet J, Sanchez E, Val A, Torremade E, Espinosa C, Bergonon S: Cost effectiveness of a clinical decision support system based on the recommendations of the European Society of Cardiology and other societies for the management of hypercholesterolemia: Report of a cluster-randomized trial. Dis Manag Health Out. 2005, 13 (6): 421-432. 10.2165/00115677-200513060-00007.CrossRef
35.
go back to reference Plaza V, Cobos A, Ignacio-Garcia JM, Molina J, Bergonon S, Garcia-Alonso F, Espinosa C, Grupo Investigador A: [Cost-effectiveness of an intervention based on the Global INitiative for Asthma (GINA) recommendations using a computerized clinical decision support system: a physicians randomized trial]. Med Clin (Barc). 2005, 124 (6): 201-206. 10.1157/13071758.CrossRef Plaza V, Cobos A, Ignacio-Garcia JM, Molina J, Bergonon S, Garcia-Alonso F, Espinosa C, Grupo Investigador A: [Cost-effectiveness of an intervention based on the Global INitiative for Asthma (GINA) recommendations using a computerized clinical decision support system: a physicians randomized trial]. Med Clin (Barc). 2005, 124 (6): 201-206. 10.1157/13071758.CrossRef
36.
go back to reference Sequist TD, Gandhi TK, Karson AS, Fiskio JM, Bugbee D, Sperling M, Cook EF, Orav EJ, Fairchild DG, Bates DW: A randomized trial of electronic clinical reminders to improve quality of care for diabetes and coronary artery disease. J Am Med Inform Assoc. 2005, 12 (4): 431-437. 10.1197/jamia.M1788.CrossRefPubMedPubMedCentral Sequist TD, Gandhi TK, Karson AS, Fiskio JM, Bugbee D, Sperling M, Cook EF, Orav EJ, Fairchild DG, Bates DW: A randomized trial of electronic clinical reminders to improve quality of care for diabetes and coronary artery disease. J Am Med Inform Assoc. 2005, 12 (4): 431-437. 10.1197/jamia.M1788.CrossRefPubMedPubMedCentral
37.
go back to reference Tierney WM, Overhage JM, Murray MD, Harris LE, Zhou XH, Eckert GJ, Smith FE, Nienaber N, McDonald CJ, Wolinsky FD: Can computer-generated evidence-based care suggestions enhance evidence-based management of asthma and chronic obstructive pulmonary disease? A randomized, controlled trial. Health Serv Res. 2005, 40 (2): 477-497. 10.1111/j.1475-6773.2005.0t369.x.CrossRefPubMedPubMedCentral Tierney WM, Overhage JM, Murray MD, Harris LE, Zhou XH, Eckert GJ, Smith FE, Nienaber N, McDonald CJ, Wolinsky FD: Can computer-generated evidence-based care suggestions enhance evidence-based management of asthma and chronic obstructive pulmonary disease? A randomized, controlled trial. Health Serv Res. 2005, 40 (2): 477-497. 10.1111/j.1475-6773.2005.0t369.x.CrossRefPubMedPubMedCentral
38.
go back to reference Mitchell E, Sullivan F, Watt G, Grimshaw JM, Donnan PT: Using electronic patient records to inform strategic decision making in primary care. Stud Health Technol Inform. 2004, 107 (Pt2): 1157-1161.PubMed Mitchell E, Sullivan F, Watt G, Grimshaw JM, Donnan PT: Using electronic patient records to inform strategic decision making in primary care. Stud Health Technol Inform. 2004, 107 (Pt2): 1157-1161.PubMed
39.
go back to reference Eccles M, McColl E, Steen N, Rousseau N, Grimshaw J, Parkin D, Purves I: Effect of computerised evidence based guidelines on management of asthma and angina in adults in primary care: cluster randomised controlled trial. BMJ. 2002, 325 (7370): 941-10.1136/bmj.325.7370.941.CrossRefPubMedPubMedCentral Eccles M, McColl E, Steen N, Rousseau N, Grimshaw J, Parkin D, Purves I: Effect of computerised evidence based guidelines on management of asthma and angina in adults in primary care: cluster randomised controlled trial. BMJ. 2002, 325 (7370): 941-10.1136/bmj.325.7370.941.CrossRefPubMedPubMedCentral
40.
go back to reference Rousseau N, McColl E, Newton J, Grimshaw J, Eccles M: Practice based, longitudinal, qualitative interview study of computerised evidence based guidelines in primary care. BMJ. 2003, 326 (7384): 314-10.1136/bmj.326.7384.314.CrossRefPubMedPubMedCentral Rousseau N, McColl E, Newton J, Grimshaw J, Eccles M: Practice based, longitudinal, qualitative interview study of computerised evidence based guidelines in primary care. BMJ. 2003, 326 (7384): 314-10.1136/bmj.326.7384.314.CrossRefPubMedPubMedCentral
41.
go back to reference Demakis JG, Beauchamp C, Cull WL, Denwood R, Eisen SA, Lofgren R, Nichol K, Woolliscroft J, Henderson WG: Improving residents' compliance with standards of ambulatory care: results from the VA Cooperative Study on Computerized Reminders. JAMA. 2000, 284 (11): 1411-1416. 10.1001/jama.284.11.1411.CrossRefPubMed Demakis JG, Beauchamp C, Cull WL, Denwood R, Eisen SA, Lofgren R, Nichol K, Woolliscroft J, Henderson WG: Improving residents' compliance with standards of ambulatory care: results from the VA Cooperative Study on Computerized Reminders. JAMA. 2000, 284 (11): 1411-1416. 10.1001/jama.284.11.1411.CrossRefPubMed
42.
go back to reference Hetlevik I, Holmen J, Krüger O: Implementing clinical guidelines in the treatment of hypertension in general practice. Evaluation of patient outcome related to implementation of a computer-based clinical decision support system. Scand J Prim Health Care. 1999, 17 (1): 35-40. 10.1080/028134399750002872.CrossRefPubMed Hetlevik I, Holmen J, Krüger O: Implementing clinical guidelines in the treatment of hypertension in general practice. Evaluation of patient outcome related to implementation of a computer-based clinical decision support system. Scand J Prim Health Care. 1999, 17 (1): 35-40. 10.1080/028134399750002872.CrossRefPubMed
43.
go back to reference Hetlevik I, Holmen J, Kruger O, Kristensen P, Iversen H: Implementing clinical guidelines in the treatment of hypertension in general practice. Blood Press. 1998, 7 (5-6): 270-276. 10.1080/080370598437114.CrossRefPubMed Hetlevik I, Holmen J, Kruger O, Kristensen P, Iversen H: Implementing clinical guidelines in the treatment of hypertension in general practice. Blood Press. 1998, 7 (5-6): 270-276. 10.1080/080370598437114.CrossRefPubMed
44.
go back to reference Hetlevik I, Holmen J, Krüger O, Kristensen P, Iversen H, Furuseth K: Implementing clinical guidelines in the treatment of diabetes mellitus in general practice. Evaluation of effort, process, and patient outcome related to implementation of a computer-based decision support system. Int J Technol Assess. 2000, 16 (1): 210-227. 10.1017/S0266462300161185.CrossRef Hetlevik I, Holmen J, Krüger O, Kristensen P, Iversen H, Furuseth K: Implementing clinical guidelines in the treatment of diabetes mellitus in general practice. Evaluation of effort, process, and patient outcome related to implementation of a computer-based decision support system. Int J Technol Assess. 2000, 16 (1): 210-227. 10.1017/S0266462300161185.CrossRef
45.
go back to reference Lobach DF, Hammond W: Computerized decision support based on a clinical practice guideline improves compliance with care standards. Am J Med. 1997, 102 (1): 89-98. 10.1016/S0002-9343(96)00382-8.CrossRefPubMed Lobach DF, Hammond W: Computerized decision support based on a clinical practice guideline improves compliance with care standards. Am J Med. 1997, 102 (1): 89-98. 10.1016/S0002-9343(96)00382-8.CrossRefPubMed
46.
go back to reference Mazzuca SA, Vinicor F, Einterz RM, Tierney WM, Norton JA, Kalasinski LA: Effects of the clinical environment on physicians' response to postgraduate medical education. Am Educ Res J. 1990, 27 (3): 473-488.CrossRef Mazzuca SA, Vinicor F, Einterz RM, Tierney WM, Norton JA, Kalasinski LA: Effects of the clinical environment on physicians' response to postgraduate medical education. Am Educ Res J. 1990, 27 (3): 473-488.CrossRef
47.
go back to reference Rogers JL, Haring OM, Goetz JP: Changes in patient attitudes following the implementation of a medical information system. QRB Qual Rev Bull. 1984, 10 (3): 65-74.PubMed Rogers JL, Haring OM, Goetz JP: Changes in patient attitudes following the implementation of a medical information system. QRB Qual Rev Bull. 1984, 10 (3): 65-74.PubMed
48.
go back to reference Rogers JL, Haring OM: The impact of a computerized medical record summary system on incidence and length of hospitalization. Med Care. 1979, 17 (6): 618-630. 10.1097/00005650-197906000-00006.CrossRefPubMed Rogers JL, Haring OM: The impact of a computerized medical record summary system on incidence and length of hospitalization. Med Care. 1979, 17 (6): 618-630. 10.1097/00005650-197906000-00006.CrossRefPubMed
49.
go back to reference Rogers JL, Haring OM, Wortman PM, Watson RA, Goetz JP: Medical information systems: assessing impact in the areas of hypertension, obesity and renal disease. Med Care. 1982, 20 (1): 63-74. 10.1097/00005650-198201000-00005.CrossRefPubMed Rogers JL, Haring OM, Wortman PM, Watson RA, Goetz JP: Medical information systems: assessing impact in the areas of hypertension, obesity and renal disease. Med Care. 1982, 20 (1): 63-74. 10.1097/00005650-198201000-00005.CrossRefPubMed
50.
go back to reference Lo HG, Matheny ME, Seger DL, Bates DW, Gandhi TK: Impact of non-interruptive medication laboratory monitoring alerts in ambulatory care. J Am Med Inform Assoc. 2009, 16 (1): 66-71.CrossRefPubMedPubMedCentral Lo HG, Matheny ME, Seger DL, Bates DW, Gandhi TK: Impact of non-interruptive medication laboratory monitoring alerts in ambulatory care. J Am Med Inform Assoc. 2009, 16 (1): 66-71.CrossRefPubMedPubMedCentral
51.
go back to reference Matheny ME, Sequist TD, Seger AC, Fiskio JM, Sperling M, Bugbee D, Bates DW, Gandhi TK: A randomized trial of electronic clinical reminders to improve medication laboratory monitoring. J Am Med Inform Assoc. 2008, 15 (4): 424-429. 10.1197/jamia.M2602.CrossRefPubMedPubMedCentral Matheny ME, Sequist TD, Seger AC, Fiskio JM, Sperling M, Bugbee D, Bates DW, Gandhi TK: A randomized trial of electronic clinical reminders to improve medication laboratory monitoring. J Am Med Inform Assoc. 2008, 15 (4): 424-429. 10.1197/jamia.M2602.CrossRefPubMedPubMedCentral
52.
go back to reference Feldstein AC, Smith DH, Perrin N, Yang X, Rix M, Raebel MA, Magid DJ, Simon SR, Soumerai SB: Improved therapeutic monitoring with several interventions: a randomized trial. Arch Intern Med. 2006, 166 (17): 1848-1854. 10.1001/archinte.166.17.1848.CrossRefPubMed Feldstein AC, Smith DH, Perrin N, Yang X, Rix M, Raebel MA, Magid DJ, Simon SR, Soumerai SB: Improved therapeutic monitoring with several interventions: a randomized trial. Arch Intern Med. 2006, 166 (17): 1848-1854. 10.1001/archinte.166.17.1848.CrossRefPubMed
53.
go back to reference Smith DH, Feldstein AC, Perrin NA, Yang X, Rix MM, Raebel MA, Magid DJ, Simon SR, Soumerai SB: Improving laboratory monitoring of medications: an economic analysis alongside a clinical trial. Am J Manag Care. 2009, 15 (5): 281-289.PubMed Smith DH, Feldstein AC, Perrin NA, Yang X, Rix MM, Raebel MA, Magid DJ, Simon SR, Soumerai SB: Improving laboratory monitoring of medications: an economic analysis alongside a clinical trial. Am J Manag Care. 2009, 15 (5): 281-289.PubMed
54.
go back to reference Palen TE, Raebel M, Lyons E, Magid DM: Evaluation of laboratory monitoring alerts within a computerized physician order entry system for medication orders. Am J Manag Care. 2006, 12 (7): 389-395.PubMed Palen TE, Raebel M, Lyons E, Magid DM: Evaluation of laboratory monitoring alerts within a computerized physician order entry system for medication orders. Am J Manag Care. 2006, 12 (7): 389-395.PubMed
55.
go back to reference Raebel MA, Lyons EE, Chester EA, Bodily MA, Kelleher JA, Long CL, Miller C, Magid DJ: Improving laboratory monitoring at initiation of drug therapy in ambulatory care: A randomized trial. Arch Intern Med. 2005, 165 (20): 2395-2401. 10.1001/archinte.165.20.2395.CrossRefPubMed Raebel MA, Lyons EE, Chester EA, Bodily MA, Kelleher JA, Long CL, Miller C, Magid DJ: Improving laboratory monitoring at initiation of drug therapy in ambulatory care: A randomized trial. Arch Intern Med. 2005, 165 (20): 2395-2401. 10.1001/archinte.165.20.2395.CrossRefPubMed
56.
go back to reference McDonald CJ, Wilson GA, McCabe GP: Physician response to computer reminders. JAMA. 1980, 244 (14): 1579-1581. 10.1001/jama.244.14.1579.CrossRefPubMed McDonald CJ, Wilson GA, McCabe GP: Physician response to computer reminders. JAMA. 1980, 244 (14): 1579-1581. 10.1001/jama.244.14.1579.CrossRefPubMed
57.
go back to reference McDonald CJ: Use of a computer to detect and respond to clinical events: its effect on clinician behavior. Ann Intern Med. 1976, 84 (2): 162-167.CrossRef McDonald CJ: Use of a computer to detect and respond to clinical events: its effect on clinician behavior. Ann Intern Med. 1976, 84 (2): 162-167.CrossRef
58.
go back to reference Sundaram V, Lazzeroni LC, Douglass LR, Sanders GD, Tempio P, Owens DK: A randomized trial of computer-based reminders and audit and feedback to improve HIV screening in a primary care setting. Int J STD AIDS. 2009, 20 (8): 527-533. 10.1258/ijsa.2008.008423.CrossRefPubMed Sundaram V, Lazzeroni LC, Douglass LR, Sanders GD, Tempio P, Owens DK: A randomized trial of computer-based reminders and audit and feedback to improve HIV screening in a primary care setting. Int J STD AIDS. 2009, 20 (8): 527-533. 10.1258/ijsa.2008.008423.CrossRefPubMed
59.
go back to reference Roukema J, Steyerberg EW, van der Lei J, Moll HA: Randomized trial of a clinical decision support system: impact on the management of children with fever without apparent source. J Am Med Inform Assoc. 2008, 15 (1): 107-113.CrossRefPubMedPubMedCentral Roukema J, Steyerberg EW, van der Lei J, Moll HA: Randomized trial of a clinical decision support system: impact on the management of children with fever without apparent source. J Am Med Inform Assoc. 2008, 15 (1): 107-113.CrossRefPubMedPubMedCentral
60.
go back to reference Downs M, Turner S, Bryans M, Wilcock J, Keady J, Levin E, O'Carroll R, Howie K, Iliffe S: Effectiveness of educational interventions in improving detection and management of dementia in primary care: cluster randomised controlled study. BMJ. 2006, 332 (7543): 692-696. 10.1136/bmj.332.7543.692.CrossRefPubMedPubMedCentral Downs M, Turner S, Bryans M, Wilcock J, Keady J, Levin E, O'Carroll R, Howie K, Iliffe S: Effectiveness of educational interventions in improving detection and management of dementia in primary care: cluster randomised controlled study. BMJ. 2006, 332 (7543): 692-696. 10.1136/bmj.332.7543.692.CrossRefPubMedPubMedCentral
61.
go back to reference Feldstein A, Elmer PJ, Smith DH, Herson M, Orwoll E, Chen C, Aickin M, Swain MC: Electronic medical record reminder improves osteoporosis management after a fracture: a randomized, controlled trial. J Am Geriatr Soc. 2006, 54 (3): 450-457. 10.1111/j.1532-5415.2005.00618.x.CrossRefPubMed Feldstein A, Elmer PJ, Smith DH, Herson M, Orwoll E, Chen C, Aickin M, Swain MC: Electronic medical record reminder improves osteoporosis management after a fracture: a randomized, controlled trial. J Am Geriatr Soc. 2006, 54 (3): 450-457. 10.1111/j.1532-5415.2005.00618.x.CrossRefPubMed
62.
go back to reference Flottorp S, Oxman AD, Havelsrud K, Treweek S, Herrin J: Cluster randomised controlled trial of tailored interventions to improve the management of urinary tract infections in women and sore throat. BMJ. 2002, 325 (7360): 367-10.1136/bmj.325.7360.367.CrossRefPubMedPubMedCentral Flottorp S, Oxman AD, Havelsrud K, Treweek S, Herrin J: Cluster randomised controlled trial of tailored interventions to improve the management of urinary tract infections in women and sore throat. BMJ. 2002, 325 (7360): 367-10.1136/bmj.325.7360.367.CrossRefPubMedPubMedCentral
63.
go back to reference Flottorp S, Havelsrud K, Oxman AD: Process evaluation of a cluster randomized trial of tailored interventions to implement guidelines in primary care--why is it so hard to change practice?. Fam Pract. 2003, 20 (3): 333-339. 10.1093/fampra/cmg316.CrossRefPubMed Flottorp S, Havelsrud K, Oxman AD: Process evaluation of a cluster randomized trial of tailored interventions to implement guidelines in primary care--why is it so hard to change practice?. Fam Pract. 2003, 20 (3): 333-339. 10.1093/fampra/cmg316.CrossRefPubMed
64.
go back to reference McDonald CJ, Hui SL, Smith DM, Tierney WM, Cohen SJ, Weinberger M, McCabe GP: Reminders to physicians from an introspective computer medical record. A two-year randomized trial. Ann Intern Med. 1984, 100 (1): 130-138.CrossRefPubMed McDonald CJ, Hui SL, Smith DM, Tierney WM, Cohen SJ, Weinberger M, McCabe GP: Reminders to physicians from an introspective computer medical record. A two-year randomized trial. Ann Intern Med. 1984, 100 (1): 130-138.CrossRefPubMed
65.
go back to reference Javitt JC, Steinberg G, Locke T, Couch JB, Jacques J, Juster I, Reisman L: Using a claims data-based sentinel system to improve compliance with clinical guidelines: results of a randomized prospective study. Am J Manag Care. 2005, 11 (2): 93-102.PubMed Javitt JC, Steinberg G, Locke T, Couch JB, Jacques J, Juster I, Reisman L: Using a claims data-based sentinel system to improve compliance with clinical guidelines: results of a randomized prospective study. Am J Manag Care. 2005, 11 (2): 93-102.PubMed
66.
go back to reference Bates DW, Kuperman GJ, Rittenberg E, Teich JM, Fiskio J, Ma'Luf N, Onderdonk A, Wybenga D, Winkelman J, Brennan TA: A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests. Am J Med. 1999, 106 (2): 144-150. 10.1016/S0002-9343(98)00410-0.CrossRefPubMed Bates DW, Kuperman GJ, Rittenberg E, Teich JM, Fiskio J, Ma'Luf N, Onderdonk A, Wybenga D, Winkelman J, Brennan TA: A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests. Am J Med. 1999, 106 (2): 144-150. 10.1016/S0002-9343(98)00410-0.CrossRefPubMed
67.
go back to reference Tierney WM, McDonald CJ, Hui SL, Martin DK: Computer predictions of abnormal test results: effects on outpatient testing. JAMA. 1988, 259 (8): 1194-1198. 10.1001/jama.259.8.1194.CrossRefPubMed Tierney WM, McDonald CJ, Hui SL, Martin DK: Computer predictions of abnormal test results: effects on outpatient testing. JAMA. 1988, 259 (8): 1194-1198. 10.1001/jama.259.8.1194.CrossRefPubMed
68.
go back to reference Overhage JM, Tierney WM, Zhou XH, McDonald CJ: A randomized trial of 'corollary orders' to prevent errors of omission. J Am Med Inform Assoc. 1997, 4 (5): 364-375. 10.1136/jamia.1997.0040364.CrossRefPubMedPubMedCentral Overhage JM, Tierney WM, Zhou XH, McDonald CJ: A randomized trial of 'corollary orders' to prevent errors of omission. J Am Med Inform Assoc. 1997, 4 (5): 364-375. 10.1136/jamia.1997.0040364.CrossRefPubMedPubMedCentral
69.
go back to reference Berner ES: Clinical decision support systems: State of the Art. 2009, Rockville, Maryland: Agency for Healthcare Research and Quality Berner ES: Clinical decision support systems: State of the Art. 2009, Rockville, Maryland: Agency for Healthcare Research and Quality
70.
go back to reference Tierney WM, Miller ME, Overhage JM, McDonald CJ: Physician inpatient order writing on microcomputer workstations. Effects on resource utilization. JAMA. 1993, 269 (3): 379-383. 10.1001/jama.269.3.379.CrossRefPubMed Tierney WM, Miller ME, Overhage JM, McDonald CJ: Physician inpatient order writing on microcomputer workstations. Effects on resource utilization. JAMA. 1993, 269 (3): 379-383. 10.1001/jama.269.3.379.CrossRefPubMed
71.
go back to reference Field TS, Rochon P, Lee M, Gavendo L, Baril JL, Gurwitz JH: Computerized clinical decision support during medication ordering for long-term care residents with renal insufficiency. J Am Med Inform Assoc. 2009, 16 (4): 480-485. 10.1197/jamia.M2981.CrossRefPubMedPubMedCentral Field TS, Rochon P, Lee M, Gavendo L, Baril JL, Gurwitz JH: Computerized clinical decision support during medication ordering for long-term care residents with renal insufficiency. J Am Med Inform Assoc. 2009, 16 (4): 480-485. 10.1197/jamia.M2981.CrossRefPubMedPubMedCentral
72.
go back to reference Thomas JC, Moore A, Qualls PE: The effect on cost of medical care for patients treated with an automated clinical audit system. J Med Syst. 1983, 7 (3): 307-313. 10.1007/BF00993294.CrossRefPubMed Thomas JC, Moore A, Qualls PE: The effect on cost of medical care for patients treated with an automated clinical audit system. J Med Syst. 1983, 7 (3): 307-313. 10.1007/BF00993294.CrossRefPubMed
73.
go back to reference Hedges LV, Olkin I: Statistical methods for meta-analysis. 1985, Orlando: Academic Press Hedges LV, Olkin I: Statistical methods for meta-analysis. 1985, Orlando: Academic Press
74.
go back to reference Kawamoto K, Houlihan CA, Balas EA, Lobach DF: Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005, 330 (7494): 765-10.1136/bmj.38398.500764.8F.CrossRefPubMedPubMedCentral Kawamoto K, Houlihan CA, Balas EA, Lobach DF: Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005, 330 (7494): 765-10.1136/bmj.38398.500764.8F.CrossRefPubMedPubMedCentral
75.
go back to reference Jaspers MW, Smeulers M, Vermeulen H, Peute LW: Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings. J Am Med Inform Assoc. 2011, 18 (3): 327-334. 10.1136/amiajnl-2011-000094.CrossRefPubMedPubMedCentral Jaspers MW, Smeulers M, Vermeulen H, Peute LW: Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings. J Am Med Inform Assoc. 2011, 18 (3): 327-334. 10.1136/amiajnl-2011-000094.CrossRefPubMedPubMedCentral
76.
go back to reference Brenner DJ, Hall EJ: Computed tomography--an increasing source of radiation exposure. N Engl J Med. 2007, 357 (22): 2277-2284. 10.1056/NEJMra072149.CrossRefPubMed Brenner DJ, Hall EJ: Computed tomography--an increasing source of radiation exposure. N Engl J Med. 2007, 357 (22): 2277-2284. 10.1056/NEJMra072149.CrossRefPubMed
77.
go back to reference Fazel R, Krumholz HM, Wang Y, Ross JS, Chen J, Ting HH, Shah ND, Nasir K, Einstein AJ, Nallamothu BK: Exposure to low-dose ionizing radiation from medical imaging procedures. N Engl J Med. 2009, 361 (9): 849-857. 10.1056/NEJMoa0901249.CrossRefPubMedPubMedCentral Fazel R, Krumholz HM, Wang Y, Ross JS, Chen J, Ting HH, Shah ND, Nasir K, Einstein AJ, Nallamothu BK: Exposure to low-dose ionizing radiation from medical imaging procedures. N Engl J Med. 2009, 361 (9): 849-857. 10.1056/NEJMoa0901249.CrossRefPubMedPubMedCentral
Metadata
Title
Can computerized clinical decision support systems improve practitioners' diagnostic test ordering behavior? A decision-maker-researcher partnership systematic review
Authors
Pavel S Roshanov
John J You
Jasmine Dhaliwal
David Koff
Jean A Mackay
Lorraine Weise-Kelly
Tamara Navarro
Nancy L Wilczynski
R Brian Haynes
the CCDSS Systematic Review Team
Publication date
01-12-2011
Publisher
BioMed Central
Published in
Implementation Science / Issue 1/2011
Electronic ISSN: 1748-5908
DOI
https://doi.org/10.1186/1748-5908-6-88

Other articles of this Issue 1/2011

Implementation Science 1/2011 Go to the issue