Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
Implementation Science
Published in: Implementation Science 1/2018

Open Access 01-12-2018 | Systematic review

A systematic review of trials evaluating success factors of interventions with computerised clinical decision support

Authors: Stijn Van de Velde, Annemie Heselmans, Nicolas Delvaux, Linn Brandt, Luis Marco-Ruiz, David Spitaels, Hanne Cloetens, Tiina Kortteisto, Pavel Roshanov, Ilkka Kunnamo, Bert Aertgeerts, Per Olav Vandvik, Signe Flottorp

Published in: Implementation Science | Issue 1/2018

Login to get access

Abstract

Background

Computerised clinical decision support (CDS) can potentially better inform decisions, and it can help with the management of information overload. It is perceived to be a key component of a learning health care system. Despite its increasing implementation worldwide, it remains uncertain why the effect of CDS varies and which factors make CDS more effective.

Objective

To examine which factors make CDS strategies more effective on a number of outcomes, including adherence to recommended practice, patient outcome measures, economic measures, provider or patient satisfaction, and medical decision quality.

Methods

We identified randomised controlled trials, non-randomised trials, and controlled before-and-after studies that directly compared CDS implementation with a given factor to CDS without that factor by searching CENTRAL, MEDLINE, EMBASE, and CINAHL and checking reference lists of relevant studies. We considered CDS with any objective for any condition in any healthcare setting. We included CDS interventions that were either displayed on screen or provided on paper and that were directed at healthcare professionals or targeted at both professionals and patients. The reviewers screened the potentially relevant studies in duplicate. They extracted data and assessed risk of bias in independent pairs or individually followed by a double check by another reviewer. We summarised results using medians and interquartile ranges and rated our certainty in the evidence using the GRADE system.

Results

We identified 66 head-to-head trials that we synthesised across 14 comparisons of CDS intervention factors. Providing CDS automatically versus on demand led to large improvements in adherence. Displaying CDS on-screen versus on paper led to moderate improvements and making CDS more versus less patient-specific improved adherence modestly. When CDS interventions were combined with professional-oriented strategies, combined with patient-oriented strategies, or combined with staff-oriented strategies, then adherence improved slightly. Providing CDS to patients slightly increased adherence versus CDS aimed at the healthcare provider only. Making CDS advice more explicit and requiring users to respond to the advice made little or no difference. The CDS intervention factors made little or no difference to patient outcomes. The results for economic outcomes and satisfaction outcomes were sparse.

Conclusion

Multiple factors may affect the success of CDS interventions. CDS may be more effective when the advice is provided automatically and displayed on-screen and when the suggestions are more patient-specific. CDS interventions combined with other strategies probably also improves adherence. Providing CDS directly to patients may also positively affect adherence. The certainty of the evidence was low to moderate for all factors.

Trial registration

PROSPERO, CRD42016033738
Appendix
Available only for authorised users
Literature
1.
Institute of Medicine (US) Roundtable on Evidence-Based Medicine; Olsen LA, Aisner D, McGinnis JM,editors. The learning healthcare system: workshop summary. Olsen LA, Aisner D, McGinnis JM, editors. Washington (DC); National Academies Press; 2007.
2.
3.
Middleton B, Sittig DF, Wright A. Clinical decision support: a 25 year retrospective and a 25 year vision. Yearb Med Inform. 2016;(Suppl 1):S103–16.
4.
Fretheim A, Flottorp S, Oxman A. Effect of interventions for implementing clinical practice guidelines. Oslo: Knowledge Centre for the Health Services at The Norwegian Institute of Publich Health (NIPH); 2015. Report from Norwegian Knowledge Centre for the Health Services (NOKC) No. 10-2015. Available from: https://​www.​ncbi.​nlm.​nih.​gov/​books/​NBK390571/​.
5.
Moja L, Kwag KH, Lytras T, Bertizzolo L, Brandt L, Pecoraro V, Rigon G, Vaona A, Ruggiero F, Mangia M, et al. Effectiveness of computerized decision support systems linked to electronic health records: a systematic review and meta-analysis. Am J Public Health. 2014;104(12):e12–22.CrossRefPubMedPubMedCentral
6.
Lobach D, Sanders GD, Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux R, Samsa G, Hasselblad V, et al. Enabling health care decisionmaking through clinical decision support and knowledge management. Evid Rep Technol Assess. 2012;203:1–784.
7.
Myers RB, Jones SL, Sittig DF. Review of reported clinical information system adverse events in US Food and Drug Administration databases. Appl Clin inform. 2011;2(1):63–74.CrossRefPubMedPubMedCentral
8.
Ash JS, Sittig DF, Campbell EM, Guappone KP, Dykstra RH. Some unintended consequences of clinical decision support systems. AMIA Annu Symp Proc. 2007:26–30.
9.
10.
Sikka R, Morath JM, Leape L. The quadruple aim: care, health, cost and meaning in work. BMJ Qual Saf. 2015;24(10):608–10.CrossRefPubMed
11.
Roshanov PS, Fernandes N, Wilczynski JM, Hemens BJ, You JJ, Handler SM, Nieuwlaat R, Souza NM, Beyene J, Van Spall HG, et al. Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ. 2013;346:f657.CrossRefPubMed
12.
Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005;330(7494):765.CrossRefPubMedPubMedCentral
13.
Damiani G, Pinnarelli L, Colosimo SC, Almiento R, Sicuro L, Galasso R, Sommella L, Ricciardi W. The effectiveness of computerized clinical guidelines in the process of care: a systematic review. BMC Health Serv Res. 2010;10:2.CrossRefPubMedPubMedCentral
14.
Nuckols TK, Smith-Spangler C, Morton SC, Asch SM, Patel VM, Anderson LJ, Deichsel EL, Shekelle PG. The effectiveness of computerized order entry at reducing preventable adverse drug events and medication errors in hospital settings: a systematic review and meta-analysis. Syst Rev. 2014;3:56.CrossRefPubMedPubMedCentral
15.
Fillmore CL, Rommel CA, Welch BM, Zhang M, Kawamoto K. The perils of meta-regression to identify clinical decision support system success factors. J Biomed Inform. 2015;56:65–8.CrossRefPubMedPubMedCentral
16.
17.
Van de Velde S, Roshanov P, Heselmans A, Delvaux N, Brandt L, Cloetens H, Spitaels D, Marco-Ruiz L, Kortteisto T, Kunnamo I, et al. A systematic review of trials evaluating success features of computerised clinical decision support systems. In: PROSPERO International prospective register of systematic reviews; 2016.
18.
Van de Velde S, Kunnamo I, Roshanov P, Kortteisto T, Aertgeerts B, Vandvik PO, Flottorp S and GUIDES expert panel. The GUIDES checklist: development of a tool to improve the successful use of guideline-based computerised clinical decision support. Implementation Science 2018;13:86.
19.
20.
Shojania KG, Jennings A, Mayhew A, Ramsay CR, Eccles MP, Grimshaw J. The effects of on-screen, point of care computer reminders on processes and outcomes of care. Cochrane Database Syst Rev. 2009;(3):CD001096.
21.
Arditi C, Rege-Walther M, Wyatt JC, Durieux P, Burnand B. Computer-generated reminders delivered on paper to healthcare professionals; effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2012;12:CD001175.PubMed
22.
Balshem H, Helfand M, Schunemann HJ, Oxman AD, Kunz R, Brozek J, Vist GE, Falck-Ytter Y, Meerpohl J, Norris S, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64(4):401–6.CrossRefPubMed
23.
Calhoun AW, Guyatt GH, Cabana MD, Lu D, Turner DA, Valentine S, Randolph AG. Addressing the unit of analysis in medical care studies: a systematic review. Med Care. 2008;46(6):635–43.CrossRefPubMed
24.
Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, Whitty P, Eccles MP, Matowe L, Shirran L, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004;8(6):iii–v. 1-72CrossRefPubMed
25.
Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, O'Brien MA, Johansen M, Grimshaw J, Oxman AD. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;(6):CD000259.
26.
O'Brien MA, Rogers S, Jamtvedt G, Oxman AD, Odgaard-Jensen J, Kristoffersen DT, Forsetlund L, Bainbridge D, Freemantle N, Davis DA, et al. Educational outreach visits: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2007;(4):CD000409.
27.
Forsetlund L, Bjorndal A, Rashidian A, Jamtvedt G, O'Brien MA, Wolf F, Davis D, Odgaard-Jensen J, Oxman AD. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009;(2):CD003030.
28.
Rashidian A, Omidvari AH, Vali Y, Sturm H, Oxman AD. Pharmaceutical policies: effects of financial incentives for prescribers. Cochrane Database Syst Rev. 2015;(8):CD006731.
29.
Pantoja T, Green ME, Grimshaw J, Denig P, Durieux P, Gill P, Colomer N, Castañon C, Leniz J. Manual paper reminders: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2014;(9):CD001174.
30.
Arditi C, Rege-Walther M, Durieux P, Burnand B. Computer-generated reminders delivered on paper to healthcare professionals: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2017;7:CD001175.PubMed
31.
Lobach DF, Kawamoto K, Anstrom KJ, Silvey GM, Willis JM, Johnson FS, Edwards R, Simo J, Phillips P, Crosslin DR, et al. A randomized trial of population-based clinical decision support to manage health and resource use for Medicaid beneficiaries. J Med Syst. 2013;37(1):9922.CrossRefPubMed
32.
Del Fiol G, Haug PJ, Cimino JJ, Narus SP, Norlin C, Mitchell JA. Effectiveness of topic-specific infobuttons: a randomized controlled trial. J Am Med Inform Assoc. 2008;15(6):752–9.CrossRefPubMedPubMedCentral
33.
Denig P, Schuling J, Haaijer-Ruskamp F, Voorham J. Effects of a patient oriented decision aid for prioritising treatment goals in diabetes: pragmatic randomised controlled trial. BMJ. 2014;349:g5651.CrossRefPubMed
34.
Rosenbloom ST, Geissbuhler AJ, Dupont WD, Giuse DA, Talbert DA, Tierney WM, Plummer WD, Stead WW, Miller RA. Effect of CPOE user interface design on user-initiated access to educational and patient information during clinical care. J Am Med Inform Assoc. 2005;12(4):458–73.CrossRefPubMedPubMedCentral
35.
Sintchenko V, Magrabi F, Tipper S. Are we measuring the right end-points? Variables that affect the impact of computerised decision support on patient outcomes: a systematic review. Med Inform Internet Med. 2007;32(3):225–40.CrossRefPubMed
36.
Jaspers MW, Smeulers M, Vermeulen H, Peute LW. Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings. J Am Med Inform Assoc. 2011;18(3):327–34.CrossRefPubMedPubMedCentral
37.
Thompson SG, Higgins J. How should meta-regression analyses be undertaken and interpreted? Stat Med. 2002;21(11):1559–73.CrossRefPubMed
38.
Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, Alonso-Coello P, Glasziou P, Jaeschke R, Akl EA, et al. GRADE guidelines: 7. Rating the quality of evidence--inconsistency. J Clin Epidemiol. 2011;64(12):1294–302.CrossRefPubMed
39.
Shojania KG, Grimshaw JM. Still no magic bullets: pursuing more rigorous research in quality improvement. Am J Med. 2004;116(11):778–80.
40.
Brandt L, Vandvik PO, Alonso-Coello P, Akl EA, Thornton J, Rigau D, Adams K, O'Connor P, Guyatt G, Kristiansen A. Multilayered and digitally structured presentation formats of trustworthy recommendations: a combined survey and randomised trial. BMJ Open. 2017;7(2):e011569.CrossRefPubMedPubMedCentral
Metadata
Title
A systematic review of trials evaluating success factors of interventions with computerised clinical decision support
Authors
Stijn Van de Velde
Annemie Heselmans
Nicolas Delvaux
Linn Brandt
Luis Marco-Ruiz
David Spitaels
Hanne Cloetens
Tiina Kortteisto
Pavel Roshanov
Ilkka Kunnamo
Bert Aertgeerts
Per Olav Vandvik
Signe Flottorp
Publication date
01-12-2018
Publisher
BioMed Central
Published in
Implementation Science / Issue 1/2018
Electronic ISSN: 1748-5908
DOI
https://doi.org/10.1186/s13012-018-0790-1

Other articles of this Issue 1/2018

Implementation Science 1/2018 Go to the issue

Research

Associations among unit leadership and unit climates for implementation in acute care: a cross-sectional study

Research

To fill or not to fill: a qualitative cross-country study on dentists’ decisions in managing non-cavitated proximal caries lesions

Research

Assessing organizational implementation context in the education sector: confirmatory factor analysis of measures of implementation leadership, climate, and citizenship

Study protocol

Sustainable reduction of antibiotic-induced antimicrobial resistance (ARena) in German ambulatory care: study protocol of a cluster randomised trial

Research

Cluster randomised controlled trial of a theory-based multiple behaviour change intervention aimed at healthcare professionals to improve their management of type 2 diabetes in primary care

Systematic review

Reduction of antibiotic prescriptions for acute respiratory tract infections in primary care: a systematic review