Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
Implementation Science
Published in: Implementation Science 1/2015

Open Access 01-12-2016 | Research

Effect of a web-based audit and feedback intervention with outreach visits on the clinical performance of multidisciplinary teams: a cluster-randomized trial in cardiac rehabilitation

Authors: Wouter T. Gude, Mariëtte M. van Engen-Verheul, Sabine N. van der Veer, Hareld M. C. Kemps, Monique W. M. Jaspers, Nicolette F. de Keizer, Niels Peek

Published in: Implementation Science | Issue 1/2015

Login to get access

Abstract

Background

The objective of this study was to assess the effect of a web-based audit and feedback (A&F) intervention with outreach visits to support decision-making by multidisciplinary teams.

Methods

We performed a multicentre cluster-randomized trial within the field of comprehensive cardiac rehabilitation (CR) in the Netherlands. Our participants were multidisciplinary teams in Dutch CR centres who were enrolled in the study between July 2012 and December 2013 and received the intervention for at least 1 year. The intervention included web-based A&F with feedback on clinical performance, facilities for goal setting and action planning, and educational outreach visits. Teams were randomized either to receive feedback that was limited to psychosocial rehabilitation (study group A) or to physical rehabilitation (study group B). The main outcome measure was the difference in performance between study groups in 11 care processes and six patient outcomes, measured at patient level. Secondary outcomes included effects on guideline concordance for the four main CR therapies.

Results

Data from 18 centres (14,847 patients) were analysed, of which 12 centres (9353 patients) were assigned to group A and six (5494 patients) to group B. During the intervention, a total of 233 quality improvement goals was identified by participating teams, of which 49 (21%) were achieved during the study period. Except for a modest improvement in data completeness (4.5% improvement per year; 95% CI 0.65 to 8.36), we found no effect of our intervention on any of our primary or secondary outcome measures.

Conclusions

Within a multidisciplinary setting, our web-based A&F intervention engaged teams to define local performance improvement goals but failed to support them in actually completing the improvement actions that were needed to achieve those goals. Future research should focus on improving the actionability of feedback on clinical performance and on addressing the socio-technical perspective of the implementation process.

Trial registration

Appendix
Available only for authorised users
Literature
1.
Wolff JL, Starfield B, Anderson G. Prevalence, expenditures, and complications of multiple chronic conditions in the elderly. Arch Intern Med. 2002;162(20):2269–76.CrossRefPubMed
2.
van Oostrom SH, Picavet HS, van Gelder BM, Lemmens LC, Hoeymans N, van Dijk CE, et al. Multimorbidity and comorbidity in the Dutch population—data from general practices. BMC Public Health. 2012;12:715.CrossRefPubMedPubMedCentral
3.
Friedman C, Rubin J, Brown J, Buntin M, Corn M, Etheredge L, et al. Toward a science of learning systems: a research agenda for the high-functioning Learning Health System. J Am Med Inform Assoc. 2015;22(1):43–50.PubMed
4.
Papadakis S, Oldridge NB, Coyle D, Mayhew A, Reid RD, Beaton L, et al. Economic evaluation of cardiac rehabilitation: a systematic review. Eur J Cardiovasc Prev Rehabil. 2005;12(6):513–20.CrossRefPubMed
5.
Piepoli MF, Corra U, Adamopoulos S, Benzer W, Bjarnason-Wehrens B, Cupples M, et al. Secondary prevention in the clinical management of patients with cardiovascular diseases. Core components, standards and outcome measures for referral and delivery: a policy statement from the cardiac rehabilitation section of the European Association for Cardiovascular Prevention & Rehabilitation. Endorsed by the Committee for Practice Guidelines of the European Society of Cardiology. Eur J Prev Cardiol. 2014;21(6):664–81.CrossRefPubMed
6.
Heran BS, Chen JM, Ebrahim S, Moxham T, Oldridge N, Rees K, et al. Exercise-based cardiac rehabilitation for coronary heart disease. Cochrane Database Syst Rev. 2011;7:CD001800.
7.
Thomas RJ, King M, Lui K, Oldridge N, Pina IL, Spertus J, et al. AACVPR/ACC/AHA 2007 performance measures on cardiac rehabilitation for referral to and delivery of cardiac rehabilitation/secondary prevention services endorsed by the American College of Chest Physicians, American College of Sports Medicine, American Physical Therapy Association, Canadian Association of Cardiac Rehabilitation, European Association for Cardiovascular Prevention and Rehabilitation, Inter-American Heart Foundation, National Association of Clinical Nurse Specialists, Preventive Cardiovascular Nurses Association, and the Society of Thoracic Surgeons. J Am Coll Cardiol. 2007;50(14):1400–33.CrossRefPubMed
8.
Lawler PR, Filion KB, Eisenberg MJ. Efficacy of exercise-based cardiac rehabilitation post-myocardial infarction: a systematic review and meta-analysis of randomized controlled trials. Am Heart J. 2011;162(4):571–84.CrossRefPubMed
9.
de Vries H, Kemps HM, van Engen-Verheul MM, Kraaijenhagen RA, Peek N. Cardiac rehabilitation and survival in a large representative community cohort of Dutch patients. Eur Heart J. 2015;36(24):1519–28. doi:10.​1093/​eurheartj/​ehv111.
10.
Bjarnason-Wehrens B, McGee H, Zwisler AD, Piepoli MF, Benzer W, Schmid JP, et al. Cardiac rehabilitation in Europe: results from the European Cardiac Rehabilitation Inventory Survey. Eur J Cardiovasc Prev Rehabil. 2010;17(4):410–8. doi:10.​1097/​HJR.​0b013e328334f42d​.
11.
Arena R, Williams M, Forman DE, Cahalin LP, Coke L, Myers J, et al. Increasing referral and participation rates to outpatient cardiac rehabilitation: the valuable role of healthcare professionals in the inpatient and home health settings: a science advisory from the American Heart Association. Circulation. 2012;125(10):1321–9.CrossRefPubMed
12.
van Engen-Verheul M, de Vries H, Kemps H, Kraaijenhagen R, de Keizer N, Peek N. Cardiac rehabilitation uptake and its determinants in the Netherlands. Eur J Prev Cardiol. 2013;20(2):349–56.CrossRefPubMed
13.
Goud R, de Keizer NF, ter Riet G, Wyatt JC, Hasman A, Hellemans IM, et al. Effect of guideline based computerised decision support on decision making of multidisciplinary teams: cluster randomised trial in cardiac rehabilitation. BMJ. 2009;338:b1440.CrossRefPubMedPubMedCentral
14.
Goud R, van Engen-Verheul M, de Keizer NF, Bal R, Hasman A, Hellemans IM, et al. The effect of computerized decision support on barriers to guideline implementation: a qualitative study in outpatient cardiac rehabilitation. Int J Med Inform. 2010;79(6):430–7.CrossRefPubMed
15.
Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;6:CD000259.
16.
17.
Gardner B, Whittington C, McAteer J, Eccles MP, Michie S. Using theory to synthesise evidence from behaviour change interventions: the example of audit and feedback. Soc Sci Med. 2010;70(10):1618–25.CrossRefPubMed
18.
Foy R, MacLennan G, Grimshaw J, Penney G, Campbell M, Grol R. Attributes of clinical recommendations that influence change in practice following audit and feedback. J Clin Epidemiol. 2002;55(7):717–22.CrossRefPubMed
19.
van der Veer SN, de Keizer NF, Ravelli AC, Tenkink S, Jager KJ. Improving quality of care. A systematic review on how medical registries provide information feedback to health care providers. Int J Med Inform. 2010;79(5):305–23.CrossRefPubMed
20.
van Engen-Verheul MM, de Keizer NF, van der Veer SN, Kemps H, Scholte op Reimer W, Jaspers M, et al. Evaluating the effect of a web-based quality improvement system with feedback and outreach visits on guideline concordance in the field of cardiac rehabilitation: rationale and study protocol. Implement Sci. 2014;9(1):780.CrossRefPubMedPubMedCentral
21.
Leon AS, Franklin BA, Costa F, Balady GJ, Berra KA, Stewart KJ, et al. cardiac rehabilitation and secondary prevention of coronary heart disease: an American Heart Association Scientific Statement from the Council on Clinical Cardiology (Subcommittee on Exercise, Cardiac Rehabilitation, and Prevention) and the Council on Nutrition, Physical Activity, and Metabolism (Subcommittee on Physical Activity), in collaboration with the American Association of Cardiovascular and Pulmonary Rehabilitation. Circulation. 2005;111(3):369–76.CrossRefPubMed
22.
Rehabilitation Committee. Netherlands Society for Cardiology (NVVC) and Netherlands Heart Foundation (NHS) (both Guidelines on Cardiac Rehabilitation 2004) and Working Group PAAHR (partial revision 2011). Multidisciplinary guidelines for cardiac rehabilitation (in Dutch). Utrecht: Netherlands Society for Cardiology (NVVC); 2011. Available at http://​www.​nvvc.​nl/​hr.​ Last accessed January 2015.
23.
van Engen-Verheul MM, Kemps HM, de Keizer NF, Hellemans IM, Goud R, Kraaijenhagen RA, et al. Revision of the Dutch clinical algorithm for assessing patient needs in cardiac rehabilitation based on identified implementation problems. Eur J Prev Cardiol. 2012;19(3):504–14.CrossRefPubMed
24.
van Engen-Verheul MM, van der Veer SN, de Keizer NF, Tjon Sjoe Sjoe W, van der Zwan EP, Peek N. A web-based system to facilitate local, systematic quality improvement by multidisciplinary care teams: development and first experiences of CARDSS Online. Stud Health Technol Inform. 2013;192:248–52. 248-52.PubMed
25.
Langley GL, Nolan KM, Nolan TW, Norman CL, Provost LP. The improvement guide: a practical approach to enhancing organizational performance (2nd edition). San Francisco: Jossey-Bass Publishers; 2009.
26.
van Engen-Verheul M, Kemps H, Kraaijenhagen R, De Keizer N, Peek N. Modified Rand method to derive quality indicators: a case study in cardiac rehabilitation. Stud Health Technol Inform. 2011;169:88–92.PubMed
27.
Goud R, Hasman A, Peek N. Development of a guideline-based decision support system with explanation facilities for outpatient therapy. Comput Methods Programs Biomed. 2008;91(2):145–53.CrossRefPubMed
28.
Pinheiro JC, Bates DM. Mixed effects models in S and S-plus. New-York: Springer; 2000. p. 2000.CrossRef
29.
White IR, Royston P, Wood AM. Multiple imputation using chained equations: issues and guidance for practice. Stat Med. 2011;30(4):377–99.CrossRefPubMed
30.
Taylor MJ, McNicholas C, Nicolay C, Darzi A, Bell D, Reed JE. Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Qual Saf. 2014;23(4):290–8.CrossRefPubMed
31.
Carver CS, Scheier MF. Control theory: a useful conceptual framework for personality-social, clinical, and health psychology. Psychol Bull. 1982;92(1):111–35.CrossRefPubMed
32.
Gude WT, van Engen-Verheul MM, van der Veer SN, de Keizer NF, Peek N. How does audit and feedback influence intentions of health professionals to improve practice? A laboratory experiment and field study in cardiac rehabilitation. BMJ Qual Saf. 2016. pii: bmjqs-2015-004795. doi:10.​1136/​bmjqs-2015-004795.
33.
de Vos MLG, van der Veer SN, Graafmans WC, de Keizer NF, Jager KJ, Westert GP, et al. Process evaluation of a tailored multifaceted feedback program to improve the quality of intensive care by using quality indicators. BMJ Qual Saf. 2013;22(3):233–41.CrossRef
34.
Sinuff T, Muscedere J, Rozmovits L, Dale CM, Scales DC. A qualitative study of the variable effects of audit and feedback in the ICU. BMJ Qual Saf. 2015;24(6):393–9.CrossRefPubMed
35.
Gardner KL, Sibthorpe B, Longstaff D. National quality and performance system for Divisions of General Practice: early reflections on a system under development. Aust New Zealand Health Policy. 2008;5:8.CrossRefPubMedPubMedCentral
36.
Ivers N, Barnsley J, Upshur R, Tu K, Shah B, Grimshaw J, et al. “My approach to this job is…one person at a time”: perceived discordance between population-level quality targets and patient-centred care. Can Fam Physician. 2014;60(3):258–66.PubMedPubMedCentral
37.
Wahlstrom R, Kounnavong S, Sisounthone B, Phanyanouvong A, Southammavong T, Eriksson B, et al. Effectiveness of feedback for improving case management of malaria, diarrhoea and pneumonia—a randomized controlled trial at provincial hospitals in Lao PDR. Trop Med Int Health. 2003;8(10):901–9.CrossRefPubMed
38.
Raasch BA, Hays R, Buettner PG. An educational intervention to improve diagnosis and management of suspicious skin lesions. J Contin Educ Heal Prof. 2000;20(1):39–51.CrossRef
39.
van Engen-Verheul MM, Peek N, Haafkens JA, Joukes E, Vromen T, Jaspers MWM, et al. What is needed to implement a web-based audit and feedback intervention with outreach visits to improve care quality: a concept mapping study among cardiac rehabilitation teams. Int J Med Inform. 2017;97:76-85. doi:10.​1016/​j.​ijmedinf.​2016.​10.​003.
40.
Ancker JS, Kern LM, Abramson E, Kaushal R. The Triangle Model for evaluating the effect of health information technology on healthcare quality and safety. J Am Med Inform Assoc. 2012;19(1):61–5.CrossRefPubMed
41.
King HB, Battles J, Baker DP, Alonso A, Salas E, Webster J, et al. Advances in patient safety TeamSTEPPS: team strategies and tools to enhance performance and patient safety. In: Henriksen K, Battles JB, Keyes MA, Grady ML, editors. Advances in patient safety: new directions and alternative approaches (vol 3: performance and tools). Rockville: Agency for Healthcare Research and Quality (US); 2008.
42.
Carayon P, Schoofs Hundt A, Karsh BT, Gurses AP, Alvarado CJ, Smith M, et al. Work system design for patient safety: the SEIPS model. Qual Saf Health Care. 2006;15 Suppl 1:i50–8.CrossRefPubMedPubMedCentral
43.
Sittig DF, Kahol K, Singh H. Sociotechnical evaluation of the safety and effectiveness of point-of-care mobile computing devices: a case study conducted in India. Stud Health Technol Inform. 2013;192:515–9.PubMed
44.
Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PA, et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;282(15):1458–65.CrossRefPubMed
Metadata
Title
Effect of a web-based audit and feedback intervention with outreach visits on the clinical performance of multidisciplinary teams: a cluster-randomized trial in cardiac rehabilitation
Authors
Wouter T. Gude
Mariëtte M. van Engen-Verheul
Sabine N. van der Veer
Hareld M. C. Kemps
Monique W. M. Jaspers
Nicolette F. de Keizer
Niels Peek
Publication date
01-12-2016
Publisher
BioMed Central
Published in
Implementation Science / Issue 1/2015
Electronic ISSN: 1748-5908
DOI
https://doi.org/10.1186/s13012-016-0516-1

Other articles of this Issue 1/2015

Implementation Science 1/2015 Go to the issue

Study protocol

The 4KEEPS study: identifying predictors of sustainment of multiple practices fiscally mandated in children’s mental health services

Systematic review

A systematic review of the use of the Consolidated Framework for Implementation Research

Systematic review

Strategies for disseminating recommendations or guidelines to patients: a systematic review

Research

A theory-based process evaluation alongside a randomised controlled trial of printed educational messages to increase primary care physicians’ prescription of thiazide diuretics for hypertension [ISRCTN72772651]

Methodology

Expanding the evidence base for global recommendations on health systems: strengths and challenges of the OptimizeMNH guidance process

Study protocol

Researching effective approaches to cleaning in hospitals: protocol of the REACH study, a multi-site stepped-wedge randomised trial