Top

Published in:

Open Access 01-12-2018 | Research

Health professionals’ perceptions about their clinical performance and the influence of audit and feedback on their intentions to improve practice: a theory-based study in Dutch intensive care units

Authors: Wouter T. Gude, Marie-José Roos-Blom, Sabine N. van der Veer, Dave A. Dongelmans, Evert de Jonge, Jill J. Francis, Niels Peek, Nicolette F. de Keizer

Published in: Implementation Science | Issue 1/2018

Abstract

Background

Audit and feedback aims to guide health professionals in improving aspects of their practice that need it most. Evidence suggests that feedback fails to increase accuracy of professional perceptions about clinical performance, which likely reduces audit and feedback effectiveness. This study investigates health professionals’ perceptions about their clinical performance and the influence of feedback on their intentions to change practice.

Methods

We conducted an online laboratory experiment guided by Control Theory with 72 intensive care professionals from 21 units. For each of four new pain management indicators, we collected professionals’ perceptions about their clinical performance; peer performance; targets; and improvement intentions before and after receiving first-time feedback. An electronic audit and feedback dashboard provided ICU’s own performance, median and top 10% peer performance, and improvement recommendations. The experiment took place approximately 1 month before units enrolled into a cluster-randomised trial assessing the impact of adding a toolbox with suggested actions and materials to improve intensive care pain management. During the experiment, the toolbox was inaccessible; all participants accessed the same version of the dashboard.

Results

We analysed 288 observations. In 53.8%, intensive care professionals overestimated their clinical performance; but in only 13.5%, they underestimated it. On average, performance was overestimated by 22.9% (on a 0–100% scale). Professionals similarly overestimated peer performance, and set targets 20.3% higher than the top performance benchmarks. In 68.4% of cases, intentions to improve practice were consistent with actual gaps in performance, even before professionals had received feedback; which increased to 79.9% after receiving feedback (odds ratio, 2.41; 95% CI, 1.53 to 3.78). However, in 56.3% of cases, professionals still wanted to improve care aspects at which they were already top performers. Alternatively, in 8.3% of cases, they lacked improvement intentions because they did not consider indicators important; did not trust the data; or deemed benchmarks unrealistic.

Conclusions

Audit and feedback helps health professionals to work on aspects for which improvement is recommended. Given the abundance of professionals’ prior good improvement intentions, the limited effects typically found by audit and feedback studies are likely predominantly caused by barriers to translation of intentions into actual change in clinical practice.

Trial registration

ClinicalTrials.gov NCT02922101. Registered 26 September 2016.

Available only for authorised users

Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;6:CD000259. https://doi.org/10.1002/14651858.CD000259.pub3.

Ivers NM, Sales A, Colquhoun H, Michie S, Foy R, Francis JJ, et al. No more “business as usual” with audit and feedback interventions: towards an agenda for a reinvigorated intervention. Implement Sci. 2014;9:14. https://doi.org/10.1186/1748-5908-9-14.CrossRefPubMedCentralPubMed

Gude WT, van der Veer SN, de Keizer NF, Coiera E, Peek N. Optimizing digital health informatics interventions through unobtrusive quantitative process evaluations. Stud Health Technol Inform. 2016;228:594–8.PubMed

Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence a systematic review. JAMA. 2006;296:1094–102. https://doi.org/10.1001/jama.296.9.1094.CrossRefPubMed

Carver CS, Scheier MF. Control theory: a useful conceptual framework for personality-social, clinical, and health psychology. Psychol Bull. 1982;92:111–35. https://doi.org/10.1037/0033-2909.92.1.111.CrossRefPubMed

Gude WT, van Engen-Verheul MM, van der Veer SN, de Keizer NF, Peek N. How does audit and feedback influence intentions of health professionals to improve practice? A laboratory experiment and field study in cardiac rehabilitation. BMJ Qual Saf. 2017;26:279–87. https://doi.org/10.1136/bmjqs-2015-004795.CrossRefPubMed

van der Veer SN, de Vos MLG, Jager KJ, van der Voort PHJ, Peek N, Westert GP, et al. Evaluating the effectiveness of a tailored multifaceted performance feedback intervention to improve the quality of care: protocol for a cluster randomized trial in intensive care. Implement Sci. 2011;6:119. https://doi.org/10.1186/1748-5908-6-119.CrossRefPubMedCentralPubMed

Brown B, Jameson D, Daker-White G, Buchan I, Ivers N, Peek N, et al. A meta-synthesis of findings from qualitative studies of audit and feedback interventions. PROSPERO. 2015;CRD4201501. http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD420150175

Eva KW, Armson H, Holmboe E, Lockyer J, Loney E, Mann K, et al. Factors influencing responsiveness to feedback: on the interplay between fear, confidence, and reasoning processes. Adv Health Sci Educ Theory Pract. 2012;17:15–26. https://doi.org/10.1007/s10459-011-9290-7.CrossRefPubMed

10.

Kluger AN, DeNisi A. The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol Bull. 1996;119:254–84. https://doi.org/10.1037/0033-2909.119.2.254.CrossRef

11.

Ajzen I, Netemeyer R, Van Ryn M, Ajzen I. The theory of planned behaviour. Organ Behav Hum Dec. 1991;50:179–211. https://doi.org/10.1016/0749-5978(91)90020-T.CrossRef

12.

Eccles MP, Hrisos S, Francis J, Kaner EF, Dickinson HO, Beyer F, et al. Do self-reported intentions predict clinicians’ behaviour: a systematic review. Implement Sci. 2006;1:28. https://doi.org/10.1186/1748-5908-1-28.CrossRefPubMedCentralPubMed

13.

Gude WT, Roos-Blom M-J, van der Veer SN, de Jonge E, Peek N, Dongelmans DA, et al. Electronic audit and feedback intervention with action implementation toolbox to improve pain management in intensive care: protocol for a laboratory experiment and cluster randomised trial. Implement Sci. 2017;12:68. https://doi.org/10.1186/s13012-017-0594-8.CrossRefPubMedCentralPubMed

14.

Gardner B, Whittington C, McAteer J, Eccles MP, Michie S. Using theory to synthesise evidence from behaviour change interventions: the example of audit and feedback. Soc Sci Med. 2010;70:1618–25. https://doi.org/10.1016/j.socscimed.2010.01.039.CrossRefPubMed

15.

van der Veer SN, de Keizer NF, Ravelli ACJ, Tenkink S, Jager KJ. Improving quality of care. A systematic review on how medical registries provide information feedback to health care providers. Int J Med Inform. 2010;79:305–23. https://doi.org/10.1016/j.ijmedinf.2010.01.011.CrossRefPubMed

16.

Kiefe CI, Allison JJ, Williams OD, Person SD, Weaver MT, Weissman NW. Improving quality improvement using achievable benchmarks for physician feedback: a randomized controlled trial. JAMA. 2001;285:2871–9.CrossRefPubMed

17.

Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PA, et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;282:1458–65. https://doi.org/10.1001/jama.282.15.1458.CrossRefPubMed

18.

Gude WT, Van Der Veer SN, Van Engen-Verheul MM, De Keizer NF, Peek N. Inside the black box of audit and feedback: a laboratory study to explore determinants of improvement target selection by healthcare professionals in cardiac rehabilitation. vol. 216. 2015. doi:https://doi.org/10.3233/978-1-61499-564-7-424.

19.

Presseau J, Sniehotta FF, Francis JJ, Campbell NC. Multiple goals and time constraints: perceived impact on physicians’ performance of evidence-based behaviours. Implement Sci. 2009;4:77. https://doi.org/10.1186/1748-5908-4-77.CrossRefPubMedCentralPubMed

20.

Pannick S, Sevdalis N, Athanasiou T. Beyond clinical engagement: a pragmatic model for quality improvement interventions, aligning clinical and managerial priorities. BMJ Qual Saf. 2015;bmjqs-2015-004453. doi:https://doi.org/10.1136/bmjqs-2015-004453.

21.

Colquhoun HL, Carroll K, Eva KW, Grimshaw JM, Ivers N, Michie S, et al. Advancing the literature on designing audit and feedback interventions: identifying theory-informed hypotheses. Implement Sci. 2017;12:117. https://doi.org/10.1186/s13012-017-0646-0.CrossRefPubMedCentralPubMed

22.

Schneider A, Wensing M, Biessecker K, Quinzler R, Kaufmann-Kolle P, Szecsenyi J. Impact of quality circles for improvement of asthma care: results of a randomized controlled trial. J Eval Clin Pract. 2008;14:185–90. https://doi.org/10.1111/j.1365-2753.2007.00827.x.CrossRefPubMedCentralPubMed

23.

Brehaut JC, Colquhoun HL, Eva KW, Carroll K, Sales A, Michie S, et al. Practice feedback interventions: 15 suggestions for optimizing effectiveness. Ann Intern Med. 2016;164:435–41. https://doi.org/10.7326/M15-2248.CrossRefPubMed

24.

Tuti T, Nzinga J, Njoroge M, Brown B, Peek N, English M, et al. A systematic review of use of behaviour change theory in electronic audit and feedback interventions. Implement Sci. 2017;12(1):61. https://doi.org/10.1186/s13012-017-0590-z.

25.

Colquhoun HL, Brehaut JC, Sales A, Ivers N, Grimshaw J, Michie S, et al. A systematic review of the use of theory in randomized controlled trials of audit and feedback. Implement Sci. 2013;8:66. https://doi.org/10.1186/1748-5908-8-66.CrossRefPubMedCentralPubMed

26.

Hartley S, Foy R, Walwyn REA, Cicero R, Farrin AJ, Francis JJ, et al. The evaluation of enhanced feedback interventions to reduce unnecessary blood transfusions (AFFINITIE): protocol for two linked cluster randomised factorial controlled trials. Implement Sci. 2017;12:84. https://doi.org/10.1186/s13012-017-0614-8.CrossRefPubMedCentralPubMed

27.

de Vos Maartje LG, van der Veer SN, Graafmans WC, de Keizer NF, Jager KJ, Westert GP, et al. Process evaluation of a tailored multifaceted feedback program to improve the quality of intensive care by using quality indicators. BMJ Qual Saf. 2013;22:233–41. https://doi.org/10.1136/bmjqs-2012-001375.CrossRefPubMed

28.

Arts DGT, de Keizer NF, Scheffer G-J, Formulation M. Defining and improving data quality in medical registries: a literature review, case study, and generic framework. J Am Med Inf Assoc. 2002;9:600–11. https://doi.org/10.1197/jamia.M1087.CrossRef

29.

Brown B, Balatsoukas P, Williams R, Sperrin M, Buchan I. Interface design recommendations for computerised clinical audit and feedback: hybrid usability evidence from a research-led system. Int J Med Inform. 2016;94:191–206. https://doi.org/10.1016/j.ijmedinf.2016.07.010.CrossRefPubMedCentralPubMed

30.

Ivers NM, Grimshaw JM. Reducing research waste with implementation laboratories. Lancet (London, England). 2016;388:547–8. https://doi.org/10.1016/S0140-6736(16)31256-9.CrossRef

Title: Health professionals’ perceptions about their clinical performance and the influence of audit and feedback on their intentions to improve practice: a theory-based study in Dutch intensive care units
Authors: Wouter T. Gude
Marie-José Roos-Blom
Sabine N. van der Veer
Dave A. Dongelmans
Evert de Jonge
Jill J. Francis
Niels Peek
Nicolette F. de Keizer
Publication date: 01-12-2018
Publisher: BioMed Central
Published in: Implementation Science / Issue 1/2018
Electronic ISSN: 1748-5908
DOI: https://doi.org/10.1186/s13012-018-0727-8

Keynote webinar | Spotlight on sleep in brain health

Springer Medicine

Health professionals’ perceptions about their clinical performance and the influence of audit and feedback on their intentions to improve practice: a theory-based study in Dutch intensive care units

Abstract

Background

Methods

Results

Conclusions

Trial registration

Keynote webinar | Spotlight on sleep in brain health

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Trial registration

Please log in to get access to this content

Other articles of this Issue 1/2018

A systematic review of trials evaluating success factors of interventions with computerised clinical decision support

Longitudinal evaluation of a course to build core competencies in implementation practice

Human papillomavirus vaccine delivery in Mozambique: identification of implementation performance drivers using the Consolidated Framework for Implementation Research (CFIR)

Leveraging routine clinical materials and mobile technology to assess CBT fidelity: the Innovative Methods to Assess Psychotherapy Practices (imAPP) study

Mixed-method analysis of program leader perspectives on the sustainment of multiple child evidence-based practices in a system-driven implementation

Implementation and use of computerised clinical decision support (CCDS) in emergency pre-hospital care: a qualitative study of paramedic views and experience using Strong Structuration Theory