Skip to main content
Top
Published in: Journal of General Internal Medicine 6/2022

01-05-2022 | Care | Original Research

Effect of Peer Benchmarking on Specialist Electronic Consult Performance in a Los Angeles Safety-Net: a Cluster Randomized Trial

Authors: Daniella Meeker, PhD, Mark W. Friedberg, MD, MPP, Tara K. Knight, PhD, Jason N. Doctor, PhD, Dina Zein, MHA, Nancy Cayasso-McIntosh, MPH, Noah J. Goldstein, PhD, Craig R. Fox, PhD, Jeffrey A. Linder, MD, MPH, Stephen D. Persell, MD, MPH, Stanley Dea, MD, Paul Giboney, MD, Hal F. Yee, MD, PhD, The Behavioral Economics and eConsult Steering Committee

Published in: Journal of General Internal Medicine | Issue 6/2022

Login to get access

ABSTRACT

Background

Since the advent of COVID-19, accelerated adoption of systems that reduce face-to-face encounters has outpaced training and best practices. Electronic consultations (eConsults), structured communications between PCPs and specialists regarding a case, have been effective in reducing face-to-face specialist encounters. As the health system rapidly adapts to multiple new practices and communication tools, new mechanisms to measure and improve performance in this context are needed.

Objective

To test whether feedback comparing physicians to top performing peers using co-specialists’ ratings improves performance.

Design

Cluster-randomized controlled trial

Participants

Eighty facility-specialty clusters and 214 clinicians

Intervention

Providers in the feedback arms were sent messages that announced their membership in an elite group of “Top Performers” or provided actionable recommendations with feedback for providers that were “Not Top Performers.”

Main Measures

The primary outcomes were changes in peer ratings in the following performance dimensions after feedback was received: (1) elicitation of information from primary care practitioners; (2) adherence to institutional clinical guidelines; (3) agreement with peer’s medical decision-making; (4) educational value; (5) relationship building.

Key Results

Specialists showed significant improvements on 3 of the 5 consultation performance dimensions: medical decision-making (odds ratio 1.52, 95% confidence interval 1.08–2.14, p<.05), educational value (1.86, 1.17–2.96) and relationship building (1.63, 1.13–2.35) (both p<.01).

Conclusions

The pandemic has shed light on clinicians’ commitment to professionalism and service as we rapidly adapt to changing paradigms. Interventions that appeal to professional norms can help improve the efficacy of new systems of practice. We show that specialists’ performance can be measured and improved with feedback using aspirational norms.

Trial Registration

Appendix
Available only for authorised users
Literature
2.
go back to reference Forrest CB, Glade GB, Baker AE, Bocian A, von Schrader S, Starfield B. Coordination of specialty referrals and physician satisfaction with referral care. Arch Pediatr Adolesc Med. 2000;154(5):499-506.CrossRef Forrest CB, Glade GB, Baker AE, Bocian A, von Schrader S, Starfield B. Coordination of specialty referrals and physician satisfaction with referral care. Arch Pediatr Adolesc Med. 2000;154(5):499-506.CrossRef
3.
go back to reference Liddy C, Afkham A, Drosinis P, Joschko J, Keely E. Impact of and Satisfaction with a New eConsult Service: A Mixed Methods Study of Primary Care Providers. J Am Board Fam Med. 2015;28(3):394-403.CrossRef Liddy C, Afkham A, Drosinis P, Joschko J, Keely E. Impact of and Satisfaction with a New eConsult Service: A Mixed Methods Study of Primary Care Providers. J Am Board Fam Med. 2015;28(3):394-403.CrossRef
4.
go back to reference Keely E, Canning S, Saloojee N, Afkham A, Liddy C. Improving Access to Gastroenterologist Using eConsultation: A Way to Potentially Shorten Wait Times. J Can Assoc Gastroenterol. 2018;1(3):124-128.CrossRef Keely E, Canning S, Saloojee N, Afkham A, Liddy C. Improving Access to Gastroenterologist Using eConsultation: A Way to Potentially Shorten Wait Times. J Can Assoc Gastroenterol. 2018;1(3):124-128.CrossRef
5.
go back to reference Schettini P, Shah KP, O’Leary CP, et al. Keeping care connected: e-Consultation program improves access to nephrology care. J Telemed Telecare. Published online January 1, 2017:1357633X17748350. Schettini P, Shah KP, O’Leary CP, et al. Keeping care connected: e-Consultation program improves access to nephrology care. J Telemed Telecare. Published online January 1, 2017:1357633X17748350.
6.
go back to reference Gleason N, Prasad PA, Ackerman S, et al. Adoption and impact of an eConsult system in a fee-for-service setting. Healthc (Amst). 2017;5(1-2):40-45.CrossRef Gleason N, Prasad PA, Ackerman S, et al. Adoption and impact of an eConsult system in a fee-for-service setting. Healthc (Amst). 2017;5(1-2):40-45.CrossRef
7.
go back to reference Lee MS, Ray KN, Mehrotra A, Giboney P, Yee HF Jr, Barnett ML. Primary Care Practitioners’ Perceptions of Electronic Consult Systems: A Qualitative Analysis. JAMA Intern Med. 2018;178(6):782-789.CrossRef Lee MS, Ray KN, Mehrotra A, Giboney P, Yee HF Jr, Barnett ML. Primary Care Practitioners’ Perceptions of Electronic Consult Systems: A Qualitative Analysis. JAMA Intern Med. 2018;178(6):782-789.CrossRef
8.
go back to reference Naseriasl M, Adham D, Janati A. E-referral Solutions: Successful Experiences, Key Features and Challenges- a Systematic Review. Mater Sociomed. 2015;27(3):195-199.CrossRef Naseriasl M, Adham D, Janati A. E-referral Solutions: Successful Experiences, Key Features and Challenges- a Systematic Review. Mater Sociomed. 2015;27(3):195-199.CrossRef
9.
go back to reference Barnett ML, Yee HF Jr, Mehrotra A, Giboney P. Los Angeles Safety-Net Program eConsult System Was Rapidly Adopted And Decreased Wait Times To See Specialists. Health Aff . 2017;36(3):492-499.CrossRef Barnett ML, Yee HF Jr, Mehrotra A, Giboney P. Los Angeles Safety-Net Program eConsult System Was Rapidly Adopted And Decreased Wait Times To See Specialists. Health Aff . 2017;36(3):492-499.CrossRef
11.
go back to reference Hefzy EM, Wegdan AA, Abdel Wahed WY. Hospital outpatient clinics as a potential hazard for healthcare associated infections. J Infect Public Health. 2016;9(1):88-97.CrossRef Hefzy EM, Wegdan AA, Abdel Wahed WY. Hospital outpatient clinics as a potential hazard for healthcare associated infections. J Infect Public Health. 2016;9(1):88-97.CrossRef
13.
go back to reference Pham HH, O’Malley AS, Bach PB, Saiontz-Martinez C, Schrag D. Primary care physicians’ links to other physicians through Medicare patients: the scope of care coordination. Ann Intern Med. 2009;150(4):236-242.CrossRef Pham HH, O’Malley AS, Bach PB, Saiontz-Martinez C, Schrag D. Primary care physicians’ links to other physicians through Medicare patients: the scope of care coordination. Ann Intern Med. 2009;150(4):236-242.CrossRef
14.
go back to reference Garrick TR, Stotland NL. How to write a psychiatric consultation. Am J Psychiatry. 1982;139(7):849-855.CrossRef Garrick TR, Stotland NL. How to write a psychiatric consultation. Am J Psychiatry. 1982;139(7):849-855.CrossRef
15.
go back to reference Tran C, Archibald D, Humphrey-Murto S, Liddy C, Keely E. What makes a high-quality electronic consultation (eConsult)? A nominal group study. J Telemed Telecare. Published online February 4, 2019:1357633X18822885. Tran C, Archibald D, Humphrey-Murto S, Liddy C, Keely E. What makes a high-quality electronic consultation (eConsult)? A nominal group study. J Telemed Telecare. Published online February 4, 2019:1357633X18822885.
16.
go back to reference O’Malley AS, Reschovsky JD. Referral and consultation communication between primary care and specialist physicians: finding common ground. Arch Intern Med. 2011;171(1):56-65.CrossRef O’Malley AS, Reschovsky JD. Referral and consultation communication between primary care and specialist physicians: finding common ground. Arch Intern Med. 2011;171(1):56-65.CrossRef
17.
go back to reference Liddy C, Abu-Hijleh T, Joschko J, Archibald D, Keely E. eConsults and Learning Between Primary Care Providers and Specialists. Fam Med. 2019;51(7):567-573.CrossRef Liddy C, Abu-Hijleh T, Joschko J, Archibald D, Keely E. eConsults and Learning Between Primary Care Providers and Specialists. Fam Med. 2019;51(7):567-573.CrossRef
18.
go back to reference Janssen M, Sagasser MH, Laro EAM, de Graaf J, Scherpbier-de Haan ND. Learning intraprofessional collaboration by participating in a consultation programme: what and how did primary and secondary care trainees learn? BMC Med Educ. 2017;17(1):125.CrossRef Janssen M, Sagasser MH, Laro EAM, de Graaf J, Scherpbier-de Haan ND. Learning intraprofessional collaboration by participating in a consultation programme: what and how did primary and secondary care trainees learn? BMC Med Educ. 2017;17(1):125.CrossRef
19.
go back to reference Meijer LJ, de Groot E, Blaauw-Westerlaken M, Damoiseaux RAMJ. Intraprofessional collaboration and learning between specialists and general practitioners during postgraduate training: a qualitative study. BMC Health Serv Res. 2016;16(a):376.CrossRef Meijer LJ, de Groot E, Blaauw-Westerlaken M, Damoiseaux RAMJ. Intraprofessional collaboration and learning between specialists and general practitioners during postgraduate training: a qualitative study. BMC Health Serv Res. 2016;16(a):376.CrossRef
20.
go back to reference Thaler RH, Sunstein CR. Nudge: Improving Decisions about Health, Wealth and Happiness.; 2009. Thaler RH, Sunstein CR. Nudge: Improving Decisions about Health, Wealth and Happiness.; 2009.
21.
go back to reference Goldstein NJ, Cialdini RB, Griskevicius V. A Room with a Viewpoint: Using Social Norms to Motivate Environmental Conservation in Hotels. J Consum Res. 2008;35(3):472-482.CrossRef Goldstein NJ, Cialdini RB, Griskevicius V. A Room with a Viewpoint: Using Social Norms to Motivate Environmental Conservation in Hotels. J Consum Res. 2008;35(3):472-482.CrossRef
22.
go back to reference Kiefe CI, Allison JJ, Williams OD, Person SD, Weaver MT, Weissman NW. Improving quality improvement using achievable benchmarks for physician feedback: a randomized controlled trial. JAMA. 2001;285(22):2871-2879.CrossRef Kiefe CI, Allison JJ, Williams OD, Person SD, Weaver MT, Weissman NW. Improving quality improvement using achievable benchmarks for physician feedback: a randomized controlled trial. JAMA. 2001;285(22):2871-2879.CrossRef
24.
go back to reference Walther JB. Computer-Mediated Communication: Impersonal, Interpersonal, and Hyperpersonal Interaction. Communic Res. 1996;23(1):3-43.CrossRef Walther JB. Computer-Mediated Communication: Impersonal, Interpersonal, and Hyperpersonal Interaction. Communic Res. 1996;23(1):3-43.CrossRef
25.
go back to reference Eskreis-Winkler L, Milkman KL, Gromet DM, Duckworth AL. A large-scale field experiment shows giving advice improves academic outcomes for the advisor. Proc Natl Acad Sci U S A. 2019;116(30):14808-14810.CrossRef Eskreis-Winkler L, Milkman KL, Gromet DM, Duckworth AL. A large-scale field experiment shows giving advice improves academic outcomes for the advisor. Proc Natl Acad Sci U S A. 2019;116(30):14808-14810.CrossRef
26.
27.
go back to reference Meeker D, Linder JA, Fox CR, et al. Effect of Behavioral Interventions on Inappropriate Antibiotic Prescribing Among Primary Care Practices: A Randomized Clinical Trial. JAMA. 2016;315(6):562-570.CrossRef Meeker D, Linder JA, Fox CR, et al. Effect of Behavioral Interventions on Inappropriate Antibiotic Prescribing Among Primary Care Practices: A Randomized Clinical Trial. JAMA. 2016;315(6):562-570.CrossRef
28.
go back to reference Bond AM, Volpp KG, Emanuel EJ, et al. Real-time Feedback in Pay-for-Performance: Does More Information Lead to Improvement? J Gen Intern Med. 2019;34(9):1737-1743.CrossRef Bond AM, Volpp KG, Emanuel EJ, et al. Real-time Feedback in Pay-for-Performance: Does More Information Lead to Improvement? J Gen Intern Med. 2019;34(9):1737-1743.CrossRef
30.
go back to reference Englis BG, Solomon MR. To be and not to be: Lifestyle imagery, reference groups, and the clustering of America. J Advert. 1995;24(1):13-28.CrossRef Englis BG, Solomon MR. To be and not to be: Lifestyle imagery, reference groups, and the clustering of America. J Advert. 1995;24(1):13-28.CrossRef
31.
go back to reference Berger J, Heath C. Where Consumers Diverge from Others: Identity Signaling and Product Domains. J Consum Res. 2007;34(2):121-134.CrossRef Berger J, Heath C. Where Consumers Diverge from Others: Identity Signaling and Product Domains. J Consum Res. 2007;34(2):121-134.CrossRef
32.
go back to reference Sandelowski M. Whatever happened to qualitative description? Res Nurs Health. 2000;23(4):334-340.CrossRef Sandelowski M. Whatever happened to qualitative description? Res Nurs Health. 2000;23(4):334-340.CrossRef
33.
go back to reference Soni SM, Giboney P, Yee HF Jr. Development and Implementation of Expected Practices to Reduce Inappropriate Variations in Clinical Practice. JAMA. 2016;315(20):2163-2164.CrossRef Soni SM, Giboney P, Yee HF Jr. Development and Implementation of Expected Practices to Reduce Inappropriate Variations in Clinical Practice. JAMA. 2016;315(20):2163-2164.CrossRef
34.
go back to reference Donner A, Klar N. Design and Analysis of Cluster Randomization Trials in Health Research. Wiley; 2010. Donner A, Klar N. Design and Analysis of Cluster Randomization Trials in Health Research. Wiley; 2010.
35.
go back to reference Davidow SL, Sheth J, Sixta CS, Thomas-Hemak L. Closing the Referral Loop: Improving Ambulatory Referral Management, Electronic Health Record Connectivity, and Care Coordination Processes. J Ambul Care Manage. 2018;41(4):240-249.CrossRef Davidow SL, Sheth J, Sixta CS, Thomas-Hemak L. Closing the Referral Loop: Improving Ambulatory Referral Management, Electronic Health Record Connectivity, and Care Coordination Processes. J Ambul Care Manage. 2018;41(4):240-249.CrossRef
36.
go back to reference Patel MP, Schettini P, O’Leary CP, Bosworth HB, Anderson JB, Shah KP. Closing the Referral Loop: an Analysis of Primary Care Referrals to Specialists in a Large Health System. J Gen Intern Med. 2018;33(5):715-721.CrossRef Patel MP, Schettini P, O’Leary CP, Bosworth HB, Anderson JB, Shah KP. Closing the Referral Loop: an Analysis of Primary Care Referrals to Specialists in a Large Health System. J Gen Intern Med. 2018;33(5):715-721.CrossRef
38.
go back to reference EHRN. Epic Health Research Network. Accessed October 28, 2020. https://www.ehrn.org/articles/as-office-visits-fall-telehealth-takes-hold/ EHRN. Epic Health Research Network. Accessed October 28, 2020. https://​www.​ehrn.​org/​articles/​as-office-visits-fall-telehealth-takes-hold/​
40.
go back to reference COVID-19 E-Consult Resources. Published March 24, 2020. Accessed May 23, 2021. https://econsulttoolkit.com/econsult-covid-19-resources/ COVID-19 E-Consult Resources. Published March 24, 2020. Accessed May 23, 2021. https://​econsulttoolkit.​com/​econsult-covid-19-resources/​
Metadata
Title
Effect of Peer Benchmarking on Specialist Electronic Consult Performance in a Los Angeles Safety-Net: a Cluster Randomized Trial
Authors
Daniella Meeker, PhD
Mark W. Friedberg, MD, MPP
Tara K. Knight, PhD
Jason N. Doctor, PhD
Dina Zein, MHA
Nancy Cayasso-McIntosh, MPH
Noah J. Goldstein, PhD
Craig R. Fox, PhD
Jeffrey A. Linder, MD, MPH
Stephen D. Persell, MD, MPH
Stanley Dea, MD
Paul Giboney, MD
Hal F. Yee, MD, PhD
The Behavioral Economics and eConsult Steering Committee
Publication date
01-05-2022
Publisher
Springer International Publishing
Keyword
Care
Published in
Journal of General Internal Medicine / Issue 6/2022
Print ISSN: 0884-8734
Electronic ISSN: 1525-1497
DOI
https://doi.org/10.1007/s11606-021-07002-1

Other articles of this Issue 6/2022

Journal of General Internal Medicine 6/2022 Go to the issue
Live Webinar | 27-06-2024 | 18:00 (CEST)

Keynote webinar | Spotlight on medication adherence

Live: Thursday 27th June 2024, 18:00-19:30 (CEST)

WHO estimates that half of all patients worldwide are non-adherent to their prescribed medication. The consequences of poor adherence can be catastrophic, on both the individual and population level.

Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.

Prof. Kevin Dolgin
Prof. Florian Limbourg
Prof. Anoop Chauhan
Developed by: Springer Medicine
Obesity Clinical Trial Summary

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.

Developed by: Springer Medicine

Highlights from the ACC 2024 Congress

Year in Review: Pediatric cardiology

Watch Dr. Anne Marie Valente present the last year's highlights in pediatric and congenital heart disease in the official ACC.24 Year in Review session.

Year in Review: Pulmonary vascular disease

The last year's highlights in pulmonary vascular disease are presented by Dr. Jane Leopold in this official video from ACC.24.

Year in Review: Valvular heart disease

Watch Prof. William Zoghbi present the last year's highlights in valvular heart disease from the official ACC.24 Year in Review session.

Year in Review: Heart failure and cardiomyopathies

Watch this official video from ACC.24. Dr. Biykem Bozkurt discusses last year's major advances in heart failure and cardiomyopathies.