Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on sleep in brain health

LIVE: Thursday 2nd May 2024, 18:00-19:30 (CEST)

Quality sleep is essential for health. But what happens to our brains when sleep patterns are disturbed? Join our experts to explore the interplay between sleep disruption and neurological diseases, and the questions that you need to be asking your patients to help you prevent the harmful effects of sleep deprivation.
ADVANCED SEARCH
Log in
Top
Implementation Science
Published in: Implementation Science 1/2018

Open Access 01-12-2018 | Research

Pay-for-performance as a cost-effective implementation strategy: results from a cluster randomized trial

Authors: Bryan R. Garner, Aung K. Lwin, Gail K. Strickler, Brooke D. Hunter, Donald S. Shepard

Published in: Implementation Science | Issue 1/2018

Login to get access

Abstract

Background

Pay-for-performance (P4P) has been recommended as a promising strategy to improve implementation of high-quality care. This study examined the incremental cost-effectiveness of a P4P strategy found to be highly effective in improving the implementation and effectiveness of the Adolescent Community Reinforcement Approach (A-CRA), an evidence-based treatment (EBT) for adolescent substance use disorders (SUDs).

Methods

Building on a $30 million national initiative to implement A-CRA in SUD treatment settings, urn randomization was used to assign 29 organizations and their 105 therapists and 1173 patients to one of two conditions (implementation-as-usual (IAU) control condition or IAU+P4P experimental condition). It was not possible to blind organizations, therapists, or all research staff to condition assignment. All treatment organizations and their therapists received a multifaceted implementation strategy. In addition to those IAU strategies, therapists in the IAU+P4P condition received US $50 for each month that they demonstrated competence in treatment delivery (A-CRA competence) and US $200 for each patient who received a specified number of treatment procedures and sessions found to be associated with significantly improved patient outcomes (target A-CRA). Incremental cost-effectiveness ratios (ICERs), which represent the difference between the two conditions in average cost per treatment organization divided by the corresponding average difference in effectiveness per organization, and quality-adjusted life years (QALYs) were the primary outcomes.

Results

At trial completion, 15 organizations were randomized to the IAU condition and 14 organizations were randomized to the IAU+P4P condition. Data from all 29 organizations were analyzed. Cluster-level analyses suggested the P4P strategy led to significantly higher average total costs compared to the IAU control condition, yet this average increase of 5% resulted in a 116% increase in the average number of months therapists demonstrated competence in treatment delivery (ICER = $333), a 325% increase in the average number of patients who received the targeted dosage of treatment (ICER = $453), and a 325% increase in the number of days of abstinence per patient in treatment (ICER = $8.134). Further supporting P4P as a cost-effective implementation strategy, the cost per QALY was only $8681 (95% confidence interval $1191–$16,171).

Conclusion

This study provides experimental evidence supporting P4P as a cost-effective implementation strategy.

Trial registration

Literature
1.
National Drug Intelligence Center. The economic impact of illicit drug use on American Society. Washington, DC: United States Department of Justice; 2011.
2.
Centers for Disease Control and Prevention. National diabetes fact sheet: national estimates and general information on diabetes and prediabetes in the United States. Atlanta, GA: U.S. Department of Health and Human Services, centers for disease control and Prevention; 2011.
3.
Finkelstein EA, Trogdon JG, Cohen JW, Dietz W. Annual medical spending attributable to obesity: payer-and service-specific estimates. Health Aff (Millwood). 2009;28(5):w822–31.CrossRef
4.
5.
Garner BR. Research on the diffusion of evidence-based treatments within substance abuse treatment: a systematic review. J Subst Abus Treat. 2009;36(4):376–99.CrossRef
6.
Institute of Medicine. Bridging the gap between practice and research: forging partnerships with community-based drug and alcohol treatment. Washington, DC: National Academy Press; 1998.
7.
Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academy Press; 2001.
8.
Institute of Medicine. Improving the quality of health care for mental and substance use conditions: quality chasm series. Washington, DC: National Academy Press; 2006.
9.
10.
Hogan MF. The President's new freedom commission: recommendations to transform mental health care in America. Psychiatr Serv. 2003;54(11):1467–74.CrossRefPubMed
11.
Institute of Medicine. Rewarding provider performance: aligning incentives in Medicare. Washington, DC: National Academy Press; 2007.
12.
Dennis M, Godley SH, Diamond G, et al. The Cannabis Youth Treatment (CYT) Study: main findings from two randomized trials. J Subst Abus Treat. 2004;27(3):197–213.CrossRef
13.
Garner BR, Godley SH, Funk RR, Dennis ML, Smith JE, Godley MD. Exposure to adolescent community reinforcement approach treatment procedures as a mediator of the relationship between adolescent substance abuse treatment retention and outcome. J Subst Abus Treat. 2009;36(3):252–64.CrossRef
14.
Godley SH, Hedges K, Hunter B. Gender and racial differences in treatment process and outcome among participants in the adolescent community reinforcement approach. Psychol Addict Behav. 2011;25(1):143–54.CrossRefPubMed
15.
Slesnick N, Prestopnik JL, Meyers RJ, Glassman M. Treatment outcome for street-living, homeless youth. Addict Behav. 2007;32(6):1237–51.CrossRefPubMed
16.
Godley SH, Hunter BD, Fernandez-Artamendi S, Smith JE, Meyers RJ, Godley MD. A comparison of treatment outcomes for adolescent community reinforcement approach participants with and without co-occurring problems. J Subst Abus Treat. 2014;46(4):463–71.CrossRef
17.
Henderson CE, Wevodau AL, Henderson SE, et al. An independent replication of the adolescent-community reinforcement approach with justice-involved youth. Am J Addict. 2016;25(3):233–40.CrossRefPubMedPubMedCentral
18.
Godley MD, Passetti LL, Subramaniam GA, Funk RR, Smith JE, Meyers RJ. Adolescent community reinforcement approach implementation and treatment outcomes for youth with opioid problem use. Drug Alcohol Depend. 2017;174:9–16.CrossRefPubMedPubMedCentral
19.
Garner BR, Godley SH, Dennis ML, Hunter BD, Bair CM, Godley MD. Using pay for performance to improve treatment implementation for adolescent substance use disorders: results from a cluster randomized trial. Arch Pediatr Adolesc Med. 2012;166(10):938–44.CrossRefPubMed
20.
Godley SH, Garner BR, Smith JE, Meyers RJ, Godley MD. A large-scale dissemination and implementation model for evidence-based treatment and continuing care. Clin Psychol (New York). 2011;18(1):67–83.
21.
Forman-Hoffman VL, Middleton JC, McKeeman JL, et al. Quality improvement, implementation, and dissemination strategies to improve mental health care for children and adolescents: a systematic review. Implement Sci. 2017;12(1):93.CrossRefPubMedPubMedCentral
22.
Garner BR, Godley SH, Dennis ML, Godley MD, Shepard DS. The reinforcing therapist performance (RTP) experiment: study protocol for a cluster randomized trial. Implement Sci. 2010;5:5.CrossRefPubMedPubMedCentral
23.
Emmert M, Eijkenaar F, Kemter H, Esslinger AS, Schoffski O. Economic evaluation of pay-for-performance in health care: a systematic review. Eur J Health Econ. 2012;13(6):755–67.CrossRefPubMed
24.
Greene SE, Nash DB. Pay for performance: an overview of the literature. Am J Med Qual. 2009;24(2):140–63.CrossRefPubMed
25.
McCarty D, McConnell KJ, Schmidt LA. Priorities for policy research on treatments for alcohol and drug use disorders. J Subst Abus Treat. 2010;39(2):87–95.CrossRef
26.
27.
Garner BR, Godley MD, Funk RR, Lee MT, Garnick DW. The Washington circle continuity of care performance measure: predictive validity with adolescents discharged from residential treatment. J Subst Abus Treat. 2010;38(1):3–11.CrossRef
28.
29.
Campbell MK, Piaggio G, Elbourne DR, Altman DG. Consort 2010 statement: extension to cluster randomised trials. BMJ. 2012;345:e5661.CrossRefPubMed
30.
Powell BJ, McMillen JC, Proctor EK, et al. A compilation of strategies for implementing clinical innovations in health and mental health. Med Care Res Rev. 2012;69(2):123–57.CrossRefPubMed
31.
32.
Shepard DS, Beaston-Blaakman A, Horgan CM. The ADSS cost study: costs of substance abuse treatment in the specialty sector. DHHS publication no. SMA 03-3762. Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies; 2003.
33.
Daley M, Shepard DS, Bury-Maynard D. Changes in quality of life for pregnant women in substance user treatment: developing a quality of life index for the addictions. Subst Use Misuse. 2005;40(3):375–94.CrossRefPubMed
34.
Salomon JA, Haagsma JA, Davis A, et al. Disability weights for the Global Burden of Disease 2013 study. Lancet Glob Health. 2015;3(11):e712–23.CrossRefPubMed
35.
Drummond MF, Sculpher MJ, Torrance GW, O'Brien BJ, Stoddart GL. Methods for the economic evaluation of health care programmes. 3rd ed. Oxford: Oxford University Press; 2005.
36.
37.
Bremer RW, Scholle SH, Keyser D, Houtsinger JV, Pincus HA. Pay for performance in behavioral health. Psychiatr Serv. 2008;59(12):1419–29.CrossRefPubMed
38.
Eijkenaar F, Emmert M, Scheppach M, Schoffski O. Effects of pay for performance in health care: a systematic review of systematic reviews. Health Policy. 2013;110(2–3):115–30.CrossRefPubMed
39.
Jha AK, Joynt KE, Orav EJ, Epstein AM. The long-term effect of premier pay for performance on patient outcomes. N Engl J Med. 2012;366(17):1606–15.CrossRefPubMed
40.
Ryan AM, Blustein J, Casalino LP. Medicare’s flagship test of pay-for-performance did not spur more rapid quality improvement among low-performing hospitals. Health Aff (Millwood). 2012;31(4):797–805.CrossRef
41.
Scheffler RM. Pay for performance (P4P) programs in health services: What is the evidence. World Health Report Background Paper, No 31. Geneva: World Health Organization; 2010.
42.
Kouides RW, Bennett NM, Lewis B, Cappuccio JD, Barker WH, LaForce FM. Performance-based physician reimbursement and influenza immunization rates in the elderly. The Primary-Care Physicians of Monroe County. Am J Prev Med. 1998;14(2):89–95.CrossRefPubMed
43.
An LC, Bluhm JH, Foldes SS, et al. A randomized trial of a pay-for-performance program targeting clinician referral to a state tobacco quitline. Arch Intern Med. 2008;168(18):1993–9.CrossRefPubMed
44.
45.
Neumann PJ, Sanders GD, Russell LB, Siegel JE, Ganiats TG. Cost-effectiveness in health and medicine. 2nd ed. New York: Oxford University Press; 2017.
46.
Drummond MF, O’Brien JO, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes. 2nd ed. New York: Oxford University Press; 1997.
47.
Ryan A, Blustein J. Making the best of hospital pay for performance. N Engl J Med. 2012;366(17):1557–9.CrossRefPubMed
48.
Van Herck P, De Smedt D, Annemans L, Remmen R, Rosenthal MB, Sermeus W. Systematic review: effects, design choices, and context of pay-for-performance in health care. BMC Health Serv Res. 2010;10:247.CrossRefPubMedPubMedCentral
49.
Garner BR, Hunter SB, Funk RR, Griffin BA, Godley SH. Toward evidence-based measures of implementation: examining the relationship between implementation outcomes and client outcomes. J Subst Abus Treat. 2016;67:15–21.CrossRef
50.
Lindenauer PK, Remus D, Roman S, et al. Public reporting and pay for performance in hospital quality improvement. N Engl J Med. 2007;356(5):486–96.CrossRefPubMed
51.
Eijkenaar F. Pay for performance in health care: an international overview of initiatives. Med Care Res Rev. 2012;69(3):251–76.CrossRefPubMed
52.
53.
Mehrotra A, Sorbero ME, Damberg CL. Using the lessons of behavioral economics to design more effective pay-for-performance programs. Am J Manag Care. 2010;16(7):497–503.PubMedPubMedCentral
54.
Shepard DS, Calabro JA, Love CT, McKay JR, Tetreault J, Yeom HS. Counselor incentives to improve client retention in an outpatient substance abuse aftercare program. Admin Pol Ment Health. 2006;33(6):629–35.CrossRef
55.
Skinner BF. The behavior of organisms: an experimental analysis. New York: Appleton-Century; 1938.
56.
57.
Squires DD, Gumbley SJ, Storti SA. Training substance abuse treatment organizations to adopt evidence-based practices: the addiction technology transfer center of New England science to service laboratory. J Subst Abus Treat. 2008;34(3):293–301.CrossRef
58.
Davis D, Barrington T, Phoenix U, et al. Evaluation and technical assistance for successful HIV program delivery. AIDS Educ Prev. 2000;12(Suppl A 5):115–25.PubMed
59.
Wandersman A, Chien VH, Katz J. Toward an evidence-based system for innovation support for implementing innovations with quality: tools, training, technical assistance, and quality assurance/quality improvement. Am J Community Psychol. 2012;50(3–4):445–59.CrossRefPubMed
60.
Chaple M, Sacks S. The impact of technical assistance and implementation support on program capacity to deliver integrated services. J Behav Health Serv Res. 2016;43(1):3–17.CrossRefPubMed
61.
Godley SH, Meyers RJ, Smith JE, et al. The Adolescent Community Reinforcement Approach (ACRA) for adolescent cannabis users, Cannabis Youth Treatment Manual Series Vol. 4. Rockville, MD: Center for Subst Abus Treatment, Subst Abus and Mental Health Services Administration; 2001. DHHS Publication No. (SMA) 01–3489
62.
Smith JE, Lundy SL, Gianini L. Community reinforcement approach (CRA) and adolescent community reinforcement approach (A-CRA) coding manual. Albuquerque, NM: University of New Mexico; 2007.
63.
Gibbons CJ, Carroll KM, Ball SA, Nich C, Frankforter TL, Martino S. Community program therapist adherence and competence in a motivational interviewing assessment intake session. Am J Drug Alcohol Abuse. 2010;36(6):342–9.CrossRefPubMedPubMedCentral
64.
Martino S, Ball S, Nich C, Frankforter TL, Carroll KM. Correspondence of motivational enhancement treatment integrity ratings among therapists, supervisors, and observers. Psychother Res. 2009;19(2):181–93.CrossRefPubMedPubMedCentral
65.
Martino S, Ball SA, Nich C, Frankforter TL, Carroll KM. Community program therapist adherence and competence in motivational enhancement therapy. Drug Alcohol Depend. 2008;96(1–2):37–48.CrossRefPubMedPubMedCentral
66.
Madson MB, Loignon AC, Lane C. Training in motivational interviewing: a systematic review. J Subst Abus Treat. 2009;36(1):101–9.CrossRef
67.
Lane C, Hood K, Rollnick S. Teaching motivational interviewing: using role play is as effective as using simulated patients. Med Educ. 2008;42(6):637–44.CrossRefPubMed
68.
Martino S, Haeseler F, Belitsky R, Pantalon M, AHt F. Teaching brief motivational interviewing to year three medical students. Med Educ. 2007;41(2):160–7.CrossRefPubMed
69.
Haeseler F, AHt F, Pfeiffer C, Walters C, Martino S. Assessment of a motivational interviewing curriculum for year 3 medical students using a standardized patient case. Patient Educ Couns. 2011;84(1):27–30.CrossRefPubMed
70.
Haine-Schlagel R, Brookman-Frazee L, Janis B, Gordon J. Evaluating a learning collaborative to implement evidence-informed engagement strategies in community-based services for young children. Child Youth Care Forum. 2013;42(5):457–73.CrossRef
71.
Roosa M, Scripa JS, Zastowny TR, Ford JH 2nd. Using a NIATx based local learning collaborative for performance improvement. Eval Program Plann. 2011;34(4):390–8.CrossRefPubMedPubMedCentral
72.
Stephan SH, Connors EH, Arora P, et al. A learning collaborative approach to training school-based health providers in evidence-based mental health treatment. Child Youth Serv Rev. 2013;35(12):1970–8.CrossRef
73.
Farmer AP, Legare F, Turcot L, et al. Printed educational materials: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2008;3(3):CD004398.
74.
Giguere A, Legare F, Grimshaw J, et al. Printed educational materials: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;(10):CD004398.
75.
Schneider EC, Zaslavsky AM, Landon BE, Lied TR, Sheingold S, Cleary PD. National quality monitoring of Medicare health plans: the relationship between enrollees’ reports and the quality of clinical care. Med Care. 2001;39(12):1313–25.CrossRefPubMed
76.
Miller WR, Moyers TB, Arciniega L, Ernst D, Forcehimes A. Training, supervision and quality monitoring of the COMBINE Study behavioral interventions. J Stud Alcohol. 2005;(Suppl 15):188–95. discussion 168–189
77.
McCormack L, Sheridan S, Lewis M, et al. Communication and Dissemination Strategies To Facilitate the Use of Health-Related Evidence. Rockville, MD: Evidence Report/Technology Assessment No. 213. (Prepared by the RTI International-University of North Carolina Evidence-based Practice Center under Contract No. 290–2007-10056-I.) November 2013. AHRQ Publication No. 13(14)-E003-EF.
78.
Miller WR, Yahne CE, Moyers TB, Martinez J, Pirritano M. A randomized trial of methods to help clinicians learn motivational interviewing. J Consult Clin Psychol. 2004;72(6):1050–62.CrossRefPubMed
79.
Sholomskas DE, Syracuse-Siewert G, Rounsaville BJ, Ball SA, Nuro KF, Carroll KM. We don’t train in vain: a dissemination trial of three strategies of training clinicians in cognitive-behavioral therapy. J Consult Clin Psychol. 2005;73(1):106–15.CrossRefPubMedPubMedCentral
80.
Jamtvedt G, Young JM, Kristoffersen DT, et al. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2006;2(2):CD000259.
81.
82.
Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;(6):CD000259.
83.
Colquhoun HL, Brehaut JC, Sales A, et al. A systematic review of the use of theory in randomized controlled trials of audit and feedback. Implement Sci. 2013;8:66.CrossRefPubMedPubMedCentral
Metadata
Title
Pay-for-performance as a cost-effective implementation strategy: results from a cluster randomized trial
Authors
Bryan R. Garner
Aung K. Lwin
Gail K. Strickler
Brooke D. Hunter
Donald S. Shepard
Publication date
01-12-2018
Publisher
BioMed Central
Published in
Implementation Science / Issue 1/2018
Electronic ISSN: 1748-5908
DOI
https://doi.org/10.1186/s13012-018-0774-1

Other articles of this Issue 1/2018

Implementation Science 1/2018 Go to the issue

Research

Development and testing of the Measure of Innovation-Specific Implementation Intentions (MISII) using Rasch measurement theory

Study protocol

Sustainable reduction of antibiotic-induced antimicrobial resistance (ARena) in German ambulatory care: study protocol of a cluster randomised trial

Research

Reporting quality of European and Croatian health practice guidelines according to the RIGHT reporting checklist

Methodology

Standardizing an approach to the evaluation of implementation science proposals

Research

T-CaST: an implementation theory comparison and selection tool

Research

Human papillomavirus vaccine delivery in Mozambique: identification of implementation performance drivers using the Consolidated Framework for Implementation Research (CFIR)