Skip to main content
Top
Published in: Implementation Science 1/2019

Open Access 01-12-2019 | Research

The clinician crowdsourcing challenge: using participatory design to seed implementation strategies

Authors: Rebecca E. Stewart, Nathaniel Williams, Y. Vivian Byeon, Alison Buttenheim, Sriram Sridharan, Kelly Zentgraf, David T. Jones, Katelin Hoskins, Molly Candon, Rinad S. Beidas

Published in: Implementation Science | Issue 1/2019

Login to get access

Abstract

Background

In healthcare settings, system and organization leaders often control the selection and design of implementation strategies even though frontline workers may have the most intimate understanding of the care delivery process, and factors that optimize and constrain evidence-based practice implementation within the local system. Innovation tournaments, a structured participatory design strategy to crowdsource ideas, are a promising approach to participatory design that may increase the effectiveness of implementation strategies by involving end users (i.e., clinicians). We utilized a system-wide innovation tournament to garner ideas from clinicians about how to enhance the use of evidence-based practices (EBPs) within a large public behavioral health system.

Methods

Our innovation tournament occurred in three phases. First, we invited over 500 clinicians to share, through a web-based platform, their ideas regarding how their organizations could best support use of EBPs. Clinicians could rate and comment on ideas submitted by others. Second, submissions were judged by an expert panel (including behavioral scientists, system leaders, and payers) based on their rated enthusiasm for the idea. Third, we held a community-facing event during which the six clinicians who submitted winning ideas presented their strategies to 85 attendees representing a cross-section of clinicians and system and organizational leaders.

Results

We had a high rate of participation (12.3%), more than double the average rate of previous tournaments conducted in other settings (5%). A total of 65 ideas were submitted by 55 participants representing 38 organizations. The most common categories of ideas pertained to training (42%), financing and compensation (26%), clinician support and preparation tools (22%), and EBP-focused supervision (17%). The expert panel and clinicians differed on their ratings of the ideas, highlighting value of seeking input from multiple stakeholder groups when developing implementation strategies.

Conclusions

Innovation tournaments are a useful and feasible methodology for engaging end users, system leaders, and behavioral scientists through a structured approach to developing implementation strategies. The process and resultant strategies engendered significant enthusiasm and engagement from participants at all levels of a healthcare system. Research is needed to compare the effectiveness of strategies developed through innovation tournaments to strategies developed through design approaches.
Literature
1.
go back to reference McHugh RK, Barlow DH. The dissemination and implementation of evidence-based psychological treatments: a review of current efforts. Am Psychol. 2010;652:73–84.CrossRef McHugh RK, Barlow DH. The dissemination and implementation of evidence-based psychological treatments: a review of current efforts. Am Psychol. 2010;652:73–84.CrossRef
2.
go back to reference Aarons GA, Wells RS, Zagursky K, Fettes DL, Palinkas LA. Implementing evidence-based practice in community mental health agencies: a multiple stakeholder analysis. Am J Public Health. 2009;99:2087–95.CrossRef Aarons GA, Wells RS, Zagursky K, Fettes DL, Palinkas LA. Implementing evidence-based practice in community mental health agencies: a multiple stakeholder analysis. Am J Public Health. 2009;99:2087–95.CrossRef
3.
go back to reference Hoagwood K, Burns BJ, Kiser L, Ringeisen H, Schoenwald SK. Evidence-based practice in child and adolescent mental health services. Psychiatr Serv. 2001;52:1179–89.CrossRef Hoagwood K, Burns BJ, Kiser L, Ringeisen H, Schoenwald SK. Evidence-based practice in child and adolescent mental health services. Psychiatr Serv. 2001;52:1179–89.CrossRef
4.
go back to reference Kazdin AE. Evidence-based treatment and practice: new opportunities to bridge clinical research and practice, enhance the knowledge base, and improve patient care. Am Psychol. 2008;63:146–59.CrossRef Kazdin AE. Evidence-based treatment and practice: new opportunities to bridge clinical research and practice, enhance the knowledge base, and improve patient care. Am Psychol. 2008;63:146–59.CrossRef
5.
go back to reference Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21.CrossRef Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10:21.CrossRef
6.
go back to reference Beidas RS, Stewart RE, Adams DR, Fernandez T, Lustbader S, Powell BJ, et al. A multi-level examination of stakeholder perspectives of implementation of evidence-based practices in a large urban publicly-funded mental health system. Admin Pol Ment Health. 2016;43:893–908.CrossRef Beidas RS, Stewart RE, Adams DR, Fernandez T, Lustbader S, Powell BJ, et al. A multi-level examination of stakeholder perspectives of implementation of evidence-based practices in a large urban publicly-funded mental health system. Admin Pol Ment Health. 2016;43:893–908.CrossRef
7.
go back to reference Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.CrossRef Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.CrossRef
8.
go back to reference Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Admin Pol Ment Health. 2011;38:4–23.CrossRef Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Admin Pol Ment Health. 2011;38:4–23.CrossRef
9.
go back to reference Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK, et al. Methods to improve the selection and tailoring of implementation strategies. J Behav Health Serv Res. 2017;44:177–94.CrossRef Powell BJ, Beidas RS, Lewis CC, Aarons GA, McMillen JC, Proctor EK, et al. Methods to improve the selection and tailoring of implementation strategies. J Behav Health Serv Res. 2017;44:177–94.CrossRef
11.
go back to reference Lindhiem O, Bennett CB, Trentacosta CJ, McLear C. Client preferences affect treatment satisfaction, completion, and clinical outcome: a meta-analysis. Clin Psychol Rev. 2014;34:506–17.CrossRef Lindhiem O, Bennett CB, Trentacosta CJ, McLear C. Client preferences affect treatment satisfaction, completion, and clinical outcome: a meta-analysis. Clin Psychol Rev. 2014;34:506–17.CrossRef
12.
go back to reference Fleurence R, Selby JV, Odom-Walker K, Hunt G, Meltzer D, Slutsky JR, et al. How the patient-centered outcomes research institute is engaging patients and others in shaping its research agenda. Health Aff (Millwood). 2013;32:393–400.CrossRef Fleurence R, Selby JV, Odom-Walker K, Hunt G, Meltzer D, Slutsky JR, et al. How the patient-centered outcomes research institute is engaging patients and others in shaping its research agenda. Health Aff (Millwood). 2013;32:393–400.CrossRef
13.
go back to reference Williams NJ. Multilevel mechanisms of implementation strategies in mental health: integrating theory, research, and practice. Admin Pol Ment Health. 2016;43:783–98.CrossRef Williams NJ. Multilevel mechanisms of implementation strategies in mental health: integrating theory, research, and practice. Admin Pol Ment Health. 2016;43:783–98.CrossRef
14.
go back to reference Simonsen J, Robertson T. Routledge international handbook of participatory design. New York: Routledge; 2013. Simonsen J, Robertson T. Routledge international handbook of participatory design. New York: Routledge; 2013.
16.
go back to reference Green LW. Making research relevant: if it is an evidence-based practice, where’s the practice-based evidence? Fam Pract. 2008;25:i20–4.CrossRef Green LW. Making research relevant: if it is an evidence-based practice, where’s the practice-based evidence? Fam Pract. 2008;25:i20–4.CrossRef
17.
go back to reference Casey M, O’ Leary D, Coghlan D. Unpacking action research and implementation science: implications for nursing. J Adv Nurs. 2018;74:1051–8.CrossRef Casey M, O’ Leary D, Coghlan D. Unpacking action research and implementation science: implications for nursing. J Adv Nurs. 2018;74:1051–8.CrossRef
18.
go back to reference Sikkens JJ, van Agtmael MA, Peters EJG, Lettinga KD, van der Kuip M, Vandenbroucke-Grauls CMJE, et al. Behavioral approach to appropriate antimicrobial prescribing in hospitals: the Dutch Unique Method for Antimicrobial Stewardship (DUMAS) participatory intervention study. JAMA Intern Med. 2017;177:1130–8.CrossRef Sikkens JJ, van Agtmael MA, Peters EJG, Lettinga KD, van der Kuip M, Vandenbroucke-Grauls CMJE, et al. Behavioral approach to appropriate antimicrobial prescribing in hospitals: the Dutch Unique Method for Antimicrobial Stewardship (DUMAS) participatory intervention study. JAMA Intern Med. 2017;177:1130–8.CrossRef
19.
go back to reference Frank L, Forsythe L, Ellis L, Schrandt S, Sheridan S, Gerson J, et al. Conceptual and practical foundations of patient engagement in research at the patient-centered outcomes research institute. Qual Life Res Int J Qual Life Asp Treat Care Rehab. 2015;24:1033–41. Frank L, Forsythe L, Ellis L, Schrandt S, Sheridan S, Gerson J, et al. Conceptual and practical foundations of patient engagement in research at the patient-centered outcomes research institute. Qual Life Res Int J Qual Life Asp Treat Care Rehab. 2015;24:1033–41.
20.
go back to reference Lyon AR, Koerner K. User-centered design for psychosocial intervention development and implementation. Clin Psychol Publ Div Clin Psychol Am Psychol Assoc. 2016;23:180–200. Lyon AR, Koerner K. User-centered design for psychosocial intervention development and implementation. Clin Psychol Publ Div Clin Psychol Am Psychol Assoc. 2016;23:180–200.
21.
go back to reference Chambers DA, Azrin ST. Research and services partnerships: partnership: a fundamental component of dissemination and implementation research. Psychiatr Serv. 2013;64:509–11.CrossRef Chambers DA, Azrin ST. Research and services partnerships: partnership: a fundamental component of dissemination and implementation research. Psychiatr Serv. 2013;64:509–11.CrossRef
22.
go back to reference Ranard BL, Ha YP, Meisel ZF, Asch DA, Hill SS, Becker LB, et al. Crowdsourcing--harnessing the masses to advance health and medicine, a systematic review. J Gen Intern Med. 2014;29:187–203.CrossRef Ranard BL, Ha YP, Meisel ZF, Asch DA, Hill SS, Becker LB, et al. Crowdsourcing--harnessing the masses to advance health and medicine, a systematic review. J Gen Intern Med. 2014;29:187–203.CrossRef
23.
go back to reference Asch DA, Terwiesch C, Mahoney KB, Rosin R. Insourcing health care innovation. N Engl J Med. 2014;370:1775–7.CrossRef Asch DA, Terwiesch C, Mahoney KB, Rosin R. Insourcing health care innovation. N Engl J Med. 2014;370:1775–7.CrossRef
24.
go back to reference Terwiesch C, Mehta SJ, Volpp KG. Innovating in health delivery: the Penn medicine innovation tournament. Healthcare. 2013;1:37–41.CrossRef Terwiesch C, Mehta SJ, Volpp KG. Innovating in health delivery: the Penn medicine innovation tournament. Healthcare. 2013;1:37–41.CrossRef
25.
go back to reference Stewart RE, Adams DR, Mandell DS, Hadley TR, Evans AC, Rubin R, et al. The perfect storm: collision of the business of mental health and the implementation of evidence-based practices. Psychiatr Serv. 2016;67:159–61.CrossRef Stewart RE, Adams DR, Mandell DS, Hadley TR, Evans AC, Rubin R, et al. The perfect storm: collision of the business of mental health and the implementation of evidence-based practices. Psychiatr Serv. 2016;67:159–61.CrossRef
26.
go back to reference Beidas RS, Marcus S, Aarons GA, Hoagwood KE, Schoenwald S, Evans AC, et al. Predictors of community therapists’ use of therapy techniques in a large public mental health system. JAMA Pediatr. 2015;169:374–82.CrossRef Beidas RS, Marcus S, Aarons GA, Hoagwood KE, Schoenwald S, Evans AC, et al. Predictors of community therapists’ use of therapy techniques in a large public mental health system. JAMA Pediatr. 2015;169:374–82.CrossRef
27.
go back to reference Powell BJ, Beidas RS, Rubin RM, Stewart RE, Wolk CB, Matlin SL, et al. Applying the policy ecology framework to Philadelphia’s behavioral health transformation efforts. Admin Pol Ment Health. 2016;43:909–26.CrossRef Powell BJ, Beidas RS, Rubin RM, Stewart RE, Wolk CB, Matlin SL, et al. Applying the policy ecology framework to Philadelphia’s behavioral health transformation efforts. Admin Pol Ment Health. 2016;43:909–26.CrossRef
28.
go back to reference Creed TA, Stirman SW, Evans AC, Beck AT. A model for implementation of cognitive therapy in community mental health: the Beck Initiative. Behav Ther. 2010;37:56–65. Creed TA, Stirman SW, Evans AC, Beck AT. A model for implementation of cognitive therapy in community mental health: the Beck Initiative. Behav Ther. 2010;37:56–65.
29.
go back to reference Stirman SW, Buchhofer R, McLaulin JB, Evans AC, Beck AT. The beck initiative: a partnership to implement cognitive therapy in a community behavioral health system. Psychiatr Serv. 2009;60:1302–4.CrossRef Stirman SW, Buchhofer R, McLaulin JB, Evans AC, Beck AT. The beck initiative: a partnership to implement cognitive therapy in a community behavioral health system. Psychiatr Serv. 2009;60:1302–4.CrossRef
30.
go back to reference Cohen JA, Deblinger E, Mannarino AP, Steer R. A multi-site, randomized controlled trial for children with abuse-related PTSD symptoms. J Am Acad Child Adolesc Psychiatry. 2004;43:393–402.CrossRef Cohen JA, Deblinger E, Mannarino AP, Steer R. A multi-site, randomized controlled trial for children with abuse-related PTSD symptoms. J Am Acad Child Adolesc Psychiatry. 2004;43:393–402.CrossRef
31.
go back to reference Foa EB, Hembree EA, Cahill SP, Rauch SAM, Riggs DS, Feeny NC, et al. Randomized trial of prolonged exposure for posttraumatic stress disorder with and without cognitive restructuring: outcome at academic and community clinics. J Consult Clin Psychol. 2005;735:953–64.CrossRef Foa EB, Hembree EA, Cahill SP, Rauch SAM, Riggs DS, Feeny NC, et al. Randomized trial of prolonged exposure for posttraumatic stress disorder with and without cognitive restructuring: outcome at academic and community clinics. J Consult Clin Psychol. 2005;735:953–64.CrossRef
32.
go back to reference Linehan MM, Comtois KA, Murray AM, Brown MZ, Gallop RJ, Heard HL, et al. Two-year randomized controlled trial and follow-up of dialectical behavior therapy vs therapy by experts for suicidal behaviors and borderline personality disorder. Arch Gen Psychiatry. 2006;63:757–66.CrossRef Linehan MM, Comtois KA, Murray AM, Brown MZ, Gallop RJ, Heard HL, et al. Two-year randomized controlled trial and follow-up of dialectical behavior therapy vs therapy by experts for suicidal behaviors and borderline personality disorder. Arch Gen Psychiatry. 2006;63:757–66.CrossRef
35.
go back to reference Dillman DA. The design and administration of mail surveys. Annu Rev Sociol. 1991;17:225–49.CrossRef Dillman DA. The design and administration of mail surveys. Annu Rev Sociol. 1991;17:225–49.CrossRef
40.
go back to reference Ajslev JZN, Persson R, Andersen LL. Contradictory individualized self-blaming: a cross-sectional study of associations between expectations to managers, coworkers, one-self and risk factors for musculoskeletal disorders among construction workers. BMC Musculoskelet Disord. 2017;18 [cited 19 Mar 2019]. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5223457/. Ajslev JZN, Persson R, Andersen LL. Contradictory individualized self-blaming: a cross-sectional study of associations between expectations to managers, coworkers, one-self and risk factors for musculoskeletal disorders among construction workers. BMC Musculoskelet Disord. 2017;18 [cited 19 Mar 2019]. Available from: https://​www.​ncbi.​nlm.​nih.​gov/​pmc/​articles/​PMC5223457/​.
41.
go back to reference Beidas RS, Kendall PC. Training therapists in evidence-based practice: a critical review of studies from a systems-contextual perspective. Clin Psychol Sci Pract. 2010;17:1–30.CrossRef Beidas RS, Kendall PC. Training therapists in evidence-based practice: a critical review of studies from a systems-contextual perspective. Clin Psychol Sci Pract. 2010;17:1–30.CrossRef
42.
go back to reference Sholomskas D, Syracuse-Siewert G, Rousanville B, Ball S, Nuro K, Carroll K. We don’t train in vain: a dissemination trial of three strategies of training clinicians in cognitive behavioral therapy. J Consult Clin Psychol. 2005;73:106–15.CrossRef Sholomskas D, Syracuse-Siewert G, Rousanville B, Ball S, Nuro K, Carroll K. We don’t train in vain: a dissemination trial of three strategies of training clinicians in cognitive behavioral therapy. J Consult Clin Psychol. 2005;73:106–15.CrossRef
43.
go back to reference Mulkens S, de Vos C, de Graaff A, Waller G. To deliver or not to deliver cognitive behavioral therapy for eating disorders: replication and extension of our understanding of why therapists fail to do what they should do. Behav Res Ther. 2018;106:57–63.CrossRef Mulkens S, de Vos C, de Graaff A, Waller G. To deliver or not to deliver cognitive behavioral therapy for eating disorders: replication and extension of our understanding of why therapists fail to do what they should do. Behav Res Ther. 2018;106:57–63.CrossRef
44.
go back to reference Matjasko JL, Cawley JH, Baker-Goering MM, Yokum DV. Applying behavioral economics to public health policy. Am J Prev Med. 2016;50:S13–9.CrossRef Matjasko JL, Cawley JH, Baker-Goering MM, Yokum DV. Applying behavioral economics to public health policy. Am J Prev Med. 2016;50:S13–9.CrossRef
46.
go back to reference Patel MS, Volpp KG, Asch DA. Nudge units to improve the delivery of health care. N Engl J Med. 2018;378:214–6.CrossRef Patel MS, Volpp KG, Asch DA. Nudge units to improve the delivery of health care. N Engl J Med. 2018;378:214–6.CrossRef
47.
go back to reference Gong CL, Hay JW, Meeker D, Doctor JN. Prescriber preferences for behavioural economics interventions to improve treatment of acute respiratory infections: a discrete choice experiment. BMJ Open. 2016;6:e012739.CrossRef Gong CL, Hay JW, Meeker D, Doctor JN. Prescriber preferences for behavioural economics interventions to improve treatment of acute respiratory infections: a discrete choice experiment. BMJ Open. 2016;6:e012739.CrossRef
Metadata
Title
The clinician crowdsourcing challenge: using participatory design to seed implementation strategies
Authors
Rebecca E. Stewart
Nathaniel Williams
Y. Vivian Byeon
Alison Buttenheim
Sriram Sridharan
Kelly Zentgraf
David T. Jones
Katelin Hoskins
Molly Candon
Rinad S. Beidas
Publication date
01-12-2019
Publisher
BioMed Central
Published in
Implementation Science / Issue 1/2019
Electronic ISSN: 1748-5908
DOI
https://doi.org/10.1186/s13012-019-0914-2

Other articles of this Issue 1/2019

Implementation Science 1/2019 Go to the issue