Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
Implementation Science
Published in: Implementation Science 1/2018

Open Access 01-12-2018 | Research

Leading for the long haul: a mixed-method evaluation of the Sustainment Leadership Scale (SLS)

Authors: Mark G. Ehrhart, Elisa M. Torres, Amy E. Green, Elise M. Trott, Cathleen E. Willging, Joanna C. Moullin, Gregory A. Aarons

Published in: Implementation Science | Issue 1/2018

Login to get access

Abstract

Background

Despite our progress in understanding the organizational context for implementation and specifically the role of leadership in implementation, its role in sustainment has received little attention. This paper took a mixed-method approach to examine leadership during the sustainment phase of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Utilizing the Implementation Leadership Scale as a foundation, we sought to develop a short, practical measure of sustainment leadership that can be used for both applied and research purposes.

Methods

Data for this study were collected as a part of a larger mixed-method study of evidence-based intervention, SafeCare®, sustainment. Quantitative data were collected from 157 providers using web-based surveys. Confirmatory factor analysis was used to examine the factor structure of the Sustainment Leadership Scale (SLS). Qualitative data were collected from 95 providers who participated in one of 15 focus groups. A framework approach guided qualitative data analysis. Mixed-method integration was also utilized to examine convergence of quantitative and qualitative findings.

Results

Confirmatory factor analysis supported the a priori higher order factor structure of the SLS with subscales indicating a single higher order sustainment leadership factor. The SLS demonstrated excellent internal consistency reliability. Qualitative analyses offered support for the dimensions of sustainment leadership captured by the quantitative measure, in addition to uncovering a fifth possible factor, available leadership.

Conclusions

This study found qualitative and quantitative support for the pragmatic SLS measure. The SLS can be used for assessing leadership of first-level leaders to understand how staff perceive leadership during sustainment and to suggest areas where leaders could direct more attention in order to increase the likelihood that EBIs are institutionalized into the normal functioning of the organization.
Literature
1.
Wandersman A, Alia K, Cook BS, Hsu L, Ramaswamy V. Evidence-based interventions are necessary but not sufficient for achieving outcomes in each setting in a complex world empowerment evaluation, getting to outcomes, and demonstrating accountability. Am J Eval. 2016;37(4):544–61.CrossRef
2.
Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Hlth. 2011;38(1):4–23.CrossRef
3.
Damschroder L, Aron D, Keith R, Kirsh S, Alexander J, Lowery J. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.CrossRefPubMedPubMedCentral
4.
Ehrhart MG, Schneider B, Macey WH. Organizational climate and culture: an introduction to theory, research, and practice. New York, NY: Routledge; 2014.
5.
Glisson C, Schoenwald SK, Hemmelgarn A, Green P, Dukes D, Armstrong KS, Chapman JE. Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. J Consult Clin Psychol. 2010;78(4):537–50.CrossRefPubMedPubMedCentral
6.
Helfrich C, Li Y, Sharp N, Sales A. Organizational readiness to change assessment (ORCA): development of an instrument based on the promoting action on research in health services (PARIHS) framework. Implement Sci. 2009;4:38.CrossRefPubMedPubMedCentral
7.
8.
Shea CM, Jacobs SR, Esserman DA, Bruce K, Weiner BJ. Organizational readiness for implementing change: a psychometric assessment of a new measure. Implement Sci. 2014;9:7.CrossRefPubMedPubMedCentral
9.
Edmondson AC, Bohmer RM, Pisano GP. Disrupted routines: team learning and new technology implementation in hospitals. Admin Sci Quart. 2001;46(4):685–716.CrossRef
10.
Pisano GP, Bohmer R, Edmondson A. Organizational differences in rates of learning: evidence from the adoption of minimally invasive cardiac surgery. Manag Sci. 2001;47(6):752–68.CrossRef
11.
Yukl G. Leadership in organizations. 8th ed. Upper Saddle River, NJ: Prentice Hall; 2013.
12.
13.
Stirman SW, Kimberly J, Cook N, Calloway A, Castro F. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012;7:17.CrossRef
14.
Aarons GA. Transformational and transactional leadership: association with attitudes toward evidence-based practice. Psychiatr Serv. 2006;57(8):1162–9.CrossRefPubMedPubMedCentral
15.
Aarons GA, Sommerfeld DH, Willging CE. The soft underbelly of system change: the role of leadership and organizational climate in turnover during statewide behavioral health reform. Psychol Serv. 2011;8(4):269–81.CrossRefPubMedPubMedCentral
16.
Guerrero EG, Padwa H, Fenwick K, Harris L, Aarons G. Identifying and ranking implicit leadership strategies to promote evidence-based practice implementation in addiction health services. Implement Sci. 2016;11:69.CrossRefPubMedPubMedCentral
17.
Michaelis B, Stegmaier R, Sonntag K. Affective commitment to change and innovation implementation behavior: the role of charismatic leadership and employees’ trust in top management. J Change Manage. 2009;9(4):399–417.CrossRef
18.
Michaelis B, Stegmaier R, Sonntag K. Shedding light on followers’ innovation implementation behavior: the role of transformational leadership, commitment to change, and climate for initiative. J Manage Psychol. 2010;25(4):408–29.CrossRef
19.
Sloan R, Gruman J. Participation in workplace health promotion programs: the contribution of health and organizational factors. Health Educ Behav. 1988;15(3):269–88.
20.
Aarons GA, Ehrhart MG, Farahnak LR. The implementation leadership scale (ILS): development of a brief measure of unit level implementation leadership. Implement Sci. 2014;9:157.CrossRefPubMedPubMedCentral
21.
Schein E. Organizational culture and leadership. 4th ed. San Francisco: Wiley; 2010.
22.
Hong Y, Liao H, Hu J, Jiang K. Missing link in the service profit chain: a meta-analytic review of the antecedents, consequences, and moderators of service climate. J Appl Psychol. 2013;98(2):237–67.CrossRefPubMed
23.
Schneider B, Ehrhart MG, Mayer DM, Saltz JL, Niles-Jolly K. Understanding organization-customer links in service settings. Acad Manag J. 2005;48(6):1017–32.CrossRef
24.
Zohar D. Modifying supervisory practices to improve subunit safety: a leadership-based intervention model. J Appl Psychol. 2002;87(1):156–63.CrossRefPubMed
25.
Neal A, Griffin MA. A study of the lagged relationships among safety climate, safety motivation, safety behavior, and accidents at the individual and group levels. J Appl Psychol. 2006;91(4):946–53.CrossRefPubMed
26.
Shediac-Rizkallah MC, Bone LR. Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Educ Res. 1998;13(1):87–108.CrossRefPubMed
27.
Proctor E, Luke D, Calhoun A, McMillen C, Brownson RC, McCrary S, Padek M. Sustainability of evidence-based healthcare: research agenda, methodological advances, and infrastructure support. Implement Sci. 2015;10:88.CrossRefPubMedPubMedCentral
28.
Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8:117.CrossRefPubMedPubMedCentral
29.
Scheirer MA. Is sustainability possible? A review and commentary on empirical studies of program sustainability. Am J Eval. 2005;26(3):320–47.CrossRef
30.
Novins DK, Green AE, Legha RK, Aarons GA. Dissemination and implementation of evidence-based practices for child and adolescent mental health: a systematic review. Journal of the American Academy of Child & Adolescent Psychiatry. 2013;52(10):1009–25. e1018CrossRef
31.
Aarons GA, Sommerfeld DH, Hecht DB, Silovsky JF, Chaffin MJ. The impact of evidence-based practice implementation and fidelity monitoring on staff turnover: evidence for a protective effect. J Consult Clin Psychol. 2009;77(2):270–80.CrossRefPubMedPubMedCentral
32.
Muthén LK, Muthén BO. Mplus user's guide. 8th ed. Muthén & Muthén: Los Angeles, CA; 1998-2017.
33.
Hu L-T, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Modeling. 1999;6(1):1–55.CrossRef
34.
QSR International Pty Ltd. NVivo qualitative data analysis software. 10th ed; 2012.
35.
Pope C, Mays N: Qualitative research in health care. Malden, MA: Blackwell; 2006.
36.
Patton MQ. Qualitative analysis and interpretation. In: Qualitative research and evaluation methods. 3rd ed. Thousand Oaks, CA: Sage Publications; 2002. p. 431–539.
37.
Corbin J, Strauss A. Basics of qualitative research: techniques and procedures for developing grounded theory (3rd ed). Thousand Oaks, CA: Sage Publications; 2008.CrossRef
38.
Aarons GA, Fettes DL, Sommerfeld DH, Palinkas LA. Mixed methods for implementation research: application to evidence-based practice implementation and staff turnover in community-based organizations providing child welfare services. Child Maltreat. 2012;17(1):67–79.CrossRefPubMed
39.
Creswell JW, Plano Clark VL. Designing and conducting mixed methods research. 2nd ed. Thousand Oaks, CA: Sage; 2011.
40.
Teddlie C, Tashakkori A. Major issues and controversies in the use of mixed methods in the social and behavioral sciences. In: Teddlie C, Tashakkori A, editors. Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage Publications; 2003. p. 3–50.
41.
JM LeBreton, Senter JL. Answers to twenty questions about interrater reliability and interrater agreement. Organ Res Methods. 2008;11:815–52.CrossRef
42.
Aarons GA, Ehrhart MG, Farahnak LR, Sklar M. Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annual Review of Public Health. 2014;35:255–74.CrossRefPubMedPubMedCentral
43.
Aarons GA. Mental health provider attitudes toward adoption of evidence-based practice: the evidence-based practice attitude scale (EBPAS). Ment Health Serv Res. 2004;6:61–74.CrossRefPubMedPubMedCentral
44.
Schoenwald SK, Garland AF, Chapman JE, Frazier SL, Sheidow AJ, Southam-Gerow MA. Toward the effective and efficient measurement of implementation fidelity. Adm Policy Ment Hlth. 2011;38(1):32–43.CrossRef
45.
Dyer NG, Hanges PJ, Hall RJ. Applying multilevel confirmatory factor analysis techniques to the study of leadership. Leadership Quart. 2005;16:149–67.CrossRef
46.
Martin R, Thomas G, Legood A, Dello Russo S. Leader–member exchange (LMX) differentiation and work outcomes: conceptual clarification and critical review. J Organ Behav. In Press
47.
Feinberg BJ, Ostroff C, Burke WW. The role of within-group agreement in understanding transformational leadership. J Occup Organ Psychol. 2005;78(3):471–88.CrossRef
48.
Nielsen K, Daniels K. Does shared and differentiated transformational leadership predict followers' working conditions and well-being? Leadership Quart. 2012;23(3):383–97.CrossRef
49.
Klein KJ, Kozlowski SW. Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions. San Francisco: Jossey-Bass; 2000.
50.
Aarons GA, Ehrhart MG, Farahnak LR, Sklar M, Horowitz J. Discrepancies in leader and follower ratings of transformational leadership: relationship with organizational culture in mental health. Adm Policy Ment Hlth. 2017;44:480–91.CrossRef
51.
Aarons GA, Ehrhart MG, Torres EM, Finn NK, Beidas RS. The humble leader: association of discrepancies in leader and follower ratings of implementation leadership with organizational climate in mental health. Psychiatr Serv. 2017;68:115–22.CrossRefPubMed
52.
Gill P, Stewart K, Treasure E, Chadwick B. Methods of data collection in qualitative research: interviews and focus groups. Br Dent J. 2008;204:291–5.CrossRefPubMed
Metadata
Title
Leading for the long haul: a mixed-method evaluation of the Sustainment Leadership Scale (SLS)
Authors
Mark G. Ehrhart
Elisa M. Torres
Amy E. Green
Elise M. Trott
Cathleen E. Willging
Joanna C. Moullin
Gregory A. Aarons
Publication date
01-12-2018
Publisher
BioMed Central
Published in
Implementation Science / Issue 1/2018
Electronic ISSN: 1748-5908
DOI
https://doi.org/10.1186/s13012-018-0710-4

Other articles of this Issue 1/2018

Implementation Science 1/2018 Go to the issue

Methodology

To what extent can behaviour change techniques be identified within an adaptable implementation package for primary care? A prospective directed content analysis

Research

How do physicians behave when they participate in audit and feedback activities in a group with their peers?

Study protocol

Toward optimal implementation of cancer prevention and control programs in public health: a study protocol on mis-implementation

Study protocol

Leveraging routine clinical materials and mobile technology to assess CBT fidelity: the Innovative Methods to Assess Psychotherapy Practices (imAPP) study

Study protocol

Supporting teams to optimize function and independence in Veterans: a multi-study program and mixed methods protocol

Methodology

A methodology for generating a tailored implementation blueprint: an exemplar from a youth residential setting