Top

BMC Health Services Research

Published in:

Open Access 01-12-2019 | Climate Change | Research article

A realist evaluation to identify contexts and mechanisms that enabled and hindered implementation and had an effect on sustainability of a lean intervention in pediatric healthcare

Authors: Rachel Flynn, Thomas Rotter, Dawn Hartfield, Amanda S. Newton, Shannon D. Scott

Published in: BMC Health Services Research | Issue 1/2019

Abstract

Background

In 2012, the Saskatchewan Ministry for Health mandated a system-wide Lean transformation. Research has been conducted on the implementation processes of this system-wide Lean implementation. However, no research has been done on the sustainability of these Lean efforts. We conducted a realist evaluation on the sustainability of Lean in pediatric healthcare. We used the context (C) + mechanism (M) = outcome (O) configurations (CMOcs) heuristic to explain under what contexts, for whom, how and why Lean efforts are sustained or not sustained in pediatric healthcare.

Methods

We employed a case study research design. Guided by a realist evaluation framework, we conducted qualitative realist interviews with various stakeholder groups across four pediatric hospital units ‘cases’ at one acute hospital. Interview data was analyzed using an integrated approach of CMOc categorization coding, CMOc connecting and pattern matching.

Results

We conducted thirty-two interviews across the four cases. Five CMOcs emerged from our realist interview data. These configurations illustrated a ‘ripple-effect’ from implementation outcomes to contexts for sustainability. Sense-making and staff engagement were prominent mechanisms to the sustainment of Lean efforts. Failure to trigger these mechanisms resulted in resistance. The implementation approach used influenced mechanisms and outcomes for sustainability, more so than Lean itself. Specifically, the language, messaging and training approaches used triggered mechanisms of innovation fatigue, poor ‘sense-making’ and a lack of engagement for frontline staff. The mandated, top-down, externally led nature of implementation and lack of customization to context served as potential pitfalls. Overall, there was variation between leadership and frontline staff’s perceptions on how embedded Lean was in their contexts, and the degree to which participants supported Lean sustainability.

Conclusions

This research illuminates important contextual factors and mechanisms to the process of Lean sustainment that can be applicable to those implementing systems changes. Future work is needed to continue to develop the science on the sustainability of interventions for healthcare improvement.

Available only for authorised users

Rotter T, Plishka C, Lawal A, et al. What is Lean management in health care? Development of an operational definition for a Cochrane systematic review. Eval Health Prof. 2018:1–25. https://doi.org/10.1177/0163278718756992 PMID:29635950.CrossRef

Womack JP, Jones DT, Roos D. The machine that changed the world: the story of lean production. New York: Harper Collins; 1990.

Womack JP, Jones DT. Lean thinking: banish the waste and create wealth in your corporation. London: Simon and Schulster; 2003.

Mann D. Creating a lean culture: tools to sustain lean conversions. 2nd ed. New York: Productivity Press; 2010. https://doi.org/10.1201/EBK1439811412.CrossRef

Goodridge D, Westhorp G, Rotter T, Dobson R, Bath B. Lean and leadership practices: development of an initial realist program theory. BMC Health Serv Res. 2015;15(1):362–77. https://doi.org/10.1186/s12913-015-1030-x.CrossRefPubMedPubMedCentral

Mazzocato P, Savage C, Brommels M, Aronsson H, Thor J. Lean thinking in healthcare: a realist review of the literature. Qual Saf Health Care. 2010;19(5):376–82 PMID:20724397.PubMed

Mazzocato P, Thor J, Bäckman U, et al. Complexity complicates lean: lessons from seven emergency services. J Health Organ Manag. 2014;28(2):266–88. https://doi.org/10.1108/JHOM-03-2013-0060.CrossRefPubMed

Holden RJ. Lean thinking in emergency departments: a critical review. Ann Emerg Med. 2011;57(3):265–78. https://doi.org/10.1016/j.annemergmed.2010.08.001.CrossRefPubMed

Grove AL, Meredith JO, Macintyre M, Angelis J, Neailey K. Lean implementation in primary care health visiting services in National Health Service UK. Qual Saf Health Care. 2010;19(5):e43 PMID:20511600.PubMed

10.

McCann L, Hassard J, Granter E, Hyde P. Casting the lean spell: the promotion, dilution and erosion of lean management in the NHS. Hum Relat. 2015;68(10):1557–77. https://doi.org/10.1177/0018726714561697.CrossRef

11.

Bhasin S, Burcher P. Lean viewed as a philosophy. J Manuf Tech Manag. 2006;17(1):56–72. https://doi.org/10.1108/17410380610639506.CrossRef

12.

Burgess N, Radnor Z. Evaluating lean in healthcare. Int J Health Care Qual Assur. 2013;26(3):220–35. https://doi.org/10.1108/09526861311311418.CrossRefPubMed

13.

Smith G, Poteat-Godwin A, Harrison LM, Randolph GD. Applying lean principles and kaizen rapid improvement events in public health practice. J Public Health Manag Pract. 2012;18(1):52–4. https://doi.org/10.1097/PHH.0b013e31823f57c0.CrossRefPubMed

14.

Young TP, McClean SI. A critical look at lean thinking in healthcare. Qual Saf Health Care. 2008;17(5):382–6. https://doi.org/10.1136/qshc.2006.020131.CrossRefPubMed

15.

Radnor Z, Osborne SP. Lean: a failed theory for public services. Public Manage Rev. 2013;15(2):265–87. https://doi.org/10.1080/14719037.2012.748820.CrossRef

16.

Andersen H, Røvik KA, Ingebrigtsen T. Lean thinking in hospitals: is there a cure for the absence of evidence? A systematic review of reviews. BMJ Open. 2014;4(1):e003873. https://doi.org/10.1136/bmjopen-2013-003873.CrossRefPubMedPubMedCentral

17.

DelliFraine JL, Langabeer JR II, Nembhard IM. Assessing the evidence of six sigma and lean in the health care industry. Qual Manag Health Care. 2010;19(3):211–25. https://doi.org/10.1097/QMH.0b013e3181eb140e.CrossRefPubMed

18.

Waring JJ, Bishop S. Lean healthcare: rhetoric, ritual and resistance. Soc Sci Med. 2010;71(7):1332–40. https://doi.org/10.1016/j.socscimed.2010.06.028.CrossRefPubMed

19.

Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8(1):117–28. https://doi.org/10.1186/1748-5908-8-117.CrossRefPubMedPubMedCentral

20.

Wiltsey Stirman S, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012;7(1):17–36. https://doi.org/10.1186/1748-5908-7-17.CrossRefPubMedPubMedCentral

21.

Proctor E, Luke D, Calhoun A, et al. Sustainability of evidence-based healthcare: research agenda, methodological advances, and infrastructure support. Implement Sci. 2015;10(1):88–101. https://doi.org/10.1186/s13012-015-0274-5.CrossRefPubMedPubMedCentral

22.

Flynn R, Newton AS, Rotter T, Hartfield D, Walton S, Fiander M, Scott SD. The sustainability of Lean in pediatric healthcare: a realist review. BMC Syst Rev. 2018;7:137. https://doi.org/10.1186/s13643-018-0800-z.CrossRef

23.

Maher L, Gustafson D, Evans A. NHS Sustainability Model. NHS Institute for Innovation and Improvement; 2010. http://webarchive.nationalarchives.gov.uk/20160805122935/http://www.nhsiq.nhs.uk/media/2757778/nhs_sustainability_model_-_february_2010_1_.pdf. Accessed 31 May 2018.

24.

Pawson R, Tilley N. Realistic evaluation. Thousand Oaks: SAGE Publications Ltd; 1997.

25.

Yin RK. Case study research: design and methods. 3rd ed. Thousand Oaks: Sage Publications; 2003.

26.

Mackenzie J, Hall W. “Lean” in Canadian health care: doing less while achieving more, The Conference Board of Canada. Ottawa: Publication 6262; 2014. Accessed 18 Jan 2018

27.

Kinsman L, Rotter T, Stevenson K, et al. “The largest lean transformation in the world”: the implementation and evaluation of lean in Saskatchewan healthcare. Healthc Q. 2014;17(2):29–32. https://doi.org/10.12927/hcq.2014.23880.CrossRefPubMed

28.

Government of Saskatchewan. Ministry of Health: Plan for 2013–14; 2013. http://publications.gov.sk.ca/documents/13/81400-ministry_of_health_ar_2013-2014-web.pdf. Accessed 3 May 2018.

29.

Sari N, Rotter T, Goodridge D, Harrison L, Kinsman L. An economic analysis of a system wide lean approach: cost estimations for the implementation of lean in the Saskatchewan healthcare system for 2012-2014. BMC Health Serv Res. 2017;17(1):523. https://doi.org/10.1186/s12913-017-2477-8.CrossRefPubMedPubMedCentral

30.

Saskatchewan-Health-Quality-Council. Lean Reform: Saskatchewan Healthcare Adopts Lean Management for Big Benefits; 2014. http://blog.hqc.sk.ca/2014/2008/2028/lean-reform-saskatchewan-healthcare-adopts-lean-management-for-big-benefits/. Accessed 3 May 2018.

31.

Rotter T, Kinsman L, Bath B, Goodridge D, Harrison L, Dobson R. et al. A first phase evaluation of Saskatchewan’s lean health care transformation: final report; 2014. http://research-groups.usask.ca/rotter/documents/select-publications/Lean%20Report%20-%20Full.pdf. Accessed 6 Nov 2017.

32.

Manzano A. The craft of interviewing in realist evaluation. Evaluation. 2016;22(3):342–60. https://doi.org/10.1177/1356389016638615.CrossRef

33.

Salter KL, Kothari A. Using realist evaluation to open the black box of knowledge translation: a state-of-the-art review. Implement Sci. 2014;9(1):115–40. https://doi.org/10.1186/s13012-014-0115-y.CrossRefPubMedPubMedCentral

34.

Pawson R, Manzano-Santaella A. A realist diagnostic workshop. Evaluation. 2012;18(2):176–91. https://doi.org/10.1177/1356389012440912.CrossRef

35.

Punton M, Vogel I, Lloyd R. Reflections from a realist evaluation in progress: scaling ladders and stitching theory. CDI Pract Pap. 2016;18:18–29.

36.

Jagosh J, Bush PL, Salsberg J, et al. A realist evaluation of community-based participatory research: partnership synergy, trust building and related ripple effects. BMC Public Health. 2015;15(1):725–36. https://doi.org/10.1186/s12889-015-1949-1.CrossRefPubMedPubMedCentral

37.

Dopson S, Fitzgerald LA. The active role of context. In: Dopson S, Fitzgerald LA, editors. Knowledge to action? Evidence-based health care in context. Oxford: Oxford University Press; 2005. p. 79–103. https://doi.org/10.1093/acprof:oso/9780199259014.003.0005.CrossRef

38.

Astbury B, Leeuw F. Unpacking black boxes: mechanisms and theory building in evaluation. Am J Eval. 2010;31(3):363–81. https://doi.org/10.1177/1098214010371972.CrossRef

39.

Doyle C, Howe C, Woodcock T, et al. Making change last: applying the NHS institute for innovation and improvement sustainability model to healthcare improvement. Implement Sci. 2013;8(1):127–37. https://doi.org/10.1186/1748-5908-8-127.CrossRefPubMedPubMedCentral

40.

May CR, Mair F, Finch T, et al. Development of a theory of implementation and integration: normalization process theory. Implement Sci. 2009;4(1):29. https://doi.org/10.1186/1748-5908-4-29.CrossRefPubMedPubMedCentral

41.

May C, Finch T. Implementing, embedding, and integrating practices: an outline of normalization process theory. Sociology. 2009;43(3):535–54. https://doi.org/10.1177/0038038509103208.CrossRef

42.

Maxwell JA. A realist approach for qualitative research. Thousand Oaks: Sage Publications; 2012.

43.

Porter S. Validity, trustworthiness and rigour: reasserting realism in qualitative research. J Adv Nurs. 2007;60(1):79–86. https://doi.org/10.1111/j.1365-2648.2007.04360.x.CrossRefPubMed

44.

Pawson R, Boaz A, Grayson L, Long A, Barnes C. Types and quality of knowledge in social care. London: Social Care Institute for Excellence; 2003.

45.

Wong G, Westhorp G, Manzano A, Greenhalgh J, Jagosh J, Greenhalgh T. RAMESES II reporting standards for realist evaluations. BMC Med. 2016;14:1–18. https://doi.org/10.1186/s12916-016-0643-1.CrossRef

46.

Hawe P, Shiell A, Riley T. Theorising interventions as events in systems. Am J Community Psychol. 2009;43(3–4):267–76. https://doi.org/10.1007/s10464-009-9229-9.CrossRefPubMed

47.

Goodridge D, Rana M, Harrison EL, et al. Assessing the implementation processes of a large-scale, multi-year quality improvement initiative: survey of health care providers. BMC Health Serv Res. 2018;18(1):237–59. https://doi.org/10.1186/s12913-018-3045-6.CrossRefPubMedPubMedCentral

48.

Wong AM, During D, Hartman M, Lappan-Gracon S, Hicks M, Bajwa S. Lean transformation of the eye clinic at the hospital for sick children: challenging an implicit mental model and lessons learned. Healthc Q. 2016;19(1):36–41. https://doi.org/10.12927/hcq.2016.24611.CrossRefPubMed

49.

Hung DY. Spreading lean: taking efficiency interventions in health services delivery to scale. In: Agency for Healthcare Research and Quality; 2016. https://www.ahrq.gov/sites/default/files/publications/files/execsumm-lean-redesign.pdf. Accessed 6 Nov 2017.

50.

Braithwaite J. Changing how we think about healthcare improvement. BMJ. 2018;361:k2014. https://doi.org/10.1136/bmj.k2014.CrossRefPubMedPubMedCentral

51.

Braithwaite J, Churruca K, Long JC, Ellis LA, Herkes J. When complexity science meets implementation science: a theoretical and empirical analysis of systems change. BMC Med. 2018;16(1):63–77. https://doi.org/10.1186/s12916-018-1057-z.CrossRefPubMedPubMedCentral

52.

Cresswell KM, Bates DW, Sheikh A. Ten key considerations for the successful implementation and adoption of large-scale health information technology. J Am Med Inform Assoc. 2013;20(e1):e9–e13. https://doi.org/10.1136/amiajnl-2013-001684.CrossRefPubMedPubMedCentral

53.

Paré G, Sicotte C, Jaana M, Girouard D. Prioritizing the risk factors influencing the success of clinical information system projects. A Delphi study in Canada. Methods Inf Med. 2008;47(3):251–9 PMID:18473092.CrossRef

54.

Ash JS, Stavri PZ, Kuperman GJ. A consensus statement on considerations for a successful CPOE implementation. J Am Med Inform Assoc. 2003;10(3):229–34. https://doi.org/10.1197/jamia.M1204.CrossRefPubMedPubMedCentral

55.

Cummings A, Lund S, Campling N, May CR, Richardson A, Myall M. Implementing communication and decision-making interventions directed at goals of care: a theory-led scoping review. BMJ Open. 2017;7(10):e017056. https://doi.org/10.1136/bmjopen-2017-017056.CrossRefPubMedPubMedCentral

56.

Fine BA, Golden B, Hannam R, Morra D. Leading lean: a Canadian healthcare leader’s guide. Healthc Q. 2009;12(3):32–41. https://doi.org/10.12927/hcq.2013.20877.CrossRefPubMed

57.

Mazzocato P, Holden RJ, Brommels M, et al. How does lean work in emergency care? A case study of a lean-inspired intervention at the Astrid Lindgren Children’s hospital, Stockholm, Sweden. BMC Health Serv Res. 2012;12(1):28–41. https://doi.org/10.1186/1472-6963-12-28.CrossRefPubMedPubMedCentral

58.

Northway T, Krahn G, Thibault K, et al. Surgical suite to pediatric intensive care unit handover protocol: implementation process and long-term sustainability. J Nurs Care Qual. 2015;30(2):113–20. https://doi.org/10.1097/NCQ.0000000000000093 PMID:25426649.CrossRefPubMed

59.

Øvretveit J. Understanding the conditions for improvement: research to discover which context influences affect improvement success. BMJ Qual Saf. 2011;20(suppl 1):i18–23. https://doi.org/10.1136/bmjqs.2010.045955.CrossRefPubMedPubMedCentral

60.

Robinson S, Radnor ZJ, Burgess N, Worthington C. SimLean: utilising simulation in the implementation of lean in healthcare. Eur J Oper Res. 2012;219(1):188–97. https://doi.org/10.1016/j.ejor.2011.12.029.CrossRef

61.

Plsek PE, Greenhalgh T. Complexity science: the challenge of complexity in health care. BMJ. 2001;323(7313):625–8. https://doi.org/10.1136/bmj.323.7313.625.CrossRefPubMedPubMedCentral

62.

May C, Johnson M, Finch T. Implementation, context and complexity. Implement Sci. 2016;11:1–12. https://doi.org/10.1186/s13012-016-0506-3.CrossRef

63.

Radnor ZJ, Holweg M, Waring J. Lean in healthcare: the unfilled promise? Soc Sci Med. 2012;74(3):364–71. https://doi.org/10.1016/j.socscimed.2011.02.011.CrossRefPubMed

64.

McCormack B, Kitson A, Harvey G, Rycroft-Malone J, Titchen A, Seers K. Getting evidence into practice: the meaning of ‘context’. J Adv Nurs. 2002;38(1):94–104. https://doi.org/10.1046/j.1365-2648.2002.02150.x.CrossRefPubMed

65.

Curran JA, Grimshaw JM, Hayden JA, Campbell B. Knowledge translation research: the science of moving research into policy and practice. J Contin Educ Heal Prof. 2011;31(3):174–80. https://doi.org/10.1002/chp.20124.CrossRef

66.

Walshe K, Freeman T. Effectiveness of quality improvement: learning from evaluations. Qual Saf Health Care. 2002;11(1):85–7. https://doi.org/10.1136/qhc.11.1.85.CrossRefPubMedPubMedCentral

67.

Lau R, Stevenson F, Ong BN, et al. Achieving change in primary care—causes of the evidence to practice gap: systematic reviews of reviews. Implement Sci. 2016;11(1):40. https://doi.org/10.1186/s13012-016-0396-4.CrossRefPubMedPubMedCentral

68.

Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629. https://doi.org/10.1111/j.0887-378X.2004.00325.x.CrossRefPubMedPubMedCentral

69.

Fritz Z, Fuld JP. Development of the universal form of treatment options (UFTO) as an alternative to do not attempt cardiopulmonary resuscitation (DNACPR) orders: a cross-disciplinary approach. J Eval Clin Pract. 2015;21(1):109–17. https://doi.org/10.1111/jep.12256.CrossRefPubMed

70.

Brimblecombe C, Crosbie D, Lim WK, Hayes B. The goals of patient care project: implementing a proactive approach to patient-centred decision-making. Intern Med J. 2014;44(10):961–6. https://doi.org/10.1111/imj.12511.CrossRefPubMed

71.

Weiner BJ. A theory of organizational readiness for change. Implement Sci. 2009;4(1):67–76. https://doi.org/10.1186/1748-5908-4-67.CrossRefPubMedPubMedCentral

72.

Kim CS, Spahlinger DA, Kin JM, Billi JE. Lean health care: what can hospitals learn from a world-class automaker? J Hosp Med. 2006;1(3):191–9. https://doi.org/10.1002/jhm.68.CrossRefPubMed

73.

Braithwaite J, Runciman WB, Merry AF. Towards safer, better healthcare: harnessing the natural properties of complex sociotechnical systems. Qual Saf Health Care. 2009;18(1):37–41. https://doi.org/10.1136/qshc.2007.023317.CrossRefPubMedPubMedCentral

74.

Zander U, Kogut B. Knowledge and the speed of the transfer and imitation of organizational capabilities: an empirical test. Organ Sci. 1995;6(1):76–92. https://doi.org/10.1287/orsc.6.1.76.CrossRef

75.

Saskatchewan Union of Nurses. Lean: A Safe and Effective Tool for Registered Nursing? 2014. http://sun-nurses.sk.ca/+pub/document/Communications/SUNSpots/2014/SUNSpots%2040-3%20w%20Feature%20(July%2015-14).pdf. Accessed 3 May 2018.

76.

Shortell SM, Bennett CL, Byck GR. Assessing the impact of continuous quality improvement on clinical practice: what it will take to accelerate progress. Milbank Q. 1998;76(4):593–624. https://doi.org/10.1111/1468-0009.00107.CrossRefPubMedPubMedCentral

77.

Atkinson P. Creating and implementing lean strategies. Manag Serv. 2004;48(2):18–33.

78.

Van Vliet EJ, Bredenhoff E, Sermeus W, Kop LM, Sol JC, Van Harten WH. Exploring the relation between process design and efficiency in high-volume cataract pathways from a lean thinking perspective. Int J Qual Health Care. 2011;23(1):83–93. https://doi.org/10.1093/intqhc/mzq071.CrossRefPubMed

79.

Radnor Z. Implementing lean in health care: making the link between the approach, readiness and sustainability. Int J Ind Eng Manag. 2011;2(1):1–12.

80.

Maijala R, Eloranta S, Reunanen T, Ikonen TS. Successful implementation of lean as a managerial principle in health care: a conceptual analysis from systematic literature review. Int J Technol Assess Health Care. 2018;34(2):134–46. https://doi.org/10.1017/S0266462318000193 PMID:29642955.CrossRefPubMed

81.

Radnor Z, Walley P, Stephens A, Bucci G. Evaluation of the Lean Approach to Business Management and its Use in the Public Sector (full report). Edinburgh: Office of Chief Researcher, Scottish Executive; 2006. http://www.gov.scot/Resource/Doc/129627/0030899.pdf . Accessed 3 May 2018.

82.

Brandao de Souza L, Pidd M. Exploring the barriers to lean health care implementation. Public Money Manag. 2011;31(1):59–66. https://doi.org/10.1080/09540962.2011.545548.CrossRef

83.

Lukas CV, Holmes SK, Cohen AB, et al. Transformational change in health care systems: an organizational model. Health Care Manag Rev. 2007;32(4):309–20. https://doi.org/10.1097/01.HMR.0000296785.29718.5d.CrossRef

84.

Ben-Tovim DI, Bassham JE, Bolch D, Martin MA, Dougherty M, Szwarcbord M. Lean thinking across a hospital: redesigning care at the flinders medical Centre. Aust Health Rev. 2007;31(1):10–5. https://doi.org/10.1071/AH070010.CrossRefPubMed

85.

Brandao de Souza L. Trends and approaches in lean healthcare. Leadersh Health Serv. 2009;22(2):121–39. https://doi.org/10.1108/17511870910953788.CrossRef

Title: A realist evaluation to identify contexts and mechanisms that enabled and hindered implementation and had an effect on sustainability of a lean intervention in pediatric healthcare
Authors: Rachel Flynn
Thomas Rotter
Dawn Hartfield
Amanda S. Newton
Shannon D. Scott
Publication date: 01-12-2019
Publisher: BioMed Central
Keyword: Climate Change
Published in: BMC Health Services Research / Issue 1/2019
Electronic ISSN: 1472-6963
DOI: https://doi.org/10.1186/s12913-019-4744-3

At a glance: The STEP trials

Springer Medicine

A realist evaluation to identify contexts and mechanisms that enabled and hindered implementation and had an effect on sustainability of a lean intervention in pediatric healthcare

Abstract

Background

Methods

Results

Conclusions

At a glance: The STEP trials

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2019

Developing a toolkit to implement the Statin Choice Conversation Aid at scale: application of a work reduction model

Impacts of working environment and benefits packages on the health professionals’ job satisfaction in selected public health facilities in eastern Ethiopia: using principal component analysis

Cost-utility analysis of palonosetron in the antiemetic regimen for cisplatin-containing highly emetogenic chemotherapy in Japan

A mixed methods process evaluation of a person-centred falls prevention program

Correction to: Effectiveness of quality incentive payments in general practice (EQuIP-GP): a study protocol for a cluster-randomised trial of an outcomes-based funding model in Australian general practice to improve patient care

The effect of a pharmaceutical transitional care program on rehospitalisations in internal medicine patients: an interrupted-time-series study