Skip to main content
Top
Published in: BMC Health Services Research 1/2019

Open Access 01-12-2019 | Research article

Can quality improvement improve the quality of care? A systematic review of reported effects and methodological rigor in plan-do-study-act projects

Authors: Søren Valgreen Knudsen, Henrik Vitus Bering Laursen, Søren Paaske Johnsen, Paul Daniel Bartels, Lars Holger Ehlers, Jan Mainz

Published in: BMC Health Services Research | Issue 1/2019

Login to get access

Abstract

Background

The Plan-Do-Study-Act (PDSA) method is widely used in quality improvement (QI) strategies. However, previous studies have indicated that methodological problems are frequent in PDSA-based QI projects. Furthermore, it has been difficult to establish an association between the use of PDSA and improvements in clinical practices and patient outcomes. The aim of this systematic review was to examine whether recently published PDSA-based QI projects show self-reported effects and are conducted according to key features of the method.

Methods

A systematic literature search was performed in the PubMed, Embase and CINAHL databases. QI projects using PDSA published in peer-reviewed journals in 2015 and 2016 were included. Projects were assessed to determine the reported effects and the use of the following key methodological features; iterative cyclic method, continuous data collection, small-scale testing and use of a theoretical rationale.

Results

Of the 120 QI projects included, almost all reported improvement (98%). However, only 32 (27%) described a specific, quantitative aim and reached it. A total of 72 projects (60%) documented PDSA cycles sufficiently for inclusion in a full analysis of key features. Of these only three (4%) adhered to all four key methodological features.

Conclusion

Even though a majority of the QI projects reported improvements, the widespread challenges with low adherence to key methodological features in the individual projects pose a challenge for the legitimacy of PDSA-based QI. This review indicates that there is a continued need for improvement in quality improvement methodology.
Appendix
Available only for authorised users
Literature
1.
go back to reference Nicolay CR, Purkayastha S, Greenhalgh A, Benn J, Chaturvedi S, Phillips N, et al. Systematic review of the application of quality improvement methodologies from the manufacturing industry to surgical healthcare. Br J Surg. 2012;99(3):324–35.CrossRef Nicolay CR, Purkayastha S, Greenhalgh A, Benn J, Chaturvedi S, Phillips N, et al. Systematic review of the application of quality improvement methodologies from the manufacturing industry to surgical healthcare. Br J Surg. 2012;99(3):324–35.CrossRef
2.
go back to reference Speroff T, O’Connor GT. Study designs for PDSA quality improvement research. Qual Manag Health Care. 2004;13(1):17–32.CrossRef Speroff T, O’Connor GT. Study designs for PDSA quality improvement research. Qual Manag Health Care. 2004;13(1):17–32.CrossRef
4.
go back to reference Reed JE, Card AJ. The problem with plan-do-study-act cycles. BMJ Qual Saf. 2016;25(3):147–52.CrossRef Reed JE, Card AJ. The problem with plan-do-study-act cycles. BMJ Qual Saf. 2016;25(3):147–52.CrossRef
5.
go back to reference Portela MC, Lima SML, Martins M, Travassos C. Improvement Science: conceptual and theoretical foundations for its application to healthcare quality improvement. Cad Saude Publica. 2016;32(sup 2):e00105815. Portela MC, Lima SML, Martins M, Travassos C. Improvement Science: conceptual and theoretical foundations for its application to healthcare quality improvement. Cad Saude Publica. 2016;32(sup 2):e00105815.
6.
go back to reference Langley GJ, Moen R, Nolan KM, Nolan TW, Norman CL, Provost LP. The improvement guide, 2nd edition. Jossey-Bass. 2009. Langley GJ, Moen R, Nolan KM, Nolan TW, Norman CL, Provost LP. The improvement guide, 2nd edition. Jossey-Bass. 2009.
7.
go back to reference Berwick DM. The science of improvement. JAMA - J Am Med Assoc. 2008;299(10):1182–4.CrossRef Berwick DM. The science of improvement. JAMA - J Am Med Assoc. 2008;299(10):1182–4.CrossRef
8.
go back to reference Berwick DM, Nolan TW. Developing and testing changes in delivery of care. Ann Intern Med. 1998;128(1):651–6.CrossRef Berwick DM, Nolan TW. Developing and testing changes in delivery of care. Ann Intern Med. 1998;128(1):651–6.CrossRef
9.
go back to reference Perla RJ, Provost LP, Parry GJ. Seven propositions of the science of improvement: exploring foundations. Qual Manag Health Care. 2013;22(3):170–86.CrossRef Perla RJ, Provost LP, Parry GJ. Seven propositions of the science of improvement: exploring foundations. Qual Manag Health Care. 2013;22(3):170–86.CrossRef
10.
go back to reference Taylor MJ, McNicholas C, Nicolay C, Darzi A, Bell D, Reed JE. Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Qual Saf 2013;0:1–9. Taylor MJ, McNicholas C, Nicolay C, Darzi A, Bell D, Reed JE. Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Qual Saf 2013;0:1–9.
11.
go back to reference Ogrinc G. Building knowledge, asking questions. BMJ Qual Saf. 2014;23(4):265–7.CrossRef Ogrinc G. Building knowledge, asking questions. BMJ Qual Saf. 2014;23(4):265–7.CrossRef
12.
go back to reference Portela MC, Pronovost PJ, Woodcock T, Carter P, Dixon-Woods M. How to study improvement interventions: a brief overview of possible study types. Postgrad Med J. 2015;91(1076):343–54.CrossRef Portela MC, Pronovost PJ, Woodcock T, Carter P, Dixon-Woods M. How to study improvement interventions: a brief overview of possible study types. Postgrad Med J. 2015;91(1076):343–54.CrossRef
13.
go back to reference Thor J, Lundberg J, Ask J, Olsson J, Carli C, Härenstam KP, et al. Application of statistical process control in healthcare improvement: systematic review. Qual Saf Heal Care. 2007;16(5):387–99.CrossRef Thor J, Lundberg J, Ask J, Olsson J, Carli C, Härenstam KP, et al. Application of statistical process control in healthcare improvement: systematic review. Qual Saf Heal Care. 2007;16(5):387–99.CrossRef
14.
go back to reference Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50.CrossRef Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50.CrossRef
15.
go back to reference Etchells E, Ho M, Shojania KG. Value of small sample sizes in rapid-cycle quality improvement projects. BMJ Qual Saf. 2016;25(3):202–6.CrossRef Etchells E, Ho M, Shojania KG. Value of small sample sizes in rapid-cycle quality improvement projects. BMJ Qual Saf. 2016;25(3):202–6.CrossRef
16.
go back to reference Berwick DM. A primer on leading the improvement of systems. BMJ Br Med J. 1996;312(7031):619.CrossRef Berwick DM. A primer on leading the improvement of systems. BMJ Br Med J. 1996;312(7031):619.CrossRef
17.
go back to reference Speroff T, James BC, Nelson EC, Headrick LA, Brommels M. Guidelines for appraisal and publication of PDSA quality improvement. Qual Manag Health Care. 2004;13(1):33–9.CrossRef Speroff T, James BC, Nelson EC, Headrick LA, Brommels M. Guidelines for appraisal and publication of PDSA quality improvement. Qual Manag Health Care. 2004;13(1):33–9.CrossRef
18.
go back to reference Ogrinc G, Davies L, Goodman D, Batalden P, Davidoff F, Stevens D. Standards for QUality Improvement Reporting Excellence 2.0: revised publication guidelines from a detailed consensus process. BMJ Qual Saf. 2016;25:986–92.CrossRef Ogrinc G, Davies L, Goodman D, Batalden P, Davidoff F, Stevens D. Standards for QUality Improvement Reporting Excellence 2.0: revised publication guidelines from a detailed consensus process. BMJ Qual Saf. 2016;25:986–92.CrossRef
19.
go back to reference Moonesinghe SR, Peden CJ. Theory and context: putting the science into improvement. Br J Anaesth. 2017;118(4):482–4.CrossRef Moonesinghe SR, Peden CJ. Theory and context: putting the science into improvement. Br J Anaesth. 2017;118(4):482–4.CrossRef
20.
go back to reference Davidoff F, Dixon-Woods M, Leviton L, Michie S. Demystifying theory and its use in improvement. BMJ Qual Saf. 2015;24(3):228–38.CrossRef Davidoff F, Dixon-Woods M, Leviton L, Michie S. Demystifying theory and its use in improvement. BMJ Qual Saf. 2015;24(3):228–38.CrossRef
21.
go back to reference Foy R, Ovretveit J, Shekelle PG, Pronovost PJ, Taylor SL, Dy S, et al. The role of theory in research to develop and evaluate the implementation of patient safety practices. BMJ Qual Saf. 2011;20(5):453–9.CrossRef Foy R, Ovretveit J, Shekelle PG, Pronovost PJ, Taylor SL, Dy S, et al. The role of theory in research to develop and evaluate the implementation of patient safety practices. BMJ Qual Saf. 2011;20(5):453–9.CrossRef
22.
go back to reference Walshe K. Pseudoinnovation: the development and spread of healthcare quality improvement methodologies. Int J Qual Heal Care. 2009;21(3):153–9.CrossRef Walshe K. Pseudoinnovation: the development and spread of healthcare quality improvement methodologies. Int J Qual Heal Care. 2009;21(3):153–9.CrossRef
23.
go back to reference Walshe K. Understanding what works-and why-in quality improvement: the need for theory-driven evaluation. Int J Qual Heal Care. 2007;19(2):57–9.CrossRef Walshe K. Understanding what works-and why-in quality improvement: the need for theory-driven evaluation. Int J Qual Heal Care. 2007;19(2):57–9.CrossRef
24.
go back to reference Powell AE, Rushmer RK, Davies HT. A systematic narrative review of quality improvement models in health care. Glasgow: Quality Improvement Scotland (NHS QIS ). 2009. Powell AE, Rushmer RK, Davies HT. A systematic narrative review of quality improvement models in health care. Glasgow: Quality Improvement Scotland (NHS QIS ). 2009.
25.
go back to reference Groene O. Does quality improvement face a legitimacy crisis? Poor quality studies, small effects. J Heal Serv Res Policy. 2011;16(3):131–2.CrossRef Groene O. Does quality improvement face a legitimacy crisis? Poor quality studies, small effects. J Heal Serv Res Policy. 2011;16(3):131–2.CrossRef
26.
go back to reference Dixon-Woods M, Martin G. Does quality improvement improve quality? Futur Hosp J. 2016;3(3):191–4.CrossRef Dixon-Woods M, Martin G. Does quality improvement improve quality? Futur Hosp J. 2016;3(3):191–4.CrossRef
27.
go back to reference Blumenthal D, Kilo CM. A report Card on continuous quality improvement. Milbank Q. 1998;76(4):625–48.CrossRef Blumenthal D, Kilo CM. A report Card on continuous quality improvement. Milbank Q. 1998;76(4):625–48.CrossRef
28.
go back to reference Dellifraine JL, Langabeer JR, Nembhard IM. Assessing the evidence of six sigma and lean in the health care industry. Qual Manag Health Care. 2010;19(3):211–25.CrossRef Dellifraine JL, Langabeer JR, Nembhard IM. Assessing the evidence of six sigma and lean in the health care industry. Qual Manag Health Care. 2010;19(3):211–25.CrossRef
29.
go back to reference D’Andreamatteo A, Ianni L, Lega F, Sargiacomo M. Lean in healthcare: a comprehensive review. Health Policy (New York). 2015;119(9):1197–209.CrossRef D’Andreamatteo A, Ianni L, Lega F, Sargiacomo M. Lean in healthcare: a comprehensive review. Health Policy (New York). 2015;119(9):1197–209.CrossRef
30.
go back to reference Moraros J, Lemstra M, Nwankwo C. Lean interventions in healthcare: do they actually work? A systematic literature review. Int J Qual Heal Care. 2016:150–65. Moraros J, Lemstra M, Nwankwo C. Lean interventions in healthcare: do they actually work? A systematic literature review. Int J Qual Heal Care. 2016:150–65.
31.
go back to reference Shojania KG, Grimshaw JM. Evidence-based quality improvement: the state of the science. Health Aff. 2005;24(1):138–50.CrossRef Shojania KG, Grimshaw JM. Evidence-based quality improvement: the state of the science. Health Aff. 2005;24(1):138–50.CrossRef
32.
go back to reference Marshall M, Pronovost P, Dixon-Woods M. Promotion of improvement as a science. Lancet. 2013;381(9881):419–21.CrossRef Marshall M, Pronovost P, Dixon-Woods M. Promotion of improvement as a science. Lancet. 2013;381(9881):419–21.CrossRef
34.
go back to reference Davidoff F, Batalden P, Stevens D, Ogrinc G, Mooney S. Publication guidelines for quality improvement in health care: evolution of the SQUIRE project. Qual Saf Heal Care. 2008;17(SUPPL. 1):i3–9.CrossRef Davidoff F, Batalden P, Stevens D, Ogrinc G, Mooney S. Publication guidelines for quality improvement in health care: evolution of the SQUIRE project. Qual Saf Heal Care. 2008;17(SUPPL. 1):i3–9.CrossRef
35.
go back to reference Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. BMJ Br Med J. 2009;339:b2700.CrossRef Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. BMJ Br Med J. 2009;339:b2700.CrossRef
36.
go back to reference Auerbach AD, Landefeld CS, Shojania KG. The tension between needing to improve care and knowing how to do it. N Engl J Med. 2007;357(6):608–13.CrossRef Auerbach AD, Landefeld CS, Shojania KG. The tension between needing to improve care and knowing how to do it. N Engl J Med. 2007;357(6):608–13.CrossRef
38.
go back to reference Dixon-Woods M, Bosk CL, Aveling EL, Goeschel CA, Pronovost PJ. Explaining Michigan: developing an ex post theory of a quality improvement program. Milbank Q. 2011;89(2):167–205.CrossRef Dixon-Woods M, Bosk CL, Aveling EL, Goeschel CA, Pronovost PJ. Explaining Michigan: developing an ex post theory of a quality improvement program. Milbank Q. 2011;89(2):167–205.CrossRef
39.
go back to reference Oxman AD, Thomson MA, Davis DA, Haynes B. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. Cmaj. 1995;153(10):1423–31.PubMedPubMedCentral Oxman AD, Thomson MA, Davis DA, Haynes B. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. Cmaj. 1995;153(10):1423–31.PubMedPubMedCentral
Metadata
Title
Can quality improvement improve the quality of care? A systematic review of reported effects and methodological rigor in plan-do-study-act projects
Authors
Søren Valgreen Knudsen
Henrik Vitus Bering Laursen
Søren Paaske Johnsen
Paul Daniel Bartels
Lars Holger Ehlers
Jan Mainz
Publication date
01-12-2019
Publisher
BioMed Central
Published in
BMC Health Services Research / Issue 1/2019
Electronic ISSN: 1472-6963
DOI
https://doi.org/10.1186/s12913-019-4482-6

Other articles of this Issue 1/2019

BMC Health Services Research 1/2019 Go to the issue