Skip to main content
Top
Published in: Globalization and Health 1/2019

Open Access 01-12-2019 | Public Health | Review

Getting the most from after action reviews to improve global health security

Authors: Michael A. Stoto, Christopher Nelson, Rachael Piltch-Loeb, Landry Ndriko Mayigane, Frederik Copper, Stella Chungong

Published in: Globalization and Health | Issue 1/2019

Login to get access

Abstract

Background

After Action Reviews (AARs) provide a means to observe how well preparedness systems perform in real world conditions and can help to identify – and address – gaps in national and global public health emergency preparedness (PHEP) systems. WHO has recently published guidance for voluntary AARs. This analysis builds on this guidance by reviewing evidence on the effectiveness of AARs as tools for system improvement and by summarizing some key lessons about ensuring that AARs result in meaningful learning from experience.

Results

Empirical evidence from a variety of fields suggests that AARs hold considerable promise as tools of system improvement for PHEP. Our review of the literature and practical experience demonstrates that AARs are most likely to result in meaningful learning if they focus on incidents that are selected for their learning value, involve an appropriately broad range of perspectives, are conducted with appropriate time for reflection, employ systems frameworks and rigorous tools such as facilitated lookbacks and root cause analysis, and strike a balance between attention to incident specifics vs. generalizable capacities and capabilities.

Conclusions

Employing these practices requires a PHEP system that facilitates the preparation of insightful AARs, and more generally rewards learning. The barriers to AARs fall into two categories: concerns about the cultural sensitivity and context, liability, the political response, and national security; and constraints on staff time and the lack of experience and the requisite analytical skills. Ensuring that AARs fulfill their promise as tools of system improvement will require ongoing investment and a change in mindset. The first step should be to clarify that the goal of AARs is organizational learning, not placing blame or punishing poor performance. Based on experience in other fields, the buy-in of agency and political leadership is critical in this regard. National public health systems also need support in the form of toolkits, guides, and training, as well as research on AAR methods. An AAR registry could support organizational improvement through careful post-event analysis of systems’ own events, facilitate identification and sharing of best practices across jurisdictions, and enable cross-case analyses.
Literature
1.
go back to reference Lederberg J, Hamburg MA, Smolinski MS. Microbial threats to health: emergence, detection, and response: National Academies Press; 2003. Lederberg J, Hamburg MA, Smolinski MS. Microbial threats to health: emergence, detection, and response: National Academies Press; 2003.
2.
go back to reference World Health Organization. International health regulations (2005): World Health Organization; 2008. World Health Organization. International health regulations (2005): World Health Organization; 2008.
3.
go back to reference World Health OrganizationH. Monitoring and evaluation framework joint external evaluation tool (JEE tool)- second. Edition Geneva, Switzerland; 2018. World Health OrganizationH. Monitoring and evaluation framework joint external evaluation tool (JEE tool)- second. Edition Geneva, Switzerland; 2018.
4.
go back to reference Piltch-Loeb R, Kraemer JD, Nelson C, Stoto MA. A public health emergency preparedness critical incident registry. Biosecur Bioterror. 2014;12(3):132–43.CrossRef Piltch-Loeb R, Kraemer JD, Nelson C, Stoto MA. A public health emergency preparedness critical incident registry. Biosecur Bioterror. 2014;12(3):132–43.CrossRef
5.
go back to reference World Health Organization. Country implementation guidance: after action reviews and simulation exercises under the international health regulations 2005 monitoring and evaluation framework (IHR MEF). World Health Organization; 2018. World Health Organization. Country implementation guidance: after action reviews and simulation exercises under the international health regulations 2005 monitoring and evaluation framework (IHR MEF). World Health Organization; 2018.
6.
go back to reference Garvin DA. Learning in action: a guide to putting the learning organization to work: Harvard Business Review Press; 2003. Garvin DA. Learning in action: a guide to putting the learning organization to work: Harvard Business Review Press; 2003.
7.
go back to reference Salter MS, Klein GE. After action reviews: current observations and recommendations. Vienna Va: Wexford Group International Inc; 2007.CrossRef Salter MS, Klein GE. After action reviews: current observations and recommendations. Vienna Va: Wexford Group International Inc; 2007.CrossRef
8.
go back to reference United States Army Office of Regulations. Army Lessons Learned Program (ALLP). 2006. United States Army Office of Regulations. Army Lessons Learned Program (ALLP). 2006.
9.
go back to reference United States Marine Corps. MCO 3504.1 Marine Corps Lessons Learned Program (MCCLP). 2006. United States Marine Corps. MCO 3504.1 Marine Corps Lessons Learned Program (MCCLP). 2006.
10.
go back to reference Rawal V, Fautin C, Moore J-L, Kalonge S, Walden VM, Bhattacharjee A. Multi-agency evaluation of tsunami response: India and Sri Lanka. CARE international, world vision (WV), Oxfam GB, Catholic relief services (CRS); 2005. Rawal V, Fautin C, Moore J-L, Kalonge S, Walden VM, Bhattacharjee A. Multi-agency evaluation of tsunami response: India and Sri Lanka. CARE international, world vision (WV), Oxfam GB, Catholic relief services (CRS); 2005.
11.
go back to reference Sexton RM, Isobel A. Comparative study of after action review (AAR) in the context of the southern Africa. Crisis. 2003. Sexton RM, Isobel A. Comparative study of after action review (AAR) in the context of the southern Africa. Crisis. 2003.
12.
go back to reference United States Department of Homeland Security. Homeland security exercise and evaluation program: the department; 2007. United States Department of Homeland Security. Homeland security exercise and evaluation program: the department; 2007.
13.
go back to reference Neto M, Pimentel J, Morais A, Santos C, Ferreira AJ, Ferreira PL. Preparação das famílias para fazer face a emergências e catástrofes: avaliação após o incêndio de 2017 ocorrido nos concelhos de Pedrógão Grande. Figueiró dos Vinhos e Castanheira de Pera; 2018. Neto M, Pimentel J, Morais A, Santos C, Ferreira AJ, Ferreira PL. Preparação das famílias para fazer face a emergências e catástrofes: avaliação após o incêndio de 2017 ocorrido nos concelhos de Pedrógão Grande. Figueiró dos Vinhos e Castanheira de Pera; 2018.
14.
go back to reference European Union Conference. Lessons learned for public health from the Ebola outbreak in West Africa – how to improve preparedness and response in the EU for future outbreaks. Luxembourg; October 2015. p. 12–4. European Union Conference. Lessons learned for public health from the Ebola outbreak in West Africa – how to improve preparedness and response in the EU for future outbreaks. Luxembourg; October 2015. p. 12–4.
15.
go back to reference Fineberg H, Aavitsland P, Aditama T, Bino S, Carmo E. Implementation of the international health regulations (2005): Report of the review committee on the functioning of the international health regulations (2005) and on pandemic influenza a (H1N1) 2009. Geneva: World Health Organization. Geneva: World Health Organization. p. 2011. Fineberg H, Aavitsland P, Aditama T, Bino S, Carmo E. Implementation of the international health regulations (2005): Report of the review committee on the functioning of the international health regulations (2005) and on pandemic influenza a (H1N1) 2009. Geneva: World Health Organization. Geneva: World Health Organization. p. 2011.
16.
go back to reference Hine D. The 2009 influenza pandemic: an independent review of the UK response to the2009 influenza pandemic; 2010. Hine D. The 2009 influenza pandemic: an independent review of the UK response to the2009 influenza pandemic; 2010.
17.
go back to reference World Health Organization. Revue Après Action de la Réponse D’urgence á la Flambée Épidémique de Peste Pulmonaire. Organisation mondiale de la Santé; 2018 2 AU 6 Juillet 2018. World Health Organization. Revue Après Action de la Réponse D’urgence á la Flambée Épidémique de Peste Pulmonaire. Organisation mondiale de la Santé; 2018 2 AU 6 Juillet 2018.
18.
go back to reference World Health Organization. WHO guidance for surveillance during an influenza pandemic. 2017. World Health Organization. WHO guidance for surveillance during an influenza pandemic. 2017.
19.
go back to reference Savoia E, Agboola F, Biddinger PD. Use of after action reports (AARs) to promote organizational and systems learning in emergency preparedness. Int J Environ Res Public Health. 2012;9(8):2949–63.CrossRef Savoia E, Agboola F, Biddinger PD. Use of after action reports (AARs) to promote organizational and systems learning in emergency preparedness. Int J Environ Res Public Health. 2012;9(8):2949–63.CrossRef
20.
go back to reference Stoto MA, Nelson C, Higdon MA, Kraemer J, Singleton C-M. Learning about after action reporting from the 2009 H1N1 pandemic: a workshop summary. J Public Health Manag Pract. 2013;19(5):420–7.CrossRef Stoto MA, Nelson C, Higdon MA, Kraemer J, Singleton C-M. Learning about after action reporting from the 2009 H1N1 pandemic: a workshop summary. J Public Health Manag Pract. 2013;19(5):420–7.CrossRef
21.
go back to reference Davies R, Vaughan E, Fraser G, Cook R, Ciotti M, Suk JE. Enhancing reporting of after action reviews of public health emergencies to strengthen preparedness: a literature review and methodology appraisal. Disaster Med Public Health Prep. 2018:1–8. Davies R, Vaughan E, Fraser G, Cook R, Ciotti M, Suk JE. Enhancing reporting of after action reviews of public health emergencies to strengthen preparedness: a literature review and methodology appraisal. Disaster Med Public Health Prep. 2018:1–8.
22.
go back to reference World Health Organization. Guidance for after action review (AAR): World Health Organization; 2019. World Health Organization. Guidance for after action review (AAR): World Health Organization; 2019.
23.
go back to reference Scott C, Dunn AM, Williams EB, Allen JA. Implementing after-action review systems in organizations: key principles and practical considerations; 2015. Scott C, Dunn AM, Williams EB, Allen JA. Implementing after-action review systems in organizations: key principles and practical considerations; 2015.
24.
go back to reference Tannenbaum SI, Cerasoli CP. Do team and individual debriefs enhance performance? A meta-analysis. Hum Factors. 2013;55(1):231–45.CrossRef Tannenbaum SI, Cerasoli CP. Do team and individual debriefs enhance performance? A meta-analysis. Hum Factors. 2013;55(1):231–45.CrossRef
25.
go back to reference Allen JA, Baran BE, Scott CW. After-action reviews: a venue for the promotion of safety climate. Accid Anal Prev. 2010;42(2):750–7.CrossRef Allen JA, Baran BE, Scott CW. After-action reviews: a venue for the promotion of safety climate. Accid Anal Prev. 2010;42(2):750–7.CrossRef
26.
go back to reference Ellis S, Davidi I. After-event reviews: drawing lessons from successful and failed experience. J Appl Psychol. 2005;90(5):857.CrossRef Ellis S, Davidi I. After-event reviews: drawing lessons from successful and failed experience. J Appl Psychol. 2005;90(5):857.CrossRef
27.
go back to reference Wu AW, Lipshutz AK, Pronovost PJ. Effectiveness and efficiency of root cause analysis in medicine. Jama. 2008;299(6):685–7.CrossRef Wu AW, Lipshutz AK, Pronovost PJ. Effectiveness and efficiency of root cause analysis in medicine. Jama. 2008;299(6):685–7.CrossRef
28.
go back to reference Zhang Y, Lopez-Gatell H, Alpuche-Aranda CM, Stoto MA. Did advances in global surveillance and notification systems make a difference in the 2009 H1N1 pandemic?–a retrospective analysis. PLoS One. 2013;8(4):e59893.CrossRef Zhang Y, Lopez-Gatell H, Alpuche-Aranda CM, Stoto MA. Did advances in global surveillance and notification systems make a difference in the 2009 H1N1 pandemic?–a retrospective analysis. PLoS One. 2013;8(4):e59893.CrossRef
29.
go back to reference World Health Organization. Swine Influenza Statement [press release] Geneva, Switzerland. 2009. World Health Organization. Swine Influenza Statement [press release] Geneva, Switzerland. 2009.
30.
go back to reference United States Centers for Disease Control and Prevention. Public health preparedness capabilities: national standards for state and local planning, March 2011: Centers for Disease Control and Prevention, Office of Public Health Preparedness and Response 2011. United States Centers for Disease Control and Prevention. Public health preparedness capabilities: national standards for state and local planning, March 2011: Centers for Disease Control and Prevention, Office of Public Health Preparedness and Response 2011.
31.
go back to reference United States Centers for Disease Control and Prevention. 2017–2022 Health care preparedness and response capabilities. 2016. United States Centers for Disease Control and Prevention. 2017–2022 Health care preparedness and response capabilities. 2016.
32.
go back to reference Stoto MA, Nelson C, Savoia E, Ljungqvist I, Ciotti M. A public health preparedness logic model: assessing preparedness for cross-border threats in the European region. Health security. 2017;15(5):473–82.CrossRef Stoto MA, Nelson C, Savoia E, Ljungqvist I, Ciotti M. A public health preparedness logic model: assessing preparedness for cross-border threats in the European region. Health security. 2017;15(5):473–82.CrossRef
33.
go back to reference World Health Organization. Actions taken to reduce plague burden in Madagascar and recent Cases Show Decline [press release] Geneva, Switzerland, 5 June 2019 2019. World Health Organization. Actions taken to reduce plague burden in Madagascar and recent Cases Show Decline [press release] Geneva, Switzerland, 5 June 2019 2019.
34.
go back to reference Morrison JE, Meliza LL. Foundations of the after action review process: Institute For Defense Analyses Alexandria Va; 1999. Morrison JE, Meliza LL. Foundations of the after action review process: Institute For Defense Analyses Alexandria Va; 1999.
35.
go back to reference Lipshitz R, Popper M, Friedman VJ. A multifacet model of organizational learning. J Appl Behav Sci. 2002;38(1):78–98.CrossRef Lipshitz R, Popper M, Friedman VJ. A multifacet model of organizational learning. J Appl Behav Sci. 2002;38(1):78–98.CrossRef
36.
go back to reference Piltch-Loeb RN, Kraemer JD, Nelson C, Savoia E, Osborn DR, Stoto MA. Root Cause Analysis for enhancing public health emergency preparedness: A case study of Salmonella in the Alamosa, Colorado water supply. Journal of Public Health Management and Practice. 2018;24(6):542–45. CrossRef Piltch-Loeb RN, Kraemer JD, Nelson C, Savoia E, Osborn DR, Stoto MA. Root Cause Analysis for enhancing public health emergency preparedness: A case study of Salmonella in the Alamosa, Colorado water supply. Journal of Public Health Management and Practice. 2018;24(6):542–45. CrossRef
37.
go back to reference Brown JS, Duguid P. Organizational learning and communities-of-practice: toward a unified view of working, learning, and innovation. Organ Sci. 1991;2(1):40–57.CrossRef Brown JS, Duguid P. Organizational learning and communities-of-practice: toward a unified view of working, learning, and innovation. Organ Sci. 1991;2(1):40–57.CrossRef
38.
go back to reference Ron N, Lipshitz R, Popper M. How organizations learn: post-flight reviews in an F-16 fighter squadron. Organ Stud. 2006;27(8):1069–89.CrossRef Ron N, Lipshitz R, Popper M. How organizations learn: post-flight reviews in an F-16 fighter squadron. Organ Stud. 2006;27(8):1069–89.CrossRef
39.
go back to reference Carroll JS. Organizational learning activities in high-hazard industries: the logics underlying self-analysis. J Manag Stud. 1998;35(6):699–717.CrossRef Carroll JS. Organizational learning activities in high-hazard industries: the logics underlying self-analysis. J Manag Stud. 1998;35(6):699–717.CrossRef
40.
go back to reference Aledort JE, Lurie N, Ricci K, Dausey DJ, Stern S. Facilitated look backs: a new quality improvement tool for management of routine annual and pandemic influenza. RAND Technical report: RAND Corporation; 2006. Aledort JE, Lurie N, Ricci K, Dausey DJ, Stern S. Facilitated look backs: a new quality improvement tool for management of routine annual and pandemic influenza. RAND Technical report: RAND Corporation; 2006.
41.
go back to reference Morris MW, Moore PC. The lessons we (don't) learn: counterfactual thinking and organizational accountability after a close call. Adm Sci Q. 2000;45(4):737–65.CrossRef Morris MW, Moore PC. The lessons we (don't) learn: counterfactual thinking and organizational accountability after a close call. Adm Sci Q. 2000;45(4):737–65.CrossRef
42.
go back to reference Upadhyay DK, Sittig DF, Singh H. Ebola US patient zero: lessons on misdiagnosis and effective use of electronic health records. Diagnosis. 2014;1(4):283–7.CrossRef Upadhyay DK, Sittig DF, Singh H. Ebola US patient zero: lessons on misdiagnosis and effective use of electronic health records. Diagnosis. 2014;1(4):283–7.CrossRef
43.
go back to reference Foote M, Daver R, Quinn C. Using “mystery patient” drills to assess hospital ebola preparedness in new York City, 2014-2015. Health security. 2017;15(5):500–8.CrossRef Foote M, Daver R, Quinn C. Using “mystery patient” drills to assess hospital ebola preparedness in new York City, 2014-2015. Health security. 2017;15(5):500–8.CrossRef
44.
go back to reference Croteau RJ. Root cause analysis in health care: tools and techniques. Joint Commission Resources. 2010. Croteau RJ. Root cause analysis in health care: tools and techniques. Joint Commission Resources. 2010.
45.
go back to reference Gilson L, Hanson K, Sheikh K, Agyepong IA, Ssengooba F, Bennett S. Building the field of health policy and systems research: social science matters. PLoS Med. 2011;8(8):e1001079.CrossRef Gilson L, Hanson K, Sheikh K, Agyepong IA, Ssengooba F, Bennett S. Building the field of health policy and systems research: social science matters. PLoS Med. 2011;8(8):e1001079.CrossRef
46.
go back to reference Stoto MA, Nelson CD, Klaiman T. Getting from what to why: using qualitative methods in public health systems research. Academy Health Issue Brief. 2013. Stoto MA, Nelson CD, Klaiman T. Getting from what to why: using qualitative methods in public health systems research. Academy Health Issue Brief. 2013.
47.
go back to reference Stoto MA, Higdon MA. The public health response to 2009 H1N1: a systems perspective: Oxford University Press; 2015. Stoto MA, Higdon MA. The public health response to 2009 H1N1: a systems perspective: Oxford University Press; 2015.
48.
go back to reference Stufflebeam DL, Shinkfield AJ. Stufflebeam’s improvement-oriented evaluation. Systematic evaluation: springer; 1985. p. 151–207.CrossRef Stufflebeam DL, Shinkfield AJ. Stufflebeam’s improvement-oriented evaluation. Systematic evaluation: springer; 1985. p. 151–207.CrossRef
49.
go back to reference Biddinger PD, Baggish A, Harrington L, d'Hemecourt P, Hooley J, Jones J, et al. Be prepared—the Boston Marathon and mass-casualty events. N Engl J Med. 2013;368(21):1958–60.CrossRef Biddinger PD, Baggish A, Harrington L, d'Hemecourt P, Hooley J, Jones J, et al. Be prepared—the Boston Marathon and mass-casualty events. N Engl J Med. 2013;368(21):1958–60.CrossRef
50.
go back to reference Billings C, Lauber J, Funkhouser H, Lyman E, Huff E. NASA aviation safety reporting system; 1976. Billings C, Lauber J, Funkhouser H, Lyman E, Huff E. NASA aviation safety reporting system; 1976.
51.
go back to reference Kandel N, Sreedharan R, Chungong S, Sliter K, Nikkari S, Ijaz K, et al. Joint external evaluation process: bringing multiple sectors together for global health security. Lancet Glob Health. 2017;5(9):e857–e8.CrossRef Kandel N, Sreedharan R, Chungong S, Sliter K, Nikkari S, Ijaz K, et al. Joint external evaluation process: bringing multiple sectors together for global health security. Lancet Glob Health. 2017;5(9):e857–e8.CrossRef
52.
go back to reference European Centers for Disease Control and Prevention. Ebola emergency preparedness in EU member states – conclusions from peer-review visits to Belgium, Portugal and Romania. Stockholm; 2015. European Centers for Disease Control and Prevention. Ebola emergency preparedness in EU member states – conclusions from peer-review visits to Belgium, Portugal and Romania. Stockholm; 2015.
Metadata
Title
Getting the most from after action reviews to improve global health security
Authors
Michael A. Stoto
Christopher Nelson
Rachael Piltch-Loeb
Landry Ndriko Mayigane
Frederik Copper
Stella Chungong
Publication date
01-12-2019
Publisher
BioMed Central
Keyword
Public Health
Published in
Globalization and Health / Issue 1/2019
Electronic ISSN: 1744-8603
DOI
https://doi.org/10.1186/s12992-019-0500-z

Other articles of this Issue 1/2019

Globalization and Health 1/2019 Go to the issue