Monitoring and evaluation of disaster response efforts undertaken by local health departments: a rapid realist review

Local health departments are often at the forefront of a disaster response, attending to the immediate trauma inflicted by the disaster and also the long term health consequences. As the frequency and severity of disasters are projected to rise, monitoring and evaluation (M&E) efforts are critical to help local health departments consolidate past experiences and improve future response efforts. Local health departments often conduct M&E work post disaster, however, many of these efforts fail to improve response procedures.

We undertook a rapid realist review (RRR) to examine why M&E efforts undertaken by local health departments do not always result in improved disaster response efforts. We aimed to complement existing frameworks by focusing on the most basic and pragmatic steps of a M&E cycle targeted towards continuous system improvements. For these purposes, we developed a theoretical framework that draws on the quality improvement literature to ‘frame’ the steps in the M&E cycle. This framework encompassed a M&E cycle involving three stages (i.e., document and assess, disseminate and implement) that must be sequentially completed to learn from past experiences and improve future disaster response efforts. We used this framework to guide our examination of the literature and to identify any context-mechanism-outcome (CMO) configurations which describe how M&E may be constrained or enabled at each stage of the M&E cycle.

This RRR found a number of explanatory CMO configurations that provide valuable insights into some of the considerations that should be made when using M&E to improve future disaster response efforts. Firstly, to support the accurate documentation and assessment of a disaster response, local health departments should consider how they can: establish a culture of learning within health departments; use embedded training methods; or facilitate external partnerships. Secondly, to enhance the widespread dissemination of lessons learned and facilitate inter-agency learning, evaluation reports should use standardised formats and terminology. Lastly, to increase commitment to improvement processes, local health department leaders should possess positive leadership attributes and encourage shared decision making.

This study is among the first to conduct a synthesis of the CMO configurations which facilitate or hinder M&E efforts aimed at improving future disaster responses. It makes a significant contribution to the disaster literature and provides an evidence base that can be used to provide pragmatic guidance for improving M&E efforts of local health departments.

PROSPERO 2015:CRD42015023526.