Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
Implementation Science
Published in: Implementation Science 1/2017

Open Access 01-12-2017 | Research

Understanding causal pathways within health systems policy evaluation through mediation analysis: an application to payment for performance (P4P) in Tanzania

Authors: Laura Anselmi, Peter Binyaruka, Josephine Borghi

Published in: Implementation Science | Issue 1/2017

Login to get access

Abstract

Background

The evaluation of payment for performance (P4P) programmes has focused mainly on understanding contributions to health service coverage, without unpacking causal mechanisms. The overall aim of the paper is to test the causal pathways through which P4P schemes may (or may not) influence maternal care outcomes.

Methods

We used data from an evaluation of a P4P programme in Tanzania. Data were collected from a sample of 3000 women who delivered in the 12 months prior to interview and 200 health workers at 150 health facilities from seven intervention and four comparison districts in Tanzania in January 2012 and in February 2013. We applied causal mediation analysis using a linear structural equation model to identify direct and indirect effects of P4P on institutional delivery rates and on the uptake of two doses of an antimalarial drug during pregnancy. We first ran a series of linear difference-in-difference regression models to test the effect of P4P on potential mediators, which we then included in a linear difference-in-difference model evaluating the impact of P4P on the outcome. We tested the robustness of our results to unmeasured confounding using semi-parametric methods.

Results

P4P reduced the probability of women paying for delivery care (−4.5 percentage points) which mediates the total effect of P4P on institutional deliveries (by 48%) and on deliveries in a public health facility (by 78%). P4P reduced the stock-out rate for some essential drugs, specifically oxytocin (−36 percentage points), which mediated the total effect of P4P on institutional deliveries (by 22%) and deliveries in a public health facility (by 30%). P4P increased kindness at delivery (5 percentage points), which mediated the effect of P4P on institutional deliveries (by 48%) and on deliveries in a public health facility (by 49%). P4P increased the likelihood of supervision visits taking place within the last 90 days (18 percentage points), which mediated 15% of the total P4P effect on the uptake of two antimalarial doses during antenatal care (IPT2). Kindness during deliveries and the probability of paying out of pocket for delivery care were the mediators most robust to unmeasured confounding.

Conclusions

The effect of P4P on institutional deliveries is mediated by financing and human resources factors, while uptake of antimalarials in pregnancy is mediated by governance factors. Further research is required to explore additional and more complex causal pathways.
Appendix
Available only for authorised users
Literature
1.
Ludwig J, Kling JR, Mullainathan S. Mechanism experiments and policy evaluations. J Econ Perspect. 2011;25(3):17–38.CrossRef
2.
Craig P et al. Developing and evaluating complex interventions: New guidance. BMJ. 2008;337:a1655.
3.
Bonell C, et al. Realist randomised controlled trials: a new approach to evaluating complex public health interventions. Soc Sci Med. 2012;75(12):2299–306.CrossRefPubMed
4.
Oakley A, et al. Process evaluation in randomised controlled trials of complex interventions. Br Med J. 2006;18(332):413–6.CrossRef
5.
Moore GF, et al. Process evaluation of complex interventions: Medical Research Council guidance. Br Med J. 2015;350:h1258.CrossRef
6.
MacKinnon DP, Fairchild AJ, Fritz MS. Mediation analysis. Annu Rev Psychol. 2007;58(593):59.
7.
Grant A, et al. Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials. 2013;12(14):15.CrossRef
8.
Imai K, et al. Unpacking the black box of causality: learning about causal mechanisms from experimental and observational studies. Am Polit Sci Rev. 2011;105(4):765–89.CrossRef
9.
Gunzler D, et al. Introduction to mediation analysis with structural equation modeling. Shanghai Arch Psychiatry. 2013;25(6):390–4.PubMedPubMedCentral
10.
Heckman J, Pinto R, Savelyev PA. Understanding the mechanisms through which an influential early childhood program boosted adult outcomes. Am Econ Rev. 2013;103:2052–86.CrossRefPubMedPubMedCentral
11.
Heckman J, Pinto R. Econometric mediation analyses: identifying the sources of treatment effects from experimentally estimated production technologies with unmeasured and mismeasured inputs. Econ Rev. 2015;34(1-2):6–31.CrossRef
12.
13.
DeStavola BL, et al. Mediation analysis with intermediate confounding: structural equation modeling viewed through the causal inference lens. Am J Epidemiol. 2015;181(1):64–80.CrossRef
14.
Abramsky T et al. Ecological pathways to prevention: how does the SASA! community mobilisation model work to prevent physical intimate partner violence against women? BMC Public Health. 2016;16(339):1–21.
15.
Ngo D, S Tisamarie, and S Bauhoff,. Health system changes under pay-for-performance: the effects of Rwanda’s national programme on facility inputs. Health Policy and Plan. 2016. doi:10.​1093/​heapol/​czw091. first published online July 19, 2016.
16.
Scott A et al. The effect of financial incentives on the quality of health care provided by primary care physicians. Cochrane Database Syst Rev. 2011;9.
17.
Witter S, et al. Paying for performance to improve the delivery of health interventions in low- and middle-income countries. Cochrane Database Syst Rev. 2012;2:CD007899.
18.
Ogundeji Y, Bland J, Sheldon T. The effectiveness of payment for performance in health care: a meta-analysis and exploration of variation in outcomes. Health Policy. 2016;120(10):1141–50.CrossRefPubMed
19.
Hasnain Z, N Manning, and JH Pierskalla. Performance-related pay in the public sector: a review of theory and evidence. World Bank Policy Research Working Paper. 2012; 6043.
20.
Basinga P, et al. Effect on maternal and child health services in Rwanda of payment to primary health-care providers for performance: an impact evaluation. Lancet. 2011;377(9775):1421–8.CrossRefPubMed
21.
Binyaruka P et al. Effect of paying for performance on utilisation, quality, and user costs of health services in Tanzania: a controlled before and after study. PLoS ONE. 2015;10(8):1–16.
22.
Bonfrer I, Van de Poel E, Van Doorslaer E. The effects of performance incentives on the utilization and quality of maternal and child care in Burundi. Soc Sci Med. 2014;123:96–104.CrossRefPubMed
23.
Peabody J, et al. Financial incentives and measurement improved physicians’ quality of care in the Philippines. Health Affairs (Millwood). 2011;30(4):773–81.CrossRef
24.
Yip W, et al. Capitation combined with pay-for-performance improves antibiotic prescribing practices in rural China. Health Affairs (Millwood). 2014;33(3):502–10.CrossRef
25.
Van de Poel E et al. Impact of performance-based financing in a low-resource setting: a decade of experience in Cambodia Health Economics. Health Economics. 2016;25(6):688–705.
26.
Witter S, et al. Performance-based financing as a health system reform: mapping the key dimensions for monitoring and evaluation. BMC Health Serv Res. 2013;13:367.CrossRefPubMedPubMedCentral
27.
Meessen P, et al. Reviewing institutions of rural health centres: the Performance Initiative in Butare, Rwanda. Trop Med Int Health. 2006;11(8):1303–17.CrossRefPubMed
28.
Bertone MP, Meessen B. Studying the link between institutions and health system performance: a framework and an illustration with the analysis of two performance-based financing schemes in Burundi. Health Policy Plan. 2013;28(8):847–57.CrossRefPubMed
29.
Ssengooba F, McPake B, Palmer N. Why performance-based contracting failed in Uganda—an “open-box” evaluation of a complex health system intervention. Soc Sci Med. 2012;75(2):377–83.CrossRefPubMed
30.
Bonfrer I, et al. Introduction of performance-based financing in Burundi was associated with improvements in care and quality. Health Aff. 2014;33(12):2179–87.CrossRef
31.
Borghi J, et al. Protocol for the evaluation of a pay for performance programme in Pwani region in Tanzania: a controlled before and after study. Implement Sci. 2013;8:80. 2013. 8(80).CrossRefPubMedPubMedCentral
32.
Falisse JB et al. Community participation and voice mechanisms under performance-based financing schemes in Burundi. Trop Med Int Health. 2012;17(5):674-682.
33.
Waddimba AC, et al. Provider attitudes associated with adherence to evidence-based clinical guidelines in a managed care setting. Med Care Res Rev. 2010;67(1):93–116.CrossRefPubMed
34.
Gertler P, Vermeersch C. Using performance incentives to improve medical care productivity and health outcomes. 2013, NBER Working Paper No. 19046.
35.
36.
Wang H, et al. An experiment in payment reform for doctors in rural China reduced some unnecessary care but did not lower total costs. Health Affairs (Millwood). 2011;30(12):2427–36.CrossRef
37.
Binyaruka P, M Mamdani, and J Borghi. Improving quality of care through payment for performance: examining effects on the availability and stock out of essential commodities in Tanzania. Trop Med Int Health. 2017;22(1):92–102.
38.
Baron RM, Kenny DA. The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. J Pers Soc Psychol. 1986;51:1173–82.CrossRefPubMed
39.
Manski CF. Identification for prediction and decision. Boston: H.U. Press; 2009.
40.
Imai K, Keele L, Yamamoto T. Identication, inference, and sensitivity analysis for causal mediation effects. Stat Sci. 2010;25:51–71.CrossRef
41.
Keele L, Tingley D, Yamamoto T. Identifying mechanisms behind policy interventions via causal mediation analysis. J Policy Anal Manage. 2015;34(4):937–63.CrossRef
42.
Imai K, Keele L, Tingley D. A general approach to causal mediation analysis. Psychol Methods. 2010;15:309–34.CrossRefPubMed
43.
Hicks R, Tingley D. Causal mediation analysis. Stata J. 2011;11(4):1–15.
44.
Cameron A, Gelbach J, Miller D. Bootstrap-based improvements for inference with clustered errors. Rev Econ Stat. 2008;90(3):414–27.CrossRef
45.
Cameron AC, Miller DL. A practitioner’s guide to cluster-robust inference. J Hum Resour. 2015;50(2):317–72.CrossRef
46.
Falisse JB, et al. Performance-based financing in the context of selective free health-care: an evaluation of its effects on the use of primary health-care services in Burundi using routine data. Health Policy Plan. 2014;30(10):1251–60.CrossRefPubMed
47.
Westhoff G, Cotter AM, Tolosa JE. Prophylactic oxytocin for the third stage of labour to prevent postpartum haemorrhage. Cochrane Database Syst Rev. 2013;10:CD001808.
48.
Wilunda C, et al. A qualitative study on barriers to utilisation of institutional delivery services in Moroto and Napak districts, Uganda: implications for programming. BMC Pregnancy Childbirth. 2014;14:259.CrossRefPubMedPubMedCentral
49.
Moore J, et al. Moving toward patient-centered care: women’s decisions, perceptions, and experiences of the induction of laborprocess. Birth. 2014;41(2):138–46.CrossRefPubMed
50.
Kruk ME, et al. User fee exemptions are not enough: out-of-pocket payments for ‘free’ delivery services in rural Tanzania. Trop Med Int Health. 2008;13(12):1442–51.CrossRefPubMed
51.
Kruk M, et al. Women’s preferences for obstetric care in rural Ethiopia: a population-based discrete choice experiment in a region with low rates of facility delivery. J Epidemiol Community Health. 2010;64(11):984–8.CrossRefPubMed
52.
Larson E, et al. Moving toward patient-centered care in africa: a discrete choice experiment of preferences for delivery care among 3,003 Tanzanian women. PLoS One. 2015;10(8):e0135621.CrossRefPubMedPubMedCentral
Metadata
Title
Understanding causal pathways within health systems policy evaluation through mediation analysis: an application to payment for performance (P4P) in Tanzania
Authors
Laura Anselmi
Peter Binyaruka
Josephine Borghi
Publication date
01-12-2017
Publisher
BioMed Central
Published in
Implementation Science / Issue 1/2017
Electronic ISSN: 1748-5908
DOI
https://doi.org/10.1186/s13012-016-0540-1

Other articles of this Issue 1/2017

Implementation Science 1/2017 Go to the issue

Short report

The creation and validation of the Measure of Effective Attributes of Trainers (MEAT)

Research

Improving the adaptability of WHO evidence-informed guidelines for nutrition actions: results of a mixed methods evaluation

Research

Reversing the pipeline? Implementing public health evidence-based guidance in english local government

Study protocol

Test of a workforce development intervention to expand opioid use disorder treatment pharmacotherapy prescribers: protocol for a cluster randomized trial

Debate

Safety analysis over time: seven major changes to adverse event investigation

Study protocol

Addressing challenges in scaling up TB and HIV treatment integration in rural primary healthcare clinics in South Africa (SUTHI): a cluster randomized controlled trial protocol