Skip to main content
Top

13-03-2024 | Human Immunodeficiency Virus | Review

Synthetic Controls for Implementation Science: Opportunities for HIV Program Evaluation Using Routinely Collected Data

Authors: Sara Wallach, Suzue Saito, Harriet Nuwagaba-Biribonwoha, Lenhle Dube, Matthew R. Lamb

Published in: Current HIV/AIDS Reports

Login to get access

Abstract

Purpose of Review

HIV service delivery programs are some of the largest funded public health programs in the world. Timely, efficient evaluation of these programs can be enhanced with methodologies designed to estimate the effects of policy. We propose using the synthetic control method (SCM) as an implementation science tool to evaluate these HIV programs.

Recent Findings

SCM, introduced in econometrics, shows increasing utility across fields. Key benefits of this methodology over traditional design-based approaches for evaluation stem from directly approximating pre-intervention trends by weighting of candidate non-intervention units. We demonstrate SCM to evaluate the effectiveness of a public health intervention targeting HIV health facilities with high numbers of recent infections on trends in pre-exposure prophylaxis (PrEP) enrollment.

Summary

This test case demonstrates SCM’s feasibility for effectiveness evaluations of site-level HIV interventions. HIV programs collecting longitudinal, routine service delivery data for many facilities, with only some receiving a time-specified intervention, are well-suited for evaluation using SCM.
Literature
1.
go back to reference Abadie A, Gardeazabal J. The economic costs of conflict: a case study of the Basque country. American Economic Review. 2003;93(1):113–32.CrossRef Abadie A, Gardeazabal J. The economic costs of conflict: a case study of the Basque country. American Economic Review. 2003;93(1):113–32.CrossRef
2.
go back to reference Abadie A, Diamond A, Hainmueller J. Synthetic control methods for comparative case studies: estimating the effect of California’s tobacco control program. J Am Stat Assoc. 2010;105(490):493–505.MathSciNetCrossRef Abadie A, Diamond A, Hainmueller J. Synthetic control methods for comparative case studies: estimating the effect of California’s tobacco control program. J Am Stat Assoc. 2010;105(490):493–505.MathSciNetCrossRef
3.
go back to reference Athey S, Imbens GW. The state of applied econometrics: causality and policy evaluation. Journal of Economic Perspectives. 2017;31(2):3–32.CrossRef Athey S, Imbens GW. The state of applied econometrics: causality and policy evaluation. Journal of Economic Perspectives. 2017;31(2):3–32.CrossRef
6.
go back to reference Bouttell J, Craig P, Lewsey J, Robinson M, Popham F. Synthetic control methodology as a tool for evaluating population-level health interventions. J Epidemiol Community Health. 2018;72(8):673–8.CrossRefPubMed Bouttell J, Craig P, Lewsey J, Robinson M, Popham F. Synthetic control methodology as a tool for evaluating population-level health interventions. J Epidemiol Community Health. 2018;72(8):673–8.CrossRefPubMed
7.
go back to reference Kreif N, Grieve R, Hangartner D, Turner AJ, Nikolova S, Sutton M. Examination of the synthetic control method for evaluating health policies with multiple treated units. Health Econ. 2016;25(12):1514–28.CrossRefPubMed Kreif N, Grieve R, Hangartner D, Turner AJ, Nikolova S, Sutton M. Examination of the synthetic control method for evaluating health policies with multiple treated units. Health Econ. 2016;25(12):1514–28.CrossRefPubMed
8.
go back to reference Bauhoff S. The effect of school district nutrition policies on dietary intake and overweight: a synthetic control approach. Econ Hum Biol. 2014;12:45–55.CrossRefPubMed Bauhoff S. The effect of school district nutrition policies on dietary intake and overweight: a synthetic control approach. Econ Hum Biol. 2014;12:45–55.CrossRefPubMed
9.
go back to reference Kiesel K, Villas-Boas SB. Can information costs affect consumer choice? Nutritional labels in a supermarket experiment. Int J Ind Organ. 2013;31(2):153–63.CrossRef Kiesel K, Villas-Boas SB. Can information costs affect consumer choice? Nutritional labels in a supermarket experiment. Int J Ind Organ. 2013;31(2):153–63.CrossRef
10.
go back to reference Garber MD. Precision and weighting of effects estimated by the generalized synthetic control and related methods: the case of Medicaid expansion. Epidemiology. 2024;35(2):273–7.CrossRefPubMed Garber MD. Precision and weighting of effects estimated by the generalized synthetic control and related methods: the case of Medicaid expansion. Epidemiology. 2024;35(2):273–7.CrossRefPubMed
11.
go back to reference Reinbold GW. Effects of the Convention on the Rights of the Child on child mortality and vaccination rates: a synthetic control analysis. BMC Int Health Hum Rights. 2019;19(1):24.CrossRefPubMedPubMedCentral Reinbold GW. Effects of the Convention on the Rights of the Child on child mortality and vaccination rates: a synthetic control analysis. BMC Int Health Hum Rights. 2019;19(1):24.CrossRefPubMedPubMedCentral
12.
go back to reference Kim SY, Kim H, Lee JT. Health Effects of air-quality regulations in Seoul metropolitan area: applying synthetic control method to controlled-interrupted time-series analysis. Atmosphere. 2020;11(8):868.CrossRefADS Kim SY, Kim H, Lee JT. Health Effects of air-quality regulations in Seoul metropolitan area: applying synthetic control method to controlled-interrupted time-series analysis. Atmosphere. 2020;11(8):868.CrossRefADS
13.
go back to reference Karlsson M, Pichler S. Demographic consequences of HIV. J Popul Econ. 2015;28(4):1097–135.CrossRef Karlsson M, Pichler S. Demographic consequences of HIV. J Popul Econ. 2015;28(4):1097–135.CrossRef
16.
go back to reference Lim JT, Bansal S, Chong CS, Dickens B, Ng Y, Deng L, et al. Efficacy of Wolbachia-mediated sterility to reduce the incidence of dengue: a synthetic control study in Singapore. The Lancet Microbe. 2024 Feb;S266652472300397X. Lim JT, Bansal S, Chong CS, Dickens B, Ng Y, Deng L, et al. Efficacy of Wolbachia-mediated sterility to reduce the incidence of dengue: a synthetic control study in Singapore. The Lancet Microbe. 2024 Feb;S266652472300397X.
18.
19.
go back to reference Mader S, Rüttenauer T. The effects of non-pharmaceutical interventions on COVID-19 mortality: a generalized synthetic control approach across 169 countries. Front Public Health. 2022;4(10): 820642.CrossRef Mader S, Rüttenauer T. The effects of non-pharmaceutical interventions on COVID-19 mortality: a generalized synthetic control approach across 169 countries. Front Public Health. 2022;4(10): 820642.CrossRef
20.
go back to reference Lang D, Esbenshade L, Willer R. Did Ohio’s vaccine lottery increase vaccination rates? A pre-registered, synthetic control study. J Exp Polit Sci. 2023;10(2):242–60.CrossRef Lang D, Esbenshade L, Willer R. Did Ohio’s vaccine lottery increase vaccination rates? A pre-registered, synthetic control study. J Exp Polit Sci. 2023;10(2):242–60.CrossRef
21.
go back to reference Cole MA, Elliott RJR, Liu B. The impact of the Wuhan Covid-19 lockdown on air pollution and health: a machine learning and augmented synthetic control approach. Environ Resource Econ. 2020;76(4):553–80.CrossRef Cole MA, Elliott RJR, Liu B. The impact of the Wuhan Covid-19 lockdown on air pollution and health: a machine learning and augmented synthetic control approach. Environ Resource Econ. 2020;76(4):553–80.CrossRef
22.
go back to reference Bruhn CAW, Hetterich S, Schuck-Paim C, Kürüm E, Taylor RJ, Lustig R, et al. Estimating the population-level impact of vaccines using synthetic controls. Proc Natl Acad Sci USA. 2017;114(7):1524–9.CrossRefPubMedPubMedCentralADS Bruhn CAW, Hetterich S, Schuck-Paim C, Kürüm E, Taylor RJ, Lustig R, et al. Estimating the population-level impact of vaccines using synthetic controls. Proc Natl Acad Sci USA. 2017;114(7):1524–9.CrossRefPubMedPubMedCentralADS
23.
go back to reference Buggs SA, Webster DW, Crifasi CK. Using synthetic control methodology to estimate effects of a Cure Violence intervention in Baltimore. Maryland Inj Prev. 2022;28(1):61–7.CrossRefPubMed Buggs SA, Webster DW, Crifasi CK. Using synthetic control methodology to estimate effects of a Cure Violence intervention in Baltimore. Maryland Inj Prev. 2022;28(1):61–7.CrossRefPubMed
24.••
go back to reference Abadie A. Using synthetic controls: feasibility, data requirements, and methodological aspects. Journal of Economic Literature. 2021;59(2):391–425. This work provides an overview of the synthetic control method, its extensions, and related methods, as well as its strengths and limitations.CrossRef Abadie A. Using synthetic controls: feasibility, data requirements, and methodological aspects. Journal of Economic Literature. 2021;59(2):391–425. This work provides an overview of the synthetic control method, its extensions, and related methods, as well as its strengths and limitations.CrossRef
26.
go back to reference Kim AA, Behel S, Northbrook S, Parekh BS. Tracking with recency assays to control the epidemic: real-time HIV surveillance and public health response. AIDS. 2019;33(9):1527–9.CrossRefPubMed Kim AA, Behel S, Northbrook S, Parekh BS. Tracking with recency assays to control the epidemic: real-time HIV surveillance and public health response. AIDS. 2019;33(9):1527–9.CrossRefPubMed
27.
go back to reference Abadie A, Diamond A, Hainmueller J. Comparative politics and the synthetic control method: comparative politics and the synthetic control method. American Journal of Political Science. 2015;59(2):495–510.CrossRef Abadie A, Diamond A, Hainmueller J. Comparative politics and the synthetic control method: comparative politics and the synthetic control method. American Journal of Political Science. 2015;59(2):495–510.CrossRef
28.
go back to reference Hainmueller J. Entropy balancing for causal effects: a multivariate reweighting method to produce balanced samples in observational studies. Polit anal. 2012;20(1):25–46.CrossRef Hainmueller J. Entropy balancing for causal effects: a multivariate reweighting method to produce balanced samples in observational studies. Polit anal. 2012;20(1):25–46.CrossRef
29.
go back to reference Robbins MW, Saunders J, Kilmer B. A Framework for synthetic control methods with high-dimensional, micro-level data: evaluating a neighborhood-specific crime intervention. J Am Stat Assoc. 2017;112(517):109–26.MathSciNetCrossRef Robbins MW, Saunders J, Kilmer B. A Framework for synthetic control methods with high-dimensional, micro-level data: evaluating a neighborhood-specific crime intervention. J Am Stat Assoc. 2017;112(517):109–26.MathSciNetCrossRef
30.••
go back to reference Ben-Michael E, Feller A, Rothstein J. Synthetic controls with staggered adoption. Journal of the Royal Statistical Society Series B: Statistical Methodology. 2022;84(2):351–81. This article provides an overview of how to use the synthetic control method with staggered adoption units and provides an example of its use.MathSciNetCrossRef Ben-Michael E, Feller A, Rothstein J. Synthetic controls with staggered adoption. Journal of the Royal Statistical Society Series B: Statistical Methodology. 2022;84(2):351–81. This article provides an overview of how to use the synthetic control method with staggered adoption units and provides an example of its use.MathSciNetCrossRef
35.••
go back to reference Ben-Michael E, Feller A, Rothstein J. The augmented synthetic control method. Journal of the American Statistical Association. 2021;116(536):1789–803. This article provides an overview of the augmented synthetic control extension of the synthetic control method and illustrates an example of its use.MathSciNetCrossRef Ben-Michael E, Feller A, Rothstein J. The augmented synthetic control method. Journal of the American Statistical Association. 2021;116(536):1789–803. This article provides an overview of the augmented synthetic control extension of the synthetic control method and illustrates an example of its use.MathSciNetCrossRef
36.
go back to reference Lipsitch M, TchetgenTchetgen E, Cohen T. Negative controls: a tool for detecting confounding and bias in observational studies. Epidemiology. 2010;21(3):383–8.CrossRefPubMedPubMedCentral Lipsitch M, TchetgenTchetgen E, Cohen T. Negative controls: a tool for detecting confounding and bias in observational studies. Epidemiology. 2010;21(3):383–8.CrossRefPubMedPubMedCentral
40.
go back to reference Wing C, Simon K, Bello-Gomez RA. Designing difference in difference studies: best practices for public health policy research. Annu Rev Public Health. 2018;39(1):453–69.CrossRefPubMed Wing C, Simon K, Bello-Gomez RA. Designing difference in difference studies: best practices for public health policy research. Annu Rev Public Health. 2018;39(1):453–69.CrossRefPubMed
Metadata
Title
Synthetic Controls for Implementation Science: Opportunities for HIV Program Evaluation Using Routinely Collected Data
Authors
Sara Wallach
Suzue Saito
Harriet Nuwagaba-Biribonwoha
Lenhle Dube
Matthew R. Lamb
Publication date
13-03-2024
Publisher
Springer US
Published in
Current HIV/AIDS Reports
Print ISSN: 1548-3568
Electronic ISSN: 1548-3576
DOI
https://doi.org/10.1007/s11904-024-00695-z
Live Webinar | 27-06-2024 | 18:00 (CEST)

Keynote webinar | Spotlight on medication adherence

Live: Thursday 27th June 2024, 18:00-19:30 (CEST)

WHO estimates that half of all patients worldwide are non-adherent to their prescribed medication. The consequences of poor adherence can be catastrophic, on both the individual and population level.

Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.

Prof. Kevin Dolgin
Prof. Florian Limbourg
Prof. Anoop Chauhan
Developed by: Springer Medicine
Obesity Clinical Trial Summary

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.

Developed by: Springer Medicine

Highlights from the ACC 2024 Congress

Year in Review: Pediatric cardiology

Watch Dr. Anne Marie Valente present the last year's highlights in pediatric and congenital heart disease in the official ACC.24 Year in Review session.

Year in Review: Pulmonary vascular disease

The last year's highlights in pulmonary vascular disease are presented by Dr. Jane Leopold in this official video from ACC.24.

Year in Review: Valvular heart disease

Watch Prof. William Zoghbi present the last year's highlights in valvular heart disease from the official ACC.24 Year in Review session.

Year in Review: Heart failure and cardiomyopathies

Watch this official video from ACC.24. Dr. Biykem Bozkurt discusses last year's major advances in heart failure and cardiomyopathies.