Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
Applied Health Economics and Health Policy
Published in: Applied Health Economics and Health Policy 4/2016

Open Access 01-08-2016 | Original Research Article

Difference-in-Differences Method in Comparative Effectiveness Research: Utility with Unbalanced Groups

Authors: Huanxue Zhou, Christopher Taber, Steve Arcona, Yunfeng Li

Published in: Applied Health Economics and Health Policy | Issue 4/2016

Login to get access

Abstract

Background

Comparative effectiveness research (CER) often includes observational studies utilizing administrative data. Multiple conditioning methods can be used for CER to adjust for group differences, including difference-in-differences (DiD) estimation.

Objective

This study presents DiD and demonstrates how to apply this conditioning method to estimate treatment outcomes in the CER setting by utilizing the MarketScan® Databases for multiple sclerosis (MS) patients receiving different therapies.

Methods

The sample included 6762 patients, with 363 in the Test Cohort [glatiramer acetate (GA) switched to fingolimod (FTY)] and 6399 in the Control Cohort (GA only, no switch) from a US administrative claims database. A trend analysis was conducted to rule out concerns regarding regression to the mean and to compare relapse rates among treatment cohorts. DiD analysis was used to enable comparisons among the Test and Control Cohorts. Logistic regression was used to estimate the probability of relapse after switching from GA to FTY, and to compare group differences in the pre- and post-index periods.

Results

Crude DiD analysis showed that in the pre-index period more patients in the Test Cohort experienced an MS relapse and had a higher mean number of relapses than in the Control Cohort. During the pre-index period, numeric and relative data for MS relapses in patients in the Test Cohort were significantly higher than in the Control Cohort, while no significant between-group differences emerged during the post-index period. Generalized linear modeling with DiD regression estimation showed that the mean number of MS relapses decreased significantly in the post-index period among patients in the Test Cohort compared with patients in the Control Cohort.

Conclusion

In this study, an MS population was utilized to demonstrate how DiD can be applied to estimate treatment effects in a heterogeneous population, where the Test and Control Cohorts varied greatly. The results show that DiD offers a robust method for comparing diverse cohorts when other risk-adjustment methods may not be adequate.
Appendix
Available only for authorised users
Literature
1.
Concato J, Lawler EV, Lew RA, et al. Observational methods in comparative effectiveness research. Am J Med. 2010;123(12 Suppl 1):e16–23.CrossRefPubMed
2.
Austin PC. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivar Behav Res. 2011;46(3):399–424.CrossRef
3.
Curtis LH, Hammill BG, Eisenstein EL, et al. Using inverse probability-weighted estimators in comparative effectiveness analyses with observational databases. Medical Care. 2007;45(10 Supl 2):S103–7.CrossRefPubMed
4.
Lanehart RE, Rodriguez de Gil P, Kim ES, Bellara AP, Kromrey JD, Lee SR. Paper 314-2012: propensity score analysis and assessment of proposensity score approaches using SAS procedures. SAS Global Forum 2012; 2012.
5.
Rubin DB. Estimating causal effects from large data sets using propensity scores. Ann Intern Med. 1997;127(8 Pt 2):757–63.CrossRefPubMed
6.
Hirano K, Imbens GW. Estimation of causal effects using propensity score weighting: an application to data on right heart catheterization. Health Serv Outcomes Res Methodol. 2001;2:259–78.CrossRef
7.
Austin PC. A critical appraisal of propensity-score matching in the medical literature between 1996 and 2003. Stat Med. 2008;27(12):2037–49.CrossRefPubMed
8.
Austin PC. Some methods of propensity-score matching had superior performance to others: results of an empirical investigation and Monte Carlo simulations. Biometrical J. 2009;51(1):171–84.CrossRef
9.
10.
Dimick JB, Ryan AM. Methods for evaluating changes in health care policy: the difference-in-differences approach. Jama. 2014;312(22):2401–2.CrossRefPubMed
11.
Bertrand M, Duflo E, Mullainathan S. How much should we trust difference-in-differences estimates? Q J Econ. 2004;119:249–75.CrossRef
12.
Crown WH. Propensity-score matching in economic analyses: comparison with regression models, instrumental variables, residual inclusion, differences-in-differences, and decomposition methods. Appl Health Econ Health Policy. 2014;12(1):7–18.CrossRefPubMed
13.
Disanto G, Berlanga AJ, Handel AE, et al. Heterogeneity in multiple sclerosis: scratching the surface of a complex disease. Autoimmun Dis. 2010;2011:932351.
14.
Abadie A. Semiparametric difference-in-differences estimators. Rev Econ Stud. 2005;72(1):1–19.CrossRef
15.
Meyer B. Natural and quas-experiments in economics. J Bus Econ Stat. 1995;13(2):151–61.
16.
Butler Quint J. White paper health research data for the real world: the MarketScan® Databases. Ann Arbor: Truven Health Analytics Inc; 2015.
17.
Chastek BJ, Oleen Burkey M, Lopez-Bresnahan MV. Medical chart validation of an algorithm for identifying multiple sclerosis relapse in healthcare claims. J Med Econ. 2010;13(4):618–25.CrossRefPubMed
18.
Ashenfelter O. Estimating the effect of training programs on earnings. Rev Econ Stat. 1978;60:47–57.CrossRef
19.
Heckman JJ, Smith JA. The pre-programme earnings dip and the determinants of participation in a social programme. Implications for simple programme evaluation strategies. Econ J. 1999;109:313–48.CrossRef
20.
Weiss J, Makonnen R, Sula D. Shifting management of a community volunteer system for improved child health outcomes: results from an operations research study in Burundi. BMC Health Serv Res. 2015;15(Suppl 1):S2.CrossRefPubMedPubMedCentral
21.
Brenner S, Muula AS, Robyn PJ, et al. Design of an impact evaluation using a mixed methods model—an explanatory assessment of the effects of results-based financing mechanisms on maternal healthcare services in Malawi. BMC Health Serv Res. 2014;14:180.CrossRefPubMedPubMedCentral
22.
Colla CH, Lewis VA, Gottlieb DJ, et al. Cancer spending and accountable care organizations: evidence from the Physician Group Practice Demonstration. Healthcare (Amst). 2013;1(3–4):100–7.CrossRefPubMedCentral
23.
Dubay L, Kenney G. Expanding public health insurance to parents: effects on children’s coverage under Medicaid. Health Serv Res. 2003;38(5):1283–301.CrossRefPubMedPubMedCentral
24.
McAdam-Marx C, Dahal A, Jennings B, et al. The effect of a diabetes collaborative care management program on clinical and economic outcomes in patients with type 2 diabetes. J Manag Care Spec Pharm. 2015;21(6):452–68.CrossRefPubMed
25.
Pereira SK, Kumar P, Dutt V, et al. Protocol for the evaluation of a social franchising model to improve maternal health in Uttar Pradesh, India. Implement Sci. 2015;10:77.CrossRefPubMedPubMedCentral
26.
Salinas-Rodriguez A, Torres-Pereda Mdel P, Manrique-Espinoza B, et al. Impact of the non-contributory social pension program 70 y mas on older adults’ mental well-being. PLoS One. 2014;9(11):e113085.CrossRefPubMedPubMedCentral
27.
Siddiqui M, Roberts ET, Pollack CE. The effect of emergency department copayments for Medicaid beneficiaries following the Deficit Reduction Act of 2005. JAMA Intern Med. 2015;175(3):393–8.CrossRefPubMedPubMedCentral
28.
Rajaram R, Chung JW, Jones AT, et al. Association of the 2011 ACGME resident duty hour reform with general surgery patient outcomes and with resident examination performance. Jama. 2014;312(22):2374–84.CrossRefPubMed
29.
Patel MS, Volpp KG, Small DS, et al. Association of the 2011 ACGME resident duty hour reforms with mortality and readmissions among hospitalized Medicare patients. Jama. 2014;312(22):2364–73.CrossRefPubMed
30.
Jain RMC, Lee H, Wong W. Use of rosiglitazone and pioglitazone immediately after the cardiovascular risk warnings. Res Soc Adm Pharm. 2012;8(1):47–59.CrossRef
Metadata
Title
Difference-in-Differences Method in Comparative Effectiveness Research: Utility with Unbalanced Groups
Authors
Huanxue Zhou
Christopher Taber
Steve Arcona
Yunfeng Li
Publication date
01-08-2016
Publisher
Springer International Publishing
Published in
Applied Health Economics and Health Policy / Issue 4/2016
Print ISSN: 1175-5652
Electronic ISSN: 1179-1896
DOI
https://doi.org/10.1007/s40258-016-0249-y

Other articles of this Issue 4/2016

Applied Health Economics and Health Policy 4/2016 Go to the issue

Original Research Article

Cost-Effectiveness Analysis of Bendamustine Plus Rituximab as a First-Line Treatment for Patients with Follicular Lymphoma in Spain

Original Research Article

A Trial-Based Predictive Microsimulation Assessing the Public Health Benefits of Nalmefene and Psychosocial Support for the Reduction of Alcohol Consumption in Alcohol Dependence

Current Opinion

Governments Need Better Guidance to Maximise Value for Money: The Case of Australia’s Pharmaceutical Benefits Advisory Committee

Commentary

The Potential Impact of the Trans-Pacific Partnership Agreement on Health Equity, with Illustration From New Zealand

Original Research Article

Analysis of Driving Factors of Willingness to Use and Willingness to Pay for Existing Pharmacological Smoking Cessation Aids Among Young and Middle-Aged Adults in Germany

Original Research Article

Measuring the Volume-Outcome Relation for Complex Hospital Surgery