Top

BMC Medical Research Methodology

Published in:

Open Access 01-12-2016 | Research article

Head to head comparison of the propensity score and the high-dimensional propensity score matching methods

Authors: Jason R. Guertin, Elham Rahme, Colin R. Dormuth, Jacques LeLorier

Published in: BMC Medical Research Methodology | Issue 1/2016

Abstract

Background

Comparative performance of the traditional propensity score (PS) and high-dimensional propensity score (hdPS) methods in the adjustment for confounding by indication remains unclear. We aimed to identify which method provided the best adjustment for confounding by indication within the context of the risk of diabetes among patients exposed to moderate versus high potency statins.

Method

A cohort of diabetes-free incident statins users was identified from the Quebec’s publicly funded medico-administrative database (Full Cohort). We created two matched sub-cohorts by matching one patient initiated on a lower potency to one patient initiated on a high potency either on patients’ PS or hdPS. Both methods’ performance were compared by means of the absolute standardized differences (ASDD) regarding relevant characteristics and by means of the obtained measures of association.

Results

Eight out of the 18 examined characteristics were shown to be unbalanced within the Full Cohort. Although matching on either method achieved balance within all examined characteristic, matching on patients’ hdPS created the most balanced sub-cohort. Measures of associations and confidence intervals obtained within the two matched sub-cohorts overlapped.

Conclusion

Although ASDD suggest better matching with hdPS than with PS, measures of association were almost identical when adjusted for either method. Use of the hdPS method in adjusting for confounding by indication within future studies should be recommended due to its ability to identify confounding variables which may be unknown to the investigators.

Available only for authorised users

Groenwold RH, Hak E, Hoes AW. Quantitative assessment of unobserved confounding is mandatory in nonrandomized intervention studies. J Clin Epidemiol. 2009;62(1):22–8.CrossRefPubMed

Austin PC. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivar Behav Res. 2011;46(3):399–424.CrossRef

Shrank WH, Patrick AR, Brookhart MA. Healthy user and related biases in observational studies of preventive interventions: a primer for physicians. J Gen Intern Med. 2011;26(5):546–50.PubMedCentralCrossRefPubMed

Austin PC. The relative ability of different propensity score methods to balance measured covariates between treated and untreated subjects in observational studies. Med Decis Making. 2009;29(6):661–77.CrossRefPubMed

Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70(1):41–55.CrossRef

Black CM, Tadrous M, Cadarette SM. Diffusion of methodological innovation in pharmacoepidemiology: high-dimensional propensity score co-authorship network analysis. CAPT; Toronto: J Popul Ther Clin Pharmacol. 2013;21(1):e138.

Schneeweiss S, Rassen JA, Glynn RJ, Avorn J, Mogun H, Brookhart MA. High-dimensional propensity score adjustment in studies of treatment effects using health care claims data. Epidemiology. 2009;20(4):512–22.PubMedCentralCrossRefPubMed

Belitser SV, Martens EP, Pestman WR, Groenwold RHH, de Boer A, Klungel OH. Measuring balance and model selection in propensity score methods. Pharmacoepidemiol Drug Saf. 2011;20(11):1115–29.CrossRefPubMed

Austin PC. Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. Stat Med. 2009;28(25):3083–107.PubMedCentralCrossRefPubMed

10.

Mamdani M, Sykora K, Li P, Normand SL, Streiner DL, Austin PC, et al. Reader’s guide to critical appraisal of cohort studies: 2. Assessing potential for confounding. BMJ. 2005;330(7497):960–2.PubMedCentralCrossRefPubMed

11.

Ali MS, Groenwold RHH, Pestman WR, Belitser SV, Roes KCB, Hoes AW, et al. Propensity score balance measures in pharmacoepidemiology: a simulation study. Pharmacoepidemiol Drug Saf. 2014;23(8):802–11.PubMed

12.

Garbe E, Kloss S, Suling M, Pigeot I, Schneeweiss S. High-dimensional versus conventional propensity scores in a comparative effectiveness study of coxibs and reduced upper gastrointestinal complications. Eur J Clin Pharmacol. 2012;69:549–57.CrossRefPubMed

13.

Polinski JM, Schneeweiss S, Glynn RJ, Lii J, Rassen JA. Confronting “confounding by health system use” in Medicare Part D: comparative effectiveness of propensity score approaches to confounding adjustment. Pharmacoepidemiol Drug Saf. 2012;21:90–8.PubMedCentralCrossRefPubMed

14.

Rassen JA, Glynn RJ, Brookhart MA, Schneeweiss S. Covariate selection in high-dimensional propensity score analyses of treatment effects in small samples. Am J Epidemiol. 2011;173(12):1404–13.PubMedCentralCrossRefPubMed

15.

Preiss D, Seshasai SR, Welsh P, Murphy SA, Ho JE, Waters DD, et al. Risk of incident diabetes with intensive-dose compared with moderate-dose statin therapy: a meta-analysis. JAMA. 2011;305(24):2556–64.CrossRefPubMed

16.

Ko DT, Wijeysundera HC, Jackevicius CA, Yousef A, Wang J, Tu JV. Diabetes and cardiovascular events in older myocardial infarction patients prescribed intensive-dose and moderate-dose statins. Circ Cardiovasc Qual Outcomes. 2013;6:315–22.CrossRefPubMed

17.

Carter AA, Gomes T, Camacho X, Juurlink DN, Shah BR, Mamdani MM. Risk of incident diabetes among patients treated with statins: population based study. BMJ. 2013;346:f2610.PubMedCentralCrossRefPubMed

18.

Dormuth CR, Filion KB, Paterson JM, James MT, Teare GF, Raymond CB, et al. Higher potency statins and the risk of new diabetes: multicentre, observational study of administrative databases. BMJ. 2014;348:g3244.PubMedCentralCrossRefPubMed

19.

Wang K-L, Liu C-J, Chao T-F, Huang C-M, Wu C-H, Chen S-J, et al. Statins, risk of diabetes, and implications on outcomes in the general population. J Am Coll Cardiol. 2012;60(14):1231–8.CrossRefPubMed

20.

Zaharan NL, Williams D, Bennett K. Statins and risk of treated incident diabetes in a primary care population. Br J Clin Pharmacol. 2013;75(4):1118–24.PubMedCentralCrossRefPubMed

21.

Soucy A. Québec Handy Numbers, 2015 Edition. Québec: Institut de la statistique du Québec; 2015.

22.

Blais C, Lambert L, Hamel D, Brown K, Rinfret S, Cartier R, et al. Évaluation des soins et surveillance des maladies cardiovasculaires: Pouvons-nous faire confiance aux données médico-administratives hospitalières ? Montreal: Institut national d’excellence en santé et en services sociaux (INESSS); 2012.

23.

Lambert L, Blais C, Hamel D, Brown K, Rinfret S, Cartier R, et al. Evaluation of care and surveillance of cardiovascular disease: can we trust medico-administrative hospital data? Can J Cardiol. 2012;28(2):162–8.CrossRefPubMed

24.

Tamblyn R, Lavoie G, Petrella L, Monette J. The use of prescription claims databases in pharmacoepidemiological research: the accuracy and comprehensiveness of the prescription claims database in Quebec. J Clin Epidemiol. 1995;48(8):999–1009.CrossRefPubMed

25.

Tamblyn R, Reid T, Mayo N, McLeod P, Churchill-Smith M. Using medical services claims to assess injuries in the elderly: sensitivity of diagnostic and procedure codes for injury ascertainment. J Clin Epidemiol. 2000;53(2):183–94.CrossRefPubMed

26.

World Health Organisation Collaborating Centre for Drug Statistics Methodology. ATC/DDD Index [July 23rd 2014]. Available from: http://www.whocc.no/atc_ddd_index/.

27.

Law MR, Wald NJ, Rudnicka AR. Quantifying effect of statins on low density lipoprotein cholesterol, ischaemic heart disease, and stroke: systematic review and meta-analysis. BMJ. 2003;326(7404):1423.PubMedCentralCrossRefPubMed

28.

Sturmer T, Rothman KJ, Avorn J, Glynn RJ. Treatment effects in the presence of unmeasured confounding: dealing with observations in the tails of the propensity score distribution--a simulation study. Am J Epidemiol. 2010;172(7):843–54.PubMedCentralCrossRefPubMed

29.

Austin PC. Optimal caliper widths for propensity-score matching when estimating differences in means and differences in proportions in observational studies. Pharm Stat. 2011;10(2):150–61.PubMedCentralCrossRefPubMed

30.

Franklin JM, Schneeweiss S, Polinski JM, Rassen JA. Plasmode simulation for the evaluation of pharmacoepidemiologic methods in complex healthcare databases. Comput Stat Data Anal. 2014;72:219–26.PubMedCentralCrossRefPubMed

31.

Westreich D, Lessler J, Funk MJ. Propensity score estimation: neural networks, support vector machines, decision trees (CART), and meta-classifiers as alternatives to logistic regression. J Clin Epidemiol. 2010;63(8):826–33.PubMedCentralCrossRefPubMed

32.

Lee BK, Lessler J, Stuart EA. Improving propensity score weighting using machine learning. Stat Med. 2010;29(3):337–46.PubMedCentralPubMed

Title: Head to head comparison of the propensity score and the high-dimensional propensity score matching methods
Authors: Jason R. Guertin
Elham Rahme
Colin R. Dormuth
Jacques LeLorier
Publication date: 01-12-2016
Publisher: BioMed Central
Published in: BMC Medical Research Methodology / Issue 1/2016
Electronic ISSN: 1471-2288
DOI: https://doi.org/10.1186/s12874-016-0119-1

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Head to head comparison of the propensity score and the high-dimensional propensity score matching methods

Abstract

Background

Method

Results

Conclusion

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Method

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 1/2016

Bias of health estimates obtained from chronic disease and risk factor surveillance systems using telephone population surveys in Australia: results from a representative face-to-face survey in Australia from 2010 to 2013

No rationale for 1 variable per 10 events criterion for binary logistic regression analysis

Methods for network meta-analysis of continuous outcomes using individual patient data: a case study in acupuncture for chronic pain

The existence of standard-biased mortality ratios due to death certificate misclassification - a simulation study based on a true story

Confidence intervals for the between-study variance in random-effects meta-analysis using generalised heterogeneity statistics: should we use unequal tails?

Validating the generic quality of life tool “QOL10” in a substance use disorder treatment cohort exposes a unique social construct