Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2012

Open Access 01-12-2012 | Research article

Practical considerations for sensitivity analysis after multiple imputation applied to epidemiological studies with incomplete data

Authors: Vanina Héraud-Bousquet, Christine Larsen, James Carpenter, Jean-Claude Desenclos, Yann Le Strat

Published in: BMC Medical Research Methodology | Issue 1/2012

Login to get access

Abstract

Background

Multiple Imputation as usually implemented assumes that data are Missing At Random (MAR), meaning that the underlying missing data mechanism, given the observed data, is independent of the unobserved data. To explore the sensitivity of the inferences to departures from the MAR assumption, we applied the method proposed by Carpenter et al. (2007).
This approach aims to approximate inferences under a Missing Not At random (MNAR) mechanism by reweighting estimates obtained after multiple imputation where the weights depend on the assumed degree of departure from the MAR assumption.

Methods

The method is illustrated with epidemiological data from a surveillance system of hepatitis C virus (HCV) infection in France during the 2001–2007 period. The subpopulation studied included 4343 HCV infected patients who reported drug use. Risk factors for severe liver disease were assessed. After performing complete-case and multiple imputation analyses, we applied the sensitivity analysis to 3 risk factors of severe liver disease: past excessive alcohol consumption, HIV co-infection and infection with HCV genotype 3.

Results

In these data, the association between severe liver disease and HIV was underestimated, if given the observed data the chance of observing HIV status is high when this is positive. Inference for two other risk factors were robust to plausible local departures from the MAR assumption.

Conclusions

We have demonstrated the practical utility of, and advocate, a pragmatic widely applicable approach to exploring plausible departures from the MAR assumption post multiple imputation. We have developed guidelines for applying this approach to epidemiological studies.
Appendix
Available only for authorised users
Literature
1.
go back to reference Sterne JA, White IR, Carlin JB, Spratt M, Royston P, Kenward MG, Wood AM, Carpenter JR: Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ. 2009, 338: b2393-10.1136/bmj.b2393.CrossRefPubMedPubMedCentral Sterne JA, White IR, Carlin JB, Spratt M, Royston P, Kenward MG, Wood AM, Carpenter JR: Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ. 2009, 338: b2393-10.1136/bmj.b2393.CrossRefPubMedPubMedCentral
2.
go back to reference Kenward MG, Carpenter J: Multiple imputation: current perspectives. Stat Methods Med Res. 2007, 16: 199-218. 10.1177/0962280206075304.CrossRefPubMed Kenward MG, Carpenter J: Multiple imputation: current perspectives. Stat Methods Med Res. 2007, 16: 199-218. 10.1177/0962280206075304.CrossRefPubMed
3.
go back to reference Horton NJ, Kleinman KP: Much ado about nothing: A comparison of missing data methods and software to fit incomplete data regression models. Am Stat. 2007, 61: 79-90. 10.1198/000313007X172556.CrossRefPubMedPubMedCentral Horton NJ, Kleinman KP: Much ado about nothing: A comparison of missing data methods and software to fit incomplete data regression models. Am Stat. 2007, 61: 79-90. 10.1198/000313007X172556.CrossRefPubMedPubMedCentral
4.
go back to reference Royston P: Multiple imputation of missing values: furter update of ice, with an emphasis on categorical variables. Stata J. 2009, 9: 466-477. Royston P: Multiple imputation of missing values: furter update of ice, with an emphasis on categorical variables. Stata J. 2009, 9: 466-477.
5.
go back to reference Little RJA, Rubin DB: Statistical analysis with missing data. Wiley series in Probability and Statistics. 2002, New York: Wiley, 2 Little RJA, Rubin DB: Statistical analysis with missing data. Wiley series in Probability and Statistics. 2002, New York: Wiley, 2
6.
go back to reference Spratt M, Carpenter J, Sterne JA, Carlin JB, Heron J, Henderson J, Tilling K: Strategies for multiple imputation in longitudinal studies. Am J Epidemiol. 2010, 172: 478-487. 10.1093/aje/kwq137.CrossRefPubMed Spratt M, Carpenter J, Sterne JA, Carlin JB, Heron J, Henderson J, Tilling K: Strategies for multiple imputation in longitudinal studies. Am J Epidemiol. 2010, 172: 478-487. 10.1093/aje/kwq137.CrossRefPubMed
8.
go back to reference Vach W, Blettner M: Biased estimation of the odds ratio in case–control studies due to the use of ad hoc methods of correcting for missing values for confounding variables. Am J Epidemiol. 1991, 134: 895-907.PubMed Vach W, Blettner M: Biased estimation of the odds ratio in case–control studies due to the use of ad hoc methods of correcting for missing values for confounding variables. Am J Epidemiol. 1991, 134: 895-907.PubMed
9.
go back to reference Potthoff RF, Tudor GE, Pieper KS, Hasselblad V: Can one assess whether missing data are missing at random in medical studies?. Stat Methods Med Res. 2006, 15: 213-234. 10.1191/0962280206sm448oa.CrossRefPubMed Potthoff RF, Tudor GE, Pieper KS, Hasselblad V: Can one assess whether missing data are missing at random in medical studies?. Stat Methods Med Res. 2006, 15: 213-234. 10.1191/0962280206sm448oa.CrossRefPubMed
10.
go back to reference White IR, Carlin JB: Bias and efficiency of multiple imputation compared with complete-case analysis for missing covariate values. Stat Med. 2010, 29: 2920-2931. 10.1002/sim.3944.CrossRefPubMed White IR, Carlin JB: Bias and efficiency of multiple imputation compared with complete-case analysis for missing covariate values. Stat Med. 2010, 29: 2920-2931. 10.1002/sim.3944.CrossRefPubMed
11.
go back to reference Collins LM, Schafer JL, Kam CM: A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychol Methods. 2001, 6: 330-351.CrossRefPubMed Collins LM, Schafer JL, Kam CM: A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychol Methods. 2001, 6: 330-351.CrossRefPubMed
12.
go back to reference Raghunathan TE: What do we do with missing data? Some options for analysis of incomplete data. Annu Rev Public Health. 2004, 25: 99-117. 10.1146/annurev.publhealth.25.102802.124410.CrossRefPubMed Raghunathan TE: What do we do with missing data? Some options for analysis of incomplete data. Annu Rev Public Health. 2004, 25: 99-117. 10.1146/annurev.publhealth.25.102802.124410.CrossRefPubMed
13.
go back to reference Schafer JL: Multiple imputation in multivariate problems when the imputation and analysis model differ. Stat Neer. 2003, 57: 19-35. 10.1111/1467-9574.00218.CrossRef Schafer JL: Multiple imputation in multivariate problems when the imputation and analysis model differ. Stat Neer. 2003, 57: 19-35. 10.1111/1467-9574.00218.CrossRef
14.
go back to reference Wood AM, White IR, Royston P: How should variable selection be performed with multiply imputed data?. Stat Med. 2008, 27: 3227-3246. 10.1002/sim.3177.CrossRefPubMed Wood AM, White IR, Royston P: How should variable selection be performed with multiply imputed data?. Stat Med. 2008, 27: 3227-3246. 10.1002/sim.3177.CrossRefPubMed
15.
go back to reference Diggle MG, Kenward MG: Informative drop-out in longitudinal data analysis. Appl Statist. 1994, 43: 49-93. 10.2307/2986113.CrossRef Diggle MG, Kenward MG: Informative drop-out in longitudinal data analysis. Appl Statist. 1994, 43: 49-93. 10.2307/2986113.CrossRef
16.
go back to reference Hogan JW, Laird NM: Mixture models for the joint distribution of repeated measures and event times. Stat Med. 1997, 16: 239-257. 10.1002/(SICI)1097-0258(19970215)16:3<239::AID-SIM483>3.0.CO;2-X.CrossRefPubMed Hogan JW, Laird NM: Mixture models for the joint distribution of repeated measures and event times. Stat Med. 1997, 16: 239-257. 10.1002/(SICI)1097-0258(19970215)16:3<239::AID-SIM483>3.0.CO;2-X.CrossRefPubMed
17.
go back to reference Curran D, Molenberghs G, Thijs H, Verbeke G: Sensitivity analysis for pattern mixture models. J Biopharm Stat. 2004, 14: 125-143. 10.1081/BIP-120028510.CrossRefPubMed Curran D, Molenberghs G, Thijs H, Verbeke G: Sensitivity analysis for pattern mixture models. J Biopharm Stat. 2004, 14: 125-143. 10.1081/BIP-120028510.CrossRefPubMed
18.
go back to reference Molenberghs G, Verbeke G: Models for discrete longitudinal data. 2005, New York: Springer Molenberghs G, Verbeke G: Models for discrete longitudinal data. 2005, New York: Springer
19.
go back to reference Carpenter J, Rucker G, Schwarzer G: Assessing the Sensitivity of Meta-analysis to Selection Bias: A Multiple Imputation Approach. Biometrics. 2011, 67: 1066-1072. 10.1111/j.1541-0420.2010.01498.x.CrossRefPubMed Carpenter J, Rucker G, Schwarzer G: Assessing the Sensitivity of Meta-analysis to Selection Bias: A Multiple Imputation Approach. Biometrics. 2011, 67: 1066-1072. 10.1111/j.1541-0420.2010.01498.x.CrossRefPubMed
20.
go back to reference Carpenter JR, Kenward MG, White IR: Sensitivity analysis after multiple imputation under missing at random: a weighting approach. Stat Methods Med Res. 2007, 16: 259-275. 10.1177/0962280206075303.CrossRefPubMed Carpenter JR, Kenward MG, White IR: Sensitivity analysis after multiple imputation under missing at random: a weighting approach. Stat Methods Med Res. 2007, 16: 259-275. 10.1177/0962280206075303.CrossRefPubMed
21.
go back to reference Troxel AB, Ma G, Heitjan DF: An index of local sensitivity to nonignorability. Stat Sin. 2004, 14: 1221-1237. Troxel AB, Ma G, Heitjan DF: An index of local sensitivity to nonignorability. Stat Sin. 2004, 14: 1221-1237.
22.
go back to reference Larsen C, Bousquet V, Delarocque-Astagneau E, Pioche C, Roudot-Thoraval F, Desenclos JC: Hepatitis C virus genotype 3 and the risk of severe liver disease in a large population of drug users in France. J Med Virol. 2010, 82: 1647-1654. 10.1002/jmv.21850.CrossRefPubMed Larsen C, Bousquet V, Delarocque-Astagneau E, Pioche C, Roudot-Thoraval F, Desenclos JC: Hepatitis C virus genotype 3 and the risk of severe liver disease in a large population of drug users in France. J Med Virol. 2010, 82: 1647-1654. 10.1002/jmv.21850.CrossRefPubMed
23.
go back to reference Delarocque-Astagneau E, Roudot-Thoraval F, Campese C, Desenclos JC, The Hepatitis CSSS: Past excessive alcohol consumption: a major determinant of severe liver disease among newly referred hepatitis C virus infected patients in hepatology reference centers, France, 2001. Ann Epidemiol. 2005, 15: 551-557. 10.1016/j.annepidem.2004.12.006.CrossRefPubMed Delarocque-Astagneau E, Roudot-Thoraval F, Campese C, Desenclos JC, The Hepatitis CSSS: Past excessive alcohol consumption: a major determinant of severe liver disease among newly referred hepatitis C virus infected patients in hepatology reference centers, France, 2001. Ann Epidemiol. 2005, 15: 551-557. 10.1016/j.annepidem.2004.12.006.CrossRefPubMed
24.
go back to reference Schuppan D: Afdhal NH: Liver cirrhosis. Lancet. 2008, 371: 838-851.PubMed Schuppan D: Afdhal NH: Liver cirrhosis. Lancet. 2008, 371: 838-851.PubMed
25.
go back to reference Lee KJ, Carlin JB: Multiple imputation for missing data: fully conditional specification versus multivariate normal imputation. Am J Epidemiol. 2010, 171: 624-632. 10.1093/aje/kwp425.CrossRefPubMed Lee KJ, Carlin JB: Multiple imputation for missing data: fully conditional specification versus multivariate normal imputation. Am J Epidemiol. 2010, 171: 624-632. 10.1093/aje/kwp425.CrossRefPubMed
26.
go back to reference Van Buuren S, Boshuizen HC, Knook DL: Multiple imputation of missing blood pressure covariates in survival analysis. Stat Med. 1999, 18: 681-694. 10.1002/(SICI)1097-0258(19990330)18:6<681::AID-SIM71>3.0.CO;2-R.CrossRefPubMed Van Buuren S, Boshuizen HC, Knook DL: Multiple imputation of missing blood pressure covariates in survival analysis. Stat Med. 1999, 18: 681-694. 10.1002/(SICI)1097-0258(19990330)18:6<681::AID-SIM71>3.0.CO;2-R.CrossRefPubMed
27.
go back to reference Van Buuren S: Multiple imputation of discrete and continuous data by fully conditional specification. Stat Methods Med Res. 2007, 16: 219-242. 10.1177/0962280206074463.CrossRefPubMed Van Buuren S: Multiple imputation of discrete and continuous data by fully conditional specification. Stat Methods Med Res. 2007, 16: 219-242. 10.1177/0962280206074463.CrossRefPubMed
28.
go back to reference White IR, Carpenter J, Evans S, Schroter S: Eliciting and using expert opinions about dropout bias in randomized controlled trials. Clin Trials. 2007, 4: 125-139. 10.1177/1740774507077849.CrossRefPubMed White IR, Carpenter J, Evans S, Schroter S: Eliciting and using expert opinions about dropout bias in randomized controlled trials. Clin Trials. 2007, 4: 125-139. 10.1177/1740774507077849.CrossRefPubMed
29.
go back to reference Rubin DB: The calculation of posterior distributions by data augmentation (in discussion of Tanner MA, Wong WH). J Am Stat Assoc. 1987, 82: 543-546. Rubin DB: The calculation of posterior distributions by data augmentation (in discussion of Tanner MA, Wong WH). J Am Stat Assoc. 1987, 82: 543-546.
30.
go back to reference Carpenter JR, Kenward MG: Missing data in randomised clinical trials-a practical guide. Methodology Portfolio, Public Health, Epidemiology & Biostatistics, University of Birmingham, Edgbaston, Birmingham, UK. 2007 Carpenter JR, Kenward MG: Missing data in randomised clinical trials-a practical guide. Methodology Portfolio, Public Health, Epidemiology & Biostatistics, University of Birmingham, Edgbaston, Birmingham, UK. 2007
31.
go back to reference Pernanen K: Validity of survey data on alcohol use. 1974, New York: Wiley Pernanen K: Validity of survey data on alcohol use. 1974, New York: Wiley
32.
go back to reference Knibbe R: Measuring drinking context. Alcohol Clin Exp Res. 1998, 22: 15S-20S. 10.1111/j.1530-0277.1998.tb04369.x.CrossRefPubMed Knibbe R: Measuring drinking context. Alcohol Clin Exp Res. 1998, 22: 15S-20S. 10.1111/j.1530-0277.1998.tb04369.x.CrossRefPubMed
33.
go back to reference Lemmens PH, Tan ES, Knibbe RA: Bias due to non-response in a Dutch survey on alcohol consumption. Br J Addict. 1988, 83: 1069-1077. 10.1111/j.1360-0443.1988.tb00534.x.CrossRefPubMed Lemmens PH, Tan ES, Knibbe RA: Bias due to non-response in a Dutch survey on alcohol consumption. Br J Addict. 1988, 83: 1069-1077. 10.1111/j.1360-0443.1988.tb00534.x.CrossRefPubMed
34.
go back to reference Lahaut VM, Jansen HA, de MD Van, Garretsen HF: Non-response bias in a sample survey on alcohol consumption. Alcohol Alcohol. 2002, 37: 256-260.CrossRefPubMed Lahaut VM, Jansen HA, de MD Van, Garretsen HF: Non-response bias in a sample survey on alcohol consumption. Alcohol Alcohol. 2002, 37: 256-260.CrossRefPubMed
35.
go back to reference Van Oers JA, Bongers IM, van de Goor LA, Garretsen HF: Alcohol consumption, alcohol-related problems, problem drinking, and socioeconomic status. Alcohol Alcohol. 1999, 34: 78-88.CrossRefPubMed Van Oers JA, Bongers IM, van de Goor LA, Garretsen HF: Alcohol consumption, alcohol-related problems, problem drinking, and socioeconomic status. Alcohol Alcohol. 1999, 34: 78-88.CrossRefPubMed
36.
go back to reference Lewden C, Jacqmin-Gadda H, Vilde JL, Bricaire F, Waldner-Combernoux A, May T, Cuzin L, Lang JM, Leport C, Chene G: An example of nonrandom missing data for hepatitis C virus status in a prognostic study among HIV-infected patients. HIV Clin Trials. 2004, 5: 224-231. 10.1310/X9AE-5F0Y-AE92-G1NA.CrossRefPubMed Lewden C, Jacqmin-Gadda H, Vilde JL, Bricaire F, Waldner-Combernoux A, May T, Cuzin L, Lang JM, Leport C, Chene G: An example of nonrandom missing data for hepatitis C virus status in a prognostic study among HIV-infected patients. HIV Clin Trials. 2004, 5: 224-231. 10.1310/X9AE-5F0Y-AE92-G1NA.CrossRefPubMed
Metadata
Title
Practical considerations for sensitivity analysis after multiple imputation applied to epidemiological studies with incomplete data
Authors
Vanina Héraud-Bousquet
Christine Larsen
James Carpenter
Jean-Claude Desenclos
Yann Le Strat
Publication date
01-12-2012
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2012
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/1471-2288-12-73

Other articles of this Issue 1/2012

BMC Medical Research Methodology 1/2012 Go to the issue