Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2019

Open Access 01-12-2019 | Research article

Imputation strategies when a continuous outcome is to be dichotomized for responder analysis: a simulation study

Authors: Lysbeth Floden, Melanie L. Bell

Published in: BMC Medical Research Methodology | Issue 1/2019

Login to get access

Abstract

Background

In many clinical trials continuous outcomes are dichotomized to compare proportions of patients who respond. A common and recommended approach to handling missing data in responder analysis is to impute as non-responders, despite known biases. Multiple imputation is another natural choice but when a continuous outcome is ultimately dichotomized, the specifications of the imputation model come into question. Practitioners can either impute the missing outcome before dichotomizing or dichotomize then impute. In this study we compared multiple imputation of the continuous and dichotomous forms of the outcome, and imputing responder status as non-response in responder analysis.

Methods

We simulated four response profiles representing a two-arm randomized controlled trial with a continuous outcome at four time points. We omitted data using six missing at random mechanisms, and imputed missing observations three ways: 1) replacing as non-responder; 2) multiply imputing before dichotomizing; and 3) multiply imputing the dichotomized response. Imputation models included the continuous response at all timepoints, and additional auxiliary variables for some scenarios. We assessed bias, power, coverage of the 95% confidence interval, and type 1 error. Finally, we applied these methods to a longitudinal trial for patients with major depressive disorder.

Results

Both forms of multiple imputation performed better than non-response imputation in terms of bias and type 1 error. When approximately 30% of responses were missing, bias was less than 7.3% for multiple imputation scenarios but when 50% of responses were missing, imputing before dichotomizing generally had lower bias compared to dichotomizing before imputing. Non-response imputation resulted in biased estimates, both underestimates and overestimates. In the example trial data, non-response imputation estimated a smaller difference in proportions than multiply imputed approaches.

Conclusions

With moderate amounts of missing data, multiply imputing the continuous outcome variable prior to dichotomizing performed similar to multiply imputing the binary responder status. With higher rates of missingness, multiply imputing the continuous variable was less biased and had well-controlled coverage probabilities of the 95% confidence interval compared to imputing the dichotomous response. In general, multiple imputation using the longitudinally measured continuous outcome in the imputation model performed better than imputing missing observations as non-responders.
Literature
1.
go back to reference Bell ML, Fiero M, Horton NJ, Hsu C-H. Handling missing data in RCTs; a review of the top medical journals. BMC Med Res Methodol. 2014;14:1–8.CrossRef Bell ML, Fiero M, Horton NJ, Hsu C-H. Handling missing data in RCTs; a review of the top medical journals. BMC Med Res Methodol. 2014;14:1–8.CrossRef
2.
go back to reference LaVange LM, Permutt T. A regulatory perspective on missing data in the aftermath of the NRC report. Stat Med Wiley-Blackwell. 2016;35:2853–64.CrossRef LaVange LM, Permutt T. A regulatory perspective on missing data in the aftermath of the NRC report. Stat Med Wiley-Blackwell. 2016;35:2853–64.CrossRef
3.
go back to reference Brundage M, Osoba D, Bezjak A, Tu D, Palmer M, Pater J. Lessons learned in the assessment of health-related quality of life: selected examples from the National Cancer Institute of Canada clinical trials group. J Clin Oncol American Society of Clinical Oncology. 2007:5078–81.CrossRef Brundage M, Osoba D, Bezjak A, Tu D, Palmer M, Pater J. Lessons learned in the assessment of health-related quality of life: selected examples from the National Cancer Institute of Canada clinical trials group. J Clin Oncol American Society of Clinical Oncology. 2007:5078–81.CrossRef
4.
go back to reference Moore AR, Straube S, Eccleston C, Derry S, Aldington D, Wiffen P, et al. Estimate at your peril: imputation methods for patient withdrawal can bias efficacy outcomes in chronic pain trials using responder analyses. Pain. 2012;153:265–8.CrossRef Moore AR, Straube S, Eccleston C, Derry S, Aldington D, Wiffen P, et al. Estimate at your peril: imputation methods for patient withdrawal can bias efficacy outcomes in chronic pain trials using responder analyses. Pain. 2012;153:265–8.CrossRef
5.
go back to reference Cordoba G, Schwartz L, Woloshin S, Bae H, Gøtzsche PC. Definition, reporting, and interpretation of composite outcomes in clinical trials: systematic review. BMJ British Medical Journal Publishing Group. 2010;341:c3920.CrossRef Cordoba G, Schwartz L, Woloshin S, Bae H, Gøtzsche PC. Definition, reporting, and interpretation of composite outcomes in clinical trials: systematic review. BMJ British Medical Journal Publishing Group. 2010;341:c3920.CrossRef
6.
go back to reference Ferreira-González I, Permanyer-Miralda G, Busse JW, Bryant DM, Montori VM, Alonso-Coello P, et al. Methodologic discussions for using and interpreting composite endpoints are limited, but still identify major concerns. J Clin Epidemiol Pergamon. 2007;60:651–7.CrossRef Ferreira-González I, Permanyer-Miralda G, Busse JW, Bryant DM, Montori VM, Alonso-Coello P, et al. Methodologic discussions for using and interpreting composite endpoints are limited, but still identify major concerns. J Clin Epidemiol Pergamon. 2007;60:651–7.CrossRef
7.
go back to reference Hall SM, Delucchi KL, Velicer WF, Kahler CW, Ranger-Moore J, Hedeker D, et al. Statistical analysis of randomized trials in tobacco treatment: longitudinal designs with dichotomous outcome. Nicotine Tob Res. 2001;3:193–202.CrossRef Hall SM, Delucchi KL, Velicer WF, Kahler CW, Ranger-Moore J, Hedeker D, et al. Statistical analysis of randomized trials in tobacco treatment: longitudinal designs with dichotomous outcome. Nicotine Tob Res. 2001;3:193–202.CrossRef
8.
go back to reference Hedeker D, Mermelstein RJ, Demirtas H. Analysis of binary outcomes with missing data: missing = smoking, last observation carried forward, and a little multiple imputation. Addiction. 2007;102:1564–73.CrossRef Hedeker D, Mermelstein RJ, Demirtas H. Analysis of binary outcomes with missing data: missing = smoking, last observation carried forward, and a little multiple imputation. Addiction. 2007;102:1564–73.CrossRef
9.
10.
go back to reference Molenberghs G, Thijs H, Jansen I, Beunckens C, Kenward MG, Mallinckrodt C, et al. Analyzing incomplete longitudinal clinical trial data. Biostatistics. 2004;5:445–64.CrossRef Molenberghs G, Thijs H, Jansen I, Beunckens C, Kenward MG, Mallinckrodt C, et al. Analyzing incomplete longitudinal clinical trial data. Biostatistics. 2004;5:445–64.CrossRef
11.
go back to reference Molenberghs G, Kenward MG. Missing data in clinical studies. Chichester, UK: John Wiley & Sons, Ltd; 2007.CrossRef Molenberghs G, Kenward MG. Missing data in clinical studies. Chichester, UK: John Wiley & Sons, Ltd; 2007.CrossRef
13.
go back to reference Bell ML, Fairclough DL. Practical and statistical issues in missing data for longitudinal patient reported outcomes. Stat Methods Med Res. 2013;625:1–20. Bell ML, Fairclough DL. Practical and statistical issues in missing data for longitudinal patient reported outcomes. Stat Methods Med Res. 2013;625:1–20.
14.
go back to reference Carpenter JR, Kenward MG. Missing data in randomised controlled trials — a practical guide. 2007;1–206. Carpenter JR, Kenward MG. Missing data in randomised controlled trials — a practical guide. 2007;1–206.
15.
go back to reference Rubin DB. Multiple imputation for nonresponse in surveys. Vol. 81. Wiley, 2004. Rubin DB. Multiple imputation for nonresponse in surveys. Vol. 81. Wiley, 2004.
16.
go back to reference Demirtas H. Practical advice on how to impute continuous data when the ultimate interest centers on dichotomized outcomes through pre-specified thresholds. Commun Stat Simul Comput Taylor & Francis Group. 2007;36:871–89.CrossRef Demirtas H. Practical advice on how to impute continuous data when the ultimate interest centers on dichotomized outcomes through pre-specified thresholds. Commun Stat Simul Comput Taylor & Francis Group. 2007;36:871–89.CrossRef
17.
go back to reference Yoo B. The impact of dichotomization in longitudinal data analysis: a simulation study. Pharm Stat Pharm Stat. 2010;9:298–312.CrossRef Yoo B. The impact of dichotomization in longitudinal data analysis: a simulation study. Pharm Stat Pharm Stat. 2010;9:298–312.CrossRef
18.
go back to reference Von Hippel PT. How to impute interactions, squares, and other transformed variables. Sociol Methodol. Wiley/Blackwell (10.1111); 2009;39:265–91. Von Hippel PT. How to impute interactions, squares, and other transformed variables. Sociol Methodol. Wiley/Blackwell (10.1111); 2009;39:265–91.
19.
go back to reference Seaman SR, Bartlett JW, White IR. Multiple imputation of missing covariates with non-linear effects and interactions: an evaluation of statistical methods. BMC Med Res Methodol BioMed Central. 2012;12:46.CrossRef Seaman SR, Bartlett JW, White IR. Multiple imputation of missing covariates with non-linear effects and interactions: an evaluation of statistical methods. BMC Med Res Methodol BioMed Central. 2012;12:46.CrossRef
20.
go back to reference Van Buuren S, Brand JPL, Groothuis-Oudshoorn CGM, Rubin DB. Fully conditional specification in multivariate imputation. J Stat Comput Simul Taylor & Francis. 2006;76:1049–64.CrossRef Van Buuren S, Brand JPL, Groothuis-Oudshoorn CGM, Rubin DB. Fully conditional specification in multivariate imputation. J Stat Comput Simul Taylor & Francis. 2006;76:1049–64.CrossRef
21.
go back to reference Azur MJ, Stuart EA, Frangakis C, Leaf PJ. Multiple imputation by chained equations: what is it and how does it work? Int J Methods Psychiatr Res NIH Public Access. 2011;20:40–9.CrossRef Azur MJ, Stuart EA, Frangakis C, Leaf PJ. Multiple imputation by chained equations: what is it and how does it work? Int J Methods Psychiatr Res NIH Public Access. 2011;20:40–9.CrossRef
22.
go back to reference White IR, Royston P, Wood AM. Multiple imputation using chained equations: issues and guidance for practice. Stat Med. 2011;30:377–99.CrossRef White IR, Royston P, Wood AM. Multiple imputation using chained equations: issues and guidance for practice. Stat Med. 2011;30:377–99.CrossRef
23.
go back to reference Bell ML, McKenzie JE. Designing psycho-oncology randomised trials and cluster randomised trials: variance components and intra-cluster correlation of commonly used psychosocial measures. Psychooncology. 2013;22:1738–47.CrossRef Bell ML, McKenzie JE. Designing psycho-oncology randomised trials and cluster randomised trials: variance components and intra-cluster correlation of commonly used psychosocial measures. Psychooncology. 2013;22:1738–47.CrossRef
24.
go back to reference Lipkovich I, Duan Y, Ahmed S. Multiple imputation compared with restricted pseudo-likelihood and generalized estimating equations for analysis of binary repeated measures in clinical studies. Pharm Stat Wiley-Blackwell. 2005;4:267–85.CrossRef Lipkovich I, Duan Y, Ahmed S. Multiple imputation compared with restricted pseudo-likelihood and generalized estimating equations for analysis of binary repeated measures in clinical studies. Pharm Stat Wiley-Blackwell. 2005;4:267–85.CrossRef
25.
go back to reference Fairclough D. Design and analysis of quality of life studies in clinical trials; 2010.CrossRef Fairclough D. Design and analysis of quality of life studies in clinical trials; 2010.CrossRef
26.
go back to reference Bell ML, Kenward MG, Fairclough DL, Horton NJ. Differential dropout and bias in randomised controlled trials: when it matters and when it may not. BMJ BMJ Group. 2013;346:e8668. Bell ML, Kenward MG, Fairclough DL, Horton NJ. Differential dropout and bias in randomised controlled trials: when it matters and when it may not. BMJ BMJ Group. 2013;346:e8668.
27.
go back to reference Leucht S, Corves C, Arbter D, Engel RR, Li C, Davis JM. Second-generation versus first-generation antipsychotic drugs for schizophrenia: a meta-analysis. Lancet. 2009;373:31–41.CrossRef Leucht S, Corves C, Arbter D, Engel RR, Li C, Davis JM. Second-generation versus first-generation antipsychotic drugs for schizophrenia: a meta-analysis. Lancet. 2009;373:31–41.CrossRef
28.
go back to reference Morris TP, White IR, Crowther MJ. Using simulation studies to evaluate statistical methods. Stat Med. 2019 Jan 16. Morris TP, White IR, Crowther MJ. Using simulation studies to evaluate statistical methods. Stat Med. 2019 Jan 16.
29.
30.
go back to reference Goldstein DJ, Lu Y, Detke MJ, Wiltse C, Mallinckrodt C, Demitrack MA. Duloxetine in the treatment of depression: a double-blind placebo-controlled comparison with paroxetine. J Clin Psychopharmacol. 2004;24:389–99.CrossRef Goldstein DJ, Lu Y, Detke MJ, Wiltse C, Mallinckrodt C, Demitrack MA. Duloxetine in the treatment of depression: a double-blind placebo-controlled comparison with paroxetine. J Clin Psychopharmacol. 2004;24:389–99.CrossRef
32.
go back to reference Yamaguchi Y, Misumi T, Maruo K. A comparison of multiple imputation methods for incomplete longitudinal binary data. J Biopharm Stat Taylor & Francis. 2017 Sep 8:1–23. Yamaguchi Y, Misumi T, Maruo K. A comparison of multiple imputation methods for incomplete longitudinal binary data. J Biopharm Stat Taylor & Francis. 2017 Sep 8:1–23.
33.
go back to reference Nelson DB, Partin MR, Fu SS, Joseph AM, An LC. Why assigning ongoing tobacco use is not necessarily a conservative approach to handling missing tobacco cessation outcomes. Nicotine Tob Res. 2009;11:77–83.CrossRef Nelson DB, Partin MR, Fu SS, Joseph AM, An LC. Why assigning ongoing tobacco use is not necessarily a conservative approach to handling missing tobacco cessation outcomes. Nicotine Tob Res. 2009;11:77–83.CrossRef
34.
go back to reference Mallinckrodt CH. Preventing and treating missing data in longitudinal clinical trials: a practical guide. Cambridge University Press, 2013. Mallinckrodt CH. Preventing and treating missing data in longitudinal clinical trials: a practical guide. Cambridge University Press, 2013.
Metadata
Title
Imputation strategies when a continuous outcome is to be dichotomized for responder analysis: a simulation study
Authors
Lysbeth Floden
Melanie L. Bell
Publication date
01-12-2019
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2019
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-019-0793-x

Other articles of this Issue 1/2019

BMC Medical Research Methodology 1/2019 Go to the issue