Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2017

Open Access 01-12-2017 | Research article

When and how should multiple imputation be used for handling missing data in randomised clinical trials – a practical guide with flowcharts

Authors: Janus Christian Jakobsen, Christian Gluud, Jørn Wetterslev, Per Winkel

Published in: BMC Medical Research Methodology | Issue 1/2017

Login to get access

Abstract

Background

Missing data may seriously compromise inferences from randomised clinical trials, especially if missing data are not handled appropriately. The potential bias due to missing data depends on the mechanism causing the data to be missing, and the analytical methods applied to amend the missingness. Therefore, the analysis of trial data with missing values requires careful planning and attention.

Methods

The authors had several meetings and discussions considering optimal ways of handling missing data to minimise the bias potential. We also searched PubMed (key words: missing data; randomi*; statistical analysis) and reference lists of known studies for papers (theoretical papers; empirical studies; simulation studies; etc.) on how to deal with missing data when analysing randomised clinical trials.

Results

Handling missing data is an important, yet difficult and complex task when analysing results of randomised clinical trials. We consider how to optimise the handling of missing data during the planning stage of a randomised clinical trial and recommend analytical approaches which may prevent bias caused by unavoidable missing data. We consider the strengths and limitations of using of best-worst and worst-best sensitivity analyses, multiple imputation, and full information maximum likelihood. We also present practical flowcharts on how to deal with missing data and an overview of the steps that always need to be considered during the analysis stage of a trial.

Conclusions

We present a practical guide and flowcharts describing when and how multiple imputation should be used to handle missing data in randomised clinical.
Appendix
Available only for authorised users
Literature
1.
Groenwold RHH, Moons KGM, Vandenbroucke JP. Randomized trials with missing outcome data: how to analyze and what to report. CMAJ Can Med Assoc J. 2014;186(15):1153–7.CrossRef
2.
Nguyen TL, Collins GS, Lamy A, Devereaux PJ, Daures JP, Landais P, Le Manach Y. Simple randomization did not protect against bias in smaller trials. J Clin Epidemiol. 2017;84:105–13.CrossRefPubMed
3.
Little RJ, D'Agostino R, Cohen ML, Dickersin K, Emerson SS, Farrar JT, Frangakis C, Hogan JW, Molenberghs G, Murphy SA, et al. The prevention and treatment of missing data in clinical trials. N Engl J Med. 2012;367(14):1355–60.CrossRefPubMedPubMedCentral
4.
Sterne JAC, White IR, Carlin JB, Spratt M, Royston P, Kenward MG, Wood AM, Carpenter JR: Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ. 2009;338:157–60.
5.
Dziura JD, Post LA, Zhao Q, Fu Z, Peduzzi P. Strategies for dealing with missing data in clinical trials: from design to analysis. Yale J Biol Med. 2013;86(3):343–58.PubMedPubMedCentral
6.
7.
Skoog M, Saarimäki JM, Gluud C, Sheinin M, Erlendsson K, Aamdal S, et al. Transparency and registration in clinical research in the Nordic countries. Oslo: NordForsk: Nordic Trial Alliance; 2015. p. 1–108.
8.
Ebrahim S, Sohani ZN, Montoya L, Agarwal A, Thorlund K, Mills EJ, Ioannidis JP. Reanalyses of randomized clinical trial data. JAMA. 2014;312(10):1024–32.CrossRefPubMed
9.
Mayo-Wilson E, Fusco N, Li T, Hong H, Canner JK, Dickersin K, Investigators M. Multiple outcomes and analyses in clinical trials create challenges for interpretation and research synthesis. J Clin Epidemiol. 2017;86:39–50.CrossRefPubMed
10.
Mayo-Wilson E, Li T, Fusco N, Bertizzolo L, Canner JK, Cowley T, Doshi P, Ehmsen J, Gresham G, Guo N et al: Cherry-picking by trialists and meta-analysts can drive conclusions about intervention efficacy. J Clin Epidemiol. 2017;91:95–110.
11.
Kahan BC, Morris TP. Reporting and analysis of trials using stratified randomisation in leading medical journals: review and reanalysis. BMJ. 2012;345:e5840.CrossRefPubMedPubMedCentral
12.
Kahan BC, Morris TP. Improper analysis of trials randomised using stratified blocks or minimisation. Stat Med. 2011;31:328–40.CrossRefPubMed
13.
14.
Hróbjartsson A, Thomsen ASS, Emanuelsson F, Tendal B, Hilden J, Boutron I, Ravaud P, Brorson S. Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. BMJ. 2012;344:e1119.CrossRefPubMed
15.
Hróbjartsson A, Thomsen ASS, Emanuelsson F, Tendal B, Hilden J, Boutron I, Ravaud P, Brorson S. Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors. CMAJ. 2013;185(4):E201–11.CrossRefPubMedPubMedCentral
16.
Hrobjartsson A, Emanuelsson F, Skou Thomsen AS, Hilden J, Brorson S. Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub-studies. Int J Epidemiol. 2014;43(4):1272–83.CrossRefPubMedPubMedCentral
17.
18.
Zhang Y, Alyass A, Vanniyasingam T, Sadeghirad B, Florez ID, Pichika SC, Kennedy SA, Abdulkarimova U, Zhang Y, Iljon T, et al. A systematic survey of the methods literature on the reporting quality and optimal methods of handling participants with missing outcome data for continuous outcomes in randomized controlled trials. J Clin Epidemiol. 2017;88:67–80.CrossRefPubMed
19.
Jorgensen AW, Lundstrom LH, Wetterslev J, Astrup A, Gotzsche PC. Comparison of results from different imputation techniques for missing data from an anti-obesity drug trial. PLoS One. 2014;9(11):e111964.CrossRefPubMedPubMedCentral
20.
Garson GD: Missing Values Analysis and Data Imputation. North Carolina: Statistical Associates Publishers: Statistical Associates Publishers; 2015.
21.
Horton NJ, Lipsitz SR. Multiple imputation in practice: comparison of software packages for regression models with missing variables. Am Stat. 2001;55(3):244–54.CrossRef
22.
StataCorp. Stata: release 14. Statistical software. College Station, TX: StataCorp LP; 2015.
23.
Jakobsen JC, Gluud C, Winkel P, Lange T, Wetterslev J. The thresholds for statistical and clinical significance - a five-step procedure for evaluation of intervention effects in randomised clinical trials. BMC Med Res Methodol. 2014;14:34.CrossRefPubMedPubMedCentral
24.
Jakobsen JC, Wetterslev J, Winkel P, Lange T, Gluud C. Thresholds for statistical and clinical significance in systematic reviews with meta-analytic methods. BMC Med Res Methodol. 2014;14(1):120.CrossRefPubMedPubMedCentral
25.
Perner A, Haase N, Guttormsen AB, Tenhunen J, Klemenzson G, Aneman A, Madsen KR, Moller MH, Elkjaer JM, Poulsen LM, et al. Hydroxyethyl starch 130/0.42 versus Ringer's acetate in severe sepsis. N Engl J Med. 2012;367(2):124–34.CrossRefPubMed
26.
Clark TG, Altman DG. Developing a prognostic model in the presence of missing data: an ovarian cancer case study. J Clin Epidemiol. 2003;56(1):28–37.CrossRefPubMed
27.
Higgins JPT, Green S: The Cochrane Handbook for Systematic Reviews of Interventions, Version 5.1.0. The Cochrane Collaboration 2011, Available from www.​cochrane-handbook.​org. Accessed 1 Oct 2017.
28.
Allison PD. Handling missing data by maximum likelihood, statistical horizons. In: SAS global forum 2012 statistics and data analysis; 2012.
29.
StataCorp.: STATA Multiple imputation. Reference manual 2013, Release 13.
30.
U.S. Department of Health and Human Services Food and Drug Administration. Guidance for industry E9 statistical principles for clinical trials. In: Center for Biologics Evaluation and Research (CBER); 1998.
31.
32.
33.
Birhanu T, Molenberghs G, Sotto C, Kenward MG. Doubly robust and multiple-imputation-based generalized estimating equations. J Biopharm Stat. 2011;21(2):202–25.CrossRefPubMed
Metadata
Title
When and how should multiple imputation be used for handling missing data in randomised clinical trials – a practical guide with flowcharts
Authors
Janus Christian Jakobsen
Christian Gluud
Jørn Wetterslev
Per Winkel
Publication date
01-12-2017
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2017
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-017-0442-1

Other articles of this Issue 1/2017

BMC Medical Research Methodology 1/2017 Go to the issue

Research article

Predicting the multi-domain progression of Parkinson’s disease: a Bayesian multivariate generalized linear mixed-effect model

Research Article

A DAG-based comparison of interventional effect underestimation between composite endpoint and multi-state analysis in cardiovascular trials

Research article

Generating evidence on a risk-based monitoring approach in the academic setting – lessons learned

Research article

Measuring the burden of treatment for chronic disease: implications of a scoping review of the literature

Research article

Methodologic considerations in the design and analysis of nested case-control studies: association between cytokines and postoperative delirium

Research article

Patients’ beliefs regarding informed consent for low-risk pragmatic trials