Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2019

Open Access 01-12-2019 | Plasmodium Falciparum | Research article

Dealing with indeterminate outcomes in antimalarial drug efficacy trials: a comparison between complete case analysis, multiple imputation and inverse probability weighting

Authors: Prabin Dahal, Kasia Stepniewska, Philippe J. Guerin, Umberto D’Alessandro, Ric N. Price, Julie A. Simpson

Published in: BMC Medical Research Methodology | Issue 1/2019

Login to get access

Abstract

Background

Antimalarial clinical efficacy studies for uncomplicated Plasmodium falciparum malaria frequently encounter situations in which molecular genotyping is unable to discriminate between parasitic recurrence, either new infection or recrudescence. The current WHO guideline recommends excluding these individuals with indeterminate outcomes in a complete case (CC) analysis. Data from the four artemisinin-based combination (4ABC) trial was used to compare the performance of multiple imputation (MI) and inverse probability weighting (IPW) against the standard CC analysis for dealing with indeterminate recurrences.

Methods

3369 study participants from the multicentre study (4ABC trial) with molecularly defined parasitic recurrence treated with three artemisinin-based combination therapies were used to represent a complete dataset. A set proportion of recurrent infections (10, 30 and 45%) were reclassified as missing using two mechanisms: a completely random selection (mechanism 1); missingness weakly dependent (mechanism 2a) and strongly dependent (mechanism 2b) on treatment and transmission intensity. The performance of MI, IPW and CC approaches in estimating the Kaplan-Meier (K-M) probability of parasitic recrudescence at day 28 was then compared. In addition, the maximum likelihood estimate of the cured proportion was presented for further comparison (analytical solution). Performance measures (bias, relative bias, standard error and coverage) were reported as an average from 1000 simulation runs.

Results

The CC analyses resulted in absolute underestimation of K-M probability of day 28 recrudescence by up to 1.7% and were associated with reduced precision and poor coverage across all the scenarios studied. Both MI and IPW method performed better (greater consistency and greater efficiency) compared to CC analysis. In the absence of censoring, the analytical solution provided the most consistent and accurate estimate of cured proportion compared to the CC analyses.

Conclusions

The widely used CC approach underestimates antimalarial failure; IPW and MI procedures provided efficient and consistent estimates and should be considered when reporting the results of antimalarial clinical trials, especially in areas of high transmission, where the proportion of indeterminate outcomes could be large. The analytical solution estimating the cured proportion could provide an alternative approach, in scenarios with minimal censoring due to loss to follow-up or new infections.
Appendix
Available only for authorised users
Literature
1.
2.
World Health Organization. Methods for Surveillance of Antimalarial Drug Efficacy. Geneva; 2009.
3.
Heitjan DF, Little RJA. Multiple imputation for the fatal accident reporting system. J R Stat Soc Stat. 1991;40:13–29.
4.
Donders ART, van der Heijden GJMG, Stijnen T, Moons KGM. Review: a gentle introduction to imputation of missing values. J Clin Epidemiol. 2006;59:1087–91.CrossRefPubMed
5.
Greenland S, Finkle WD. A critical look at methods for handling missing covariates in epidemiologic regression analyses. Am J Epidemiol. 1995;142:1255–64.CrossRefPubMed
6.
Little RJ, Rubin DB. Chapter 13: Models for partially classified Contigency tables, ignoring the missing-data mechanism. In Statistical Analysis with Missing Data. Wiley Series in Probability and Statistics. 2002. pp. 266-278.
7.
Horvitz DG, Thompson D. A generalization of sampling without replacement from a finite universe. J Am Stat Assoc. 1952;44:663–85.CrossRef
8.
Sterne JAC, White IR, Carlin JB, Spratt M, Royston P, Kenward MG, et al. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ. 2009;338:b2393–3.
9.
Seaman SR, White IR. Review of inverse probability weighting for dealing with missing data. Stat Methods Med Res. 2013;22:278–95.CrossRefPubMed
10.
Hayati Rezvan P, Lee KJ, Simpson JA. The rise of multiple imputation: a review of the reporting and implementation of the method in medical research. BMC Med Res Methodol. 2015;15:30.CrossRefPubMedPubMedCentral
11.
White IR, Royston P, Wood AM. Multiple imputation using chained equations: issues and guidance for practice. Stat Med. 2011;30:377–99.CrossRefPubMed
12.
13.
Rubin DB. Basic ideas of multiple imputation for nonresponse. Surv Methodol. 1986;12:37–47.
14.
Lee KJ, Simpson JA. Introduction to multiple imputation for dealing with missing data. Respirology. 2014;19:162–7.CrossRefPubMed
15.
White IR, Carlin JB. Bias and efficiency of multiple imputation compared with complete-case analysis for missing covariate values. Stat Med. 2010;29:2920–31.CrossRefPubMed
16.
Machekano R, Dorsey G, Hubbard A. Efficacy studies of malaria treatments in Africa: efficient estimation with missing indicators of failure. Stat Methods Med Res. 2007;17:191–206.CrossRefPubMed
17.
Mukaka M, White SA, Terlouw DJ, Mwapasa V, Kalilani-Phiri L, Faragher EB. Is using multiple imputation better than complete case analysis for estimating a prevalence (risk) difference in randomized controlled trials when binary outcome observations are missing? Trials. 2016;17:341.CrossRefPubMedPubMedCentral
18.
The PREGACT Study Group. Four artemisinin-based treatments in African pregnant women with malaria. N Engl J Med. 2016;374:913–27.CrossRef
19.
20.
Brand JPL, van Buuren S, Groothuis-Oudshoorn K, Gelsema ES. A toolkit in SAS for the evaluation of multiple imputation methods. Statistica Neerlandica. 2003;57:36–45.CrossRef
21.
Rodwell L, Lee KJ, Romaniuk H, Carlin JB. Comparison of methods for imputing limited-range variables: a simulation study. BMC Med Res Methodol. 2014;14:1–11.CrossRef
22.
Rombach I, Gray AM, Jenkinson C, Murray DW, Rivero-Arias O. Multiple imputation for patient reported outcome measures in randomised controlled trials: advantages and disadvantages of imputing at the item, subscale or composite score level. BMC med res Methodol. BMC Med Res Methodol. 2018;18:1–16.CrossRef
23.
Gething PW, Patil AP, Smith DL, Guerra CA, Elyazar IRF, Johnston GL, et al. A new world malaria map: plasmodium falciparum endemicity in 2010. Malar J. 2011;10:378.CrossRefPubMedPubMedCentral
24.
Shaukat AM, Gilliams EA, Kenefic LJ, Laurens MB, Dzinjalamala FK, Nyirenda OM, et al. Clinical manifestations of new versus recrudescent malaria infections following anti-malarial drug treatment. Malar J. 2012;11:207.CrossRefPubMedPubMedCentral
25.
The WorldWide Antimalarial Resistance Network (WWARN) DP Study Group. The Effect of Dosing Regimens on the Antimalarial Efficacy of Dihydroartemisinin-Piperaquine: A Pooled Analysis of Individual Patient Data. PLoS Med. Public Libr Sci. 2013;10:1–17.CrossRef
26.
Worldwide Antimalarial Resistance Network (WWARN) AL Dose Impact Study Group. The effect of dose on the antimalarial efficacy of artemether–lumefantrine: a systematic review and pooled analysis of individual patient data. Lancet Infect Dis. 2015;15:692–702.CrossRef
27.
The WorldWide Antimalarial Resistance Network (WWARN) AS-AQ Study Group. The effect of dosing strategies on the therapeutic efficacy of artesunate-amodiaquine for uncomplicated malaria: a meta-analysis of individual patient data. BMC Med. 2015;13:66.CrossRef
28.
Collins LM, Schafer JL, Kam C-M. A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychol Methods. 2001;6:330–51.CrossRefPubMed
29.
Lee M, Cronin KA, Gail MH, Dignam JJ, Feuer EJ. Multiple imputation methods for nonparametric inference on cumulative incidence with missing cause of failure. Biom J. 2011;6:974–93.CrossRef
30.
Lee M, Dignam JJ, Han J. Multiple imputation methods for nonparametric inference on cumulative incidence with missing cause of failure. Stat Med. 2014;33:4605–26.CrossRefPubMedPubMedCentral
31.
Bodner TE. What improves with increased missing data imputations? Struct Equ Model A Multidiscip J. 2008;15:651–75.CrossRef
32.
Molenberghs G, Kenward MG. 9. Multiple Imputation. Missing Data Clin Stud. First, vol. 108. Chichester: John Wiley & Sons, Inc; 2007.CrossRef
33.
van Buuren S. Flexible imputation of missing data. Ser: Chapman Hall/CRC Interdiscip. Stat; 2012.CrossRef
34.
35.
36.
White IR. Simsum: analyses of simulation studies including Monte Carlo error. Stata J. 2010;10:369–85.CrossRef
37.
38.
Harel O, Zhou X-H. Multiple imputation: review of theory, implementation and software. Stat Med. 2007;26:3057–77.CrossRefPubMed
39.
Allison PD. Multiple imputation for missing data: a cautionary tale. Sociol Methods Res. 2000;28:301–9.CrossRef
40.
Carpenter JR, Kenward MG, Vansteelandt S. A comparison of multiple imputation and inverse probability weighting for analyses with missing data. J R Stat Soc Ser A. 2006;169:571–84.CrossRef
41.
Dondorp A, Nosten F, Yi P, Das D, Phyo AP, Tarning J, et al. Artemisinin Resistance in Plasmodium falciparum Malaria. N Engl J Med. 2009;361:391.CrossRef
42.
Greenhouse B, Dokomajilar C, Hubbard A, Rosenthal PJ, Dorsey G. Impact of transmission intensity on the accuracy of genotyping to distinguish recrudescence from new infection in antimalarial clinical trials. Antimicrob Agents Chemother. 2007;51:3096–103.CrossRefPubMedPubMedCentral
43.
Juliano JJ, Gadalla N, Sutherland CJ, Meshnick SR. The perils of PCR: can we accurately “correct” antimalarial trials? Trends Parasitol. 2010;26:119–24.CrossRefPubMedPubMedCentral
44.
45.
Messerli C, Hofmann NE, Beck H-P, Felger I. Critical evaluation of molecular monitoring in malaria drug efficacy trials: pitfalls of length polymorphic markers. Antimicrob Agents Chemother. 2017;61:e01500–16.CrossRefPubMed
46.
Graham JW. Missing data analysis: making it work in the real world. Annu Rev Psychol. 2009;60:549–76.CrossRefPubMed
47.
Klein JP, Andersen PK. Regression modeling of competing risks data based on pseudovalues of the cumulative incidence function. Biometrics. 2005;61:223–9.CrossRefPubMed
48.
Klein JP, Gerster M, Andersen PK, Tarima S, Perme MP. SAS and R functions to compute pseudo-values for censored data regression. Comput Methods Prog Biomed. 2008;89:289–300.CrossRef
49.
Bakoyannis G, Siannis F, Touloumi G. Modelling competing risks data with missing cause of failure. Stat Med. 2010;29:3172–85.CrossRefPubMed
50.
Moreno-Betancur M, Latouche A. Regression modeling of the cumulative incidence function with missing causes of failure using pseudo-values. Stat Med. 2013;32:3206–23.CrossRefPubMed
51.
Dahal P, Simpson JA, Dorsey G, Guérin PJ, Price RN, Stepniewska K. Statistical methods to derive efficacy estimates of anti-malarials for uncomplicated plasmodium falciparum malaria: pitfalls and challenges. Malar J. 2017;16:430.CrossRefPubMedPubMedCentral
52.
Dahal P, Guerin PJ, Price RN, Simpson JA. Stepniewska K. Evaluating antimalarial efficacy in single-armed and comparative drug trials using competing risk survival analysis : a simulation study. BMC Med Res Methodol. 2019;19:107.CrossRefPubMedPubMedCentral
53.
The WorldWide Antimalarial Resistance Network Methodology Study Group. Competing risk events in antimalarial drug trials in uncomplicated plasmodium falciparum malaria: a WorldWide antimalarial resistance network individual participant data meta-analysis. Malar J. 2019;18:1–14.CrossRef
54.
Wittes J. Missing inaction: preventing missing outcome data in randomized clinical trials. J Biopharm Stat. 2009;19:957–68.CrossRefPubMed
55.
White IR, Horton NJ, Carpenter J, Pocock SJ. Strategy for intention to treat analysis in randomised trials with missing outcome data. Br Med J. 2011;342:910–2.
56.
Little RJ, Agostino RD, Cohen ML, Dickersin K, Emerson SS, Farrar JT, et al. The prevention and treatment of missing data in clinical trials. N Engl J Med. 2012;367:1355–60.CrossRefPubMedPubMedCentral
57.
Little RJ, Cohen ML, Dickersin K, Emerson SS, Farrar JT, Neaton JD, et al. The design and conduct of clinical trials to limit missing data. Stat Med. 2012;31:3433–43.CrossRefPubMedPubMedCentral
58.
Lourens C, Lindegardh N, Barnes KI, Guerin PJ, Sibley CH, White NJ, et al. Benefits of a pharmacology antimalarial reference standard and proficiency testing program provided by the Worldwide antimalarial resistance network (WWARN). Antimicrob Agents Chemother. 2014;58:3889–94.CrossRefPubMedPubMedCentral
59.
Mustillo S. The effects of auxiliary variables on coefficient bias and efficiency in multiple imputation. Sociol Methods Res. 2012;41:335–61.CrossRef
60.
Cattamanchi A, Kyabayinze D, Hubbard A, Rosenthal PJ, Dorsey G. Distinguishing recrudescence from reinfection in a longitudinal antimalarial drug efficacy study: comparison of results based on genotyping of MSP-1, MSP-2, and GLURP. Am J Trop Med Hyg. 2003;68:133–9.CrossRefPubMed
61.
Rouse P, Mkulama MAP, Thuma PE, Mharakurwa S. Distinction of plasmodium falciparum recrudescence and re-infection by MSP2 genotyping: a caution about unstandardized classification criteria. Malar J. 2008;7:185.CrossRefPubMedPubMedCentral
62.
Plucinski MM, Dimbu PR, Macaia AP, Ferreira CM, Samutondo C, Quivinja J, et al. Efficacy of artemether-lumefantrine, artesunate-amodiaquine, and dihydroartemisinin-piperaquine for treatment of uncomplicated plasmodium falciparum malaria in Angola, 2015. Malar J BioMed Central. 2017;16:1–10.
Metadata
Title
Dealing with indeterminate outcomes in antimalarial drug efficacy trials: a comparison between complete case analysis, multiple imputation and inverse probability weighting
Authors
Prabin Dahal
Kasia Stepniewska
Philippe J. Guerin
Umberto D’Alessandro
Ric N. Price
Julie A. Simpson
Publication date
01-12-2019
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2019
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-019-0856-z

Other articles of this Issue 1/2019

BMC Medical Research Methodology 1/2019 Go to the issue

Research article

A review and synthesis of frameworks for engagement in health research to identify concepts of knowledge user engagement

Research article

Methodology and reporting characteristics of studies using interrupted time series design in healthcare

Research article

Barriers and recruitment strategies for precarious status migrants in Montreal, Canada

Research article

A methodological systematic review of meta-ethnography conduct to articulate the complex analytical phases

Research article

Ranking hospital performance based on individual indicators: can we increase reliability by creating composite indicators?

Research article

An application of a pattern-mixture model with multiple imputation for the analysis of longitudinal trials with protocol deviations