Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress 2023 EHA Congress 2023 ASCO Annual Meeting
Clinical Collections
Advances in Alzheimer’s

At a glance: The ONWARDS insulin icodec trials

A round-up of the ONWARDS phase 3 clinical trials evaluating the novel weekly insulin icodec in people with diabetes.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2021

Open Access 01-12-2021 | Research article

Multiple imputation validation study: addressing unmeasured survey data in a longitudinal design

Authors: Claire A. Kolaja, Ben Porter, Teresa M. Powell, Rudolph P. Rull, for the Millennium Cohort Study Team

Published in: BMC Medical Research Methodology | Issue 1/2021

Login to get access

Abstract

Background

Questionnaires used in longitudinal studies may have questions added or removed over time for numerous reasons. Data missing completely at a follow-up survey is a unique issue for longitudinal studies. While such excluded questions lack information at one follow-up survey, they are collected at other follow-up surveys, and covariances observed at other follow-up surveys may allow for the recovery of the missing data. This study utilized data from a large longitudinal cohort study to assess the efficiency and feasibility of using multiple imputation (MI) to recover this type of information.

Methods

Millennium Cohort Study participants completed the 9-item Patient Health Questionnaire (PHQ) depression module at 2 time points (2004, 2007). The suicidal ideation item in the module was set to missing for the 2007 assessment. Several single-level MI models using different sets of predictors and forms of suicidal ideation were used to compare self-reported values and imputed values for this item in 2007. Additionally, associations with sleep duration and smoking status, which are related constructs, were compared between self-reported and imputed values of suicidal ideation.

Results

Among 63,028 participants eligible for imputation analysis, 4.05% reported suicidal ideation on the 2007 survey. The imputation models successfully identified suicidal ideation, with a sensitivity ranging between 34 and 66% and a positive predictive value between 36 and 42%. Specificity remained above 96% and negative predictive value above 97% for all imputed models. Similar associations were found for all imputation models on related constructs, though the dichotomous suicidal ideation imputed from the model using only PHQ depression items yielded estimates that were closest with the self-reported associations for all adjusted analyses.

Conclusions

Although sensitivity and positive predictive value were relatively low, applying MI techniques allowed for inclusion of an otherwise missing variable. Additionally, correlations with related constructs were estimated near self-reported values. Therefore, the other 8 depression items can be used to estimate suicidal ideation that was completely missing from a survey using MI. However, these imputed values should not be used to estimate population prevalence.

Literature
  1. Laird NM. Missing data in longitudinal studies. Stat Med. 1988;7(1–2):305–15.PubMedView Article
  2. Little RJ, Rubin DB. Statistical analysis with missing data. 3rd Ed. Hoboken: Wiley, Inc.; 2019.
  3. Bell ML, Fairclough DL. Practical and statistical issues in missing data for longitudinal patient-reported outcomes. Stat Methods Med Res. 2014;23(5):440–59.PubMedView Article
  4. Kroenke K, Spitzer RL. The PHQ-9: a new depression diagnostic and severity measure. Psychiatr Ann. 2002;32(9):509–15.View Article
  5. Reeves, W.C., et al., Mental illness surveillance among adults in the United States. 2011.
  6. Kroenke K, et al. The patient health questionnaire somatic, anxiety, and depressive symptom scales: a systematic review. Gen Hosp Psychiatry. 2010;32(4):345–59.PubMedView Article
  7. Tomitaka S, et al. Distributional patterns of item responses and total scores on the PHQ-9 in the general population: data from the National Health and nutrition examination survey. BMC psychiatry. 2018;18(1):108.PubMedPubMed CentralView Article
  8. Wells TS, et al. A comparison of the PRIME-MD PHQ-9 and PHQ-8 in a large military prospective study, the millennium cohort study. J Affect Disord. 2013;148(1):77–83.PubMedView Article
  9. Simon GE, et al. Does response on the PHQ-9 Depression Questionnaire predict subsequent suicide attempt or suicide death? Psychiatr Serv. 2013;64(12):1195–202.PubMedPubMed CentralView Article
  10. Reinhard MJ, Bloeser KJ. Enhanced survey methods to clinically assess suicide risk. Psychiatr Serv. 2014;65(9):1179.PubMedView Article
  11. Rubin DB. Multiple imputation for nonresponse in surveys. Hoboken: Wiley, Inc.; 2004.
  12. Schafer JL, Olsen MK. Multiple imputation for multivariate missing-data problems: a data analyst's perspective. Multivar Behav Res. 1998;33(4):545–71.View Article
  13. Jeličić H, Phelps E, Lerner RM. Use of missing data methods in longitudinal studies: the persistence of bad practices in developmental psychology. Dev Psychol. 2009;45(4):1195.PubMedView Article
  14. Huque MH, et al. A comparison of multiple imputation methods for missing data in longitudinal studies. BMC Med Res Methodol. 2018;18(1):168.PubMedPubMed CentralView Article
  15. Sterne JA, et al. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. Bmj. 2009;338:b2393.PubMedPubMed CentralView Article
  16. Van Buuren S. Multiple imputation of discrete and continuous data by fully conditional specification. Stat Methods Med Res. 2007;16(3):219–42.PubMedView Article
  17. Jolani S, et al. Imputation of systematically missing predictors in an individual participant data meta-analysis: a generalized approach using MICE. Stat Med. 2015;34(11):1841–63.PubMedView Article
  18. Ryan MA, et al. Millennium cohort: enrollment begins a 21-year contribution to understanding the impact of military service. J Clin Epidemiol. 2007;60(2):181–91.PubMedView Article
  19. Blanchard EB, et al. Psychometric properties of the PTSD checklist (PCL). Behav Res Ther. 1996;34(8):669–73.PubMedView Article
  20. Smith TC, et al. Reliability of standard health assessment instruments in a large, population-based cohort study. Ann Epidemiol. 2007;17(7):525–32.PubMedView Article
  21. American Psychiatric Association. Diagnostic and statistical manual of mental disorders 4th ed. DSM-IV. Washington, DC: American Psychiatric Association; 1994.
  22. Kazis LE, et al. Improving the response choices on the veterans SF-36 health survey role functioning scales: results from the veterans health study. J Ambulatory Care Management. 2004;27(3):263–80.View Article
  23. Kazis LE, et al. Measurement comparisons of the medical outcomes study and veterans SF-36® health survey. Health Care Financing Review. 2004;25(4):43.PubMedPubMed Central
  24. Jacobson IG, et al. Alcohol use and alcohol-related problems before and after military combat deployment. Jama. 2008;300(6):663–75.PubMedPubMed CentralView Article
  25. Jakupcak M, et al. PTSD symptom clusters in relationship to alcohol misuse among Iraq and Afghanistan war veterans seeking post-deployment VA health care. Addict Behav. 2010;35(9):840–3.PubMedView Article
  26. Williams EC, et al. Risk factors for relapse to problem drinking among current and former US military personnel: a prospective study of the millennium cohort. Drug Alcohol Depend. 2015;148:93–101.PubMedView Article
  27. Hirshkowitz M, et al. National Sleep Foundation’s sleep time duration recommendations: methodology and results summary. Sleep health. 2015;1(1):40–3.PubMedView Article
  28. Pietrzak RH, et al. Risk and protective factors associated with suicidal ideation in veterans of operations enduring freedom and Iraqi freedom. J Affect Disord. 2010;123(1–3):102–7.PubMedView Article
  29. LeardMann CA, et al. Risk factors associated with suicide in current and former US military personnel. JAMA. 2013;310(5):496–506.
  30. Reger MA, et al. Risk of suicide among US military service members following operation enduring freedom or operation Iraqi freedom deployment and separation from the US military. JAMA psychiatry. 2015;72(6):561–9..PubMedView Article
  31. Pease JL, Billera M, Gerard G. Military culture and the transition to civilian life: suicide risk and other considerations. Soc Work. 2015;61(1):83–6.View Article
  32. Grund, S., et al., Package ‘mitml’. 2019.
  33. Von Hippel PT. 8. How to impute interactions, squares, and other transformed variables. Sociol Methodol. 2009;39(1):265–91.View Article
  34. Nguyen CD, Carlin JB, Lee KJ. Model checking in multiple imputation: an overview and case study. Emerging Themes Epidemiol. 2017;14(1):8.View Article
  35. Clarke DE, et al. Increased risk of suicidal ideation in smokers and former smokers compared to never smokers: evidence from the Baltimore ECA follow-up study. Suicide Life Threat Behav. 2010;40(4):307–18.PubMedPubMed CentralView Article
  36. Goodwin RD, Marusic A. Association between short sleep and suicidal ideation and suicide attempt among adults in the general population. Sleep. 2008;31(8):1097–101.PubMedPubMed Central
  37. Glozier N, et al. Short sleep duration in prevalent and persistent psychological distress in young adults: the DRIVE study. Sleep. 2010;33(9):1139–45.PubMedPubMed CentralView Article
  38. Wong HB, Lim GH. Measures of diagnostic accuracy: sensitivity, specificity, PPV and NPV. Proceedings of Singapore Healthcare. 2011;20(4):316–8.View Article
Metadata
Title
Multiple imputation validation study: addressing unmeasured survey data in a longitudinal design
Authors
Claire A. Kolaja
Ben Porter
Teresa M. Powell
Rudolph P. Rull
for the Millennium Cohort Study Team
Publication date
01-12-2021
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2021
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-020-01158-w

Other articles of this Issue 1/2021

BMC Medical Research Methodology 1/2021 Go to the issue

Research

Exploring the impact of ineligibility on individuals expressing interest in a trial aimed at improving daily functioning regarding perceptions of self, research and likelihood of future participation

Research article

Missing not at random in end of life care studies: multiple imputation and sensitivity analysis on data from the ACTION study

Research article

A Bayesian latent class mixture model with censoring for correlation analysis in antimicrobial resistance across populations

Research

Inspection plan for COVID-19 patients for Weibull distribution using repetitive sampling under indeterminacy

Research article

Using random forests to model 90-day hometime in people with stroke

Research

Modelling of longitudinal data to predict cardiovascular disease risk: a methodological review