Skip to main content
Top
Published in: BMC Public Health 1/2012

Open Access 01-12-2012 | Research article

Attrition and generalizability in longitudinal studies: findings from a 15-year population-based study and a Monte Carlo simulation study

Authors: Kristin Gustavson, Tilmann von Soest, Evalill Karevold, Espen Røysamb

Published in: BMC Public Health | Issue 1/2012

Login to get access

Abstract

Background

Attrition is one of the major methodological problems in longitudinal studies. It can deteriorate generalizability of findings if participants who stay in a study differ from those who drop out. The aim of this study was to examine the degree to which attrition leads to biased estimates of means of variables and associations between them.

Methods

Mothers of 18-month-old children were enrolled in a population-based study in 1993 (N=913) that aimed to examine development in children and their families in the general population. Fifteen years later, 56% of the sample had dropped out. The present study examined predictors of attrition as well as baseline associations between variables among those who stayed and those who dropped out of that study. A Monte Carlo simulation study was also performed.

Results

Those who had dropped out of the study over 15 years had lower educational level at baseline than those who stayed, but they did not differ regarding baseline psychological and relationship variables. Baseline correlations were the same among those who stayed and those who later dropped out. The simulation study showed that estimates of means became biased even at low attrition rates and only weak dependency between attrition and follow-up variables. Estimates of associations between variables became biased only when attrition was dependent on both baseline and follow-up variables. Attrition rate did not affect estimates of associations between variables.

Conclusions

Long-term longitudinal studies are valuable for studying associations between risk/protective factors and health outcomes even considering substantial attrition rates.
Appendix
Available only for authorised users
Literature
1.
go back to reference Schaffer RH: Social Development. 1996, Blackwell Plublishers Inc., Oxford Schaffer RH: Social Development. 1996, Blackwell Plublishers Inc., Oxford
2.
go back to reference Tambs K, Ronning T, Prescott CA, Kendler KS, Reichborn-Kjennerud T, Torgersen S, et al: The Norwegian Institute of Public Health Twin Study of Mental Health: Examining Recruitment and Attrition Bias. Twin Res Hum Genet. 2009, 12: 158-168.CrossRefPubMedPubMedCentral Tambs K, Ronning T, Prescott CA, Kendler KS, Reichborn-Kjennerud T, Torgersen S, et al: The Norwegian Institute of Public Health Twin Study of Mental Health: Examining Recruitment and Attrition Bias. Twin Res Hum Genet. 2009, 12: 158-168.CrossRefPubMedPubMedCentral
3.
go back to reference Fischer EH, Dornelas EA, Goethe JW: Characteristics of people lost to attrition in psychiatric follow-up studies. J Nerv Ment Dis. 2001, 189: 49-55.CrossRefPubMed Fischer EH, Dornelas EA, Goethe JW: Characteristics of people lost to attrition in psychiatric follow-up studies. J Nerv Ment Dis. 2001, 189: 49-55.CrossRefPubMed
4.
go back to reference Bjerkeset O, Nordahl HM, Larsson S, Dahl AA, Linaker O: A 4-year follow-up study of syndromal and sub-syndromal anxiety and depression symptoms in the general population. Soc Psychiatry Psychiatr Epidemiol. 2008, 43: 192-199.CrossRefPubMed Bjerkeset O, Nordahl HM, Larsson S, Dahl AA, Linaker O: A 4-year follow-up study of syndromal and sub-syndromal anxiety and depression symptoms in the general population. Soc Psychiatry Psychiatr Epidemiol. 2008, 43: 192-199.CrossRefPubMed
5.
go back to reference Badawi MA, Eaton WW, Myllyluoma J, Weimer LG, Gallo J: Psychopathology and attrition in the Baltimore ECA 15-year follow-up 1981–1996. Soc Psychiatry Psychiatr Epidemiol. 1999, 34: 91-98.CrossRefPubMed Badawi MA, Eaton WW, Myllyluoma J, Weimer LG, Gallo J: Psychopathology and attrition in the Baltimore ECA 15-year follow-up 1981–1996. Soc Psychiatry Psychiatr Epidemiol. 1999, 34: 91-98.CrossRefPubMed
6.
go back to reference Goodman JS, Blum TC: Assessing the non-random sampling effects of subject attrition in longitudinal research. J Manage. 1996, 22: 627-652. Goodman JS, Blum TC: Assessing the non-random sampling effects of subject attrition in longitudinal research. J Manage. 1996, 22: 627-652.
7.
go back to reference Miller RB, Wright DW: Detecting and correcting attrition bias in longitudinal family research. J Marriage Fam. 1995, 57: 921-929.CrossRef Miller RB, Wright DW: Detecting and correcting attrition bias in longitudinal family research. J Marriage Fam. 1995, 57: 921-929.CrossRef
8.
go back to reference Nilsen RM, Vollset SE, Gjessing HK, Skjærven R, Melve KK, Schreuder P, et al: Self-selection and bias in a large prospective pregnancy cohort in Norway. Paediatr Perinat Epidemiol. 2009, 23: 597-608.CrossRefPubMed Nilsen RM, Vollset SE, Gjessing HK, Skjærven R, Melve KK, Schreuder P, et al: Self-selection and bias in a large prospective pregnancy cohort in Norway. Paediatr Perinat Epidemiol. 2009, 23: 597-608.CrossRefPubMed
9.
go back to reference Eaton WW, Anthony JC, Tepper S, Dryman A: Psychopathology and attrition in the epidemiologic catchment-area surveys. Am J Epidemiol. 1992, 135: 1051-1059.PubMed Eaton WW, Anthony JC, Tepper S, Dryman A: Psychopathology and attrition in the epidemiologic catchment-area surveys. Am J Epidemiol. 1992, 135: 1051-1059.PubMed
10.
go back to reference de Graaf R, Bijl RV, Smit F, Ravelli A, Vollebergh WAM: Psychiatric and sociodemographic predictors of attrition in a longitudinal study-The Netherlands Mental Health Survey and Incidence Study (NEMESIS). Am J Epidemiol. 2000, 152: 1039-1047.CrossRefPubMed de Graaf R, Bijl RV, Smit F, Ravelli A, Vollebergh WAM: Psychiatric and sociodemographic predictors of attrition in a longitudinal study-The Netherlands Mental Health Survey and Incidence Study (NEMESIS). Am J Epidemiol. 2000, 152: 1039-1047.CrossRefPubMed
11.
go back to reference Thygesen LC, Johansen C, Keiding N, Giovannucci E, Gronbaek M: Effects of sample attrition in a longitudinal study of the association between alcohol intake and all-cause mortality. Addiction. 2008, 103: 1149-1159.CrossRefPubMed Thygesen LC, Johansen C, Keiding N, Giovannucci E, Gronbaek M: Effects of sample attrition in a longitudinal study of the association between alcohol intake and all-cause mortality. Addiction. 2008, 103: 1149-1159.CrossRefPubMed
12.
go back to reference Torvik FA, Rognmo K, Tambs K: Alcohol use and mental distress as predictors of non-response in a general population health survey: the HUNT study. Soc Psychiatry Psychiatr Epidemiol. 2012, 47: 805-816.CrossRefPubMed Torvik FA, Rognmo K, Tambs K: Alcohol use and mental distress as predictors of non-response in a general population health survey: the HUNT study. Soc Psychiatry Psychiatr Epidemiol. 2012, 47: 805-816.CrossRefPubMed
13.
go back to reference Van Loon AJM, Tijhuis M, Picavet HSJ, Surtees PG, Ormel J: Survey non-response in the Netherlands: effects on prevalence estimates and associations. Ann Epidemiol. 2003, 13: 105-110.CrossRefPubMed Van Loon AJM, Tijhuis M, Picavet HSJ, Surtees PG, Ormel J: Survey non-response in the Netherlands: effects on prevalence estimates and associations. Ann Epidemiol. 2003, 13: 105-110.CrossRefPubMed
14.
go back to reference Allott K, Chanen A, Yuen HP: Attrition bias in longitudinal research involving adolescent psychiatric outpatients. J Nerv Ment Dis. 2006, 194: 958-961.CrossRefPubMed Allott K, Chanen A, Yuen HP: Attrition bias in longitudinal research involving adolescent psychiatric outpatients. J Nerv Ment Dis. 2006, 194: 958-961.CrossRefPubMed
15.
go back to reference Whisman MA: The association between depression and marital dissatisfaction. Marital and family processes in depression. A scientific foundation for clinical practice. Edited by: Beach SRH. 2001, American Psychological Association, Washington D.C, 3-24.CrossRef Whisman MA: The association between depression and marital dissatisfaction. Marital and family processes in depression. A scientific foundation for clinical practice. Edited by: Beach SRH. 2001, American Psychological Association, Washington D.C, 3-24.CrossRef
16.
go back to reference Cotter RB, Burke JD, Stouthamer-Loeber M, Loeber R: Contacting participants for follow-up: how much effort is required to retain participants in longitudinal studies?. Eval Program Plann. 2005, 28: 15-21.CrossRef Cotter RB, Burke JD, Stouthamer-Loeber M, Loeber R: Contacting participants for follow-up: how much effort is required to retain participants in longitudinal studies?. Eval Program Plann. 2005, 28: 15-21.CrossRef
17.
go back to reference Perez RG, Ezpeleta L, Domenech JM: Features associated with the non-participation and drop out by socially-at-risk children and adolescents in mental-health epidemiological studies. Soc Psychiatry Psychiatr Epidemiol. 2007, 42: 251-258.CrossRefPubMed Perez RG, Ezpeleta L, Domenech JM: Features associated with the non-participation and drop out by socially-at-risk children and adolescents in mental-health epidemiological studies. Soc Psychiatry Psychiatr Epidemiol. 2007, 42: 251-258.CrossRefPubMed
18.
go back to reference Caspi A, Moffitt TE, Newman DL, Silva PA: Behavioral observations at age 3 years predict adult psychiatric disorders - Longitudinal evidence from a birth cohort. Arch Gen Psychiatry. 1996, 53: 1033-1039.CrossRefPubMed Caspi A, Moffitt TE, Newman DL, Silva PA: Behavioral observations at age 3 years predict adult psychiatric disorders - Longitudinal evidence from a birth cohort. Arch Gen Psychiatry. 1996, 53: 1033-1039.CrossRefPubMed
19.
go back to reference Reinherz HZ, Giaconia RM, Hauf AMC, Wasserman MS, Silverman AB: Major depression in the transition to adulthood: Risks and impairments. J Abnorm Psychol. 1999, 108: 500-510.CrossRefPubMed Reinherz HZ, Giaconia RM, Hauf AMC, Wasserman MS, Silverman AB: Major depression in the transition to adulthood: Risks and impairments. J Abnorm Psychol. 1999, 108: 500-510.CrossRefPubMed
20.
go back to reference Schafer JL, Graham JW: Missing data: our view of the state of the art. Psychol Methods. 2002, 7: 147-177.CrossRefPubMed Schafer JL, Graham JW: Missing data: our view of the state of the art. Psychol Methods. 2002, 7: 147-177.CrossRefPubMed
21.
go back to reference Kristman VL, Manno M, Cote P: Methods to account for attrition in longitudinal data: do they work? A simulation study. Eur J Epidemiol. 2005, 20: 657-662.CrossRefPubMed Kristman VL, Manno M, Cote P: Methods to account for attrition in longitudinal data: do they work? A simulation study. Eur J Epidemiol. 2005, 20: 657-662.CrossRefPubMed
22.
go back to reference Muthén LK, Muthén BO: Mplus user’s guide. 2010, Muthén & Muthén, Los Angeles, 6 Muthén LK, Muthén BO: Mplus user’s guide. 2010, Muthén & Muthén, Los Angeles, 6
23.
go back to reference Paxton P, Curran PJ, Bollen KA, Kirby J, Chen FN: Monte Carlo experiments: design and implementation. Structural Equation Modeling-A Multidisciplinary J. 2001, 8: 287-312.CrossRef Paxton P, Curran PJ, Bollen KA, Kirby J, Chen FN: Monte Carlo experiments: design and implementation. Structural Equation Modeling-A Multidisciplinary J. 2001, 8: 287-312.CrossRef
24.
go back to reference Graham JW, Donaldson SI: Evaluating interventions with differential attrition-the importance of nonresponse mechanisms and use of follow-up data. J Appl Psychol. 1993, 78: 119-128.CrossRefPubMed Graham JW, Donaldson SI: Evaluating interventions with differential attrition-the importance of nonresponse mechanisms and use of follow-up data. J Appl Psychol. 1993, 78: 119-128.CrossRefPubMed
25.
go back to reference Soullier , et al: Multiple imputation for estimation of an occurrence rate in cohorts with attrition and discrete follow-up time points: a simulation study. BMC Medical Research Methodology. 2010, 10: 79-CrossRefPubMedPubMedCentral Soullier , et al: Multiple imputation for estimation of an occurrence rate in cohorts with attrition and discrete follow-up time points: a simulation study. BMC Medical Research Methodology. 2010, 10: 79-CrossRefPubMedPubMedCentral
26.
go back to reference Karevold E, Roysamb E, Ystrom E, Mathiesen KS: Predictors and pathways from infancy to symptoms of anxiety and depression in early adolescence. Dev Psychol. 2009, 45: 1051-1060.CrossRefPubMed Karevold E, Roysamb E, Ystrom E, Mathiesen KS: Predictors and pathways from infancy to symptoms of anxiety and depression in early adolescence. Dev Psychol. 2009, 45: 1051-1060.CrossRefPubMed
27.
go back to reference Mathiesen KS, Sanson A, Stoolmiller M, Karevold E: The nature and predictors of undercontrolled and internalizing problem trajectories across early childhood. J Abnorm Child Psychol. 2009, 37: 209-222.CrossRefPubMed Mathiesen KS, Sanson A, Stoolmiller M, Karevold E: The nature and predictors of undercontrolled and internalizing problem trajectories across early childhood. J Abnorm Child Psychol. 2009, 37: 209-222.CrossRefPubMed
28.
go back to reference Gustavson K, Røysamb E, von Soest T, Helland MJ, Mathiesen KS: Longitudinal associations between relationship problems, divorce, and life satisfaction: findings from a 15-year population-based study. J Posit Psychol. 2012, 7: 188-197.CrossRef Gustavson K, Røysamb E, von Soest T, Helland MJ, Mathiesen KS: Longitudinal associations between relationship problems, divorce, and life satisfaction: findings from a 15-year population-based study. J Posit Psychol. 2012, 7: 188-197.CrossRef
29.
go back to reference Winokur A, Winokur D, Rickels K, Cox DS: Symptoms of emotional distress in family planning service: stability over a four-week period. Br J Psychiatry. 1984, 144: 395-399.CrossRefPubMed Winokur A, Winokur D, Rickels K, Cox DS: Symptoms of emotional distress in family planning service: stability over a four-week period. Br J Psychiatry. 1984, 144: 395-399.CrossRefPubMed
30.
go back to reference Hesbacher PT, Rickels K, Morris RJ, Newman H, Rosenfeld H: Psychiatric illness in family practice. J Clin Psychiatry. 1980, 41: 6-10.PubMed Hesbacher PT, Rickels K, Morris RJ, Newman H, Rosenfeld H: Psychiatric illness in family practice. J Clin Psychiatry. 1980, 41: 6-10.PubMed
31.
go back to reference Deane FP, Leathem J, Spicer J: Clinical norms, reliability and validity for the Hopkins Symptom Checklist-21. Aust J Psychol. 1992, 44: 21-25.CrossRef Deane FP, Leathem J, Spicer J: Clinical norms, reliability and validity for the Hopkins Symptom Checklist-21. Aust J Psychol. 1992, 44: 21-25.CrossRef
32.
go back to reference Tambs K, Moum T: How well can a few questionnaire items indicate anxiety and depression. Acta Psychiatr Scand. 1993, 87: 364-367.CrossRefPubMed Tambs K, Moum T: How well can a few questionnaire items indicate anxiety and depression. Acta Psychiatr Scand. 1993, 87: 364-367.CrossRefPubMed
33.
go back to reference Buss AH, Plomin R: Temperament: Early developing personality traits. 1984, Lawrence Erlbaum Associates, New Jersey Buss AH, Plomin R: Temperament: Early developing personality traits. 1984, Lawrence Erlbaum Associates, New Jersey
34.
go back to reference Naerde A, Roysamb E, Tambs K: Temperament in adults - Reliability, stability, and factor structure of the EAS temperament survey. J Pers Assess. 2004, 82: 71-79.CrossRefPubMed Naerde A, Roysamb E, Tambs K: Temperament in adults - Reliability, stability, and factor structure of the EAS temperament survey. J Pers Assess. 2004, 82: 71-79.CrossRefPubMed
35.
go back to reference Boer F, Westenberg PM: The factor structure of the buss and plomin eas temperament survey (Parental Ratings) in a dutch sample of elementary-school-children. J Pers Assess. 1994, 62: 537-551.CrossRefPubMed Boer F, Westenberg PM: The factor structure of the buss and plomin eas temperament survey (Parental Ratings) in a dutch sample of elementary-school-children. J Pers Assess. 1994, 62: 537-551.CrossRefPubMed
36.
go back to reference Mathiesen KS, Tambs K: The EAS Temperament questionnaire-factor structure, age trends, reliability, and stability in a Norwegian sample. J Child Psychol Psychiatry. 1999, 40: 431-439.CrossRefPubMed Mathiesen KS, Tambs K: The EAS Temperament questionnaire-factor structure, age trends, reliability, and stability in a Norwegian sample. J Child Psychol Psychiatry. 1999, 40: 431-439.CrossRefPubMed
37.
go back to reference Mathiesen KS, Tambs K, Dalgard OS: The influence of social class, strain and social support on symptoms of anxiety and depression in mothers of toddlers. Soc Psychiatry Psychiatr Epidemiol. 1999, 34: 61-72.CrossRefPubMed Mathiesen KS, Tambs K, Dalgard OS: The influence of social class, strain and social support on symptoms of anxiety and depression in mothers of toddlers. Soc Psychiatry Psychiatr Epidemiol. 1999, 34: 61-72.CrossRefPubMed
38.
go back to reference Dalgard OS, Bjork S, Tambs K: Social support, negative life events and mental-health. Br J Psychiatry. 1995, 166: 29-34.CrossRefPubMed Dalgard OS, Bjork S, Tambs K: Social support, negative life events and mental-health. Br J Psychiatry. 1995, 166: 29-34.CrossRefPubMed
40.
go back to reference Demidenko E: Sample size determination for logistic regression revisited. Stat Med. 2007, 26: 3385-3397.CrossRefPubMed Demidenko E: Sample size determination for logistic regression revisited. Stat Med. 2007, 26: 3385-3397.CrossRefPubMed
41.
go back to reference Faul F, Erdfelder E, Buchner A, Lang AG: Statistical power analyses using G*Power 3.1: tests for correlation and regression analyses. Behav Res Methods. 2009, 41: 1149-1160.CrossRefPubMed Faul F, Erdfelder E, Buchner A, Lang AG: Statistical power analyses using G*Power 3.1: tests for correlation and regression analyses. Behav Res Methods. 2009, 41: 1149-1160.CrossRefPubMed
42.
go back to reference Kline RB: Principles and practice of structural equation modeling. 2011, Guilford, New York, 3 Kline RB: Principles and practice of structural equation modeling. 2011, Guilford, New York, 3
43.
go back to reference Cohen J: Statistical Power Analysis for the Behavioral Sciences. 1988, Lawrence Erlbaum Associates, New Jersey, 2 Cohen J: Statistical Power Analysis for the Behavioral Sciences. 1988, Lawrence Erlbaum Associates, New Jersey, 2
44.
go back to reference Maldonado G, Greenland S: The importance of critically interpreting simulation studies. Epidemiology. 1997, 8: 453-456.PubMed Maldonado G, Greenland S: The importance of critically interpreting simulation studies. Epidemiology. 1997, 8: 453-456.PubMed
45.
go back to reference Perneger TV: What’s wrong with Bonferroni adjustments. Br Med J. 1998, 316: 1236-1238.CrossRef Perneger TV: What’s wrong with Bonferroni adjustments. Br Med J. 1998, 316: 1236-1238.CrossRef
Metadata
Title
Attrition and generalizability in longitudinal studies: findings from a 15-year population-based study and a Monte Carlo simulation study
Authors
Kristin Gustavson
Tilmann von Soest
Evalill Karevold
Espen Røysamb
Publication date
01-12-2012
Publisher
BioMed Central
Published in
BMC Public Health / Issue 1/2012
Electronic ISSN: 1471-2458
DOI
https://doi.org/10.1186/1471-2458-12-918

Other articles of this Issue 1/2012

BMC Public Health 1/2012 Go to the issue