Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2018

Open Access 01-12-2018 | Research article

Sample size determination for mediation analysis of longitudinal data

Authors: Haitao Pan, Suyu Liu, Danmin Miao, Ying Yuan

Published in: BMC Medical Research Methodology | Issue 1/2018

Login to get access

Abstract

Background

Sample size planning for longitudinal data is crucial when designing mediation studies because sufficient statistical power is not only required in grant applications and peer-reviewed publications, but is essential to reliable research results. However, sample size determination is not straightforward for mediation analysis of longitudinal design.

Methods

To facilitate planning the sample size for longitudinal mediation studies with a multilevel mediation model, this article provides the sample size required to achieve 80% power by simulations under various sizes of the mediation effect, within-subject correlations and numbers of repeated measures. The sample size calculation is based on three commonly used mediation tests: Sobel’s method, distribution of product method and the bootstrap method.

Results

Among the three methods of testing the mediation effects, Sobel’s method required the largest sample size to achieve 80% power. Bootstrapping and the distribution of the product method performed similarly and were more powerful than Sobel’s method, as reflected by the relatively smaller sample sizes. For all three methods, the sample size required to achieve 80% power depended on the value of the ICC (i.e., within-subject correlation). A larger value of ICC typically required a larger sample size to achieve 80% power. Simulation results also illustrated the advantage of the longitudinal study design. The sample size tables for most encountered scenarios in practice have also been published for convenient use.

Conclusions

Extensive simulations study showed that the distribution of the product method and bootstrapping method have superior performance to the Sobel’s method, but the product method was recommended to use in practice in terms of less computation time load compared to the bootstrapping method. A R package has been developed for the product method of sample size determination in mediation longitudinal study design.
Appendix
Available only for authorised users
Literature
1.
go back to reference Abraham WT, Russell DW. Statistical power analysis in psychological research. Soc Personal Psychol Compass. 2008;2(1):283–301.CrossRef Abraham WT, Russell DW. Statistical power analysis in psychological research. Soc Personal Psychol Compass. 2008;2(1):283–301.CrossRef
2.
go back to reference Anderson SF, Maxwell SE. Addressing the "replication crisis": using original studies to design replication studies with appropriate statistical power. Multivar Behav Res. 2017:1–20. Anderson SF, Maxwell SE. Addressing the "replication crisis": using original studies to design replication studies with appropriate statistical power. Multivar Behav Res. 2017:1–20.
3.
go back to reference Barcikowski R. Statistical power with group mean as the unit of analysis. J Educ Stat. 1981;6:267–85.CrossRef Barcikowski R. Statistical power with group mean as the unit of analysis. J Educ Stat. 1981;6:267–85.CrossRef
4.
go back to reference Baron RM, Kenny DA. The moderator-mediator variable distinction on social psychological research: conceptual, strategic, and statistical considerations. J Pers Soc Psychol. 1986;51:1173–82.CrossRefPubMed Baron RM, Kenny DA. The moderator-mediator variable distinction on social psychological research: conceptual, strategic, and statistical considerations. J Pers Soc Psychol. 1986;51:1173–82.CrossRefPubMed
5.
go back to reference Bauer DJ, Preacher KJ, Gil KM. Conceptualizing and testing random indirect effects and moderated mediation in multilevel models: new procedures and recommendations. Psychol Methods. 2006;11:142–63.CrossRefPubMed Bauer DJ, Preacher KJ, Gil KM. Conceptualizing and testing random indirect effects and moderated mediation in multilevel models: new procedures and recommendations. Psychol Methods. 2006;11:142–63.CrossRefPubMed
6.
go back to reference Bollen KA, Stine R. Direct and indirect effects: classical and bootstrap estimates of variability. Sociol Methodol. 1990;20:115–40.CrossRef Bollen KA, Stine R. Direct and indirect effects: classical and bootstrap estimates of variability. Sociol Methodol. 1990;20:115–40.CrossRef
7.
go back to reference Collins LM, Graham JW, Flaherty BP. An alternative framework for defining mediation. Multivar Behav Res. 1998;33:295–312.CrossRef Collins LM, Graham JW, Flaherty BP. An alternative framework for defining mediation. Multivar Behav Res. 1998;33:295–312.CrossRef
8.
go back to reference Crits-Christoph P, Mintz J. Implication of therapist effects for the design and analysis of comparative studies of psychotherapies. J Consult Clin Psychol. 1991;59:20–6.CrossRefPubMed Crits-Christoph P, Mintz J. Implication of therapist effects for the design and analysis of comparative studies of psychotherapies. J Consult Clin Psychol. 1991;59:20–6.CrossRefPubMed
9.
go back to reference de Leeuw J, Meijer E. Handbook of multilevel analysis. New York: Springer; 2008.CrossRef de Leeuw J, Meijer E. Handbook of multilevel analysis. New York: Springer; 2008.CrossRef
10.
go back to reference Du H, Wang L. A Bayesian power analysis procedure considering uncertainty in effect size estimates from a meta-analysis. Multivar Behav Res. 2016;51(5):589–605.CrossRef Du H, Wang L. A Bayesian power analysis procedure considering uncertainty in effect size estimates from a meta-analysis. Multivar Behav Res. 2016;51(5):589–605.CrossRef
11.
go back to reference Efron B. Bootstrap methods: another look at the jackknife. Ann Stat. 1979;7:1–26.CrossRef Efron B. Bootstrap methods: another look at the jackknife. Ann Stat. 1979;7:1–26.CrossRef
12.
go back to reference Elkin I, Falconnier L, Martinovich Z, Mahoney C. Therapist effects in the NIMH Treatment of Depression Collaborative Research Program. Psychother Res. 2006;16:144–60.CrossRef Elkin I, Falconnier L, Martinovich Z, Mahoney C. Therapist effects in the NIMH Treatment of Depression Collaborative Research Program. Psychother Res. 2006;16:144–60.CrossRef
13.
go back to reference Frees EW. Longitudinal and panel data: analysis and applications in the social sciences. Cambridge: Cambridge University Press; 2004. Frees EW. Longitudinal and panel data: analysis and applications in the social sciences. Cambridge: Cambridge University Press; 2004.
15.
go back to reference Hox JJ. Multilevel analysis: techniques and applications. Mahwah, NJ: Erlbaum; 2002. Hox JJ. Multilevel analysis: techniques and applications. Mahwah, NJ: Erlbaum; 2002.
16.
go back to reference Judd CM, Kenny DA. Estimating the effects of social interventions. Cambridge: Cambridge University Press; 1981a. Judd CM, Kenny DA. Estimating the effects of social interventions. Cambridge: Cambridge University Press; 1981a.
17.
go back to reference Judd CM, Kenny DA. Process analysis: estimating mediation in treatment evaluations. Eval Rev. 1981b;5:602–19.CrossRef Judd CM, Kenny DA. Process analysis: estimating mediation in treatment evaluations. Eval Rev. 1981b;5:602–19.CrossRef
18.
go back to reference Kenny DA, Korchmaros JD, Bolger N. Lower level mediation in multilevel models. Psychol Methods. 2003;8:115–28.CrossRefPubMed Kenny DA, Korchmaros JD, Bolger N. Lower level mediation in multilevel models. Psychol Methods. 2003;8:115–28.CrossRefPubMed
20.
go back to reference Kim D-M, Wampold BE, Bolt DM. Therapist effects in psychotherapy: a random effects modeling of the NIMH TDCRP data. Psychother Res. 2006;16:161–72.CrossRef Kim D-M, Wampold BE, Bolt DM. Therapist effects in psychotherapy: a random effects modeling of the NIMH TDCRP data. Psychother Res. 2006;16:161–72.CrossRef
21.
go back to reference Krull JL, MacKinnon DP. Multilevel mediation modeling in group-based intervention studies. Eval Rev. 1999;23:418–44.CrossRefPubMed Krull JL, MacKinnon DP. Multilevel mediation modeling in group-based intervention studies. Eval Rev. 1999;23:418–44.CrossRefPubMed
22.
go back to reference Krull JL, Mackinnon DP. Multilevel modeling of individual and group level mediated effects. Multivar Behav Res. 2001;36:249–77.CrossRef Krull JL, Mackinnon DP. Multilevel modeling of individual and group level mediated effects. Multivar Behav Res. 2001;36:249–77.CrossRef
23.
go back to reference Lomnicki ZA. On the distribution of product of random variables. J R Stat Soc. 1967;29:513–24. Lomnicki ZA. On the distribution of product of random variables. J R Stat Soc. 1967;29:513–24.
24.
go back to reference Lutz, Wolfgang; Leon, Scott C.; Martinovich, Zoran; Lyons, John S.; Stiles, William B. Therapist effects in outpatient psychotherapy: a three-level growth curve approach. J Couns Psychol, Vol 54(1), Jan 2007, 32–39. Lutz, Wolfgang; Leon, Scott C.; Martinovich, Zoran; Lyons, John S.; Stiles, William B. Therapist effects in outpatient psychotherapy: a three-level growth curve approach. J Couns Psychol, Vol 54(1), Jan 2007, 32–39.
25.
go back to reference MacKinnon DP, Dwyer JH. Estimating mediated effects in prevention studies. Eval Rev. 1993;17:144–58.CrossRef MacKinnon DP, Dwyer JH. Estimating mediated effects in prevention studies. Eval Rev. 1993;17:144–58.CrossRef
26.
go back to reference MacKinnon DP, Lockwood CM, Williams J. Confidence limits for the indirect effect: distribution of the product and resampling methods. Multivar Behav Res. 2004;39:99–128.CrossRef MacKinnon DP, Lockwood CM, Williams J. Confidence limits for the indirect effect: distribution of the product and resampling methods. Multivar Behav Res. 2004;39:99–128.CrossRef
27.
go back to reference MacKinnon DP. Introduction to statistical mediation analysis. New York: Lawrence Erlbaum Associates; 2008. MacKinnon DP. Introduction to statistical mediation analysis. New York: Lawrence Erlbaum Associates; 2008.
28.
go back to reference MacKinnon DP, Warsi G, Dwyer JH. A simulation study of mediated effect measures. Multivar Behav Res. 1995;30:41–62.CrossRef MacKinnon DP, Warsi G, Dwyer JH. A simulation study of mediated effect measures. Multivar Behav Res. 1995;30:41–62.CrossRef
29.
go back to reference Maxwell SE. The persistence of underpowered studies in psychological research: causes, consequences, and remedies. Psychol Methods. 2004;9(2):147.CrossRefPubMed Maxwell SE. The persistence of underpowered studies in psychological research: causes, consequences, and remedies. Psychol Methods. 2004;9(2):147.CrossRefPubMed
30.
go back to reference Maxwell SE, Kelley K, Rausch JR. Sample size planning for statistical power and accuracy in parameter estimation. Annu Rev Psychol. 2008;59:537–63.CrossRefPubMed Maxwell SE, Kelley K, Rausch JR. Sample size planning for statistical power and accuracy in parameter estimation. Annu Rev Psychol. 2008;59:537–63.CrossRefPubMed
31.
go back to reference Meeker WQ Jr, Cornwell LW, Aroian LA. The product of two normally distributed random variables. In: Kennedy WJ, Odeh RE, editors. Selected tables in mathematical statistics, vol. VII. Providence, RI: American Mathematical Society; 1981. Meeker WQ Jr, Cornwell LW, Aroian LA. The product of two normally distributed random variables. In: Kennedy WJ, Odeh RE, editors. Selected tables in mathematical statistics, vol. VII. Providence, RI: American Mathematical Society; 1981.
32.
go back to reference Okiishi J, Lambert MJ, Nielsen SL, Ogles BM. Waiting for Supershrink: an empirical analysis of therapist effects. Clinical Psychology and Psychotherapy. 2003;10:361–73.CrossRef Okiishi J, Lambert MJ, Nielsen SL, Ogles BM. Waiting for Supershrink: an empirical analysis of therapist effects. Clinical Psychology and Psychotherapy. 2003;10:361–73.CrossRef
33.
go back to reference Raudenbush SW, Bryk AS. Hierarchical linear models: applications and data analysis methods. 2nd ed. Newbury Park, CA: Sage; 2002. Raudenbush SW, Bryk AS. Hierarchical linear models: applications and data analysis methods. 2nd ed. Newbury Park, CA: Sage; 2002.
34.
go back to reference Shrout PE, Bolger N. Mediation in experimental and nonexperimental studies: new procedures and recommendations. Psychology Methods. 2002;7:422–45.CrossRef Shrout PE, Bolger N. Mediation in experimental and nonexperimental studies: new procedures and recommendations. Psychology Methods. 2002;7:422–45.CrossRef
35.
go back to reference Sobel ME. Asymptotic confidence intervals for indirect effects in structural equation models. Sociol Methodol. 1982;13:290–312.CrossRef Sobel ME. Asymptotic confidence intervals for indirect effects in structural equation models. Sociol Methodol. 1982;13:290–312.CrossRef
36.
go back to reference Sobel ME. Direct and indirect effects in linear structural equation models. Sociological Methods and Research. 1987;16:155–67.CrossRef Sobel ME. Direct and indirect effects in linear structural equation models. Sociological Methods and Research. 1987;16:155–67.CrossRef
37.
go back to reference Wampold BE, Brown GS. Estimating variability in outcomes attributable to therapists: a naturalistic study of outcomes in managed care. J Consult Clin Psychol. 2005;73:914.CrossRefPubMed Wampold BE, Brown GS. Estimating variability in outcomes attributable to therapists: a naturalistic study of outcomes in managed care. J Consult Clin Psychol. 2005;73:914.CrossRefPubMed
38.
go back to reference Wang C, Xue X. Power and sample size calculations for evaluating mediation effects in longitudinal studies. Stat Methods Med Res. 2016 Apr;25(2):686–705.CrossRefPubMed Wang C, Xue X. Power and sample size calculations for evaluating mediation effects in longitudinal studies. Stat Methods Med Res. 2016 Apr;25(2):686–705.CrossRefPubMed
Metadata
Title
Sample size determination for mediation analysis of longitudinal data
Authors
Haitao Pan
Suyu Liu
Danmin Miao
Ying Yuan
Publication date
01-12-2018
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2018
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-018-0473-2

Other articles of this Issue 1/2018

BMC Medical Research Methodology 1/2018 Go to the issue