Top

Open Access 19-04-2024 | Review

Transparency in Cognitive Training Meta-analyses: A Meta-review

Authors: Alejandro Sandoval-Lentisco, Rubén López-Nicolás, Miriam Tortajada, José Antonio López-López, Julio Sánchez-Meca

Published in: Neuropsychology Review

Abstract

Meta-analyses often present flexibility regarding their inclusion criteria, outcomes of interest, statistical analyses, and assessments of the primary studies. For this reason, it is necessary to transparently report all the information that could impact the results. In this meta-review, we aimed to assess the transparency of meta-analyses that examined the benefits of cognitive training, given the ongoing controversy that exists in this field. Ninety-seven meta-analytic reviews were included, which examined a wide range of populations with different clinical conditions and ages. Regarding the reporting, information about the search of the studies, screening procedure, or data collection was detailed by most reviews. However, authors usually failed to report other aspects such as the specific meta-analytic parameters, the formula used to compute the effect sizes, or the data from primary studies that were used to compute the effect sizes. Although some of these practices have improved over the years, others remained the same. Moreover, examining the eligibility criteria of the reviews revealed a great heterogeneity in aspects such as the training duration, age cut-offs, or study designs that were considered. Preregistered meta-analyses often specified poorly how they would deal with the multiplicity of data or assess publication bias in their protocols, and some contained non-disclosed deviations in their eligibility criteria or outcomes of interests. The findings shown here, although they do not question the benefits of cognitive training, illustrate important aspects that future reviews must consider.

Available only for authorised users

PROSPERO, the leading platform for preregistering systematic reviews and meta-analyses, was launched in 2011 (Page et al., 2018). If we consider only studies that were conducted after its implementation, the prevalence of preregistration increases to 38%.

A sensitivity analysis that included only studies published after the launch of PROSPERO in 2011 also showed a significant association between year and preregistration.

Aksayli, N. D., Sala, G., & Gobet, F. (2019). The cognitive and academic benefits of Cogmed: A meta-analysis. Educational Research Review, 27, 229–243. https://doi.org/10.1016/j.edurev.2019.04.003CrossRef

Au, J., Buschkuehl, M., Duncan, G. J., & Jaeggi, S. M. (2016). There is no convincing evidence that working memory training is NOT effective: A reply to Melby-Lervåg and Hulme (2015). Psychonomic Bulletin & Review, 23(1), 331–337. https://doi.org/10.3758/s13423-015-0967-4CrossRef

Au, J., Sheehan, E., Tsai, N., Duncan, G. J., Buschkuehl, M., & Jaeggi, S. M. (2015). Improving fluid intelligence with training on working memory: A meta-analysis. Psychonomic Bulletin & Review, 22(2), 366–377. https://doi.org/10.3758/s13423-014-0699-xCrossRef

Baker, M. (2016). 1,500 scientists lift the lid on reproducibility. Nature, 533(7604), Article 7604. https://doi.org/10.1038/533452a

Bakker, M., Veldkamp, C. L. S., van Assen, M. A. L. M., Crompvoets, E. A. V., Ong, H. H., Nosek, B. A., Soderberg, C. K., Mellor, D., & Wicherts, J. M. (2020). Ensuring the quality and specificity of preregistrations. PLOS Biology, 18(12), e3000937. https://doi.org/10.1371/journal.pbio.3000937CrossRefPubMedCentralPubMed

Bartoš, F., Maier, M., Quintana, D. S., & Wagenmakers, E. J. (2022). Adjusting for publication bias in JASP and R: Selection models, PET-PEESE, and robust Bayesian meta-analysis. Advances in Methods and Practices in Psychological Science, 5(3). https://doi.org/10.1177/25152459221109259

Basak, C., Qin, S., & O’Connell, M. A. (2020). Differential effects of cognitive training modules in healthy aging and mild cognitive impairment: A comprehensive meta-analysis of randomized controlled trials. Psychology and Aging, 35(2), 220–249. https://doi.org/10.1037/pag0000442CrossRefPubMedCentralPubMed

Bonnechère, B., Langley, C., & Sahakian, B. J. (2020). The use of commercial computerised cognitive games in older adults: A meta-analysis. Scientific Reports, 10(1), Article 1. https://doi.org/10.1038/s41598-020-72281-3

Cao, Y., Huang, T., Huang, J., Xie, X., & Wang, Y. (2020). Effects and moderators of computer-based training on children’s executive functions: A systematic review and meta-analysis. Frontiers in Psychology, 11, 580329. https://doi.org/10.3389/fpsyg.2020.580329CrossRefPubMedCentralPubMed

Chen, S., Yu, J., Zhang, Q., Zhang, J., Zhang, Y., & Wang, J. (2021). Which factor is more relevant to the effectiveness of the cognitive intervention? A meta-analysis of randomized controlled trials of cognitive training on symptoms and executive function behaviors of children with attention deficit hyperactivity disorder. Frontiers in Psychology, 12, 810298. https://doi.org/10.3389/fpsyg.2021.810298CrossRefPubMed

Cheung, M.W.-L. (2019). A guide to conducting a meta-analysis with non-independent effect sizes. Neuropsychology Review, 29(4), 387–396. https://doi.org/10.1007/s11065-019-09415-6CrossRefPubMedCentralPubMed

Claesen, A., Gomes, S., Tuerlinckx, F., & Vanpaemel, W. (2021). Comparing dream to reality: An assessment of adherence of the first generation of preregistered studies. Royal Society Open Science, 8(10), 211037. https://doi.org/10.1098/rsos.211037CrossRefPubMedCentralPubMed

Clare, L., & Woods, R. T. (2004). Cognitive training and cognitive rehabilitation for people with early-stage Alzheimer’s disease: A review. Neuropsychological Rehabilitation, 14(4), 385–401. https://doi.org/10.1080/09602010443000074CrossRef

Cortese, S., Ferrin, M., Brandeis, D., Buitelaar, J., Daley, D., Dittmann, R. W., Holtmann, M., Santosh, P., Stevenson, J., Stringaris, A., Zuddas, A., & Sonuga-Barke, E. J. (2015). Cognitive training for attention-deficit/hyperactivity disorder: Meta-analysis of clinical and neuropsychological outcomes from randomized controlled trials. Journal of the American Academy of Child & Adolescent Psychiatry, 54(3), 164–174. https://doi.org/10.1016/j.jaac.2014.12.010CrossRef

Corti, C., Oldrati, V., Oprandi, M. C., Ferrari, E., Poggi, G., Borgatti, R., Urgesi, C., & Bardoni, A. (2019). Remote technology-based training programs for children with acquired brain injury: A systematic review and a meta-analytic exploration. Behavioural Neurology, 2019, 1346987. https://doi.org/10.1155/2019/1346987CrossRefPubMedCentralPubMed

Cristea, I. A., Gentili, C., Pietrini, P., & Cuijpers, P. (2017). Sponsorship bias in the comparative efficacy of psychotherapy and pharmacotherapy for adult depression: Meta-analysis. The British Journal of Psychiatry, 210(1), 16–23. https://doi.org/10.1192/bjp.bp.115.179275CrossRefPubMed

Dardiotis, E., Nousia, A., Siokas, V., Tsouris, Z., Andravizou, A., Mentis, A. A., Florou, D., Messinis, L., & Nasios, G. (2018). Efficacy of computer-based cognitive training in neuropsychological performance of patients with multiple sclerosis: A systematic review and meta-analysis. Multiple Sclerosis and Related Disorders, 20, 58–66. https://doi.org/10.1016/j.msard.2017.12.017CrossRefPubMed

das Nair, R., Cogger, H., Worthington, E., & Lincoln, N. B. (2016). Cognitive rehabilitation for memory deficits after stroke. Cochrane Database of Systematic Reviews, 9. https://doi.org/10.1002/14651858.CD002293.pub3

Ebrahim, S., Bance, S., Athale, A., Malachowski, C., & Ioannidis, J. P. A. (2016). Meta-analyses with industry involvement are massively published and report no caveats for antidepressants. Journal of Clinical Epidemiology, 70, 155–163. https://doi.org/10.1016/j.jclinepi.2015.08.021CrossRefPubMed

Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ, 315(7109), 629–634. https://doi.org/10.1136/bmj.315.7109.629

Epskamp, S. (2019). Reproducibility and replicability in a fast-paced methodological world. Advances in Methods and Practices in Psychological Science, 2(2), 145–155. https://doi.org/10.1177/2515245919847421CrossRef

Ferrell, S., Demla, S., Anderson, J. M., Weaver, M., Torgerson, T., Hartwell, M., & Vassar, M. (2022). Association between industry sponsorship and author conflicts of interest with outcomes of systematic reviews and meta-analyses of interventions for opioid use disorder. Journal of Substance Abuse Treatment, 132, 108598. https://doi.org/10.1016/j.jsat.2021.108598CrossRefPubMed

Gates, N. J., Rutjes, A. W., Nisio, M. D., Karim, S., Chong, L.-Y., March, E., Martínez, G., & Vernooij, R. W. (2020). Computerised cognitive training for 12 or more weeks for maintaining cognitive function in cognitively healthy people in late life. Cochrane Database of Systematic Reviews, 2. https://doi.org/10.1002/14651858.CD012277.pub3

Gavelin, H. M., Lampit, A., Hallock, H., Sabatés, J., & Bahar-Fuchs, A. (2020). Cognition-oriented treatments for older adults: A systematic overview of systematic reviews. Neuropsychology Review, 30(2), 167–193. https://doi.org/10.1007/s11065-020-09434-8CrossRefPubMedCentralPubMed

Giustiniani, A., Maistrello, L., Danesin, L., Rigon, E., & Burgio, F. (2022). Effects of cognitive rehabilitation in Parkinson disease: A meta-analysis. Neurological Sciences, 43(4), 2323–2337. https://doi.org/10.1007/s10072-021-05772-4CrossRefPubMed

Gobet, F., & Sala, G. (2022). Cognitive training: A field in search of a phenomenon. Perspectives on Psychological Science, 18(1), 125–141. https://doi.org/10.1177/17456916221091830

Goldberg, Z., Kuslak, B., & Kurtz, M. M. (2023). A meta-analytic investigation of cognitive remediation for mood disorders: Efficacy and the role of study quality, sample and treatment factors. Journal of Affective Disorders, 330, 74–82. https://doi.org/10.1016/j.jad.2023.02.137CrossRefPubMed

Gucciardi, D. F., Lines, R. L. J., & Ntoumanis, N. (2022). Handling effect size dependency in meta-analysis. International Review of Sport and Exercise Psychology, 15(1), 152–178. https://doi.org/10.1080/1750984X.2021.1946835CrossRef

Ha, J.-Y., & Park, H.-J. (2023). Effects of mobile-based cognitive interventions for the cognitive function in the community-dwelling older adults: A systematic review and meta-analysis. Archives of Gerontology and Geriatrics, 104, 104829. https://doi.org/10.1016/j.archger.2022.104829CrossRefPubMed

Hallock, H., Collins, D., Lampit, A., Deol, K., Fleming, J., & Valenzuela, M. (2016). Cognitive training for post-acute traumatic brain injury: A systematic review and meta-analysis. Frontiers in Human Neuroscience, 10, 537. https://doi.org/10.3389/fnhum.2016.00537CrossRefPubMedCentralPubMed

Hardwicke, T. E., & Wagenmakers, E.-J. (2021). Reducing bias, increasing transparency, and calibrating confidence with preregistration. Nature Human Behaviour, 7(1), 15–26. https://doi.org/10.1038/s41562-022-01497-2

Hardwicke, T. E., Wallach, J. D., Kidwell, M. C., Bendixen, T., Crüwell, S., & Ioannidis, J. P. A. (2020). An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014–2017). Royal Society Open Science, 7(2), 190806. https://doi.org/10.1098/rsos.190806CrossRefPubMedCentralPubMed

Harrer, M., Cuijpers, P., A, F. T., & Ebert, D. D. (2021). Doing meta-analysis with R: A hands-on guide (1.^a ed.). Chapman & Hall/CRC Press.

Harvey, P. D., McGurk, S. R., Mahncke, H., & Wykes, T. (2018). Controversies in computerized cognitive training. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 3(11), 907–915. https://doi.org/10.1016/j.bpsc.2018.06.008CrossRefPubMed

He, F., Huang, H., Ye, L., Wen, X., & Cheng, A. S. K. (2022). Meta-analysis of neurocognitive rehabilitation for cognitive dysfunction among pediatric cancer survivors. Journal of Cancer Research and Therapeutics, 18(7), 2058. https://doi.org/10.4103/jcrt.jcrt_1429_22CrossRefPubMed

Hedges, L. V., Tipton, E., & Johnson, M. C. (2010). Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods, 1(1), 39–65. https://doi.org/10.1002/jrsm.5CrossRefPubMed

Hill, N. T., Mowszowski, L., Naismith, S. L., Chadwick, V. L., Valenzuela, M., & Lampit, A. (2017). Computerized cognitive training in older adults with mild cognitive impairment or dementia: A systematic review and meta-analysis. American Journal of Psychiatry, 174(4), 329–340. https://doi.org/10.1176/appi.ajp.2016.16030360CrossRefPubMed

Hou, J., Jiang, T., Fu, J., Su, B., Wu, H., Sun, R., & Zhang, T. (2020). The long-term efficacy of working memory training in healthy older adults: A systematic review and meta-analysis of 22 randomized controlled trials. The Journals of Gerontology: Series B, 75(8), e174–e188. https://doi.org/10.1093/geronb/gbaa077CrossRef

Huntley, J. D., Gould, R. L., Liu, K., Smith, M., & Howard, R. J. (2015). Do cognitive interventions improve general cognition in dementia? A Meta-Analysis and Meta-Regression. BMJ Open, 5(4), e005247. https://doi.org/10.1136/bmjopen-2014-005247CrossRefPubMed

Ioannidis, J. P. A. (2005). Why most published research findings are false. PLOS Medicine, 2(8), e124. https://doi.org/10.1371/journal.pmed.0020124CrossRefPubMedCentralPubMed

Ioannidis, J. (2016). The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. The Milbank Quarterly, 94(3), 485–514. https://doi.org/10.1111/1468-0009.12210CrossRefPubMedCentralPubMed

Jones, W. E., Benge, J. F., & Scullin, M. K. (2021). Preserving prospective memory in daily life: A systematic review and meta-analysis of mnemonic strategy, cognitive training, external memory aid, and combination interventions. Neuropsychology, 35, 123–140. https://doi.org/10.1037/neu0000704CrossRefPubMed

Kambeitz-Ilankovic, L., Betz, L. T., Dominke, C., Haas, S. S., Subramaniam, K., Fisher, M., Vinogradov, S., Koutsouleris, N., & Kambeitz, J. (2019). Multi-outcome meta-analysis (MOMA) of cognitive remediation in schizophrenia: Revisiting the relevance of human coaching and elucidating interplay between multiple outcomes. Neuroscience & Biobehavioral Reviews, 107, 828–845. https://doi.org/10.1016/j.neubiorev.2019.09.031CrossRef

Karbach, J., & Verhaeghen, P. (2014). Making working memory work: A meta-analysis of executive-control and working memory training in older adults. Psychological Science, 25(11), 2027–2037. https://doi.org/10.1177/0956797614548725CrossRefPubMed

Karch, D., Albers, L., Renner, G., Lichtenauer, N., & von Kries, R. (2013). The efficacy of cognitive training programs in children and adolescents. Deutsches Ärzteblatt International, 110(39), 643–652. https://doi.org/10.3238/arztebl.2013.0643CrossRefPubMedCentralPubMed

Kassai, R., Futo, J., Demetrovics, Z., & Takacs, Z. K. (2019). A meta-analysis of the experimental evidence on the near- and far-transfer effects among children’s executive function skills. Psychological Bulletin, 145, 165–188. https://doi.org/10.1037/bul0000180CrossRefPubMed

Kelly, M. E., Loughrey, D., Lawlor, B. A., Robertson, I. H., Walsh, C., & Brennan, S. (2014). The impact of cognitive training and mental stimulation on cognitive and everyday functioning of healthy older adults: A systematic review and meta-analysis. Ageing Research Reviews, 15, 28–43. https://doi.org/10.1016/j.arr.2014.02.004CrossRefPubMed

Koffel, J. B., & Rethlefsen, M. L. (2016). Reproducibility of search strategies is poor in systematic reviews published in high-impact pediatrics, cardiology and surgery journals: A cross-sectional study. PLoS ONE, 11(9), e0163309. https://doi.org/10.1371/journal.pone.0163309CrossRefPubMedCentralPubMed

Kvarven, A., Strømland, E., & Johannesson, M. (2020). Comparing meta-analyses and preregistered multiple-laboratory replication projects. Nature Human Behaviour, 4(4), Article 4. https://doi.org/10.1038/s41562-019-0787-z

Lakens, D., Page-Gould, E., van Assen, M. A. L. M., Spellman, B., Schönbrodt, F., Hasselman, F., Corker, K. S., Grange, J. A., Sharples, A., Cavender, C., Augusteijn, H. E. M., Augusteijn, H., Gerger, H., Locher, C., Miller, I. D., Anvari, F., & Scheel, A. M. (2017). Examining the reproducibility of meta-analyses in psychology: A preliminary report. MetaArXiv. https://doi.org/10.31222/osf.io/xfbjf

Lampit, A., Hallock, H., & Valenzuela, M. (2014). Computerized cognitive training in cognitively healthy older adults: A systematic review and meta-analysis of effect modifiers. PLOS Medicine, 11(11), e1001756. https://doi.org/10.1371/journal.pmed.1001756CrossRefPubMedCentralPubMed

Lampit, A., Heine, J., Finke, C., Barnett, M. H., Valenzuela, M., Wolf, A., Leung, I. H. K., & Hill, N. T. M. (2019). Computerized cognitive training in multiple sclerosis: A systematic review and meta-analysis. Neurorehabilitation and Neural Repair, 33(9), 695–706. https://doi.org/10.1177/1545968319860490CrossRefPubMed

Leung, I. H. K., Walton, C. C., Hallock, H., Lewis, S. J. G., Valenzuela, M., & Lampit, A. (2015). Cognitive training in Parkinson disease: A systematic review and meta-analysis. Neurology, 85(21), 1843–1851. https://doi.org/10.1212/WNL.0000000000002145

Light, R. J., & Pillemer, D. B. (1984). Summing up: The science of reviewing research. Harvard University Press.

Loetscher, T., Potter, K.-J., Wong, D., & das Nair, R. (2019). Cognitive rehabilitation for attention deficits following stroke. Cochrane Database of Systematic Reviews, 11. https://doi.org/10.1002/14651858.CD002842.pub3

López-López, J. A., Page, M. J., Lipsey, M. W., & Higgins, J. P. T. (2018). Dealing with effect size multiplicity in systematic reviews and meta-analyses. Research Synthesis Methods, 9(3), 336–351. https://doi.org/10.1002/jrsm.1310CrossRef

López-López, J. A., Rubio-Aparicio, M., & Sánchez-Meca, J. (2022). Overviews of reviews: Concept and development. Psicothema, 34(2), 175–181. https://doi.org/10.7334/psicothema2021.586CrossRefPubMed

López-Nicolás, R., López-López, J. A., Rubio-Aparicio, M., & Sánchez-Meca, J. (2022). A meta-review of transparency and reproducibility-related reporting practices in published meta-analyses on clinical psychological interventions (2000–2020). Behavior Research Methods, 54(1), 334–349. https://doi.org/10.3758/s13428-021-01644-zCrossRefPubMed

Maassen, E., van Assen, M. A. L. M., Nuijten, M. B., Olsson-Collentine, A., & Wicherts, J. M. (2020). Reproducibility of individual effect sizes in meta-analyses in psychology. PLoS ONE, 15(5), e0233107. https://doi.org/10.1371/journal.pone.0233107CrossRefPubMedCentralPubMed

Maier, M., Bartoš, F., & Wagenmakers, E.-J. (2022). Robust Bayesian meta-analysis: Addressing publication bias with model-averaging. Psychological Methods, 28(1), 107. https://doi.org/10.1037/met0000405CrossRefPubMed

McGurk, S. R., Twamley, E. W., Sitzer, D. I., McHugo, G. J., & Mueser, K. T. (2007). A meta-analysis of cognitive remediation in schizophrenia. American Journal of Psychiatry, 164(12), 1791–1802. https://doi.org/10.1176/appi.ajp.2007.07060906CrossRefPubMed

Melby-Lervåg, M., & Hulme, C. (2016). There is no convincing evidence that working memory training is effective: A reply to Au et al. (2014) and Karbach and Verhaeghen (2014). Psychonomic Bulletin & Review, 23(1), 324–330. https://doi.org/10.3758/s13423-015-0862-zCrossRef

Melby-Lervåg, M., Redick, T. S., & Hulme, C. (2016). Working memory training does not improve performance on measures of intelligence or other measures of “Far Transfer”: Evidence from a meta-analytic review. Perspectives on Psychological Science, 11(4), 512–534. https://doi.org/10.1177/1745691616635612

Moeyaert, M., Ugille, M., Natasha Beretvas, S., Ferron, J., Bunuan, R., & Van den Noortgate, W. (2017). Methods for dealing with multiple outcomes in meta-analysis: A comparison between averaging effect sizes, robust variance estimation and multilevel meta-analysis. International Journal of Social Research Methodology, 20(6), 559–572. https://doi.org/10.1080/13645579.2016.1252189CrossRef

Moreau, D., & Gamble, B. (2022). Conducting a meta-analysis in the age of open science: Tools, tips, and practical recommendations. Psychological Methods, 27(3), 426–432. https://doi.org/10.1037/met0000351CrossRefPubMed

Nguyen, L., Murphy, K., & Andrews, G. (2019). Immediate and long-term efficacy of executive functions cognitive training in older adults: A systematic review and meta-analysis. Psychological Bulletin, 145, 698–733. https://doi.org/10.1037/bul0000196CrossRefPubMed

Nguyen, L., Murphy, K., & Andrews, G. (2022). A game a day keeps cognitive decline away? a systematic review and meta-analysis of commercially-available brain training programs in healthy and cognitively impaired older adults. Neuropsychology Review, 32(3), 601–630. https://doi.org/10.1007/s11065-021-09515-2CrossRefPubMed

Nosek, B. A., Beck, E. D., Campbell, L., Flake, J. K., Hardwicke, T. E., Mellor, D. T., van ’t Veer, A. E., & Vazire, S. (2019). Preregistration is hard, and worthwhile. Trends in Cognitive Sciences, 23(10), 815–818. https://doi.org/10.1016/j.tics.2019.07.009CrossRefPubMed

Oldrati, V., Corti, C., Poggi, G., Borgatti, R., Urgesi, C., & Bardoni, A. (2020). Effectiveness of computerized cognitive training programs (CCTP) with game-like features in children with or without neuropsychological disorders: A meta-analytic investigation. Neuropsychology Review, 30(1), 126–141. https://doi.org/10.1007/s11065-020-09429-5CrossRefPubMed

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. https://doi.org/10.1126/science.aac4716

Page, M. J., McKenzie, J. E., & Forbes, A. (2013). Many scenarios exist for selective inclusion and reporting of results in randomized trials and systematic reviews. Journal of Clinical Epidemiology, 66(5), 524–537. https://doi.org/10.1016/j.jclinepi.2012.10.010CrossRefPubMed

Page, M. J., Moher, D., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., & … McKenzie, J. E. (2021). PRISMA 2020 explanation and elaboration: Updated guidance and exemplars for reporting systematic reviews. BMJ, 372, n160. https://doi.org/10.1136/bmj.n160

Page, M. J., Shamseer, L., & Tricco, A. C. (2018). Registration of systematic reviews in PROSPERO: 30,000 records and counting. Systematic Reviews, 7(1), 32. https://doi.org/10.1186/s13643-018-0699-4CrossRefPubMedCentralPubMed

Pigott, T. D., & Polanin, J. R. (2020). Methodological guidance paper: High-quality meta-analysis in a systematic review. Review of Educational Research, 90(1), 24–46. https://doi.org/10.3102/0034654319877153CrossRef

Polanin, J. R., Hennessy, E. A., & Tsuji, S. (2020). Transparency and reproducibility of meta-analyses in psychology: A meta-review. Perspectives on Psychological Science, 15(4), 1026–1041. https://doi.org/10.1177/1745691620906416CrossRefPubMed

Pustejovsky, J. E., & Tipton, E. (2022). Meta-analysis with robust variance estimation: Expanding the range of working models. Prevention Science, 23(3), 425–438. https://doi.org/10.1007/s11121-021-01246-3CrossRefPubMed

R Core Team. (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/

Redick, T. S. (2019). The hype cycle of working memory training. Current Directions in Psychological Science, 28(5), 423–429. https://doi.org/10.1177/0963721419848668CrossRefPubMedCentralPubMed

Robinson, K. E., Kaizar, E., Catroppa, C., Godfrey, C., & Yeates, K. O. (2014). Systematic review and meta-analysis of cognitive interventions for children with central nervous system disorders and neurodevelopmental disorders. Journal of Pediatric Psychology, 39(8), 846–865. https://doi.org/10.1093/jpepsy/jsu031CrossRefPubMed

Rubio-Aparicio, M., Marín-Martínez, F., Sánchez-Meca, J., & López-López, J. A. (2018). A methodological review of meta-analyses of the effectiveness of clinical psychology treatments. Behavior Research Methods, 50(5), 2057–2073. https://doi.org/10.3758/s13428-017-0973-8CrossRefPubMed

Sala, G., & Gobet, F. (2017). Working memory training in typically developing children: A meta-analysis of the available evidence. Developmental Psychology, 53, 671–685. https://doi.org/10.1037/dev0000265CrossRefPubMed

Sala, G., & Gobet, F. (2019). Cognitive training does not enhance general cognition. Trends in Cognitive Sciences, 23(1), 9–20. https://doi.org/10.1016/j.tics.2018.10.004CrossRefPubMed

Sala, G., & Gobet, F. (2020). Working memory training in typically developing children: A multilevel meta-analysis. Psychonomic Bulletin & Review, 27(3), 423–434. https://doi.org/10.3758/s13423-019-01681-yCrossRef

Savović, J., Jones, H. E., Altman, D. G., Harris, R. J., Jüni, P., Pildal, J., Als-Nielsen, B., Balk, E. M., Gluud, C., Gluud, L. L., Ioannidis, J. P. A., Schulz, K. F., Beynon, R., Welton, N. J., Wood, L., Moher, D., Deeks, J. J., & Sterne, J. A. C. (2012). Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Annals of Internal Medicine, 157(6), 429–438. https://doi.org/10.7326/0003-4819-157-6-201209180-00537CrossRefPubMed

Schwaighofer, M., Fischer, F., & Buehner, M. (2015). Does working memory training transfer? A meta-analysis including training conditions as moderators. Educational Psychologist, 50(2), 138–166. https://doi.org/10.1080/00461520.2015.1036274CrossRef

Scionti, N., Cavallero, M., Zogmaister, C., & Marzocchi, G. M. (2020). Is cognitive training effective for improving executive functions in preschoolers? A systematic review and meta-analysis. Frontiers in Psychology, 10. https://www.frontiersin.org/articles/10.3389/fpsyg.2019.02812

Shenkin, S. D., Harrison, J. K., Wilkinson, T., Dodds, R. M., & Ioannidis, J. P. A. (2017). Systematic reviews: Guidance relevant for studies of older people. Age and Ageing, 46(5), 722–728. https://doi.org/10.1093/ageing/afx105CrossRefPubMedCentralPubMed

Shrout, P. E., & Rodgers, J. L. (2018). Psychology, science, and knowledge construction: Broadening perspectives from the replication crisis. Annual Review of Psychology, 69(1), 487–510. https://doi.org/10.1146/annurev-psych-122216-011845CrossRefPubMed

Siegel, M., Eder, J. S. N., Wicherts, J. M., & Pietschnig, J. (2021). Times are changing, bias isn’t: A meta-meta-analysis on publication bias detection practices, prevalence rates, and predictors in industrial/organizational psychology. Journal of Applied Psychology, 107(11), 2013. https://doi.org/10.1037/apl0000991CrossRefPubMed

Signorell, A., Aho, K., Alfons, A., Anderegg, N., Aragon, T., Arppe, A., ... & Borchers, H. W. (2019). DescTools: Tools for descriptive statistics. R package version 0.99, 28, 17.

Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632CrossRefPubMed

Simons, D. J., Boot, W. R., Charness, N., Gathercole, S. E., Chabris, C. F., Hambrick, D. Z., & Stine-Morrow, E. A. L. (2016). Do “brain-training” programs work? Psychological Science in the Public Interest, 17(3), 103–186. https://doi.org/10.1177/1529100616661983CrossRefPubMed

Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014). P-curve: A key to the file-drawer. Journal of Experimental Psychology: General, 143(2), 534. https://doi.org/10.1037/a0033242

Spencer-Smith, M., & Klingberg, T. (2015). Benefits of a working memory training program for inattention in daily life: A systematic review and meta-analysis. PLoS ONE, 10(3), e0119522. https://doi.org/10.1371/journal.pone.0119522CrossRefPubMedCentralPubMed

Stanley, T. D., & Doucouliagos, H. (2014). Meta‐regression approximations to reduce publication selection bias. Research Synthesis Methods, 5(1), 60–78. https://doi.org/10.1002/jrsm.1095

Tanner-Smith, E. E., Tipton, E., & Polanin, J. R. (2016). Handling complex meta-analytic data structures using robust variance estimates: A tutorial in R. Journal of Developmental and Life-Course Criminology, 2(1), 85–112. https://doi.org/10.1007/s40865-016-0026-5CrossRef

Teixeira-Santos, A. C., Moreira, C. S., Magalhães, R., Magalhães, C., Pereira, D. R., Leite, J., Carvalho, S., & Sampaio, A. (2019). Reviewing working memory training gains in healthy older adults: A meta-analytic review of transfer for cognitive outcomes. Neuroscience & Biobehavioral Reviews, 103, 163–177. https://doi.org/10.1016/j.neubiorev.2019.05.009CrossRef

Tetlow, A. M., & Edwards, J. D. (2017). Systematic literature review and meta-analysis of commercially available computerized cognitive training among older adults. Journal of Cognitive Enhancement, 1(4), 559–575. https://doi.org/10.1007/s41465-017-0051-2CrossRef

Traut, H. J., Guild, R. M., & Munakata, Y. (2021). Why does cognitive training yield inconsistent benefits? A meta-analysis of individual differences in baseline cognitive abilities and training outcomes. Frontiers in Psychology, 12. https://www.frontiersin.org/articles/10.3389/fpsyg.2021.662139

Tricco, A. C., Cogo, E., Page, M. J., Polisena, J., Booth, A., Dwan, K., MacDonald, H., Clifford, T. J., Stewart, L. A., Straus, S. E., & Moher, D. (2016). A third of systematic reviews changed or did not specify the primary outcome: A PROSPERO register study. Journal of Clinical Epidemiology, 79, 46–54. https://doi.org/10.1016/j.jclinepi.2016.03.025CrossRefPubMed

van Aert, R. C., Wicherts, J. M., & van Assen, M. A. (2016). Conducting meta-analyses based on p values: Reservations and recommendations for applying p-uniform and p-curve. Perspectives on Psychological Science, 11(5), 713–729. https://doi.org/10.1177/1745691616650874

van Aert, R. C. M., Wicherts, J. M., & van Assen, M. A. L. M. (2019). Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis. PLoS ONE, 14(4), e0215052. https://doi.org/10.1371/journal.pone.0215052CrossRefPubMedCentralPubMed

van den Akker, O. R., van Assen, M. A. L. M., Enting, M., de Jonge, M., Ong, H. H., Rüffer, F., Schoenmakers, M., Stoevenbelt, A. H., Wicherts, J. M., & Bakker, M. (2023). Selective hypothesis reporting in psychology: Comparing preregistrations and corresponding publications. Advances in Methods and Practices in Psychological Science, 6(3). https://doi.org/10.1177/25152459231187988

Van den Noortgate, W., López-López, J. A., Marín-Martínez, F., & Sánchez-Meca, J. (2015). Meta-analysis of multiple outcomes: A multilevel approach. Behavior Research Methods, 47(4), 1274–1294. https://doi.org/10.3758/s13428-014-0527-2CrossRefPubMed

Veroniki, A. A., Jackson, D., Viechtbauer, W., Bender, R., Bowden, J., Knapp, G., Kuss, O., Higgins, J. P., Langan, D., & Salanti, G. (2016). Methods to estimate the between-study variance and its uncertainty in meta-analysis. Research Synthesis Methods, 7(1), 55–79. https://doi.org/10.1002/jrsm.1164CrossRefPubMed

Vevea, J. L., & Woods, C. M. (2005). Publication bias in research synthesis: Sensitivity analysis using a priori weight functions. Psychological Methods, 10(4), 428. https://doi.org/10.1037/1082-989X.10.4.428

Voracek, M., Kossmeier, M., & Tran, U. S. (2019). Which data to meta-analyze, and how? Zeitschrift Für Psychologie, 227(1), 64–82. https://doi.org/10.1027/2151-2604/a000357CrossRef

Wallach, J. D., Boyack, K. W., & Ioannidis, J. P. A. (2018). Reproducible research practices, transparency, and open access data in the biomedical literature, 2015–2017. PLOS Biology, 16(11), e2006930. https://doi.org/10.1371/journal.pbio.2006930CrossRefPubMedCentralPubMed

Weicker, J., Villringer, A., & Thöne-Otto, A. (2016). Can impaired working memory functioning be improved by training? A meta-analysis with a special focus on brain injured patients. Neuropsychology, 30(2), 190–212. https://doi.org/10.1037/neu0000227CrossRefPubMed

Wicherts, J. M., Veldkamp, C. L. S., Augusteijn, H. E. M., Bakker, M., van Aert, R. C. M., & van Assen, M. A. L. M. (2016). Degrees of freedom in planning, running, analyzing, and reporting psychological studies: A checklist to avoid p-hacking. Frontiers in Psychology, 7. https://www.frontiersin.org/articles/10.3389/fpsyg.2016.01832

Wickham, H., Averick, M., Bryan, J., Chang, W., McGowan, L. D., François, R., Grolemund, G., Hayes, A., Henry, L., Hester, J., Kuhn, M., Pedersen, T. L., Miller, E., Bache, S. M., Müller, K., Ooms, J., Robinson, D., Seidel, D. P., Spinu, V., & … Yutani, H. (2019). Welcome to the Tidyverse. Journal of Open Source Software, 4(43), 1686. https://doi.org/10.21105/joss.01686

Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., Bouwman, J., Brookes, A. J., Clark, T., Crosas, M., Dillo, I., Dumon, O., Edmunds, S., Evelo, C. T., Finkers, R., & ... Mons, B. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3(1), Article 1. https://doi.org/10.1038/sdata.2016.18

Woolf, C., Lampit, A., Shahnawaz, Z., Sabates, J., Norrie, L. M., Burke, D., Naismith, S. L., & Mowszowski, L. (2022). A systematic review and meta-analysis of cognitive training in adults with major depressive disorder. Neuropsychology Review, 32(2), 419–437. https://doi.org/10.1007/s11065-021-09487-3

Wykes, T., Huddy, V., Cellard, C., McGurk, S. R., & Czobor, P. (2011). A meta-analysis of cognitive remediation for Schizophrenia: Methodology and effect sizes. American Journal of Psychiatry, 168(5), 472–485. https://doi.org/10.1176/appi.ajp.2010.10060855CrossRefPubMed

Yan, X., Wei, S., & Liu, Q. (2023). Effect of cognitive training on patients with breast cancer reporting cognitive changes: A systematic review and meta-analysis. BMJ Open, 13(1). https://doi.org/10.1136/bmjopen-2021-058088

Yank, V., Rennie, D., & Bero, L. A. (2007). Financial ties and concordance between results and conclusions in meta-analyses: Retrospective cohort study. BMJ, 335(7631), 1202–1205. https://doi.org/10.1136/bmj.39376.447211.BECrossRefPubMedCentralPubMed

Zhang, H., Huntley, J., Bhome, R., Holmes, B., Cahill, J., Gould, R. L., Wang, H., Yu, X., & Howard, R. (2019). Effect of computerised cognitive training on cognitive outcomes in mild cognitive impairment: A systematic review and meta-analysis. BMJ OPEN, 9(8). https://doi.org/10.1136/bmjopen-2018-027062

Title: Transparency in Cognitive Training Meta-analyses: A Meta-review
Authors: Alejandro Sandoval-Lentisco
Rubén López-Nicolás
Miriam Tortajada
José Antonio López-López
Julio Sánchez-Meca
Publication date: 19-04-2024
Publisher: Springer US
Published in: Neuropsychology Review
Print ISSN: 1040-7308
Electronic ISSN: 1573-6660
DOI: https://doi.org/10.1007/s11065-024-09638-2

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Transparency in Cognitive Training Meta-analyses: A Meta-review

Abstract

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Please log in to get access to this content