Skip to main content
Top
Published in: Neuroinformatics 3/2015

01-07-2015 | Original Article

Bootstrapping fMRI Data: Dealing with Misspecification

Authors: Sanne P. Roels, Beatrijs Moerkerke, Tom Loeys

Published in: Neuroinformatics | Issue 3/2015

Login to get access

Abstract

The validity of inference based on the General Linear Model (GLM) for the analysis of functional magnetic resonance imaging (fMRI) time series has recently been questioned. Bootstrap procedures that partially avoid modeling assumptions may offer a welcome solution. We empirically compare two voxelwise GLM-based bootstrap approaches: a semi-parametric approach, relying solely on a model for the expected signal; and a fully parametric bootstrap approach, requiring an additional parameterization of the temporal structure. While the fully parametric approach assumes independent whitened residuals, the semi-parametric approach relies on independent blocks of residuals. The evaluation is based on inferential properties and the potential to reproduce important data characteristics. Different noise structures and data-generating mechanisms for the signal are simulated. When the model for the noise and expected signal is correct, we find that the fully parametric approach works well, with respect to both inference and reproduction of data characteristics. However, in the presence of misspecification, the fully parametric approach can be improved with additional blocking. The semi-parametric approach performs worse than the (fully) parametric approach with respect to inference but achieves comparable results as the parametric approach with additional blocking with respect to image reproducibility. We demonstrate that when the expected signal is incorrect GLM-based bootstrapping can overcome the poor performance of classical (non-bootstrap) parametric inference. We illustrate both approaches on a study exploring the neural representation of object representation in the visual pathway.
Appendix
Available only for authorised users
Footnotes
1
In a pilot study we looked at alternatives such as e.g. circular bootstrap, but no remarkable differences were observed. Results are not shown.
 
2
In Table S1, in the Supplementary Material, the observed empirical test sizes are presented for both a correctly specified and an incorrectly specified expected signal.
 
3
The graphical depictions of the median values per volume via the smooth densities functions illustrate the performances to reproduce data characteristics for the fully parametric approach and the parametric and semi-parametric approach with block lengths of 20 and 40. Detailed results for all parametric and semi-parametric bootstrap approaches are in the Supplementary Tables S2S5
 
Literature
go back to reference Adolf, D., Weston, S., Baecke, S., Luchtmann, M., Bernarding, J., Kropf, S. (2014). Increasing the reliability of data analysis of functional magnetic resonance imaging by applying a new blockwise permutation method. Frontiers in neuroinformatics, 8, 72.PubMedCentralPubMedCrossRef Adolf, D., Weston, S., Baecke, S., Luchtmann, M., Bernarding, J., Kropf, S. (2014). Increasing the reliability of data analysis of functional magnetic resonance imaging by applying a new blockwise permutation method. Frontiers in neuroinformatics, 8, 72.PubMedCentralPubMedCrossRef
go back to reference Aguirre, G. K., Zarahn, E., D’esposito, M. (1998). The variability of human, BOLD hemodynamic responses. NeuroImage, 8(4), 360–369.PubMedCrossRef Aguirre, G. K., Zarahn, E., D’esposito, M. (1998). The variability of human, BOLD hemodynamic responses. NeuroImage, 8(4), 360–369.PubMedCrossRef
go back to reference Bellec, P., Perlbarg, V., Evans, A.C. (2009). Bootstrap generation and evaluation of an fMRI simulation database. Magnetic Resonance Imaging, 27(10), 1382–1396.PubMedCentralPubMedCrossRef Bellec, P., Perlbarg, V., Evans, A.C. (2009). Bootstrap generation and evaluation of an fMRI simulation database. Magnetic Resonance Imaging, 27(10), 1382–1396.PubMedCentralPubMedCrossRef
go back to reference Bullmore, E., Fadili, J., Maxim, V., Sendur, L., Whitcher, B., Suckling, J., Brammer, M., Breakspear, M. (2004). Wavelets and functional magnetic resonance imaging of the human brain. NeuroImage, 23(Suppl 1), S234–S249.PubMedCrossRef Bullmore, E., Fadili, J., Maxim, V., Sendur, L., Whitcher, B., Suckling, J., Brammer, M., Breakspear, M. (2004). Wavelets and functional magnetic resonance imaging of the human brain. NeuroImage, 23(Suppl 1), S234–S249.PubMedCrossRef
go back to reference Buxton, R.B., Uludağ, K., Dubowitz, D.J., Liu, T.T. (2004). Modeling the hemodynamic response to brain activation. NeuroImage, 23(Suppl 1), S220–S233.PubMedCrossRef Buxton, R.B., Uludağ, K., Dubowitz, D.J., Liu, T.T. (2004). Modeling the hemodynamic response to brain activation. NeuroImage, 23(Suppl 1), S220–S233.PubMedCrossRef
go back to reference Carp, J (2012). The secret lives of experiments: Methods reporting in the fMRI literature. NeuroImage, 63(1), 289–300.PubMedCrossRef Carp, J (2012). The secret lives of experiments: Methods reporting in the fMRI literature. NeuroImage, 63(1), 289–300.PubMedCrossRef
go back to reference Chatfield, C. (2000). The analysis of time series. An introduction. Texts in statistical science. Boca Raton: Chapman & Hall/CRC. Chatfield, C. (2000). The analysis of time series. An introduction. Texts in statistical science. Boca Raton: Chapman & Hall/CRC.
go back to reference Cochrane, D., & Orcutt, G. (1949). Application of least squares regression to relationships containing auto-correlated error terms. Journal of the American Statistical Association, 44(245), 32–61. Cochrane, D., & Orcutt, G. (1949). Application of least squares regression to relationships containing auto-correlated error terms. Journal of the American Statistical Association, 44(245), 32–61.
go back to reference Darki, F., & Oghabian, M. A. (2013). False positive control of activated voxels in single fMRI analysis using bootstrap resampling in comparison to spatial smoothing. Magnetic Resonance Imaging, 31(8), 1331–1337.PubMedCrossRef Darki, F., & Oghabian, M. A. (2013). False positive control of activated voxels in single fMRI analysis using bootstrap resampling in comparison to spatial smoothing. Magnetic Resonance Imaging, 31(8), 1331–1337.PubMedCrossRef
go back to reference Davison, A., & Hinkley, D. (1997). Bootstrap methods and their application. Cambridge: University Press.CrossRef Davison, A., & Hinkley, D. (1997). Bootstrap methods and their application. Cambridge: University Press.CrossRef
go back to reference Eklund, A., Andersson, M., Josephson, C., Johannesson, M., Knutsson, H. (2012). Does parametric fMRI analysis with SPM yield valid results? An empirical study of 1484 rest datasets. NeuroImage, 61(3), 565–578.PubMedCrossRef Eklund, A., Andersson, M., Josephson, C., Johannesson, M., Knutsson, H. (2012). Does parametric fMRI analysis with SPM yield valid results? An empirical study of 1484 rest datasets. NeuroImage, 61(3), 565–578.PubMedCrossRef
go back to reference Friston, K.J., Fletcher, P., Josephs, O., Holmes, A., Rugg, M.D., Turner, R. (1998). Event-related fMRI: characterizing differential responses. NeuroImage, 7(1), 30–40.PubMedCrossRef Friston, K.J., Fletcher, P., Josephs, O., Holmes, A., Rugg, M.D., Turner, R. (1998). Event-related fMRI: characterizing differential responses. NeuroImage, 7(1), 30–40.PubMedCrossRef
go back to reference Glaser, D., & Friston, K. (2007). Covariance components In Friston, K., Ashburner, J., Kiebel, S., Nichols, T., Penny, W. (Eds.), Statistical Parametric Mapping: The Analysis of Functional Brain Images, (pp. 140–147). London: Academic. Glaser, D., & Friston, K. (2007). Covariance components In Friston, K., Ashburner, J., Kiebel, S., Nichols, T., Penny, W. (Eds.), Statistical Parametric Mapping: The Analysis of Functional Brain Images, (pp. 140–147). London: Academic.
go back to reference Glover, G.H. (1999). Deconvolution of impulse response in event-related BOLD fMRI. NeuroImage, 9(4), 416–429.PubMedCrossRef Glover, G.H. (1999). Deconvolution of impulse response in event-related BOLD fMRI. NeuroImage, 9(4), 416–429.PubMedCrossRef
go back to reference Grinband, J., Wager, T.D., Lindquist, M., Ferrera, V.P., Hirsch, J. (2008). Detection of time-varying signals in event-related fMRI designs. NeuroImage, 43(3), 509–520.PubMedCentralPubMedCrossRef Grinband, J., Wager, T.D., Lindquist, M., Ferrera, V.P., Hirsch, J. (2008). Detection of time-varying signals in event-related fMRI designs. NeuroImage, 43(3), 509–520.PubMedCentralPubMedCrossRef
go back to reference Gudbjartsson, H., & Patz, S. (2005). The rician distribution of noisy MRI data. Magnetic Resonance in Medicine, 34(6), 910–914.CrossRef Gudbjartsson, H., & Patz, S. (2005). The rician distribution of noisy MRI data. Magnetic Resonance in Medicine, 34(6), 910–914.CrossRef
go back to reference Hayfield, T., & Racine, J. (2008). Nonparametric econometrics: the np package. Journal of Statistical Software, 27(5), 1–32. Hayfield, T., & Racine, J. (2008). Nonparametric econometrics: the np package. Journal of Statistical Software, 27(5), 1–32.
go back to reference Henson, R., & Friston, K. (2007). Statistical parametric mapping: the analysis of functional brain images, 193–210. Henson, R., & Friston, K. (2007). Statistical parametric mapping: the analysis of functional brain images, 193–210.
go back to reference Ishai, A., Ungerleider, L.G., Martin, A., Haxby, J.V. (2000). The representation of objects in the human occipital and temporal cortex. Journal of Cognitive Neuroscience, 12(S2), 35–51.PubMedCrossRef Ishai, A., Ungerleider, L.G., Martin, A., Haxby, J.V. (2000). The representation of objects in the human occipital and temporal cortex. Journal of Cognitive Neuroscience, 12(S2), 35–51.PubMedCrossRef
go back to reference Jenkinson, M., Pechaud, M., Smith, S.M.S. (2005). Bet2: Mr-based estimation of brain, skull and scalp surfaces. In Eleventh Annual Meeting of the Organization for Human Brain Mapping. Jenkinson, M., Pechaud, M., Smith, S.M.S. (2005). Bet2: Mr-based estimation of brain, skull and scalp surfaces. In Eleventh Annual Meeting of the Organization for Human Brain Mapping.
go back to reference Kutner, M., Nachtsheim, C., Neter, J., Li, W. (2005). Applied linear models. New York: McGraw-Hill Irwin. Kutner, M., Nachtsheim, C., Neter, J., Li, W. (2005). Applied linear models. New York: McGraw-Hill Irwin.
go back to reference Lahiri, S.N. (2003). Resampling methods for dependent data. Springer series in statistics: Springer. Lahiri, S.N. (2003). Resampling methods for dependent data. Springer series in statistics: Springer.
go back to reference Laird, A.R., Rogers, B.P., Meyerand, M.E. (2004). Comparison of Fourier and wavelet resampling methods. Magnetic resonance in medicine : official journal of the Society of Magnetic Resonance in Medicine / Society of Magnetic Resonance in Medicine, 51(2), 418–422.CrossRef Laird, A.R., Rogers, B.P., Meyerand, M.E. (2004). Comparison of Fourier and wavelet resampling methods. Magnetic resonance in medicine : official journal of the Society of Magnetic Resonance in Medicine / Society of Magnetic Resonance in Medicine, 51(2), 418–422.CrossRef
go back to reference Lazar, N. A. (2008). Noise and data preprocessing. In: The Statistical Analysis of Functional MRI data (pp 37–51), chapter 3. Springer. Lazar, N. A. (2008). Noise and data preprocessing. In: The Statistical Analysis of Functional MRI data (pp 37–51), chapter 3. Springer.
go back to reference Lenoski, B., Baxter, L C., Karam, L.J., Maisog, J., Debbins, J. (2008). On the performance of autocorrelation estimation algorithms for fmri analysis. IEEE Journal of Selected Topics in Signal Processing, 2, 828–838.CrossRef Lenoski, B., Baxter, L C., Karam, L.J., Maisog, J., Debbins, J. (2008). On the performance of autocorrelation estimation algorithms for fmri analysis. IEEE Journal of Selected Topics in Signal Processing, 2, 828–838.CrossRef
go back to reference Lieberman, M.D., & Cunningham, W.A. (2009). Type I and Type II error concerns in fMRI research: re-balancing the scale. Social Cognitive and Affective Neuroscience, 4(4), 423–428.PubMedCentralPubMedCrossRef Lieberman, M.D., & Cunningham, W.A. (2009). Type I and Type II error concerns in fMRI research: re-balancing the scale. Social Cognitive and Affective Neuroscience, 4(4), 423–428.PubMedCentralPubMedCrossRef
go back to reference Lindquist, M. (2008). The statistical analysis of fMRI data. Statistical Science, 23(4), 439–464.CrossRef Lindquist, M. (2008). The statistical analysis of fMRI data. Statistical Science, 23(4), 439–464.CrossRef
go back to reference Lindquist, M.A., Meng Loh, J., Atlas, L.Y., Wager, T.D. (2009). Modeling the hemodynamic response function in fMRI: efficiency, bias and mis-modeling. NeuroImage, 45(1 Suppl), S187–S198.PubMedCentralPubMedCrossRef Lindquist, M.A., Meng Loh, J., Atlas, L.Y., Wager, T.D. (2009). Modeling the hemodynamic response function in fMRI: efficiency, bias and mis-modeling. NeuroImage, 45(1 Suppl), S187–S198.PubMedCentralPubMedCrossRef
go back to reference Lindquist, M.A., & Wager, T.D. (2007). Validity and power in hemodynamic response modeling: a comparison study and a new approach. Human Brain Mapping, 28(8), 764–784.PubMedCentralPubMedCrossRef Lindquist, M.A., & Wager, T.D. (2007). Validity and power in hemodynamic response modeling: a comparison study and a new approach. Human Brain Mapping, 28(8), 764–784.PubMedCentralPubMedCrossRef
go back to reference Luo, W.-L., & Nichols, T.E. (2003). Diagnosis and exploration of massively univariate neuroimaging models. NeuroImage, 19, 1014–1032.PubMedCrossRef Luo, W.-L., & Nichols, T.E. (2003). Diagnosis and exploration of massively univariate neuroimaging models. NeuroImage, 19, 1014–1032.PubMedCrossRef
go back to reference Monti, M.M. (2011). Statistical analysis of fMRI time-series: A critical review of the GLM approach. Frontiers in Human Neuroscience, 5 (March), 28.PubMedCentralPubMed Monti, M.M. (2011). Statistical analysis of fMRI time-series: A critical review of the GLM approach. Frontiers in Human Neuroscience, 5 (March), 28.PubMedCentralPubMed
go back to reference Nichols, T., & Hayasaka, S. (2003). Controlling the familywise error rate in functional neuroimaging: a comparative review. Statistical Methods in Medical Research, 12(5), 419–446.PubMedCrossRef Nichols, T., & Hayasaka, S. (2003). Controlling the familywise error rate in functional neuroimaging: a comparative review. Statistical Methods in Medical Research, 12(5), 419–446.PubMedCrossRef
go back to reference Politis, D.N., & White, H. (2004). Automatic block-length selection for the dependent bootstrap. Econometric Reviews, 23(1), 53–70.CrossRef Politis, D.N., & White, H. (2004). Automatic block-length selection for the dependent bootstrap. Econometric Reviews, 23(1), 53–70.CrossRef
go back to reference Core Team, R. (2013). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. Core Team, R. (2013). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.
go back to reference Razavi, M., Grabowski, T.J., Vispoel, W.P., Monahan, P., Mehta, S., Eaton, B., Bolinger, L. (2003). Model assessment and model building in fMRI. Human Brain Mapping, 20(4), 227–238.PubMedCrossRef Razavi, M., Grabowski, T.J., Vispoel, W.P., Monahan, P., Mehta, S., Eaton, B., Bolinger, L. (2003). Model assessment and model building in fMRI. Human Brain Mapping, 20(4), 227–238.PubMedCrossRef
go back to reference Serences, J.T. (2004). A comparison of methods for characterizing the event-related BOLD timeseries in rapid fMRI. NeuroImage, 21(4), 1690–700.PubMedCrossRef Serences, J.T. (2004). A comparison of methods for characterizing the event-related BOLD timeseries in rapid fMRI. NeuroImage, 21(4), 1690–700.PubMedCrossRef
go back to reference Tabelow, K., & Polzehl, J. (2011). Statistical Parametric Maps for Functional MRI Experiments in R : The Package fmri. Journal of Statistical Software, 44(11). Tabelow, K., & Polzehl, J. (2011). Statistical Parametric Maps for Functional MRI Experiments in R : The Package fmri. Journal of Statistical Software, 44(11).
go back to reference Tang, L., Woodward, W.A., Schucany, W.R. (2008). Undercoverage of wavelet-based resampling confidence intervals. Communications in Statistics - Simulation and Computation, 37(7), 1307–1315.CrossRef Tang, L., Woodward, W.A., Schucany, W.R. (2008). Undercoverage of wavelet-based resampling confidence intervals. Communications in Statistics - Simulation and Computation, 37(7), 1307–1315.CrossRef
go back to reference Thyreau, B., Schwartz, Y., Thirion, B., Frouin, V., Loth, E., Vollstädt-Klein, S., Paus, T., Artiges, E., Conrod, P.J., Schumann, G., Whelan, R., Poline, J.-B. (2012). Very large fMRI study using the IMAGEN database: Sensitivity-specificity and population effect modeling in relation to the underlying anatomy. NeuroImage, 61(1), 295–303.PubMedCrossRef Thyreau, B., Schwartz, Y., Thirion, B., Frouin, V., Loth, E., Vollstädt-Klein, S., Paus, T., Artiges, E., Conrod, P.J., Schumann, G., Whelan, R., Poline, J.-B. (2012). Very large fMRI study using the IMAGEN database: Sensitivity-specificity and population effect modeling in relation to the underlying anatomy. NeuroImage, 61(1), 295–303.PubMedCrossRef
go back to reference Welvaert, M., Durnez, J., Moerkerke, B., Verdoolaeghe, G., Rosseel, Y. (2011). Journal of Statistical Software, 44 (10), 1–18. Welvaert, M., Durnez, J., Moerkerke, B., Verdoolaeghe, G., Rosseel, Y. (2011). Journal of Statistical Software, 44 (10), 1–18.
go back to reference Welvaert, M., & Rosseel, Y. (2012). How ignoring physiological noise can bias the conclusions from fMRI simulation results. Journal of Neuroscience Methods, 211(1), 125–132.PubMedCrossRef Welvaert, M., & Rosseel, Y. (2012). How ignoring physiological noise can bias the conclusions from fMRI simulation results. Journal of Neuroscience Methods, 211(1), 125–132.PubMedCrossRef
go back to reference Westfall, P.H., & Young, S.S. (1993). Resampling-based multiple testing. Examples and methods for p-value adjustment: Wiley. Westfall, P.H., & Young, S.S. (1993). Resampling-based multiple testing. Examples and methods for p-value adjustment: Wiley.
go back to reference Worsley, K.J. (2005). Spatial smoothing of autocorrelations to control the degrees of freedom in fMRI analysis. NeuroImage, 26(2), 635–641.PubMedCrossRef Worsley, K.J. (2005). Spatial smoothing of autocorrelations to control the degrees of freedom in fMRI analysis. NeuroImage, 26(2), 635–641.PubMedCrossRef
go back to reference Zhang, H., Luo, W.-L., Nichols, T.E. (2006). Diagnosis of single-subject and group fMRI data with SPMd. Human Brain Mapping, 27(5), 442–451.PubMedCrossRef Zhang, H., Luo, W.-L., Nichols, T.E. (2006). Diagnosis of single-subject and group fMRI data with SPMd. Human Brain Mapping, 27(5), 442–451.PubMedCrossRef
Metadata
Title
Bootstrapping fMRI Data: Dealing with Misspecification
Authors
Sanne P. Roels
Beatrijs Moerkerke
Tom Loeys
Publication date
01-07-2015
Publisher
Springer US
Published in
Neuroinformatics / Issue 3/2015
Print ISSN: 1539-2791
Electronic ISSN: 1559-0089
DOI
https://doi.org/10.1007/s12021-015-9261-x

Other articles of this Issue 3/2015

Neuroinformatics 3/2015 Go to the issue