Top

BMC Medical Research Methodology

Published in:

Open Access 01-12-2020 | Software

Bayesian alternatives to null hypothesis significance testing in biomedical research: a non-technical introduction to Bayesian inference with JASP

Author: Riko Kelter

Published in: BMC Medical Research Methodology | Issue 1/2020

Abstract

Background

Although null hypothesis significance testing (NHST) is the agreed gold standard in medical decision making and the most widespread inferential framework used in medical research, it has several drawbacks. Bayesian methods can complement or even replace frequentist NHST, but these methods have been underutilised mainly due to a lack of easy-to-use software. JASP is an open-source software for common operating systems, which has recently been developed to make Bayesian inference more accessible to researchers, including the most common tests, an intuitive graphical user interface and publication-ready output plots. This article provides a non-technical introduction to Bayesian hypothesis testing in JASP by comparing traditional tests and statistical methods with their Bayesian counterparts.

Results

The comparison shows the strengths and limitations of JASP for frequentist NHST and Bayesian inference. Specifically, Bayesian hypothesis testing via Bayes factors can complement and even replace NHST in most situations in JASP. While p-values can only reject the null hypothesis, the Bayes factor can state evidence for both the null and the alternative hypothesis, making confirmation of hypotheses possible. Also, effect sizes can be precisely estimated in the Bayesian paradigm via JASP.

Conclusions

Bayesian inference has not been widely used by now due to the dearth of accessible software. Medical decision making can be complemented by Bayesian hypothesis testing in JASP, providing richer information than single p-values and thus strengthening the credibility of an analysis. Through an easy point-and-click interface researchers used to other graphical statistical packages like SPSS can seemlessly transition to JASP and benefit from the listed advantages with only few limitations.

Available only for authorised users

Altman DG. Statistics in medical journals. Stat Med. 1982; 1(1):59–71. https://doi.org/10.1002/sim.4780010109.CrossRef

Altman DG, Gore SM, Gardner MJ, Pocock SJ. Statistical guidelines for contributors to medical journals. Br Med J (Clin Res ed.) 1983; 286(6376):1489–93. https://doi.org/10.1136/bmj.286.6376.1489.CrossRef

Altman DG. Statistics in medical journals: Developments in the 1980s. Stat Med. 1991; 10(12):1897–913. https://doi.org/10.1002/sim.4780101206.CrossRef

Altman DG. Practical Statistics for Medical Research. Boca Raton: Chapman and Hall; 1991, p. 611.

Fisher RA. Statistical Methods for Research Workers. Edinburgh: Oliver and Boyd; 1925.

Neyman J, Pearson ES. Contributions to the theory of testing statistical hypotheses. Stat Res Mem. 1936; 1:1–37.

Colquhoun D. An investigation of the false discovery rate and the misinterpretation of p-values. R Soc Open Sci. 2014; 1(3):140216. https://doi.org/10.1098/rsos.140216, http://arxiv.org/abs/1407.5296.CrossRef

Benjamin DJ, Berger JO. Three recommendations for improving the use of p-values. The Am Stat. 2019; 73(sup1):186–91. https://doi.org/10.1080/00031305.2018.1543135.CrossRef

Ioannidis JPA. Why most published research findings are false. PLoS Med. 2005; 2(8):e124. https://doi.org/10.1371/journal.pmed.0020124.CrossRef

10.

Wasserstein RL, Lazar NA. The ASA’s statement on p-values: context, process, and purpose. The Am Stat. 2016; 70(2):129–33. https://doi.org/10.1080/00031305.2016.1154108, http://arxiv.org/abs/1011.1669.CrossRef

11.

Wasserstein RL, Schirm AL, Lazar NA. Moving to a world beyond “p<0.05”. The Am Stat. 2019; 73(sup1):1–19. https://doi.org/10.1080/00031305.2019.1583913.CrossRef

12.

Matthews R, Wasserstein R, Spiegelhalter D. The ASA’s p-value statement, one year on. Significance. 2017; 14(2):38–41. https://doi.org/10.1111/j.1740-9713.2017.01021.x.CrossRef

13.

Colquhoun D. The problem with p-values. 2016. https://aeon.co/essays/it-s-time-for-science-to-abandonthe-term-statistically-significant. Accessed 11 Oct 2016.

14.

Ioannidis JPA. What have we (not) learnt from millions of scientific papers with p-values?The Am Stat. 2019; 73:20–5. https://doi.org/10.1080/00031305.2018.1447512.CrossRef

15.

Colquhoun D. The reproducibility of research and the misinterpretation of p-values. R Soc Open Sci. 2017; 4(12):171085. https://doi.org/10.1098/rsos.171085.CrossRef

16.

Ioannidis JPA. Why most clinical research is not useful. PLoS Med. 2016; 13(6):1002049. https://doi.org/10.1371/journal.pmed.1002049.CrossRef

17.

McElreath R, Smaldino PE. Replication, communication, and the population dynamics of scientific discovery. PLoS ONE. 2015; 10(8):1–16. https://doi.org/10.1371/journal.pone.0136088.CrossRef

18.

JASP Team. JASP (Version 0.12)[Computer software]. 2020. https://jasp-stats.org/.

19.

Jeffreys H. Scientific Inference. Cambridge: Cambridge University Press; 1931.

20.

R Core, Team. R: A language and environment for statistical computing. R Found Stat Comput. 2019. https://www.r-project.org/.

21.

Open Science Foundation. OSF - Open Science Foundation. https://osf.io/. Accessed 25 Oct 2019.

22.

McElreath R. Statistical Rethinking: A Bayesian Course With Examples in R and Stan. Boca Raton: Chapman & Hall, CRC Press; 2016. http://jeb.sagepub.com/cgi/doi/10.3102/1076998616659752.

23.

Robert C, Casella G. Monte Carlo Statistical Methods. New York: Springer; 2004, p. 645.CrossRef

24.

Kruschke JK. Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan, Second Edition. Oxford: Academic Press; 2015, pp. 1–759. https://doi.org/10.1016/B978-0-12-405888-0.09999-2, http://arxiv.org/abs/arXiv:1011.1669v3.

25.

Cohen J. Statistical Power Analysis for the Behavioral Sciences, 2nd edn. Hillsdale, N.J: Routledge; 1988.

26.

Rouder JN, Speckman PL, Sun D, Morey RD, Iverson G. Bayesian t tests for accepting and rejecting the null hypothesis. Psychon Bull Rev. 2009; 16(2):225–37. https://doi.org/10.3758/PBR.16.2.225.CrossRef

27.

Jeffreys H. Theory of Probability, 3rd edn. Oxford: Oxford University Press; 1961.

28.

Gelman A, Lee D, Guo J. Stan: A probabilistic programming language for Bayesian inference. J Educ Behav Stat. 2015; 40(5):530–43. https://doi.org/10.3102/1076998615606113.CrossRef

29.

van Doorn J, van den Bergh D, Bohm U, Dablander F, Derks K, Draws T, Evans NJ, Gronau QF, Hinne M, Kucharský Š, Ly A, Marsman M, Matzke D, Raj A, Sarafoglou A, Stefan A, Voelkel JG, Wagenmakers E-J. The JASP guidelines for conducting and reporting a Bayesian analysis. PsyArxiv Preprint. 2019. doi:10.31234/osf.io/yqxfr.

30.

Good IJ. Probability and the Weighing of Evidence. London: Charles Griffin; 1950.

31.

Wagenmakers EJ, Love J, Marsman M, Jamil T, Ly A, Verhagen J, Selker R, Gronau QF, Dropmann D, Boutin B, Meerhoff F, Knight P, Raj A, van Kesteren EJ, van Doorn J, Šmíra M, Epskamp S, Etz A, Matzke D, de Jong T, van den Bergh D, Sarafoglou A, Steingroever H, Derks K, Rouder JN, Morey RD. Bayesian inference for psychology. Part II: Example applications with JASP. Psychon Bull Rev. 2018; 25(1):58–76. https://doi.org/10.3758/s13423-017-1323-7.CrossRef

32.

Goss-Sampson MA. Statistical analysis in JASP 0.10.2: A guide for students; 2019.

33.

Navarro DJ, Foxcroft DR, Faulkenberry TJ. Learning statistics with JASP: A tutorial for psychology students and other beginners; 2019. https://learnstatswithjasp.com/.

34.

Etz A, Vandekerckhove J. A Bayesian perspective on the reproducibility project: Psychology. PLoS ONE. 2016; 11(2):0149794. https://doi.org/10.1371/journal.pone.0149794.CrossRef

35.

Moore DS, McCabe GP, Craig BA. Introduction to the Practice of Statistics, 9th edn. New York: Freeman, WH; 2012.

36.

Morey RD, Rouder JN. BayesFactor: Computation of Bayes factors for common designs. 2018. https://cran.r-project.org/package=BayesFactor.

37.

van den Bergh D, van Doorn J, Marsman M, Draws T, van Kesteren E, Derks K, Wagenmakers E. A Tutorial on Conducting and Interpreting a Bayesian ANOVA in JASP. 2019. https://doi.org/10.31234/osf.io/spreb.

38.

Mestek ML, Plaisance E, Grandjean P. The relationship between pedometer-determined and self-reported physical activity and body composition variables in college-aged men and women. J Am Coll Health. 2008; 57(1):39–44. https://doi.org/10.3200/JACH.57.1.39-44.CrossRef

39.

Neyman J, Pearson ES. On the problem of the most efficient tests of statistical hypotheses. Phil Trans R Soc Lond. A. 1933; 231(694-706):289–337. https://doi.org/10.1098/RSTA.1933.0009.CrossRef

40.

Clyde M. Bayesian variable selection and model averaging using Bayesian adaptive sampling. R Package Version 1.5.5.R Package Version 1.5.5. 2018.

41.

Klein JP, van Houwelingen HC, Ibrahim JG, Scheike TH. Handbook of survival analysis. Boca Raton: Taylor & Francis; 2014. https://doi.org/10.1201/b16248.

42.

Ibrahim JG, Chen M-H, Sinha D. Bayesian Survival Analysis. New York: Springer; 2001, p. 481.CrossRef

43.

Faraway JJ. Extending the Linear Model with R : Generalized Linear, Mixed Effects and Nonparametric Regression Models, 2nd edn. New York: Chapman and Hall/CRC; 2016, p. 399. https://doi.org/10.1201/9781315382722.CrossRef

Title: Bayesian alternatives to null hypothesis significance testing in biomedical research: a non-technical introduction to Bayesian inference with JASP
Author: Riko Kelter
Publication date: 01-12-2020
Publisher: BioMed Central
Published in: BMC Medical Research Methodology / Issue 1/2020
Electronic ISSN: 1471-2288
DOI: https://doi.org/10.1186/s12874-020-00980-6

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Bayesian alternatives to null hypothesis significance testing in biomedical research: a non-technical introduction to Bayesian inference with JASP

Abstract

Background

Results

Conclusions

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Background

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2020

An R package for an integrated evaluation of statistical approaches to cancer incidence projection

SARFIMA model prediction for infectious diseases: application to hemorrhagic fever with renal syndrome and comparing with SARIMA

Patient and Public Involvement (PPI) in evidence synthesis: how the PatMed study approached embedding audience responses into the expression of a meta-ethnography

Evaluation of approaches for multiple imputation of three-level data

Recruitment and retention of participants from socioeconomically deprived communities: lessons from the Awareness and Beliefs About Cancer (ABACus3) Randomised Controlled Trial

Including non-concurrent control patients in the analysis of platform trials: is it worth it?