Statistics in Brief: Interpretation and Use of p Values: All p Values Are Not Equal

Excerpt

In a formal hypothesis testing situation, a question is frequently asked about differences between groups, and based on that question an experiment is designed, data are collected, and a statistical test is performed, usually resulting in one or more p values. The p value resulting from a hypothesis test is heuristically defined as a probability measure of how much evidence there is against the null hypothesis of the test, that is, no difference exists [1]. When the p value is small (however defined), then a decision might be made to reject the null hypothesis and accept the alternative hypothesis that a difference exists. However, in many (if not most) situations today, the reader of a medical journal has made no such prior definition of what is small, or exactly what use should be made of any given p value. Thus, despite the exact definition of what a p value means, how p values in general should be interpreted or how they should influence the readers of medical journals is not clear. Although the definitions involving hypothesis testing and p values are precise, the interpretation and use of the resulting p values are much more subjective and individual processes. …