The tyranny of statistics in medicine: a critique of unthinking adherence to an arbitrary p value

Excerpt

There is no question in anyone’s mind, and certainly not in ours, that statistics are essential to ensure that clinical trials are properly designed, with sufficient numbers of patients and observations to lead to a meaningful evaluation of the outcome. From that evaluation, also based on appropriate statistical analysis, emanates the design of follow-up trials either with the same treatment (if the outcome is “positive”), or a significantly modified therapy or entirely different approach (if the results are “negative”). It is this binary emphasis on “positive” and “negative” results that this commentary will focus upon, questioning what these terms really mean and how they affect our perception of “therapeutic success”. We will specifically consider the possibility that potentially successful treatments have been overlooked or abandoned because of an inappropriate emphasis on an arbitrarily fixed level of “type 1” or “α” error (namely, declaring the existence of a treatment effect when none exists). We also will consider the case of insufficient emphasis on the possibility of declaring an absence of treatment effect when it actually exists (“type 2” or “β” error). The biological or medical importance of the treatment often seems to be given a secondary role to the statistics imposed on the analysis of the study, which is exactly opposite to what should be the order of importance¹. …