Overoptimism in the interpretation of statistics

Excerpt

A letter to the BMJ in 2000 unveiled macroscopic flaws regarding the calculations of means in a manuscript published by the journal [1]. Although such serious mistakes may be the exception, frequently statistical errors have been found in published articles, indicating reviewing system failures [2]. In our experience with intensive care medicine, however, we noticed that editors frequently involve statistical reviewers, taking full account of their revisions and requiring their final evaluation after authors have complied with reviewers’ recommendations. This allows for the filtering of poor quality articles with evident mistakes such as applying exclusion criteria after randomisation, running multiple regression analyses on very small samples without accounting for the event-to-variable ratio, performing infinite bivariate comparisons to rule out basic differences between two study groups, and calculating sensitivities without having all patients submitted to the diagnostic test or reporting areas under the receiver operating characteristic curves values less than 0.5. …