Common Errors in Statistics (and How to Avoid Them), 4th Edition

NULL OR NIL HYPOTHESIS

A major research failing seems to be the exploration of uninteresting or even trivial questions. … In the 347 sampled articles in Ecology containing null hypotheses tests, we found few examples of null hypotheses that seemed biologically plausible.

—Anderson, Burnham, and Thompson [2000].

We do not perform an experiment to find out if two varieties of wheat or two drugs are equal. We know in advance, without spending a dollar on an experiment, that they are not equal.

—Deming [1975].

Test only relevant null hypotheses.

The null hypothesis has taken on an almost mythic role in contemporary statistics. Obsession with the null (more accurately spelled and pronounced nil), has been allowed to shape the direction of our research. We have let the tool use us instead of us using the tool.³

Virtually any quantifiable hypothesis can be converted into null form. There is no excuse and no need to be content with a meaningless nil.

For example, suppose we want to test that a given treatment will decrease the need for bed rest by at least three days. Previous trials have convinced us that the treatment will reduce the need for bed rest to some degree, so merely testing that the treatment has a positive effect would yield no new information. Instead, we would subtract three from each observation and then test the nil hypothesis that the mean value is zero.

We often will want to test that an effect is inconsequential, not zero but close to it, smaller than d, say, where ...