Common Errors in Statistics (and How to Avoid Them), 4th Edition

A CONJECTURE

A great deal of publicity has heralded the arrival of new and more powerful data mining methods, among them neural networks along with dozens of unspecified proprietary algorithms. In our limited experience, none of these have lived up to expectations; see a report of our tribulations in Good [2001, Section 7.6]. Most of the experts we have consulted have attributed this failure to the small size of our test dataset: 400 observations each with 30 variables. In fact, many publishers of data mining software assert that their wares are designed solely for use with terrabytes of information.

This observation has led to our putting our experience in the form of the following conjecture.

If m points are required to determine a univariate regression line with sufficient precision, then, it will take at least mⁿ observations and perhaps n!mⁿ observations to appropriately characterize and evaluate a regression model with n variables.