O'Reilly logo
  • Victor Bos thinks this is interesting:

The curse of dimensionality will apply to our machine learning algorithms because as the number of input dimensions gets larger, we will need more data to enable the algorithm to generalise sufficiently well. Our algorithms try to separate data into classes based on the features; therefore as the number of features increases, so will the number of datapoints we need. For this reason, we will often have to be careful about what information we give to the algorithm, meaning that we need to understand something about the data in advance.


Cover of Machine Learning, 2nd Edition


Moreover, the number of data points has to grow much faster. Does it have to grow at an exponential rate of the number of features?