Mastering Machine Learning Algorithms

Smoothness assumption

Let's consider a real-valued function f(x) and the corresponding metric spaces X and Y. Such a function is said to be Lipschitz-continuous if:

In other words, if two points x₁ and x₂ are near, the corresponding output values y₁ and y₂ cannot be arbitrarily far from each other. This condition is fundamental in regression problems where a generalization is often required for points that are between training samples. For example, if we need to predict the output for a point x_t : x₁ < x_t < x₂ and the regressor is Lipschitz-continuous, we can be sure that y_t will be correctly bounded by y₁ and y₂. This condition is often called ...