Mastering Machine Learning Algorithms

Activation functions

In general, any continuous and differentiable function could be employed as activation; however, some of them have particular properties that allow achieving a good accuracy while improving the learning process speed. They are commonly used in the state-of-the-art models, and it's important to understand their properties in order to make the most reasonable choice.