Machine Learning with Apache Spark Quick Start Guide

Activation function

Once the weighted input plus a bias is calculated, the activation function, denoted by the Greek letter phi (Φ), is used to determine the output of the neuron and whether it activates or not. To make this determination, an activation function is typically a non-linear function bounded between two values, thereby adding non-linearity to ANNs. As most real-world data tends to be non-linear in nature when it comes to complex use cases, we require that ANNs have the capability to learn these non-linear concepts or representations. This is enabled by non-linear activation functions. Examples of activation functions include a Heaviside step function, a sigmoid function, and a hyperbolic tangent function.