A Heaviside step function is a basic discontinuous function that compares values against a simple threshold and is used for classification where the input data is linearly separable. The neuron is activated if the weighted sum plus a bias exceeds a certain threshold, denoted by the Greek letter theta (θ) in the equation below. If it does not, the neuron is not activated. The following step function is an example of a Heaviside step function that is bounded between 1 and -1:
This Heaviside step function is illustrated in Figure 3.6:
Figure 3.6: Heaviside step activation function
