A fully RNN is a network of neurons, each with a directed (one-way) connection to every other neuron. Each neuron has a time-varying, real-valued activation. Each connection has a modifiable real-valued weight. Input neurons, output neurons, and hidden neurons are expected. This type of network is a multilayer perceptron with the previous set of hidden unit activations feeding back into the network along with the inputs, as shown in the following figure:
At each step, each non-input unit calculates its current activation as a nonlinear function of the weighted sum of activations of all units that connect to ...
