The key to LSTMs is the cell state that is like a conveyor belt. It moves down the stream with minor linear interactions. It's straightforward for data to flow as unchanged:
LSTM networks have the ability to either remove or add information to the cell state that is carefully regulated by structures known as gates.
- The first step in an LSTM network is to determine what information we will be throwing away from the cell state. The decision is made by a sigmoid layer known as the forget gate layer. The layer looks at the previous state h(t-1) and current input x(t) and outputs a number between 0 and 1 for each number in the ...
