As our agent transitions from state to state, by taking actions, it receives a reward. The agent can learn online by using each state, action, and reward as training input. After every action, the agent will update it's neural network weights, hopefully getting smarter along the way. This is the basic idea of online learning. The agent learns as it goes, just like you and I do.
The shortcomings of this naive type of online learning are somewhat obvious and two-fold:
- We throw away our experience after we experience it.
- The experiences we work through are highly correlated to each other and we will overfit to the most recent experiences. Interestingly enough, this is something humans suffer from too, called availability bias. ...
