A state-value function is a function that represents the agent's estimate of how good it is to be in a state at time step t. It is denoted by and is usually just called the value function. It represents the agent's prediction of the future reward it would get if it were to end up in state at time step t. Mathematically, it can be represented as follows:
What this expression means is that the value of state under policy ...