When using a CNN, since one of the objective functions is to minimize the evaluated cost, we must define an optimizer. Using the most common optimizer , such as SGD, the learning rates must scale with 1/T to get convergence, where T is the number of iterations. Adam or RMSProp try to overcome this limitation automatically by adjusting the step size so that the step is on the same scale as the gradients. In addition, in the previous example, we have used Adam optimizer, which performs well in most cases.
Nevertheless, if you are training a neural network but computing the gradients is mandatory, using the RMSPropOptimizer function (which implements the RMSProp algorithm) is a better idea since it would be the faster ...
