Hyperparameter tuning works by running multiple trials in a single training job. Each trial is a complete execution of your training application, with values for your chosen hyperparameters set within the limits you specify. The Cloud ML Engine training service keeps track of the results of each trial and makes adjustments for subsequent trials. When the job is finished, you can get a summary of all the trials, along with the most effective configuration of values according to the criteria you specify.
We want to select those hyperparameters that give the best performance. This amounts to an optimization problem, specifically, the problem of optimizing a function f(x) (that is, performance as a function of ...
