Effective Amazon Machine Learning

Evaluating our model with create-evaluation

Our model is now trained and we would like to evaluate it on the evaluation subset. For that, we will use the create-evaluation CLI command:

1. Generate the skeleton:

        $ aws machinelearning create-evaluation --generate-cli-skeleton >          eval_ames_housing_001.json

2. Configure the parameter file:

        {            "EvaluationId": "ch8_ames_housing_001",            "EvaluationName": "[EVL] Ames Housing 001",            "MLModelId": "ch8_ames_housing_001",            "EvaluationDataSourceId": "ch8_ames_housing_002"        }

3. Launch the evaluation creation:

        $ aws machinelearning create-evaluation --cli-input-json         file://eval_ames_housing_001.json

4. Get the evaluation information:

        $ aws machinelearning get-evaluation --evaluation-id  ch8_ames_housing_001 ...