Agile by Design

Chapter 19Operationalizing Your Model Deployment Strategy

At the beginning of our book, we discussed three types of analytics: informational, directional, and operational. Once the model development process and the business implementation planning are complete, a new iteration of work begins. In this section, we'll select our final model and prepare it for operational deployment. In some cases, our best model for deployment may not be the best model in terms of performance: For certain business problems, simpler may be better (see Figure 19.1).

Figure 19.1 Operationalized Model Deployment Considerations

Finding the Right Model

Before we get to deployment, we need to determine which models will need to be put into production. In our modeling project, our data scientists used a number of different modeling techniques and created many different models. The models are fairly similar in terms of their ability to predict the target (likely to churn) as well as the predictive variables. If you have many models performing similarly, you have some options for considering which ones will be deployed.

The primary metric used to select the best model is lift. Lift measures how effective a predictive model is by comparing the outcomes obtained both with and without the model. Let's take a look at a decile lift chart in Table 19.1 for a simple response model.

Table 19.1 Example Lift ...