Now that we have interactive reports exposing different aspects of our data, we’re ready to make our first prediction. This forms our fourth agile sprint (Figure 7-1).
When making predictions, we take what we know about the past and use it to infer what will happen in the future. In doing so, we transition from batch processing of historical data to real-time extrapolation about the future. In real terms, our task in this chapter is to take historical flight records and use them to predict things about future flights.
Code examples for this chapter are available at https://github.com/rjurney/Agile_Data_Code_2/tree/master/ch07. Clone the repository and follow along!
git clone https://github.com/rjurney/Agile_Data_Code_2.git
We are all used to predictions in life. Some forecasts are based on statistical inference, and some are simply the opinions of pundits. Statistical inference is increasingly involved in predictions of all kinds. From weather forecasts to insurance actuaries determining rates to the point spread in sports betting or odds in poker to election forecasts, statistical predictions are a part of modern life. Sometimes forecasts are accurate, and sometimes they are inaccurate.
For instance, as I write this, pundits have repeatedly dismissed the Trump presidential candidacy as a joke, ...