Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference

5. Would You Rather Lose an Arm or a Leg?

5.1 Introduction

Statisticians can be a sour bunch. Instead of considering their winnings, they only measure how much they have lost. In fact, they consider their wins to be negative losses. But what’s interesting is how they measure their losses.

For example, consider the following:

A meteorologist is predicting the probability of a hurricane striking his city. He estimates, with 95% confidence, that the probability of it not striking is between 99% and 100%. He is very happy with his precision and advises the city that a major evacuation is unnecessary. Unfortunately, the hurricane does strike and the city is flooded.

This stylized example shows the flaw in using a pure accuracy metric to measure ...