Illuminating Statistical Analysis Using Scenarios and Simulations

1Evidence and Verdicts

Before we focus in on using statistics as evidence to be used in making judgments, let's take a look at a widely used “verdict outcomes framework.” This general framework is useful for framing judgments in a wide range of situations, including those encountered in statistical analysis.

Anytime we use evidence to arrive at a judgment, there are four generic outcomes possible, as shown in Table 1.1. Two outcomes correspond to correct judgments and two correspond to incorrect judgments, although we rarely know whether our judgments are correct or incorrect. Consider a jury trial in U.S. criminal court. Ideally, the jury is always correct, judging innocent defendants not guilty and judging guilty defendants guilty. Evidence is never perfect, though, and so juries will make erroneous judgments, judging innocent defendants guilty or guilty defendants not guilty.

Table 1.1 Verdict outcomes framework.

In U.S. criminal court, the presumption is that a defendant is innocent until “proven” guilty. Further, convention in U.S. criminal court has it that we are more afraid of punishing an innocent person (type I error) than we are of letting a guilty person go unpunished (type II error). Because of this fear, the threshold for a guilty verdict is set high: “Beyond a reasonable doubt.” So, convicting an innocent person should be a relatively unlikely outcome. ...