Web Performance Daybook Volume 2

Chapter 24. Why You’re Probably Reading Your Performance Measurement Results Wrong (At Least You’re in Good Company)

Joshua Bixby

One of my favorite books of 2011 was Thinking, Fast and Slow by the Nobel Prize-winning psychologist Daniel Kahneman. In his book, Kahneman identifies the two systems of thought that are constantly warring inside our heads:

• System 1, which is fast and intuitive

• System 2, which is slow and logical

Almost invariably, System 1 is flawed, yet we helplessly rely on it. We also have a painful tendency to think we’re applying System 2 to our thinking, when in fact it’s just an intellectually tarted up version of System 1.

Kahneman offers a nifty little test of this thinking:

A certain town is served by two hospitals. In the larger hospital about 45 babies are born each day, and in the smaller hospital about 15 babies are born each day. As you know, about 50% of all babies are boys. However the exact percentage varies from day to day. Sometimes it may be higher than 50%, sometimes lower. For a period of 1 year, each hospital recorded the days on which more than 60% of the babies born were boys. Which hospital do you think recorded more such days?

1. The larger hospital

2. The smaller hospital

3. About the same (that is, within 5% of each other)

The correct answer is B, the smaller hospital. But as Kahneman notes, “When this question was posed to a number of undergraduate students, 22% said A; 22% said B; and 56% said C. Sampling theory entails that the expected number ...