How to Measure Anything Workbook: Finding the Value of Intangibles in Business

CHAPTER 12 The Ultimate Measurement Instrument: Human Judges

LEARNING OBJECTIVES

• Describe and give an example of the anchoring bias, the halo/horns effect, the bandwagon bias, and emerging preferences.
• Describe how research by Meehl, Dawes, and others has undermined the assumed authority of experts.
• Explain the benefit of Rasch Models, and create and calculate a scenario in which they would be helpful.
• Explain the benefit of Lens Models, and create and calculate a scenario in which they would be helpful.
• Explain several problems with subjective weighted arbitrary scoring methods.
• Describe the relative efficiency of the various types of models described in the chapter.

CHAPTER OVERVIEW

Chapter begins with an overview of the advantages and disadvantages of the human mind as a measurement instrument. Despite our strengths, we are prone to a variety of biases, including anchoring, the halo/horns effect, bandwagon bias, and emerging preferences. There is a large, consistent body of research demonstrating the inferiority of unaided human intuition relative to quantitative models across a range of domains. However, human experts can still be used as a type of measurement instrument, especially after controlling for some known types of errors.

The Rasch Model provides a simple way to adjust probabilities in situations like differing standards among judges in their ratings. Similar improvements can be achieved with Lens Models, which use hypothetical scenarios that are rated by ...