Quantifying the User Experience, 2nd Edition

Chapter 5

Is there a statistical difference between designs?

Abstract

The goal of this chapter is to cover methods for determining if a difference is statistically significant and how large or small of a difference likely exists in the untested population. It is important to account for chance differences when comparing two designs or products. To do this, you need to find a p-value from the appropriate statistical test. To understand the likely range of the difference between designs or products, you should compute a confidence interval around the difference. To determine which statistical test you need to use, you need to identify whether your outcome measure is binary or continuous and whether you have the same users in each group (within-subjects) ...