O'Reilly logo

Experiment!: Website conversion rate optimization with A/B and multivariate testing by Colin McFarland

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

7. On Analysis

Trusting your findings

Understanding the results of your experiments can be a complicated affair. Luckily, there are a few simple methods to make sense of all the data your experiments accumulate and give you confidence in your findings.

Statistical significance

In your experiments, you attempt to hold all variables constant except the thing you want to investigate. The reality, of course, is that many variables are not constant. You’re comparing similar groups, but those groups themselves will have differences that introduce noise. That’s where statistical significance comes in (Figure 7.1).

Image

Figure 7.1. Statistical significance ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required