Interpreting the Results
Most A/B tools interpret the results for you; quite often, you can see early trends indicating which design performs best. But be careful: humans are pattern seekers, and it’s easy to see patterns where there are none. One thing to keep in mind here is statistical significance. You need to have some idea of how big the probability is that the results you’re seeing have occurred because of random chance.
The company User Effect[172] has a simple calculator[173] that reveals whether the results of an A/B test with two designs are statistically significant. If they’re not, the calculator also tells you how many more visitors you need to get a significant result.
If you have more than two designs in your A/B test, you can ...
Get Designed for Use now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
