Ignoring Your Baseline
During a split test conducted by SiteTuners, the company saw performance improve on the challenger recipe from a 4.63 percent conversion rate in week one to 5.03 percent during week two. Normally you would be happy to see this kind of increase in conversion during your data collection period. But was it really a good thing?
On closer inspection, it turned out that the original baseline recipe had a conversion rate of 3.59 percent during week one and 6.18 percent in week two. In other words, during week one, our challenger recipe had a 29 percent advantage over the baseline. During week two it was actually 19 percent worse!
What had changed? Week two was a holiday week. Presumably the composition of the audience had changed. ...
Get Landing Page Optimization: The Definitive Guide to Testing and Tuning for Conversions, 2nd Edition now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
