Landing Page Optimization: The Definitive Guide to Testing and Tuning for Conversions, 2nd Edition

Not Accounting for Seasonality

Remember that it is a big mistake to conduct sequential testing. In other words, you should never run one version of your landing page design followed by another one. Too many outside influences can cloud or invalidate your results. The reason that you should collect data in parallel (by randomly splitting your traffic among the alternative recipes in your test) is to account for as many of these outside factors as possible.

However, even if you properly conduct data collection in parallel, you may still run into obvious or subtle seasonality issues. Underlying all landing page testing is the statistical framework that calls for an unchanging visitor population that behaves consistently over time. Unfortunately, ...