Email Marketing by the Numbers: How to Use the World’s Greatest Marketing Tool to Take Any Organization to the Next Level

Sampling

Sampling refers to the size of your test list. Sample size is important because if your test lists are too small, then your results might not be statistically significant (they won’t accurately indicate what is going to happen with your larger list). A sample size that is too large wastes an opportunity to send the winner to more people. The goal is to find the smallest sample size that will provide relevant results. There are complex formulas that can be used to estimate sample size, but I don’t think it’s necessary if you follow a few simple guidelines:

• Test with approximately 10 percent of your list since this leaves 90 percent of your list to receive the winner.

• You should have at least 250 people in each test group, which can be less than ideal if you’re testing several versions. But if your list is less than 10,000 total, then this will provide usable results, while still giving you the opportunity to send the winner to the majority of your list.

• There is no reason to go with more than 20,000 in each test group. If you don’t see statistically significant results with these large test groups, then there isn’t a meaningful difference.

It’s also important that you pull a random sample from your segmented list. To do so, you start with the segment you want to mail to (so you are not mailing to the entire database). Then you randomly select the number of subscribers required for each test sample from this list. Most email tools will make this easy for the marketer to simply ...