Google AdWords™ For Dummies®, 3rd Edition

Un-muddying the Testing Waters: A Website Optimizer Cautionary Tale

The challenge

A client with a lead generation business model wanted to improve the effectiveness of its landing pages using Website Optimizer testing. While initial tests showed significant improvements in some page variations, after about a month, the differences disappeared, and all the pages seemed to perform about the same. This was very confusing.

The client knew that testing to improve conversion rates is one of the most powerful strategies for online success, but with the ambiguous results, it wasn't paying off. We had to dig into AdWords data and the account history to find the wrench in the testing works.

The discovery

The client had been buying traffic in its three most profitable markets only: the US, the UK, and Australia. This traffic was converting so well that they decided to get aggressive and bid on clicks in other regions, including the Middle East, the European Union, South America, and Japan. The new international traffic was about 90 percent less expensive on a per-click basis than the original traffic, and even with a much lower conversion rate, the international cost per conversion was about one-half that in the US.

We still didn't have an explanation for the collapse of Website Optimizer testing protocol. Even if the new traffic converted half as well, why would these visitors mess up our testing so badly?

We discovered the answer in a conversation with the client's Chief Operating Officer, ...