The Art of SEO, 2nd Edition

The Ongoing Evolution of Search

Search has come a long way, and will continue to progress at a faster and faster pace. Keeping up with these changes, the competitive environment, and new and emerging technologies provides both a challenge and an opportunity.

The Growth of Search Complexity

Search has been evolving rapidly over the past decade and a half. At the WSDM conference (http://www.wsdm2009.org) in February 2009, Google Fellow Jeff Dean provided some interesting metrics that tell part of the story:

• Google search volume had grown 1,000 times since 1999.

• Google has more than 1,000 times the machines it had in 1999.

• Latency dropped from less than 1,000 ms in 1999 to less than 200 ms in 2009.

• Index update latency improved by about 10,000 times over this period. Whereas updates took Google months in 1999, in 2009 Google was detecting and indexing changes on web pages in just a few minutes.

These are staggering changes in Google’s performance power, but this is just part of the changing search environment. Some of the early commercial search engines, such as Web Crawler, InfoSeek, and AltaVista, launched in the mid-1990s. At that time, web search engines’ relevancy and ranking algorithms were largely based on keyword analysis. This was a simple model to execute and initially provided pretty decent results.

However, there was (and is) too much money in search for such a simple model to stand. Spammers began abusing the weakness of the keyword algorithms by stuffing their pages with keywords, ...