A/B Testing: The Most Powerful Way to Turn Clicks Into Customers

A/B Testing: The Most Powerful Way to Turn Clicks Into Customers

by Dan Siroker, Pete Koomen
Released August 2013
Publisher(s): Wiley
ISBN: 9781118536094

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

A guide to delivering a better user experience through A/B testing

A/B Testing outlines a simple way to test several different versions of a web page with live traffic, and then measure the effect each version has on visitors. Using A/B testing you can determine the most effective way to increase conversion rate—the rate at which visitors convert into customers. The average conversion rate across the web is only 2%. That means 98% of visitors to most websites don't end up converting. Using A/B testing, companies can improve the effectiveness of their marketing and user experience and in doing so can sometimes double or triple their conversion rates.

In the past, marketing teams were dependent on IT and engineering to implement A/B tests, and that proved to be a costly bottleneck. Today, the market for A/B testing is growing very fast, in no small part due to this transition from IT to marketing.

  • Offers best practices and lessons learned from over 100,000 experiments run by over 2,000 Optimizely customers

  • Details a roadmap for how to use A/B testing to personalize your customer's web experience and a practical guide to start A/B testing today

  • Authors Dan Siroker and Pete Koomen are cofounders of Optimizely.com, an user-friendly testing system that more than 2,000 organizations use

Marketers and web professionals will become obsolete if they don't embrace a data-driven approach to decision making. This book shows you how, no matter your technical expertise.

Table of contents

  1. Cover Page
  2. Title Page
  3. Copyright
  4. Contents
  5. CHAPTER 1: How A/B Testing Helped Win the White House—Twice
    1. Optimization for Everyone
  6. PART I: Lessons Learned from 200,000 A/B Tests (and Counting)
    1. CHAPTER 2: What to Test
      1. Step One: Define Success
      2. Step Two: Identify Bottlenecks
      3. Step Three: Construct a Hypothesis
      4. Step Four: Prioritize
      5. Step Five: Test
    2. CHAPTER 3: Seek the Global Maximum
      1. Break from the Status Quo: ABC Family
      2. Learn Your Way to the New Site: Chrome Industries
      3. Rethink the Business Model: Lumosity
      4. Test Through the Redesign, Not After: Digg and Netflix
    3. CHAPTER 4: Less Is More: Reduce Choices
      1. Every Field Counts: The Clinton Bush Haiti Fund
      2. Keep It Simple: SeeClickFix
      3. Hide Options: Cost Plus World Market
      4. Remove Distractions and “Outs”: Avalanche Technology Group
      5. Lower the Slope: Obama 2012
    4. CHAPTER 5: Words Matter: Focus on Your Call to Action
      1. Reconsider “Submit”: The Clinton Bush Haiti Fund
      2. Find the Perfect Appeal: Wikipedia
      3. Why versus How : Formstack
      4. Nouns versus Verbs: LiveChat
      5. Framing Effects
    5. CHAPTER 6: Fail Fast and Learn
      1. Prime Real Estate versus Familiar Real Estate: IGN
      2. What's Good at One Scale Isn't Always Good at Another: E-Commerce
      3. What Buyers Want Isn't Always What Sellers Want: Etsy
      4. When a Win Isn't a Win (Is a Win): Chrome Industries
  7. PART II: Implementing A/B Testing: Play-by-Play Guide
    1. CHAPTER 7: Choose the Solution That's Right for Your Organization
      1. Option One: Build
      2. Option Two: Buy
      3. Option Three: Hire
      4. The Choice Is Yours (and Your Team's)
    2. CHAPTER 8: The Cure for the HiPPO Syndrome
      1. The HiPPO Syndrome Has a Cure
      2. Winning Over the Stakeholders
      3. Communicate Findings and Value to Your Team
      4. Evangelize and Galvanize: Lizzie Allen at IGN
    3. CHAPTER 9: The A/B Testing Dream Team
      1. The Centralized Team
      2. The Decentralized Team
      3. The Three Key Ingredients of a Scalable Testing Strategy
    4. CHAPTER 10: Iterate, Iterate, Iterate
      1. Multivariate Testing and Iterative Testing
      2. There Are No Universal Truths: Always Be Testing
      3. Redesigning Your Redesign: CareerBuilder and the Optimizely Website
  8. PART III: Advanced Topics in A/B Testing
    1. CHAPTER 11: How A/B Tests Can Go Awry
      1. Testing without Traffic
      2. The Beginning of the Funnel versus the End: UserVoice
      3. Off-Brand Testing: Dell
    2. CHAPTER 12: Beyond the Page: Non-Website A/B Testing
      1. The What and the When : Prezi
      2. Price Testing
      3. Testing the Perceived Price: The Last-Minute Discount
      4. Anchoring in Action: Judy's Book Club
      5. Testing the Billing
      6. Testing the Actual Price
    3. CHAPTER 13: Personalize, Personalize, Personalize
      1. Targeting versus Segmentation
      2. Using Segmentation to Drill Down into Test Results
      3. Geo-Targeting, State by State: Romney 2012
      4. When to Personalize and When Not to: Wikipedia
  9. Conclusion
  10. APPENDIX 1: 60 Things to A/B Test
    1. Calls to Action
    2. Content
    3. Copy
    4. Visual Media
    5. Funnels
    6. Site Navigation
    7. Forms
    8. Mobile Site
    9. Advertising
    10. Social
    11. Email
    12. Personalize It
    13. Pricing and Shipping
  11. APPENDIX 2: Metrics and the Statistics behind A/B Testing
    1. Confidence Intervals
    2. Confidence Levels
    3. A/B Testing Metrics Framework
    4. The Z Statistic for Two Proportions
    5. The t Statistic for Two Averages
  12. Acknowledgments
  13. Index

Product information

  • Title: A/B Testing: The Most Powerful Way to Turn Clicks Into Customers
  • Author(s): Dan Siroker, Pete Koomen
  • Release date: August 2013
  • Publisher(s): Wiley
  • ISBN: 9781118536094