You are previewing Lean Analytics.

Lean Analytics

Cover of Lean Analytics by Alistair Croll... Published by O'Reilly Media, Inc.
  1. Lean Analytics
  2. Dedication
  3. Preface
    1. Who this book is for
    2. How the book works
    3. The building blocks
      1. Customer Development
      2. Lean Startup
    4. About the authors
    5. Thanks and acknowledgements
  4. I. Stop lying to yourself
    1. 1. We’re all liars
      1. The Lean Startup movement
      2. Poking a hole in your reality distortion field
      3. Case study: AirBnB Photography—growth within growth
    2. 2. How to keep score
      1. What makes a good metric?
      2. Qualitative versus quantitative
      3. Vanity metrics versus real metrics
      4. Pattern: Eight vanity metrics to watch out for
      5. Exploration versus reporting
      6. Case study: Circle of Moms explores its way to success
      7. Leading metrics vs. lagging metrics
      8. Moving targets
      9. Case Study: HighScore House defines an “active user”
      10. Segments, cohorts, A/B testing, and multivariate analysis
      11. The Lean Analytics cycle
      12. Exercise: Evaluating the metrics you track
    3. 3. Deciding what to do with your life
      1. The Lean Canvas
      2. What should you work on?
      3. Exercise: Create a Lean Canvas
    4. 4. Data-driven vs. Data-informed
      1. Pattern: How to think like a data scientist
      2. Lean Startup and Big Vision
  5. II. Finding the right metric for right now
    1. 5. Analytics Frameworks
      1. Dave McClure’s Pirate Metrics
      2. Eric Ries’s Engines of Growth
      3. Ash Maurya’s Lean Canvas
      4. Sean Ellis’s Startup Pyramid of Growth
      5. The Long Funnel
      6. Introducing the Lean Analytics Stages and Gates
    2. 6. The Discipline of One Metric That Matters
      1. Case Study: SEOmoz tracks fewer KPIs to increase focus
      2. Four reasons to use the One Metric That Matters
      3. Case study: Solare focuses on a few key metrics
      4. Drawing lines in the sand
      5. The squeeze toy
      6. Exercise: Define your OMTM
    3. 7. What business are you in?
      1. About those people
      2. The Business Model Flipbook
      3. Six business models
      4. Exercise: Pick your business model
    4. 8. Model one: E-commerce
      1. Pattern: What mode of e-commerce are you?
      2. A practical example
      3. Conversion rate
      4. Purchases per year
      5. Shopping cart size
      6. Abandonment
      7. Cost of customer acquisition
      8. Revenue per customer
      9. Case study: WineExpress increases revenue by 41% per visitor
      10. Keywords and search terms
      11. Recommendation acceptance rate
      12. Virality
      13. Mailing list click-through rates
      14. Offline and online combinations
      15. Visualizing the e-commerce business
      16. Wrinkles: Traditional e-commerce vs. subscription e-commerce
      17. Key takeaways
    5. 9. Model two: Software-as-a-Service (SaaS)
      1. Case study: Backupify’s Customer Lifecycle Learning
      2. Measuring engagement
      3. Churn
      4. Visualizing the SaaS business
      5. Case study: ClearFit abandons monthly subscriptions for 10x growth
      6. Wrinkles: Freemium, tiers and other pricing models
      7. Key takeaways
    6. 10. Model three: Free mobile app
      1. Installation volume
      2. Average Revenue per User
      3. Percentage of users that pay
      4. Churn
      5. Visualizing the mobile app business
      6. Wrinkles: In-app monetization vs. advertising
      7. Key takeaways
    7. 11. Model four: Media site
      1. Audience and churn
      2. Inventory
      3. Pattern: Performance and the sessions-to-clicks ratio
      4. Ad rates
      5. Content/advertising trade-off
      6. Visualizing the media business
      7. Wrinkles: hidden affiliates, background noise, ad blockers, and paywalls
      8. Key takeaways
    8. 12. Model five: User-generated content
      1. Visitor engagement
      2. Content creation & interaction
      3. Engagement funnel changes
      4. Value of created content
      5. Content sharing and virality
      6. Notification effectiveness
      7. Visualizing a UCG business
      8. Wrinkles: Passive content creation
      9. Key takeaways
    9. 13. Model six: Two-sided marketplace
      1. Case study: What Duproprio watches
      2. Rate at which you’re adding buyers and sellers
      3. Rate of inventory growth
      4. Buyer searches
      5. Conversion rates and segmentation
      6. Buyer and seller ratings
      7. Visualizing a two-sided marketplace
      8. Wrinkles: Chicken and Egg; Fraud; keeping the transaction; auctions
      9. Key takeaways
    10. 14. What stage are you at?
      1. Exercise: Pick the stage that you’re at
    11. 15. Stage one: Empathy
      1. Metrics for the Empathy Stage
      2. This is the best idea I’ve ever had! (or how to discover problems worth solving)
      3. Finding a problem to fix (or how to validate a problem)
      4. Pattern: Signs you’ve found a problem worth tackling
      5. Pattern: Running Lean and how to do a good interview
      6. Pattern: How to avoid leading the witness
      7. Convergent and divergent problem interviews
      8. How do I know if the problem is really painful enough?
      9. Case study: Cloud9 IDE interviews existing customers
      10. How are people solving the problem now?
      11. Are there enough people that care about this problem? (Understanding the market)
      12. What will it take to make them aware of the problem?
      13. A ‘Day in the Life’ of Your Customer
      14. Pattern: Finding people to talk to
      15. Getting answers at scale
      16. Case Study: LikeBright Mechanical Turks its way into TechStars
      17. Pattern: Creating an answers-at-scale campaign
      18. Build it before you build it (or how to validate the solution)
      19. Case study: Localmind hacks Twitter
      20. Before you launch the MVP
      21. Deciding what goes into the MVP
      22. Measuring the MVP
      23. Case Study: Static Pixels eliminates a step in their order process
      24. A summary of the Empathy Stage
      25. Exercise: Should you move to the next stage?
    12. 16. Stage two: Stickiness
      1. Iterating the MVP
      2. Case study: qidiq changes how it adds users
      3. Premature virality
      4. The goal is retention
      5. Pattern: 7 Questions to Ask Yourself Before Building a Feature
      6. Case study: How Rally builds new features with a Lean approach
      7. How to handle user feedback
      8. The Minimum Viable Vision
      9. The Problem-Solution Canvas
      10. Case study: VNN uses the Problem-Solution Canvas to solve business problems
      11. A summary of the Stickiness Stage
      12. Exercise #1: Should you move to the next stage?
      13. Exercise #2: Have you identified your biggest problems?
    13. 17. Stage three: Virality and word of mouth
      1. The three ways things spread
      2. Metrics for the viral phase
      3. Beyond the viral coefficient
      4. Case studyComment [BY4]: This is a new case study, I’ve sent it to Jonathan Wegener for review / approval.Timehop experiments with content sharing to achieve virality
      5. Instrumenting the viral pattern
      6. Growth hacking
      7. A summary of the Virality Stage
      8. Exercise: Should you move on to the revenue stage?
    14. 18. Stage four: Revenue
      1. Metrics for the Revenue Stage
      2. The penny machine
      3. Finding your revenue groove
      4. Customer Lifetime Value > Customer Acquisition Cost
      5. Case study: and the pivot to revenue
      6. Market/Product Fit
      7. The breakeven lines in the sand
      8. Revenue Stage Summary
    15. 19. Stage five: Scale
      1. The hole in the middle
      2. Metrics for the Scale Stage
      3. Is my business model right?
      4. Case study: Buffer goes from Stickiness to Scale (through Revenue)
      5. Pattern: The three-threes model
      6. Finding discipline as you scale
      7. A summary of the Scale Stage
    16. 20. Model + Stage drives the metric you track
  6. III. Lines in the sand
    1. 21. Am I good enough?
      1. Case study: WP Engine discovers the 2% Cancellation Rate
      2. Average isn’t good enough
      3. What’s good enough?
      4. Growth rate
      5. Number of engaged visitors
      6. Pricing metrics
      7. Case study: Socialight discovers the underlying metrics of pricing
      8. Cost of customer acquisition
      9. Virality
      10. Mailing list effectiveness
      11. Uptime and reliability
      12. Site engagement
      13. Web performance
      14. Exercise: Make your own lines in the sand
    2. 22. E-commerce: lines in the sand
      1. Conversion rate
      2. Shopping cart abandonment
      3. Search effectiveness
    3. 23. SaaS: lines in the sand
      1. Paid enrollment
      2. Freemium versus paid
      3. Upselling and growing revenue
      4. Churn
      5. Case study: OfficeDrop’s key metric – paid churn
    4. 24. Free mobile app: lines in the sand
      1. Mobile downloads
      2. Mobile download size
      3. Mobile customer acquisition cost
      4. Case study: Sincerely learns the challenges of mobile customer acquisition
      5. Application launch rate
      6. Percent active mobile users/players
      7. Percentage of Mobile users who pay
      8. Average revenue per daily active user
      9. Monthly average revenue per mobile user
      10. Average revenue per paying user
      11. Mobile app ratings click-through
      12. Mobile Customer Lifetime Value
    5. 25. Media site: lines in the sand
      1. Click-through rates
      2. Sessions-to-clicks ratio
      3. Referrers
      4. Engaged time
      5. Pattern: What onsite engagement can tell you about goals & behaviors
      6. Sharing with others
      7. Case study: JFL Gags cracks up YouTube
    6. 26. User-generated content: lines in the sand
      1. Content upload success
      2. Time on site per day
      3. Case study: reddit part one—from links to a community
      4. Engagement funnel changes
      5. Case study: reddit part two—there’s gold in those users
      6. Spam and bad content
    7. 27. Two-sided marketplaces: lines in the sand
      1. Transaction size
      2. Case study: What Etsy watches
      3. Top ten lists
    8. 28. What to do when you don’t have a baseline
  7. IV. Putting Lean Analytics to work
    1. 29. Selling to businesses: Enterprise markets
      1. Why are enterprise customers different?
      2. The enterprise startup lifecycle
      3. Case study: How Coradiant found a market
      4. So what metrics matter?
      5. The bottom line: Startups are startups
    2. 30. Lean from within: Intrapreneurs
      1. Span of control and the railroads
      2. Pattern: Skunk Works for intrapreneurs
      3. Changing—or innovating to resist change?
      4. Stars, dogs, cows, and question marks
      5. Case study: Swiffer gives up on chemistry
      6. Case study: Doritos chooses a flavor
      7. Working with an executive sponsor
      8. Case study: EMI embraces data to understand its customers
      9. The stages of intrapreneur analytics
    3. 31. Conclusion: Beyond startups
      1. How to instill a culture of data in your company
      2. Ask good questions
    4. A.
      1. References and further reading
  8. About the Authors
  9. Copyright

Chapter 2. How to keep score

Analytics is about tracking the metrics that are critical to your business. Usually, those metrics matter because they relate to your business model—where money comes from, how much things cost, how many customers you have, and the effectiveness of your customer acquisition strategies.

In a startup, you don’t always know your key business metrics, because you’re not entirely sure what business you’re in. You’re frequently changing the activity you analyze. You’re still trying to find the right product, or the right target audience. In a startup, the purpose of analytics is to find your way to the right product and market before the money runs out.

What makes a good metric?

Here are some rules of thumb for what makes a good metric—a number that will drive the changes you’re looking for.

A good metric is comparative. Being able to compare a metric to other time periods, groups of users, or competitors, helps you understand which way things are moving. Increased conversion from last week is more meaningful than “2% conversion.”

A good metric is understandable. Otherwise, people won’t remember it and discuss it, and it’ll be much harder to turn a change in the data into a change in the culture.

A good metric is a ratio or a rate. Accountants and financial analysts have several ratios they look at[5] to understand, at a glance, the fundamental health of a company. You need some, too.

There are several reasons ratios tend to be the best metrics:

  • Ratios are easier ...

The best content for your career. Discover unlimited learning on demand for around $1/day.