Cover image for Statistics Hacks

Book description

Want to calculate the probability that an event will happen? Be able to spot fake data? Prove beyond doubt whether one thing causes another? Or learn to be a better gambler? You can do that and much more with 75 practical and fun hacks packed into Statistics Hacks. These cool tips, tricks, and mind-boggling solutions from the world of statistics, measurement, and research methods will not only amaze and entertain you, but will give you an advantage in several real-world situations-including business.

This book is ideal for anyone who likes puzzles, brainteasers, games, gambling, magic tricks, and those who want to apply math and science to everyday circumstances. Several hacks in the first chapter alone-such as the "central limit theorem,", which allows you to know everything by knowing just a little-serve as sound approaches for marketing and other business objectives. Using the tools of inferential statistics, you can understand the way probability works, discover relationships, predict events with uncanny accuracy, and even make a little money with a well-placed wager here and there.

Statistics Hacks presents useful techniques from statistics, educational and psychological measurement, and experimental research to help you solve a variety of problems in business, games, and life. You'll learn how to:

  • Play smart when you play Texas Hold 'Em, blackjack, roulette, dice games, or even the lottery

  • Design your own winnable bar bets to make money and amaze your friends

  • Predict the outcomes of baseball games, know when to "go for two" in football, and anticipate the winners of other sporting events with surprising accuracy

  • Demystify amazing coincidences and distinguish the truly random from the only seemingly random--even keep your iPod's "random" shuffle honest

  • Spot fraudulent data, detect plagiarism, and break codes

  • How to isolate the effects of observation on the thing observed

Whether you're a statistics enthusiast who does calculations in your sleep or a civilian who is entertained by clever solutions to interesting problems, Statistics Hacks has tools to give you an edge over the world's slim odds.

Table of Contents

  1. Statistics Hacks
  2. SPECIAL OFFER: Upgrade this ebook with O’Reilly
  3. Credits
    1. About the Author
    2. Contributors
    3. Acknowledgments
  4. Preface
    1. Why Statistics Hacks?
    2. How This Book Is Organized
    3. Conventions Used in This Book
    4. Safari Enabled
    5. How to Contact Us
    6. Got a Hack?
  5. 1. The Basics
    1. Hack #1. Know the Big Secret
      1. The Big Secret
      2. A Smaller, Dirtier Secret
    2. Hack #2. Describe the World Using Just Two Numbers
      1. Some Quick Statistics Basics
        1. Mean
        2. Standard deviation
      2. Central Limit Theorem
      3. So What?
      4. Applying the Central Limit Theorem
      5. Where Else It Works
    3. Hack #3. Figure the Odds
      1. Questions About the Future
      2. Likelihood of a Specific Outcome
      3. Likelihood of a Group of Outcomes
      4. Likelihood of a Series of Outcomes
      5. What Probability Means
        1. Analytic view
        2. Relative frequency view
    4. Hack #4. Reject the Null
      1. Hypothesis Testing
      2. Statistical Hypothesis Testing
      3. Why It Works
    5. Hack #5. Go Big to Get Small
      1. Laying Down the Law
      2. Improving Accuracy
      3. Why It Works
        1. Gambling
        2. Error
        3. Implications
      4. See Also
    6. Hack #6. Measure Precisely
      1. Classical Test Theory
      2. Standard Error of Measurement
      3. Building Confidence Intervals
      4. Why It Works
      5. What It Means
    7. Hack #7. Measure Up
      1. Using Numbers as Labels
      2. Using Numbers to Show Sequence
      3. Using Numbers to Show Distance
      4. Using Numbers to Count in Concrete Ways
      5. Choosing Your Level of Measurement
      6. Controversial Tools
    8. Hack #8. Power Up
      1. Power
      2. Conducting a Power Analysis
      3. Making Inferences About Beautiful Relationships
      4. Why It Works
      5. Where It Doesn't Work
    9. Hack #9. Show Cause and Effect
      1. Designing Effective Experiments
      2. Does Weight Cause Height?
      3. Fighting Threats to Validity
      4. See Also
    10. Hack #10. Know Big When You See It
      1. Seeing Effect Sizes Everywhere
      2. Finding or Computing Effect Sizes
      3. Interpreting Effect Sizes
      4. Interpreting Research Findings
      5. Why It Works
  6. 2. Discovering Relationships
    1. Hack #11. Discover Relationships
      1. Testing Hypotheses About Relationships
      2. Computing a Correlation Coefficient
      3. Interpreting a Correlation Coefficient
      4. Statistical Significance and Correlations
      5. Where Else It Works
      6. Dire Warning About Correlations
    2. Hack #12. Graph Relationships
      1. Drawing a Picture of the Future
      2. Connecting the Dots
      3. Playing "What If?"
      4. Why It Works
      5. Where It Doesn't Work
    3. Hack #13. Use One Variable to Predict Another
      1. Cooking Up the Equation
      2. Predicting Scores
      3. Why It Works
      4. Where Else It Works
      5. Where It Doesn't Work
    4. Hack #14. Use More Than One Variable to Predict Another
      1. Choosing Predictor Variables
      2. Building a Multiple Regression Equation
      3. Making Predictions and Understanding Relationships
      4. Why It Works
      5. Where Else It Works
    5. Hack #15. Identify Unexpected Outcomes
      1. Determining Whether Something Is Going On
      2. Calculating the Chi-Square
      3. Determining if the Chi-Square Is "Really Big"
      4. Why It Works
      5. Where Else It Works
      6. See Also
    6. Hack #16. Identify Unexpected Relationships
      1. Answering Relationship Questions
        1. Conduct preliminary analyses
        2. Compute the two-way chi-square
        3. Determine if the chi-square is big enough
      2. Why It Works
      3. See Also
    7. Hack #17. Compare Two Groups
      1. Proving Uncle Frank Wrong (or Right)
      2. Interpreting the t Value
      3. Why It Works
      4. Where Else It Works
    8. Hack #18. Find Out Just How Wrong You Really Are
      1. Calibrating Error and Calculating Precision
      2. Mean Estimates
      3. Proportion Estimates
      4. Estimates of Future Performance
      5. Using Standard Errors
      6. Uncle Frank's Campaign for Dogcatcher
      7. Why It Works
    9. Hack #19. Sample Fairly
      1. Using Samples to Make Inferences
      2. Constructing the Best Random Sample
      3. Sampling Strategies for the Real World
      4. Choosing a Sample Size
      5. See Also
    10. Hack #20. Sample with a Touch of Scotch
      1. A Sampling Problem
      2. Using Metaphor to Solve the Problem
      3. Where Else It Works
    11. Hack #21. Choose the Honest Average
      1. Measures of Central Tendency
      2. Choosing the Middle Ground
      3. Where It Doesn't Work
      4. How to Choose the Honest Average
    12. Hack #22. Avoid the Axis of Evil
      1. Choosing the Honest Graph
      2. Graphic Violence
      3. See Also
  7. 3. Measuring the World
    1. Hack #23. See the Shape of Everything
      1. Applying Areas Under the Normal Curve
        1. Predicting test performance
        2. Setting standards
      2. Appreciating the Beauty of the Normal Curve
    2. Hack #24. Produce Percentiles
      1. Producing and Reporting Percentile Ranks
        1. Organize your data
        2. Compute frequencies and percentages
        3. Determine percentile ranks
      2. Interpreting the Percentile Rank
      3. Where It Doesn't Work
      4. See Also
    3. Hack #25. Predict the Future with the Normal Curve
      1. The Table of Areas Under the Normal Curve
      2. Deciphering the Table
      3. Estimating the Chance of Scoring Above or Below Any Score
      4. Estimating the Chance of Scoring Between Any Two Scores
      5. Producing Percentile Ranks
      6. Determining Statistical Significance
      7. Why It Works
      8. See Also
    4. Hack #26. Give Raw Scores a Makeover
      1. Calculating z Scores
      2. Understanding Performance
      3. Identifying the Rarity of Your Performance
      4. Why It Works
    5. Hack #27. Standardize Scores
      1. Problems with z Scores
      2. Creating and Interpreting T Scores
      3. Creating Customized Standardized Scores
      4. Create Your Own Standardized Score
      5. Why It Works
      6. Understanding Norm-Referenced Scoring
    6. Hack #28. Ask the Right Questions
      1. Constructing a Good Question
      2. Asking a Question at the Right Level
        1. Choosing the right cognitive level
        2. Designing questions at different cognitive levels
        3. When to use Bloom's Taxonomy
      3. See Also
    7. Hack #29. Test Fairly
      1. Item Analysis
      2. Three Types of Classroom Assessment Problems
        1. Are my test questions too hard?
        2. Is each test question measuring what it is supposed to?
        3. Why did my students miss a question?
      3. Conducting Item Analyses and Interpreting Results
      4. Suggestions for Item Analysis and Test Fairness
    8. Hack #30. Improve Your Test Score While Watching Paint Dry
      1. Regressing to the Mean
      2. Why It Works
      3. Predicting the Likelihood of a Higher Score
    9. Hack #31. Establish Reliability
      1. Why Reliability Matters
      2. Calculating Reliability
        1. Internal reliability
        2. Test-retest reliability
        3. Inter-rater reliability
        4. Parallel forms reliability
      3. Interpreting Reliability Evidence
      4. Improving Test Reliability
      5. Why It Works
    10. Hack #32. Establish Validity
      1. Strategies for Winning the Validity Argument
      2. Content-Based Arguments
      3. Criterion-Based Arguments
      4. Construct-Based Arguments
      5. Consequences-Based Arguments
      6. Choosing from the Menu of Validity Options
    11. Hack #33. Predict the Length of a Lifetime
      1. In Action
      2. How It Works
      3. In Real Life
      4. Endnotes
    12. Hack #34. Make Wise Medical Decisions
      1. Statistics and Medical Screening
      2. Understanding Breast Cancer Screening
      3. Why It Works
      4. Making Informed Decisions
      5. See Also
  8. 4. Beating the Odds
    1. Hack #35. Gamble Smart
      1. The Gambler's Fallacy
      2. Casinos and Money
      3. Systems
    2. Hack #36. Know When to Hold 'Em
      1. How It Works
        1. Example 1
        2. Example 2
      2. Why It Works
      3. Other Places It Works
      4. Where It Doesn't Work
    3. Hack #37. Know When to Fold 'Em
      1. Pot Odds
      2. Why It Works
      3. Where Else It Works
      4. Where It Doesn't Work
    4. Hack #38. Know When to Walk Away
      1. Recognizing a Short-Stacked Situation
        1. Twelve times the big blind or less
        2. Eight times the big blind or less
        3. Four times the big blind or less
      2. Statistical Decision Making
      3. Getting Your Mind Right
    5. Hack #39. Lose Slowly at Roulette
      1. Basic Wagers
      2. Why It Works
    6. Hack #40. Play in the Black in Blackjack
      1. Basic Strategy
      2. Why It Works
      3. Simple Card-Counting Methods
        1. Counting Aces
        2. Counting Aces and 10s
        3. Going by the point system
    7. Hack #41. Play Smart When You Play the Lottery
      1. Powerball Odds
      2. Powerball Payoff
        1. Easiest prize
        2. Powerball only
        3. Grand prize
      3. Winning Powerball
      4. Don't Split the Prize
    8. Hack #42. Play with Cards and Get Lucky
      1. Getting a Li'l Flush
        1. The bet
        2. Why it works
      2. Finding a Match with Two Decks of Cards
        1. The bet
        2. Why it works
    9. Hack #43. Play with Dice and Get Lucky
      1. Distribution of Dice Outcomes
      2. Bar Bets with Dice
      3. Why It Works
    10. Hack #44. Sharpen Your Card-Sharping
      1. Improving Your Hand
        1. Probabilities
        2. Implications
      2. Reading the Community Cards in a Flash
    11. Hack #45. Amaze Your 23 Closest Friends
      1. Getting Started
      2. Applying the Law of Total Probability
      3. Calculating the Probability of Independent Events
      4. Solving the Birthday Problem
      5. Solving for Almost Any Group Size
    12. Hack #46. Design Your Own Bar Bet
      1. Principle 1
      2. Principle 2
      3. Rolling Your Own Bar Bet
        1. Letters of the alphabet
        2. Pick a number, any number
        3. On your own
      4. Make Sure the Sucker Isn't You!
    13. Hack #47. Go Crazy with Wild Cards
      1. The Problem with Wild Cards
      2. Why It Works
      3. The Other Problem with Wild Cards
    14. Hack #48. Never Trust an Honest Coin
      1. Shiny New Pennies
      2. Binomial Expectations
      3. Where It Doesn't Work
      4. See Also
    15. Hack #49. Know Your Limit
      1. The Game of St. Petersburg
      2. Statistical Analysis
      3. Why It Doesn't Work
        1. Infinite is a lot
        2. Decreasing marginal utility
        3. Risk versus reward
        4. Infinity doesn't exist
      4. See Also
  9. 5. Playing Games
    1. Hack #50. Avoid the Zonk
      1. The Monty Hall Problem and Game Show Strategy
      2. Why You Should Always Switch
    2. Hack #51. Pass Go, Collect $200, Win the Game
      1. Monopoly Statistical Basics
      2. Key Properties
      3. Importance of the Monopoly Prison System
      4. See Also
    3. Hack #52. Use Random Selection as Artificial Intelligence
      1. Trial-and-Error Learning
      2. Building a Tic-Tac-Toe Machine
      3. Operating the Computer
      4. Why It Works
      5. Hacking the Hack
    4. Hack #53. Do Card Tricks Through the Mail
      1. How It Works
      2. Probability of Success
      3. See Also
    5. Hack #54. Check Your iPod's Honesty
      1. Assessing iTunes's Selection Procedures
      2. Calculating the Statistics of the Selection Process
      3. Explaining Statistical Surprises
      4. See Also
    6. Hack #55. Predict the Game Winners
      1. Choosing Predictor Variables
      2. Entering the Data into a Spreadsheet
      3. Building a Regression Equation
      4. Interpreting and Applying the Regression Equation
      5. Improving Your Regression Equation
    7. Hack #56. Predict the Outcome of a Baseball Game
      1. How It Works
      2. Why It Works
      3. Proving It Works
      4. Other Places It Works
      5. Where It Doesn't Work
    8. Hack #57. Plot Histograms in Excel
      1. The Code
      2. Hacking the Hack
    9. Hack #58. Go for Two
      1. Traditional Two-Point Conversion Charts
      2. Modern Super-Scientific Chart
      3. How It Works
    10. Hack #59. Rank with the Best of Them
      1. How to Rank Fairly
        1. Total points
        2. Mean performance
        3. Total wins
      2. Comparing the Three Methods
      3. The End of the Story
    11. Hack #60. Estimate Pi by Chance
      1. Pi
      2. Pi and Falling Needles
      3. Probability and Pi
      4. Estimating Pi Using Probability
  10. 6. Thinking Smart
    1. Hack #61. Outsmart Superman
      1. Lucky Lois Lane
      2. The Guesses
      3. The Calculations
        1. Guess 1
        2. Guess 2
        3. Guess 3
        4. Guess 4
      4. Final Probability
    2. Hack #62. Demystify Amazing Coincidences
      1. Compare the Number of Possible Outcomes
      2. Figure Out the Actual Odds
      3. Remove Meaning Assigned to Meaningless Events
    3. Hack #63. Sense the Real Randomness of Life
      1. What Does Random Look Like?
      2. How to Spot Random Outcomes
      3. How to Calculate Combinations
      4. When to Be Suspicious
    4. Hack #64. Spot Faked Data
      1. How It Works
      2. Laying Down the Law
        1. Street addresses
        2. Stock prices
      3. More General Statements of Benford's Law
      4. Where Else It Works
        1. Scale invariance
        2. Base invariance
      5. Why It Works
      6. Where It Doesn't Work
      7. See Also
    5. Hack #65. Give Credit Where Credit Is Due
      1. Building a Model
      2. Factor Analysis
      3. See Also
    6. Hack #66. Play a Tune on Pascal's Triangle
      1. Presenting Pascal's Triangle
      2. Calculating Probability Using the Triangle
        1. Assessing the probability of flipped coin outcomes
        2. Assessing the probability of a bad car trip
      3. Why It Works
    7. Hack #67. Control Random Thoughts
      1. Mind Control
      2. Probability and Word Association
      3. Building a List of Word Associations
      4. Why It Works
      5. Where Else It Works
      6. Where It Doesn't Work
    8. Hack #68. Search for ESP
      1. Identifying Psychic Abilities
      2. Analyzing the Results
      3. How Much Is Enough?
    9. Hack #69. Cure Conjunctionitus
      1. The Problem
      2. Conjunction Junction, What's Your Function?
      3. The Cure
      4. See Also
    10. Hack #70. Break Codes with Etaoin Shrdlu
      1. Single Substitution Ciphers
      2. Using Probability to Decode Substitution Ciphers
      3. ETAOIN SHRDLU
      4. Statistical Analysis of Coded Texts
      5. Other Common Letter Patterns
      6. See Also
    11. Hack #71. Discover a New Species
      1. Identifying Species with Statistics
        1. Collect some data
        2. Choose a statistical method
        3. Select a hypothesis and analyze the data
        4. Interpret results
      2. Two Possum Species
      3. See Also
    12. Hack #72. Feel Connected
      1. Six Degrees of Separation
      2. Doing a Big Study
        1. Choose a target
        2. Recruit participates
        3. Train participates
        4. Collect and analyze the results
      3. Doing a Small Study
        1. Milgram via email
        2. Throw a party
      4. Just Doing the Math
      5. See Also
    13. Hack #73. Learn to Ride a Votercycle
      1. Vote Cycling
      2. How It Works
      3. Getting Off the Votercycle
    14. Hack #74. Live Life in the Fast Lane (You're Already In)
      1. Skips, Slips, and Epochs
      2. Probability and Traffic Patterns
      3. Making Wise Lane-Changing Decisions
      4. See Also
    15. Hack #75. Seek Out New Life and New Civilizations
      1. Estimating the Number of Smart Planets
      2. Applying Drake's Equation
      3. Finding our Space Chums
      4. Data Analysis
  11. Index
  12. About the Author
  13. Colophon
  14. SPECIAL OFFER: Upgrade this ebook with O’Reilly