You are previewing Data Smart: Using Data Science to Transform Information into Insight.
O'Reilly logo
Data Smart: Using Data Science to Transform Information into Insight

Book Description

Data Science gets thrown around in the press like it's magic. Major retailers are predicting everything from when their customers are pregnant to when they want a new pair of Chuck Taylors. It's a brave new world where seemingly meaningless data can be transformed into valuable insight to drive smart business decisions.

But how does one exactly do data science? Do you have to hire one of these priests of the dark arts, the "data scientist," to extract this gold from your data? Nope.

Data science is little more than using straight-forward steps to process raw data into actionable insight. And in Data Smart, author and data scientist John Foreman will show you how that's done within the familiar environment of a spreadsheet.

Why a spreadsheet? It's comfortable! You get to look at the data every step of the way, building confidence as you learn the tricks of the trade. Plus, spreadsheets are a vendor-neutral place to learn data science without the hype.

But don't let the Excel sheets fool you. This is a book for those serious about learning the analytic techniques, the math and the magic, behind big data.

Each chapter will cover a different technique in a spreadsheet so you can follow along:

  • Mathematical optimization, including non-linear programming and genetic algorithms

  • Clustering via k-means, spherical k-means, and graph modularity

  • Data mining in graphs, such as outlier detection

  • Supervised AI through logistic regression, ensemble models, and bag-of-words models

  • Forecasting, seasonal adjustments, and prediction intervals through monte carlo simulation

  • Moving from spreadsheets into the R programming language

You get your hands dirty as you work alongside John through each technique. But never fear, the topics are readily applicable and the author laces humor throughout. You'll even learn what a dead squirrel has to do with optimization modeling, which you no doubt are dying to know.

Table of Contents

  1. Cover Page
  2. Title Page
  3. Copyright
  4. Dedication
  5. Credits
  6. About the Author
  7. Acknowledgments
  8. Contents
  9. Introduction
    1. What Am I Doing Here?
    2. A Workable Definition of Data Science
    3. But Wait, What about Big Data?
    4. Who Am I?
    5. Who Are You?
    6. No Regrets. Spreadsheets Forever
    7. Conventions
    8. Let's Get Going
  10. 1: Everything You Ever Needed to Know about Spreadsheets but Were Too Afraid to Ask
    1. Some Sample Data
    2. Moving Quickly with the Control Button
    3. Copying Formulas and Data Quickly
    4. Formatting Cells
    5. Paste Special Values
    6. Inserting Charts
    7. Locating the Find and Replace Menus
    8. Formulas for Locating and Pulling Values
    9. Using VLOOKUP to Merge Data
    10. Filtering and Sorting
    11. Using PivotTables
    12. Using Array Formulas
    13. Solving Stuff with Solver
    14. OpenSolver: I Wish We Didn't Need This, but We Do
    15. Wrapping Up
  11. 2: Cluster Analysis Part I: Using K-Means to Segment Your Customer Base
    1. Girls Dance with Girls, Boys Scratch Their Elbows
    2. Getting Real: K-Means Clustering Subscribers in E-mail Marketing
    3. K-Medians Clustering and Asymmetric Distance Measurements
    4. Wrapping Up
  12. 3: Naïve Bayes and the Incredible Lightness of Being an Idiot
    1. When You Name a Product Mandrill, You're Going to Get Some Signal and Some Noise
    2. The World's Fastest Intro to Probability Theory
    3. Using Bayes Rule to Create an AI Model
    4. Let's Get This Excel Party Started
    5. Wrapping Up
  13. 4: Optimization Modeling: Because That “Fresh Squeezed” Orange Juice Ain't Gonna Blend Itself
    1. Why Should Data Scientists Know Optimization?
    2. Starting with a Simple Trade-Off
    3. Fresh from the Grove to Your Glass...with a Pit Stop through a Blending Model
    4. Modeling Risk
    5. Wrapping Up
  14. 5: Cluster Analysis Part II: Network Graphs and Community Detection
    1. What Is a Network Graph?
    2. Visualizing a Simple Graph
    3. Brief Introduction to Gephi
    4. Building a Graph from the Wholesale Wine Data
    5. How Much Is an Edge Worth? Points and Penalties in Graph Modularity
    6. Let's Get Clustering!
    7. There and Back Again: A Gephi Tale
    8. Wrapping Up
  15. 6: The Granddaddy of Supervised Artificial Intelligence—Regression
    1. Wait, What? You're Pregnant?
    2. Don't Kid Yourself
    3. Predicting Pregnant Customers at RetailMart Using Linear Regression
    4. Predicting Pregnant Customers at RetailMart Using Logistic Regression
    5. For More Information
    6. Wrapping Up
  16. 7: Ensemble Models: A Whole Lot of Bad Pizza
    1. Using the Data from Chapter 6
    2. Bagging: Randomize, Train, Repeat
    3. Boosting: If You Get It Wrong, Just Boost and Try Again
    4. Wrapping Up
  17. 8: Forecasting: Breathe Easy; You Can't Win
    1. The Sword Trade Is Hopping
    2. Getting Acquainted with Time Series Data
    3. Starting Slow with Simple Exponential Smoothing
    4. You Might Have a Trend
    5. Holt's Trend-Corrected Exponential Smoothing
    6. Multiplicative Holt-Winters Exponential Smoothing
    7. Wrapping Up
  18. 9: Outlier Detection: Just Because They're Odd Doesn't Mean They're Unimportant
    1. Outliers Are (Bad?) People, Too
    2. The Fascinating Case of Hadlum v. Hadlum
    3. Terrible at Nothing, Bad at Everything
    4. Wrapping Up
  19. 10: Moving from Spreadsheets into R
    1. Getting Up and Running with R
    2. Doing Some Actual Data Science
    3. Wrapping Up
  20. Conclusion
    1. Where Am I? What Just Happened?
    2. Before You Go-Go
    3. Get Creative and Keep in Touch!
  21. Index