You are previewing Machine Learning for Hackers.

Machine Learning for Hackers

Cover of Machine Learning for Hackers by Drew Conway... Published by O'Reilly Media, Inc.
  1. Machine Learning for Hackers
  2. Preface
    1. Machine Learning for Hackers
    2. How This Book Is Organized
    3. Conventions Used in This Book
    4. Using Code Examples
    5. Safari® Books Online
    6. How to Contact Us
    7. Acknowledgements
  3. 1. Using R
    1. R for Machine Learning
      1. Downloading and Installing R
      2. IDEs and Text Editors
      3. Loading and Installing R Packages
      4. R Basics for Machine Learning
      5. Further Reading on R
  4. 2. Data Exploration
    1. Exploration versus Confirmation
    2. What Is Data?
    3. Inferring the Types of Columns in Your Data
    4. Inferring Meaning
    5. Numeric Summaries
    6. Means, Medians, and Modes
    7. Quantiles
    8. Standard Deviations and Variances
    9. Exploratory Data Visualization
    10. Visualizing the Relationships Between Columns
  5. 3. Classification: Spam Filtering
    1. This or That: Binary Classification
    2. Moving Gently into Conditional Probability
    3. Writing Our First Bayesian Spam Classifier
      1. Defining the Classifier and Testing It with Hard Ham
      2. Testing the Classifier Against All Email Types
      3. Improving the Results
  6. 4. Ranking: Priority Inbox
    1. How Do You Sort Something When You Don’t Know the Order?
    2. Ordering Email Messages by Priority
      1. Priority Features of Email
    3. Writing a Priority Inbox
      1. Functions for Extracting the Feature Set
      2. Creating a Weighting Scheme for Ranking
      3. Weighting from Email Thread Activity
      4. Training and Testing the Ranker
  7. 5. Regression: Predicting Page Views
    1. Introducing Regression
      1. The Baseline Model
      2. Regression Using Dummy Variables
      3. Linear Regression in a Nutshell
    2. Predicting Web Traffic
    3. Defining Correlation
  8. 6. Regularization: Text Regression
    1. Nonlinear Relationships Between Columns: Beyond Straight Lines
      1. Introducing Polynomial Regression
    2. Methods for Preventing Overfitting
      1. Preventing Overfitting with Regularization
    3. Text Regression
      1. Logistic Regression to the Rescue
  9. 7. Optimization: Breaking Codes
    1. Introduction to Optimization
    2. Ridge Regression
    3. Code Breaking as Optimization
  10. 8. PCA: Building a Market Index
    1. Unsupervised Learning
  11. 9. MDS: Visually Exploring US Senator Similarity
    1. Clustering Based on Similarity
      1. A Brief Introduction to Distance Metrics and Multidirectional Scaling
    2. How Do US Senators Cluster?
      1. Analyzing US Senator Roll Call Data (101st–111th Congresses)
  12. 10. kNN: Recommendation Systems
    1. The k-Nearest Neighbors Algorithm
    2. R Package Installation Data
  13. 11. Analyzing Social Graphs
    1. Social Network Analysis
      1. Thinking Graphically
    2. Hacking Twitter Social Graph Data
      1. Working with the Google SocialGraph API
    3. Analyzing Twitter Networks
      1. Local Community Structure
      2. Visualizing the Clustered Twitter Network with Gephi
      3. Building Your Own “Who to Follow” Engine
  14. 12. Model Comparison
    1. SVMs: The Support Vector Machine
    2. Comparing Algorithms
  15. Works Cited
    1. Books
    2. Articles
  16. Index
  17. About the Authors
  18. Colophon
  19. Copyright
O'Reilly logo

Chapter 12. Model Comparison

SVMs: The Support Vector Machine

In Chapter 3, we introduced the idea of decision boundaries and noted that problems in which the decision boundary isn’t linear pose a problem for simple classification algorithms. In Chapter 6, we showed you how to perform logistic regression, a classification algorithm that works by constructing a linear decision boundary. And in both chapters, we promised to describe a technique called the kernel trick that could be used to solve problems with nonlinear decision boundaries. Let’s deliver on that promise by introducing a new classification algorithm called the support vector machine (SVM for short), which allows you to use multiple different kernels to find nonlinear decision boundaries. We’ll use an SVM to classify points from a data set with a nonlinear decision boundary. Specifically, we’ll work with the data set shown in Figure 12-1.

Looking at this data set, it should be clear that the points from Class 0 are on the periphery, whereas points from Class 1 are in the center of the plot. This sort of nonlinear decision boundary can’t be discovered using a simple classification algorithm like the logistic regression algorithm we described in Chapter 6. Let’s demonstrate that by trying to use logistic regression through the glm function. We’ll then look into the reason why logistic regression fails.

df <- read.csv('data/df.csv') logit.fit <- glm(Label ~ X + Y, family = binomial(link = 'logit'), data = df) logit.predictions ...

The best content for your career. Discover unlimited learning on demand for around $1/day.