You are previewing An Introduction to Support Vector Machines and Other Kernel-based Learning Methods.
O'Reilly logo
An Introduction to Support Vector Machines and Other Kernel-based Learning Methods

Book Description

This is the first comprehensive introduction to Support Vector Machines (SVMs), a new generation learning system based on recent advances in statistical learning theory. SVMs deliver state-of-the-art performance in real-world applications such as text categorisation, hand-written character recognition, image classification, biosequences analysis, etc., and are now established as one of the standard tools for machine learning and data mining. Students will find the book both stimulating and accessible, while practitioners will be guided smoothly through the material required for a good grasp of the theory and its applications. The concepts are introduced gradually in accessible and self-contained stages, while the presentation is rigorous and thorough. Pointers to relevant literature and web sites containing software ensure that it forms an ideal starting point for further study. Equally, the book and its associated web site will guide practitioners to updated literature, new applications, and on-line software.

Table of Contents

  1. Cover
  2. An Introduction to Support Vector Machines
  3. Title
  4. Copyright
  5. Contents
  6. Preface
  7. Notation
  8. 1 The Learning Methodology
    1. 1.1 Supervised Learning
    2. 1.2 Learning and Generalisation
    3. 1.3 Improving Generalisation
    4. 1.4 Attractions and Drawbacks of Learning
    5. 1.5 Support Vector Machines for Learning
    6. 1.6 Exercises
    7. 1.7 Further Reading and Advanced Topics
  9. 2 Linear Learning Machines
    1. 2.1 Linear Classification
      1. 2.1.1 Rosenblatt's Perceptron
      2. 2.1.2 Other Linear Classifiers
      3. 2.1.3 Multi-class Discrimination
    2. 2.2 Linear Regression
      1. 2.2.1 Least Squares
      2. 2.2.2 Ridge Regression
    3. 2.3 Dual Representation of Linear Machines
    4. 2.4 Exercises
    5. 2.5 Further Reading and Advanced Topics
  10. 3 Kernel–Induced Feature Spaces
    1. 3.1 Learning in Feature Space
    2. 3.2 The Implicit Mapping into Feature Space
    3. 3.3 Making Kernels
      1. 3.3.1 Characterisation of Kernels
      2. 3.3.2 Making Kernels from Kernels
      3. 3.3.3 Making Kernels from Features
    4. 3.4 Working in Feature Space
    5. 3.5 Kernels and Gaussian Processes
    6. 3.6 Exercises
    7. 3.7 Further Reading and Advanced Topics
  11. 4 Generalisation Theory
    1. 4.1 Probably Approximately Correct Learning
    2. 4.2 Vapnik Chervonenkis (VC) Theory
    3. 4.3 Margin–Based Bounds on Generalisation
      1. 4.3.1 Maximal Margin Bounds
      2. 4.3.2 Margin Percentile Bounds
      3. 4.3.3 Soft Margin Bounds
    4. 4.4 Other Bounds on Generalisation and Luckiness
    5. 4.5 Generalisation for Regression
    6. 4.6 Bayesian Analysis of Learning
    7. 4.7 Exercises
    8. 4.8 Further Reading and Advanced Topics
  12. 5 Optimisation Theory
    1. 5.1 Problem Formulation
    2. 5.2 Lagrangian Theory
    3. 5.3 Duality
    4. 5.4 Exercises
    5. 5.5 Further Reading and Advanced Topics
  13. 6 Support Vector Machines
    1. 6.1 Support Vector Classification
      1. 6.1.1 The Maximal Margin Classifier
      2. 6.1.2 Soft Margin Optimisation
      3. 6.1.3 Linear Programming Support Vector Machines
    2. 6.2 Support Vector Regression
      1. 6.2.1 ε-Insensitive Loss Regression
      2. 6.2.2 Kernel Ridge Regression
      3. 6.2.3 Gaussian Processes
    3. 6.3 Discussion
    4. 6.4 Exercises
    5. 6.5 Further Reading and Advanced Topics
  14. 7 Implementation Techniques
    1. 7.1 General Issues
    2. 7.2 The Naive Solution: Gradient Ascent
    3. 7.3 General Techniques and Packages
    4. 7.4 Chunking and Decomposition
    5. 7.5 Sequential Minimal Optimisation (SMO)
      1. 7.5.1 Analytical Solution for Two Points
      2. 7.5.2 Selection Heuristics
    6. 7.6 Techniques for Gaussian Processes
    7. 7.7 Exercises
    8. 7.8 Further Reading and Advanced Topics
  15. 8 Applications of Support Vector Machines
    1. 8.1 Text Categorisation
      1. 8.1.1 A Kernel from IR Applied to Information Filtering ....
    2. 8.2 Image Recognition
      1. 8.2.1 Aspect Independent Classification
      2. 8.2.2 Colour–Based Classification
    3. 8.3 Hand-written Digit Recognition
    4. 8.4 Bioinformatics
      1. 8.4.1 Protein Homology Detection
      2. 8.4.2 Gene Expression
    5. 8.5 Further Reading and Advanced Topics
  16. A Pseudocode for the SMO Algorithm
  17. B Background Mathematics
    1. B.1 Vector Spaces
    2. B.2 Inner Product Spaces
    3. B.3 Hilbert Spaces
    4. B.4 Operators, Eigenvalues and Eigenvectors
  18. References
  19. Index