- Machine Learning for Hackers
- Preface
- 1. Using R
- 2. Data Exploration
- 3. Classification: Spam Filtering
- 4. Ranking: Priority Inbox
- 5. Regression: Predicting Page Views
- 6. Regularization: Text Regression
- 7. Optimization: Breaking Codes
- 8. PCA: Building a Market Index
- 9. MDS: Visually Exploring US Senator Similarity
- 10. kNN: Recommendation Systems
- 11. Analyzing Social Graphs
- 12. Model Comparison
- Works Cited
- Index
- About the Authors
- Colophon
- Copyright

So far we’ve treated most of the algorithms in this book as partial black boxes, in that we’ve focused on understanding the inputs you’re expected to use and the outputs you’ll get. Essentially, we’ve treated machine learning algorithms as a library of functions for performing prediction tasks.

In this chapter, we’re going to examine some of the techniques
that are used to implement the most basic machine learning algorithms.
As a starting point, we’re going to put together a function for fitting
simple linear regression models with only one predictor. That example
will allow us to motivate the idea of viewing the act of fitting a model
to data as an optimization problem. An optimization problem is one in
which we have a machine with some knobs that we can turn to change the
machine’s settings and a way to measure how well the machine is
performing with the current settings. We want to find the best possible
settings, which will be those that maximize some simple measure of how
well the machine is performing. That point will be called the *optimum*.
Reaching it will be called *optimization*.

Once we have a basic understanding of how optimization works, we’ll embark on our major task: building a very simple code-breaking system that treats deciphering an encrypted text as an optimization problem.

Because we’re going to build our own linear regression function, let’s go back to our standard example data set: people’s heights ...