Thoughtful Machine Learning

Preface

This book is about approaching tough problems. Machine learning is an amazing application of computation because it tackles problems that are straight out of science fiction. These algorithms can solve voice recognition, mapping, recommendations, and disease detection. The applications are endless, which is what makes machine learning so fascinating.

This flexibility is also what makes machine learning daunting. It can solve many problems, but how do we know whether we’re solving the right problem, or actually solving it in the first place? On top of that sadly much of academic coding standards are lax.

Up until this moment there hasn’t been a lot of talk about writing good quality code when it comes to machine learning and that is unfortunate. The ability for us to disseminate an idea across an entire industry is based on our ability to communicate it effectively. And if we write bad code, it’s doubtful a lot of people will listen.

Writing this book is my answer to that problem. Teaching machine learning to people in an easier to approach way. This subject is tough, and it’s compounded by hard to read code, or ancient C implementations that make zero sense.

While a lot of people will be confused as to why this book is written in Ruby instead of Python, it’s because writing tests in Ruby is a beautiful way of explaining your code. The entire book taking this test driven approach is about communication, and communicating the beautiful world of Machine Learning.

What to Expect from This Book

This book is not an exhaustive machine learning resource. For that I’d highly recommend Peter Flach’s Machine Learning: The Art and Science of Algorithms that Make Sense of Data (Cambridge University Press) or if you are mathematically inclined, Tom Mitchell’s Machine Learning series is top notch. There are also great tidbits from Artificial Intelligence: A Modern Approach, Third Edition by Stuart Russell and Peter Norvig (Prentice Hall).

After reading this book you will not have a PhD in machine learning, but I hope to give you enough information to get working on real problems using data with machine learning. You should expect lots of examples of the approach to problems as well as how to use them at a fundamental level.

You should also find yourself learning how to approach problems that are more fuzzy than the normal unit testing scenario.

How to Read This Book

The best way to read this book is to find examples that excite you. Each chapter aims to be fairly contained, although at times they won’t be. My goal for this book is not to be purely theoretical but to introduce you to some examples of problems that machine learning can solve for you as well as some worked out samples of how I’d approach working with data.

In most of the chapters, I try to introduce some business cases in the beginning then delve into a worked out example toward the end. This book is intended as a short read because I want you to focus on working with the code and thinking about these problems instead of getting steeped up in theory.

Who This Book Is For

There are three main people I have written the book for: the developer, the CTO, and the business analyst.

The developer already knows how to write code and is interested in learning more about the exciting world of machine learning. She has some background in working out problems in a computational context and may or may not write Ruby. The book is primarily focused on this persona but there is also the CTO and the business analyst.

The CTO is someone who really wants to know how to utilize machine learning to improve his company. He might have heard of K-Means, K-Nearest Neighbors but hasn’t quite figured out how it’s applicable to him. The business analyst is similar except that she is less technically inclined. These two personas I wrote the start of every chapter for.

How to Contact Me

I love receiving emails from people who either liked a presentation I gave or need help with a problem. Feel free to email me at [e-mail]. And to cement this, I will gladly buy you a cup of coffee if you come to the Seattle area (and our schedules permit).

If you’d like to view any of the code in this book, it’s free at GitHub.

Conventions Used in This Book

The following typographical conventions are used in this book:

Italic

Indicates new terms, URLs, email addresses, filenames, and file extensions.

Constant width

Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords.

Constant width bold

Shows commands or other text that should be typed literally by the user.

Constant width italic

Shows text that should be replaced with user-supplied values or by values determined by context.

Tip

This element signifies a tip or suggestion.

Note

This element signifies a general note.

Warning

This element indicates a warning or caution.

Important

This element indicates a warning of significant importance; read carefully.

Using Code Examples

Supplemental material (code examples, exercises, etc.) is available for download at http://github.com/thoughtfulml.

This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.

We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Thoughtful Machine Learning by Matthew Kirk (O’Reilly). Copyright 2015 Matthew Kirk, 978-1-449-37406-8.”

If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at [e-mail].

Safari® Books Online

Safari Books Online is an on-demand digital library that delivers expert content in both book and video form from the world’s leading authors in technology and business.

Technology professionals, software developers, web designers, and business and creative professionals use Safari Books Online as their primary resource for research, problem solving, learning, and certification training.

Safari Books Online offers a range of plans and pricing for enterprise, government, education, and individuals.

Members have access to thousands of books, training videos, and prepublication manuscripts in one fully searchable database from publishers like O’Reilly Media, Prentice Hall Professional, Addison-Wesley Professional, Microsoft Press, Sams, Que, Peachpit Press, Focal Press, Cisco Press, John Wiley & Sons, Syngress, Morgan Kaufmann, IBM Redbooks, Packt, Adobe Press, FT Press, Apress, Manning, New Riders, McGraw-Hill, Jones & Bartlett, Course Technology, and hundreds more. For more information about Safari Books Online, please visit us online.

How to Contact Us

Please address comments and questions concerning this book to the publisher:

• O’Reilly Media, Inc.
• 1005 Gravenstein Highway North
• Sebastopol, CA 95472
• 800-998-9938 (in the United States or Canada)
• 707-829-0515 (international or local)
• 707-829-0104 (fax)

We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at http://bit.ly/thoughtful-machine-learning.

To comment or ask technical questions about this book, send email to [e-mail].

For more information about our books, courses, conferences, and news, see our website at http://www.oreilly.com.

Find us on Facebook: http://facebook.com/oreilly

Follow us on Twitter: http://twitter.com/oreillymedia

Watch us on YouTube: http://www.youtube.com/oreillymedia

Acknowledgments

• Mike Loukides, who was intrigued by my idea about using test-driven development on machine learning code.
• Ann Spencer, my editor, who over the many months of my writing the book, coached me through edits and gave great feedback to shape the book.

I would like to thank all of the O’Reilly team, who helped make this book what it is, especially the following:

My reviewers:

• Brad Ediger, who was excited by my weird idea of writing a book on test-driven machine learning code, and gave great feedback on the first draft of the book.
• Starr Horne, who offered great insight during the review process. Thanks also for the conversation on the conference circuit about machine learning, error reporting, and more.
• Aaron Sumner, who provided great feedback about the overall coding structure of the book.

My amazing coworkers and friends who offered guidance during the book writing process: Edward Carrel, Jon-Michael Deldin, Christopher Hobbs, Chris Kuttruff, Stefan Novak, Mike Perham, Max Spransy, Moxley Stratton, and Wafa Zouyed.

This book would not be a reality without the consistent and pressing support of my family:

• To my wife, Sophia, who has been the anchor to my dreams and helped me shape the idea of this book into a reality.
• To my grandmother, Gail, who instilled a love of learning in me from an early age, and asked intently about the coffee book I was reading during a road trip (it was a book on Java).
• To my parents, Jay and Carol, who taught me the most about dissecting systems and adding human emotion to them.
• To my brother, Jacob, and nieces, Zoe and Darby, who are teaching me to relearn the world through a toddler’s mind.

Lastly, I dedicate this book to science and the pursuit of knowledge.