Mastering Azure Analytics, 1st Edition

Preface

If you are building software solutions today, odds are that you have a data problem. You might even have an advanced analytics problem or one that requires machine learning. The trouble is that the world of software development and those of big data and advanced analytics seem like they are light years apart—they use different software stacks, different terminology, and often different engineering approaches, and there are lots of choices. The aim of this book is to provide you with a map of the galaxy that helps you chart your course to wrangling insights and guidance out of your data—irrespective of whether that data is arriving at warp speed from IoT sensors or at the glacial pace of decades of historical data.

The structure of this book is designed along the path of a data pipeline that aims to ingest, process, store, and deliver data along both real-time (hot data) and batch (cold data) paths. The waypoints in the map to your data pipeline are groups of Azure services, and each is covered in one or more chapters. We describe each service and tool that you should consider for a particular step in your pipeline. Another way to think about it is to look at each phase of the analytics pipeline as a toolbox onto itself: which Azure service would you use for long-term storage? We show you how to use Azure Storage and Azure Data Lake Store. What about storage of streaming data? We give you the options—including Azure Stream Analytics, Azure HDInsight with Storm or Spark, and the Event Processor Host—and show you how to program them.

Of course, without a specific destination in mind, a map is not that interesting. To motivate our journey to build analytic data pipelines in Azure, we provide a fictitious business scenario looking to manage the data for airports, and give you all the sample data and code you will need to cement your understanding of the covered services.

Approach this book as you would a tour guide: you can read it from cover to cover, but you can also pick and choose the areas of most interest to you and dive into those. The map we build provides a narrated tour of the constellations of the various Azure services and tools you can use to build your data pipeline. In some cases, these constellations are complex, deep, and robust. In other cases, they are simple purpose-built solutions to a narrow set of problems. They might contain open source code bases, or they might be proprietary innovations from Microsoft. In all cases, by the end of this book you should have built your guide to the galaxy, will know which services and tools to use for which purpose, and will be well on your way to mastering Azure analytics.

Conventions Used in This Book

The following typographical conventions are used in this book:

Italic

Indicates new terms, URLs, email addresses, filenames, and file extensions.

Constant width

Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords.

Constant width bold

Shows commands or other text that should be typed literally by the user.

Constant width italic

Shows text that should be replaced with user-supplied values or by values determined by context.

Tip

This element signifies a tip or suggestion.

Note

This element signifies a general note.

Warning

This element indicates a warning or caution.

Using Code Examples

Supplemental material (code examples, exercises, etc.) is available for download at https://github.com/ZoinerTejada/mastering-azure-analytics.

This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.

We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Mastering Azure Analytics by Zoiner Tejada (O’Reilly). Copyright 2017 Zoiner Tejada, 978-1-491-95665-6.”

If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at [e-mail].

O’Reilly Safari

Note

Safari (formerly Safari Books Online) is a membership-based training and reference platform for enterprise, government, educators, and individuals.

Members have access to thousands of books, training videos, Learning Paths, interactive tutorials, and curated playlists from over 250 publishers, including O’Reilly Media, Harvard Business Review, Prentice Hall Professional, Addison-Wesley Professional, Microsoft Press, Sams, Que, Peachpit Press, Adobe, Focal Press, Cisco Press, John Wiley & Sons, Syngress, Morgan Kaufmann, IBM Redbooks, Packt, Adobe Press, FT Press, Apress, Manning, New Riders, McGraw-Hill, Jones & Bartlett, and Course Technology, among others.

For more information, please visit http://oreilly.com/safari.

How to Contact Us

Please address comments and questions concerning this book to the publisher:

• O’Reilly Media, Inc.
• 1005 Gravenstein Highway North
• Sebastopol, CA 95472
• 800-998-9938 (in the United States or Canada)
• 707-829-0515 (international or local)
• 707-829-0104 (fax)

We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at http://bit.ly/masterAzureAnalytics.

To comment or ask technical questions about this book, send email to [e-mail].

For more information about our books, courses, conferences, and news, see our website at http://www.oreilly.com.

Find us on Facebook: http://facebook.com/oreilly

Follow us on Twitter: http://twitter.com/oreillymedia

Watch us on YouTube: http://www.youtube.com/oreillymedia

Acknowledgments

A book with a scope this large needs a village to support its creation, and I was honored to be supported by so many great experts along the way. I would like to thank Lynn Langit for her hypercritical eye for anything unclear or ambiguous—her challenges and pressure helped this book emerge from a diamond in the rough. I am honored to have had technical reviewers from Microsoft: thanks to Nishant Thacker, Ted Way, and Rama Ramani for lending their specific areas of expertise to this effort.

I would like to recognize one outstanding citizen of the Azure MVP community: Tom Kerkhove. Thank you for your meticulous attention to detail in testing the code and making modifications so that readers have a smooth implementation experience.

Thanks to Matt Winkler for lighting the way forward from those workflow days into this amazing new world of data, and teaching me both how to drink from the firehose and control it.

I would be remiss not to mention the caring, insightful, and helpful support I received from my editor Shannon Cutt on this journey of more than a year. Similarly, I am most appreciative to Kristen Brown for helping catch the places where my fingers didn’t quite transcribe my thoughts at the speed I was thinking them. Thank you both for making my first O’Reilly authoring experience an amazing one.

Finally, thank you to my wife Ashley for tolerating the many late nights and boring weekends we spent “together” writing this book. Your understanding and patience were the ultimate gesture of love.