Ethics of Big Data

Preface

Philosophy and business don’t always get along well. Philosophy is generally not much concerned with the practical implications of its investigations and, conversely, business is often deeply interested in the tactical outcomes of its operations.

And ethics is a loaded word. Preconceived notions of what ethics mean, even as a legitimate field of study, often make people shy away from it as a topic of discussion. It’s hard to talk about what we don’t fully understand and even the word itself can sometimes imply judgment: do-this-don’t-do-that kinds of directives and obligations. And we all frequently chafe when we think we’re being told what to do.

This book tries to diminish these difficulties. Not because they are difficult (ethical inquiry can be hard work) but because they create barriers to helping organizations benefit from philosophical thinking and inquiry. And there are plenty of benefits. The primary characteristic of my approach was to recognize that business contexts, markets, companies, cultures, geographic distinctions, and organizational size and maturity all contribute to an unwieldy set of complex and different circumstances. Circumstances with which you are much more familiar in your own case and therefore more qualified to determine how best to inform your organization’s operations with ethical inquiry.

People often ask me: “how did you get from a degree in philosophy to consulting?” The answer varied and evolved over the years—mostly as consequence of me learning more about how to answer the question. And it bears on the relationship between philosophy and business in general and ethics and big data in particular.

My interest in technology started in 5th grade when my grandmother gave me a 75 in One Electronic Project Kit—vintage editions are still available on eBay! It turned out that wires and batteries and capacitors and resistors could all be combined and recombined to create brand new circuits that performed all manner of fascinating and interesting functions. Through high school programming classes and working in telecommunications as a Radioman for most of my nearly 5 years in the United States Coast Guard, I came to realize that what was engaging about technology was that it spoke to the essence of some important and hard facts about our physical world. Energy flowed and could be directed. Radio waves were generated and could carry digital information. Transistors and other semiconductor materials could be combined to create powerful new computing processing and storage devices. And software could be written that would make all those devices do some amazing things.

You’d think I would have studied physics or computer science. Instead what happened is that philosophy captured my attention by offering the best of both worlds: the rigor of analysis and investigation into the essence of all things and an open and willing approach to understanding how science and technology itself works. I was sold.

A key motivation for this book is to apply the tools that philosophy in general, and ethical inquiry in particular, provide us to evolve technology and shape it into tools that can help us live better, easier lives.

Enter big data. This aspect of technology is unique in that its very nature (its essence) is to create, connect, correlate, aggregate, store, process, and report on massive amounts of information. As human beings, we have simply never seen, let alone understood, how to manage that much data. One of the implications of amassing this much information, especially about people and their behaviors, is what I’m calling big data’s “forcing function.” It is pushing us—whether we like it or not—to consider serious ethical issues including whether certain uses of big data violate fundamental civil, social, political, and legal rights.

These are long, complex, and deeply important conversations. And, as a society, we’re not having enough of them. But it’s hard to have them because we’re not accustomed to having them in business environments very much. And ethics can be a loaded word. So, the hope is that this work will help you and your organization begin to develop the capability to engage in explicit ethical inquiry in new ways and in new contexts. To begin, the methods, concepts, and intentional vocabulary in this book are intended to provide you with a better ability to determine, in your own unique circumstances, how best to execute on and utilize the results of explicit ethical inquiry to improve your organization.

Such discussions are in their infancy in terms of understanding both the issues and their outcomes. We are all just figuring it out as we go—a circumstance about which we have no other choice. Nobody in history has ever had the opportunity to innovate, or been faced with the risks of unintended consequences, that big data now provides.

I look forward to being a part of that ongoing discussion. O’Reilly has constructed a tool chain that allows this book to be easily updated and re-distributed through print-on-demand and digital channels. As the collective understanding and use of big data evolves, the work can evolve right along with it.

Conventions Used in This Book

The following typographical conventions are used in this book:

Italic

Indicates new terms, URLs, email addresses, filenames, and file extensions.

Using Code Examples

This book is here to help you get your job done. In general, you may use the code in this book in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.

We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Ethics of Big Data by Kord Davis with Doug Patterson (O’Reilly). Copyright 2012 Kord Davis, 978-1-449-31179-7.”

If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at [e-mail].

Safari® Books Online

Note

Safari Books Online (www.safaribooksonline.com) is an on-demand digital library that delivers expert content in both book and video form from the world’s leading authors in technology and business.

Technology professionals, software developers, web designers, and business and creative professionals use Safari Books Online as their primary resource for research, problem solving, learning, and certification training.

Safari Books Online offers a range of product mixes and pricing programs for organizations, government agencies, and individuals. Subscribers have access to thousands of books, training videos, and prepublication manuscripts in one fully searchable database from publishers like O’Reilly Media, Prentice Hall Professional, Addison-Wesley Professional, Microsoft Press, Sams, Que, Peachpit Press, Focal Press, Cisco Press, John Wiley & Sons, Syngress, Morgan Kaufmann, IBM Redbooks, Packt, Adobe Press, FT Press, Apress, Manning, New Riders, McGraw-Hill, Jones & Bartlett, Course Technology, and dozens more. For more information about Safari Books Online, please visit us online.

How to Contact Us

Please address comments and questions concerning this book to the publisher:

O’Reilly Media, Inc.

1005 Gravenstein Highway North

Sebastopol, CA 95472

800-998-9938 (in the United States or Canada)

707-829-0515 (international or local)

707-829-0104 (fax)

We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at http://oreil.ly/ethics-big-data.

To comment or ask technical questions about this book, send email to [e-mail].

For more information about our books, courses, conferences, and news, see our website at http://www.oreilly.com.

Find us on Facebook: http://facebook.com/oreilly

Follow us on Twitter: http://twitter.com/oreillymedia

Watch us on YouTube: http://www.youtube.com/oreillymedia

Acknowledgments

This book benefited from a large and wide variety of people, ideas, input, and efforts. I’d like to acknowledge several of them and apologize in advance to those I may have forgotten.

First, thanks to Brian Smith, Rob Wiley, and Tom Williams at Exact Target, a company who not only does a wonderful job of incorporating their values into their organizational culture, but are on the forefront of learning how to turn big data technologies into useful tools. In many ways, the experience of working with them forged the motivation for this book. Numerous conversations, interviews, dinners, and lunches yielded a great deal of great thinking and material, and I hope I’ve represented our discussions well.

Those discussions wouldn’t have been possible if not for the projects that gave them a platform through working with XPLANE, The Visual Thinking Company. There are many great folks—and talents—at XPLANE but several individuals made direct contributions to the book including Matt Adams, Stephanie Gioia, Dave King, and James Macanufo.

Aside from those with whom I work closely, there were various subject matter experts, from a wide variety of industries and disciplines, who graciously gave of their time and expertise to help work through various concepts and their implications. These include: Andrew Davis, Nigel Ballard, Jason Bobe, Erin Conroy, Pete Forsyth, Ezra Gollogly, Dion Hinchliffe, Erik Huddleston, Bill Hoffman, Max Niederhofer, Martha Koenig, and Adam Riggs.

A special individual who provided a great deal of subject matter expertise is Doug Patterson. His academic background, training, and expertise were valuable and informed much of the philosophical thinking here. His experience teaching business ethics and facilitating classroom discussion on highly conceptual topics meant he could quickly identify key ethical issues. He was a great resource to turn to in those moments when I needed clarity on more nuanced aspects of issues that had become complex.

A dedicated, informed, and rigorous group of technical reviewers gave the work the thrashing it deserved and I hope that their comments and input are reflected fairly—I know they made the work stronger. So, a special thanks to Terence Craig, Bob Gourley, Mary E. Ludloff, James Macanufo, and Cathy O’Neill.

Last, and certainly not least, are many friends and members of my community. I want to thank them from the bottom of my heart for their encouragement, faith, discussions, patience, sustenance, interest, and ongoing belief in the value of this project: Laura Allen, Jake Baker, Tad Bamford, Cari Carter, Collin Connon, Tanya Frantzen, Patrick Foss, Vincent Grace, Drew Hansen, Erica Hassinger, Geoff Rogers, Jodi Sweetman, Carisa Sprecher, Khris Soden, Ben Thompson, Paul Wille, Rob Woolsey, and Morgan Wu.

Finally, a great deal of gratitude and an explicit Thank You to the many folks at O’Reilly who have been a part of this effort.

Especially my primary editor Courtney Nash who, when I told her I was planning to write a self-published white paper on big data ethics, immediately started investigating whether anyone had signed up to do that for O’Reilly and offered to bring a proposal to the editorial group. Special thanks for recognizing the value of the topic, being its champion, and working diligently to help make sure the project continued to unfold productively—all while making the work read better in the process.

That also couldn’t have happened without interim editor (while Courtney was working on a side project of her own—which resulted in a beautiful baby girl), Julie Steele. Julie stepped in graciously, in the middle of a very busy and important time, and helped make sure I stayed between the rails as the work moved forward.

And lastly, of course, thanks to Tim O’Reilly for creating an organization that would even consider publishing work on such a topic and for his discussion and insights on technology, culture, and community.

I hope you all enjoy the book and find it useful.