Chapter 2

Working with Big Data

Complexity is the prodigy of the world. Simplicity is the sensation of the universe. Behind complexity, there is always simplicity to be revealed. Inside simplicity, there is always complexity to be discovered.

—Gang Yu

Introduction

Why is Big Data tough to understand? What makes it complex? How can multiple layers of data and processes be inferred? How can meaningful insights through all this complexity be gained? These are some pivotal questions that arise repeatedly in Big Data programs. Let us take a step back and understand how we arrived at the definition of Big Data.

In 2001, META Group (now Gartner) analyst Doug Laney, in a research report that he published, defined data growth challenges and opportunities ...

Get Data Warehousing in the Age of Big Data now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.