Explore architectural approaches to building Data Lakes that ingest, index, manage, and analyze massive amounts of data using Big Data technologies
About This Book
Comprehend the intricacies of architecting a Data Lake and build a data strategy around your current data architecture
Efficiently manage vast amounts of data and deliver it to multiple applications and systems with a high degree of performance and scalability
Packed with industry best practices and use-case scenarios to get you up-and-running
Who This Book Is For
This book is for architects and senior managers who are responsible for building a strategy around their current data architecture, helping them identify the need for a Data Lake implementation in an enterprise context. The reader will need a good knowledge of master data management and information lifecycle management, and experience of Big Data technologies.
What You Will Learn
Identify the need for a Data Lake in your enterprise context and learn to architect a Data Lake
Learn to build various tiers of a Data Lake, such as data intake, management, consumption, and governance, with a focus on practical implementation scenarios
Find out the key considerations to be taken into account while building each tier of the Data Lake
Understand Hadoop-oriented data transfer mechanism to ingest data in batch, micro-batch, and real-time modes
Explore various data integration needs and learn how to perform data enrichment and data transformations using Big Data technologies
Enable data discovery on the Data Lake to allow users to discover the data
Discover how data is packaged and provisioned for consumption
Comprehend the importance of including data governance disciplines while building a Data Lake
A Data Lake is a highly scalable platform for storing huge volumes of multistructured data from disparate sources with centralized data management services. This book explores the potential of Data Lakes and explores architectural approaches to building data lakes that ingest, index, manage, and analyze massive amounts of data using batch and real-time processing frameworks. It guides you on how to go about building a Data Lake that is managed by Hadoop and accessed as required by other Big Data applications.
This book will guide readers (using best practices) in developing Data Lake's capabilities. It will focus on architect data governance, security, data quality, data lineage tracking, metadata management, and semantic data tagging. By the end of this book, you will have a good understanding of building a Data Lake for Big Data.
Data Lakes can be viewed as having three capabilities—intake, management, and consumption. This book will take readers through each of these processes of developing a Data Lake and guide them (using best practices) in developing these capabilities. It will also explore often ignored, yet crucial considerations while building Data Lakes, with the focus on how to architect data governance, security, data quality, data lineage tracking, metadata management, and semantic data tagging. By the end of this book, you will have a good understanding of building a Data Lake for Big Data. You will be able to utilize Data Lakes for efficient and easy data processing and analytics.
Style and approach
Data Lake Development with Big Data provides architectural approaches to building a Data Lake. It follows a use case-based approach where practical implementation scenarios of each key component are explained. It also helps you understand how these use cases are implemented in a Data Lake. The chapters are organized in a way that mimics the sequential data flow evidenced in a Data Lake.
Downloading the example code for this book. You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the code file.