O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Getting Started with Kudu

Book Description

Fast data ingestion, serving, and analytics in the Hadoop ecosystem have forced developers and architects to choose solutions using the least common denominator—either fast analytics at the cost of slow data ingestion or fast data ingestion at the cost of slow analytics. There is an answer to this problem. With the Apache Kudu column-oriented data store, you can easily perform fast analytics on fast data. This practical guide shows you how.

Begun as an internal project at Cloudera, Kudu is an open source solution compatible with many data processing frameworks in the Hadoop environment. In this book, current and former solutions professionals from Cloudera provide use cases, examples, best practices, and sample code to help you get up to speed with Kudu.

  • Explore Kudu’s high-level design, including how it spreads data across servers
  • Fully administer a Kudu cluster, enable security, and add or remove nodes
  • Learn Kudu’s client-side APIs, including how to integrate Apache Impala, Spark, and other frameworks for data manipulation
  • Examine Kudu’s schema design, including basic concepts and primitives necessary to make your project successful
  • Explore case studies for using Kudu for real-time IoT analytics, predictive modeling, and in combination with another storage engine

Table of Contents

  1. Preface
    1. Conventions Used in This Book
    2. Using Code Examples
    3. O’Reilly Safari
    4. How to Contact Us
    5. Acknowledgments
  2. 1. Why Kudu?
    1. Why Does Kudu Matter?
    2. Simplicity Drives Adoption
    3. New Use Cases
      1. IoT
      2. Current Approaches to Real-Time Analytics
      3. Real-Time Processing
    4. Hardware Landscape
    5. Kudu’s Unique Place in the Big Data Ecosystem
      1. Comparing Kudu with Other Ecosystem Components
      2. Big Data—HDFS, HBase, Cassandra
    6. Conclusion
  3. 2. About Kudu
    1. Kudu High-Level Design
      1. Kudu Roles
      2. Master Server
      3. Tablet Server
    2. Kudu Concepts and Mechanisms
      1. Hotspotting
      2. Partitioning
  4. 3. Getting Up and Running
    1. Installation
      1. Apache Kudu Quickstart VM
      2. Using Cloudera Manager
      3. Building from Source
      4. Packages
      5. Cloudera Quickstart VM
    2. Quick Install: Three Minutes or Less
    3. Conclusion
  5. 4. Kudu Administration
    1. Planning for Kudu
      1. Master and Tablet Servers
      2. Write-Ahead Log
      3. Data Servers and Storage
      4. Replication Strategies
    2. Deployment Considerations: New or Existing Clusters?
      1. New Kudu-Only Cluster
      2. New Hadoop Cluster with Kudu
      3. Add Kudu to Existing Hadoop Cluster
    3. Web UI of Tablet and Master Servers
      1. Master Server UI and Tablet Server UI
      2. Master Server UI
      3. Tablet Server UI
    4. The Kudu Command-Line Interface
      1. Cluster
      2. Filesystem
      3. Tablet Replica
      4. Consensus Metadata
    5. Adding and Removing Tablet Servers
      1. Adding Tablet Servers
      2. Removing a Tablet Server
    6. Security
      1. A Simple Analogy
      2. Kudu Security Features
    7. Basic Performance Tuning
      1. Kudu Memory Limits
      2. Maintenance Manager Threads
      3. Monitoring Performance
    8. Getting Ahead and Staying Out of Trouble
      1. Avoid Running Out of Disk Space
      2. Disk Failures Tolerance
      3. Backup
    9. Conclusion
  6. 5. Common Developer Tasks for Kudu
    1. Client API
      1. Kudu Client
      2. Kudu Table
      3. Kudu DDL
      4. Kudu Scanner Read Modes
    2. C++ API
    3. Python API
      1. Preparing the Python Development Environment
      2. Python Kudu Application
    4. Java
      1. Java Application
    5. Spark
    6. Impala with Kudu
  7. 6. Table and Schema Design
    1. Schema Design Basics
    2. Schema for Hybrid Transactional/Analytical Processing
      1. Lambda Architecture
      2. OLTP/OLAP Split
    3. Primary Key and Column Design
      1. Other Column Schema Considerations
    4. Partitioning Basics
      1. Range Partitioning
      2. Hash Partitioning
    5. Schema Alteration
    6. Best Practices and Tips
      1. Partitioning
      2. Large Objects
      3. decimal
      4. Unique Strings
      5. Compression
      6. Object Names
      7. Number of Columns
      8. Binary Types
    7. Network Packet Example
    8. Conclusion
  8. 7. Kudu Use Cases
    1. Real-Time Internet of Things Analytics
    2. Predictive Modeling
    3. Mixed Platforms Solution
  9. Index