Apache Kafka 2.0

To finish off our local development environment, we will install Apache Kafka. As described in Chapter 1, The Big Data Ecosystem, Apache Kafka is a distributed streaming platform. We will use Apache Kafka in Chapter 8, Real-Time Machine Learning Using Apache Spark, to develop a real-time analytical model by combining it with Spark Streaming and MLlib.

Again, for the purposes of our single-node development cluster, Apache Kafka will be deployed on the same single node as the Apache Spark software services. We will also be installing the version of Apache Kafka 2.0.0 that has been built for Scala 2.11.

Get Machine Learning with Apache Spark Quick Start Guide now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.