How it works...

In this recipe, we begin by loading the iris dataset and using the zip() API to pair data with a unique identifier to the data for generating labeled points data structure for use with the KMeans algorithm.

Next, the mutable queues and QueueInputDStream are created for appending data to simulate streaming. Once the QueueInputDStream starts receiving data then the streaming k-mean clustering begins to dynamically cluster data and printing out results. The interesting thing you will notice here is we are streaming the training dataset on one queue stream and the test data on another queue stream. As we append data to our queues, the KMeans clustering algorithm is processing our incoming data and dynamically generating clusters. ...

Get Apache Spark 2.x Machine Learning Cookbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.