How it works...

In this example, we first set up two arrays representing signal noise and signal strength. They are simply a set of measured numbers that we could have received from the IoT platform. We then proceeded to pair the two separate arrays so each member looks like they have been input originally as a pair of (x, y). We then proceed to divide the pair and produce the noise to signal ratio using the following code snippet:

val zipRDD= parSN.zip(parSS).map(r => r._1 / r._2) 

The zip() method has many variations that involve partitions. The developers should familiarize themselves with variations of the zip() method with partition (for example, zipPartitions).

Get Apache Spark 2.x Machine Learning Cookbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.