Transforming RDDs with the super useful flatMap() API

In this recipe, we examine the flatMap() method which is often a source of confusion for beginners; however, on closer examination we demonstrate that it is a clear concept that applies the lambda function to each element just like map, and then flattens the resulting RDD as a single structure (rather than having a list of lists, we create a single list made of all sublist with sublist elements).

Get Apache Spark 2.x Machine Learning Cookbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.