In this recipe, we explore the groupBy() and reduceBy() methods, which allow us to group values corresponding to a key. It is an expensive operation due to internal shuffling. We first demonstrate groupby() in more detail and then cover reduceBy() to show the similarity in coding these while stressing the advantage of the reduceBy() operator.
RDD transformation/aggregation with groupBy() and reduceByKey()
Get Apache Spark 2.x Machine Learning Cookbook now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
