Given the direction for Spark which favors the Dataset/DataFrame paradigm over low level RDD coding, one must seriously consider the reasoning for doing groupBy() on an RDD. While there are legitimate situations for which the operation is needed, the readers are advised to reformulate their solution to take advantage of the SparkSQL subsystem and its optimizer called Catalyst.
The Catalyst optimizer takes into account Scala's powerful features such as pattern matching and quasiquotes while building an optimized query plan.
- The documentation on Scala pattern matching is available at http://docs.scala-lang.org/tutorials/tour/pattern-matching.html
- The documentation on Scala quasiquotes is available at http://docs.scala-lang.org/overviews/quasiquotes/intro.html ...
