In this recipe, we demonstrate how to get hold of SparkContext using a SparkSession object in Spark 2.0. This recipe will demonstrate the creation, usage, and back and forth conversion of RDD to Dataset. The reason this is important is that even though we prefer Dataset going forward, we must still be able to use and augment the legacy (pre-Spark 2.0) code mostly utilizing RDD.
Getting access to SparkContext vis-a-vis SparkSession object in Spark 2.0
Get Apache Spark 2.x Machine Learning Cookbook now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
