Apache Spark 2.x Machine Learning Cookbook

Creating and using RDD versus DataFrame versus Dataset from a text file in Spark 2.0

In this recipe, we explore the subtle differences in creating RDD, DataFrame, and Dataset from a text file and their relationship to each other via a short sample code:

Dataset: spark.read.textFile()RDD: spark.sparkContext.textFile()DataFrame: spark.read.text()

Assume spark is the session name