To fully understand and master the Dataset API, be sure to understand the concept of Row and Encoder.
Datasets follow the lazy execution paradigm, meaning that execution only occurs by invoking actions in Spark. When we execute an action, the Catalyst query optimizer produces a logical plan and generates a physical plan for optimized execution in a parallel distributed manner. See the figure in the introduction for all the detailed steps.
Documentation for Row is available at http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.Dataset
Documentation for Encoder is available at http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.Encoder
