We began by loading and parsing the data file into a dataset with the data type ratings, and finally converted it to a DataFrame. The DataFrame was then used to execute a Spark SQL query that grouped all the ratings by user with their totals.
We explored Dataset/DataFrame in Chapter 3, Spark's Three Data Musketeers for Machine Learning - Perfect Together, but we encourage the user to refresh and dig deeper into the Dataset/DataFrame API. A full understanding of the API and its concepts (lazy instantiation, staging, pipelining, and caching) is critical for every Spark developer.
Finally, we passed the result set of data to the ...
