Fast Data Processing with Spark 2 - Third Edition

Summary

This was a slightly longer chapter, but I am sure you have progressed to be experts in Spark by now. We started by looking at graph processing and then moved on to GraphX APIs and finally to a case study. Keep a look out for more GraphX APIs and also the new GraphFrame API, which is being developed for querying. We also have come to the end of this book. You started by installing Spark and understanding Spark from the basics, then you progressed to RDDs, Datasets, SQL, big data, and machine learning. In the process, we also discussed how Spark has matured from 1.x to 2.x, what data scientists would look for in a framework such as Spark, and the Spark architecture. We (the authors, editors, reviewers, and the rest of the gang at Packt) ...