You need to be cognizant of our use of Spark SQL because of its flexibility. More information is available at http://spark.apache.org/docs/latest/sql-programming-guide.html#running-sql-queries-programmatically.
There's more...
Get Apache Spark 2.x Machine Learning Cookbook now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.