Apache Spark 2.x Machine Learning Cookbook

How it works...

In this example, we show how to connect to a Spark cluster using local and remote options for an application. First, we create a SparkSession object which will grant us access to a Spark cluster by specifying whether the cluster is local or remote using the master() function. You can also specify the master location by passing a JVM argument when starting your client program. In addition, you can configure an application name and a working data directory. Next, you invoked the getOrCreate() method to create a new SparkSession or hand you a reference to an already existing session. Finally, we execute a small sample program to prove our SparkSession object creation is valid.