Apache Spark 2.x Machine Learning Cookbook

How to do it...

1. Start a new project in IntelliJ or in an IDE of your choice. Make sure that the necessary JAR files are included.
2. Import the necessary packages for vector and matrix manipulation:

import org.apache.spark.sql.{SparkSession}import org.apache.spark.mllib.linalg._import breeze.linalg.{DenseVector => BreezeVector}import Array._import org.apache.spark.mllib.linalg.SparseVector

3. Set up the Spark context and application parameters so Spark can run. See the first recipe in this chapter for more details and variations:

val spark = SparkSession .builder .master("local[*]") .appName("myVectorMatrix") .config("spark.sql.warehouse.dir", ".") .getOrCreate()

4. Here we look at creating a ML SparseVector that corresponds to its equivalent ...