Apache Spark 2.x Machine Learning Cookbook

How to do it...

1. Start a new project in IntelliJ or in an IDE of your choice. Make sure the necessary JAR files are included.

2. Set up the path using the package directive: package spark.ml.cookbook.chapter9.

3. Import the necessary packages.

The scala.util.control.Breaks will allow us to break out of the program. We use this during the debugging phase only when the program fails to converge or gets stuck in a never ending process (for example, when the step size is too large).

import scala.collection.mutable.ArrayBufferimport scala.util.control.Breaks._

4. This step defines the actual quadratic function that we are trying to minimize:

def quadratic_function_itself(x:Double):Double = {// the function being differentiated// f(x) = 2x^2 - 8x ...