How it works...

We read the housing data and load selected columns and use them to predict the price of a housing unit. We use the following code snippet to select the regression as LASSO and let Spark pick up the optimization on its own:

val lr = new LinearRegression().setMaxIter(1000).setElasticNetParam(0.0).setRegParam(0.01).setSolver( "auto" )

We change the setMaxIter() to 1000 for demonstration purposes. The default setting is 100 out of the box.

Get Apache Spark 2.x Machine Learning Cookbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.