Apache Spark Machine Learning Blueprints

ML workflows and Spark pipelines

In this section, we provide an introduction to machine learning workflows, and also Spark pipelines, and then discuss how Spark pipeline can serve as a good tool of computing ML workflows.

After this section, readers will master these two important concepts, and be ready to program and implement Spark pipelines for machine learning workflows.

ML as a step-by-step workflow

Almost all ML projects involve cleaning data, developing features, estimating models, evaluating models, and then interpreting results, which all can be organized into some step by step workflows. These workflows are sometimes called analytical processes.

Some people even define machine learning as workflows of turning data into actionable insights, ...