There's more...

The net result of using PCA in this recipe is that the original search space of 14 dimensions (the same as saying14 features) is reduced to 4 dimensions that explain almost all the variations in the original dataset.

PCA is not purely a ML concept and has been in use in finance for many years prior to the ML movement. At its core, PCA uses an orthogonal transformation (each component is perpendicular to the other component) to map the original features (apparent dimensions) to a set of newly derived dimensions so that most of the redundant and co-linear attributes are removed. The derived (actual latent dimension) components are linear combinations of the original attributes.

While it is easy to program PCA from scratch using ...

Get Apache Spark 2.x Machine Learning Cookbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.