There's more...

We will end up with much more efficient (low-ranked) matrices for computation based on the original dataset.

The following equation depicts the decomposition of an array of m x n, which is large and hard to work with. The right-hand side of the equation helps to solve the decomposition problem which is the basis of the SVD technique.

The following steps provides a concrete example of the SVD decomposition step by step:

  • Consider a matrix of 1,000 x 1,000 which provides 1,000,000 data points (M= users, N = Movies).
  • Assume there are 1,000 rows (number of observations) and 1,000 columns (number of movies).
  • Let's assume we use ...

Get Apache Spark 2.x Machine Learning Cookbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.