Spark for Data Science

Clustering techniques

Clustering is an unsupervised learning technique where there is no response variable to supervise the model. The idea is to cluster the data points that have some level of similarity. Apart from exploratory data analysis, it is also used as a part of a supervised pipeline where classifiers or regressors can be built on the distinct clusters. There are a bunch of clustering techniques available. Let us look into a few important ones that are supported by Spark.

K-means clustering

K-means is one of the most common clustering techniques. The k-means problem is to find cluster centers that minimize the intra-class variance, that is, the sum of squared distances from each data point being clustered to its cluster center (the center ...