Exploring Distributed IndexedRowMatrix in Spark 2.0

In this recipe, we cover the IndexRowMatrix, which is the first specialized distributed matrix that we cover in this chapter. The primary advantage of IndexedRowMatrix is that the index can be carried along with the row (RDD), which is the data itself.

In the case of IndexRowMatrix, we have an index defined by the developer which is permanently paired with a given row that is very useful for random access cases. The index not only helps with random access, but is also used for identifying the row itself when performing join() operations.

Get Apache Spark 2.x Machine Learning Cookbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.