The MapReduceTriplets operator
Prior to Spark 1.2, there was no aggregateMessages
method in Graph. Instead, the now deprecated mapReduceTriplets
was the primary aggregation operator. The API for mapReduceTriplets
is:
class Graph[VD, ED] { def mapReduceTriplets[Msg]( map: EdgeTriplet[VD, ED] => Iterator[(VertexId, Msg)], reduce: (Msg, Msg) => Msg) : VertexRDD[Msg] }
Compared to mapReduceTriplets
, the new operator aggregateMessages
is more expressive as it employs the message passing mechanism instead of returning an iterator of messages as mapReduceTriplets
does. In addition, aggregateMessages
explicitly requires the user to specify the TripletFields
object for performance improvement as we explained previously. In addition to API improvements, ...
Get Apache Spark Graph Processing now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.