O'Reilly logo

Learning Storm by Anand Nalya, Ankit Jain

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Trident repartitioning operations

By performing repartitioning operations, a user can partition tuples across multiple tasks. A repartitioning operation doesn't make any changes to the content of tuples. Also, the tuples will only pass over the network in the case of a repartitioning operation. The different types of repartitioning operations are explained in this section.

The shuffle operation

The shuffle repartitioning operation partitions the tuples in a uniform, random way across multiple tasks. This repartitioning operation is generally used when we want to distribute our processing load uniformly across tasks. The following diagram shows how the input tuples are repartitioned using the shuffle operation:

Working of the shuffle repartitioning ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required