There are some additional factors to consider when you use sparse or dense elements (vectors or block matrices). Multiplying by a local matrix is usually preferable since it doesn't require expensive shuffling.
While simplicity and control is preferred when dealing with large matrices, the four types of distributed matrices simplify the setup and operation. Each of the four types has advantages and disadvantages that have to be considered and weighed against these three criteria:
- Sparsity or Density of underlying data
- Shuffling that will take place when using these facilities.
- Network capacity utilization when dealing with edge cases
For the reasons mentioned, and especially to reduce the shuffling (that is, a network bottleneck) ...
