Learning AWS

Designing for parallel processing

It is a lot easier to design for parallelization on the cloud platform. You need to design for concurrency throughout your architecture, from data ingestion to its processing. So use multithreading to parallelize your cloud service requests, distribute load using load balancing, ensure multiple processing components or service endpoints are available via horizontal scaling, and so on.

As a best practice, you should exploit both multithreading and multi-node processing features in your designs. For example, using multiple concurrent threads for fetching objects from a cloud data storage service is a lot faster than fetching them sequentially. In the precloud or noncloud environments, parallel processing across a ...