Computation in the batch layer

Many lambda architecture systems will rely on Hadoop or Spark for the batch layer. Since we don't want to manage a cluster ourselves, we'll have to pick some other serverless system or, at the very least, a managed service. There are a variety of options here.

First, it's possible to implement our MapReduce system entirely using serverless technologies. You'll read about this in Chapter 8, The MapReduce Pattern. If you'd rather not build a MapReduce system, there are other services that you can leverage. Both Spark and Hadoop are available within AWS EMR. HDInsight from Azure provides the same or similar functionality to EMR.

Batch processing is a solved problem nowadays, and you should have no problems finding ...

Get Serverless Design Patterns and Best Practices now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.