Using a dead-letter queue

Under certain circumstances, worker processes can never successfully process a message sitting in a queue. Take for example our image classifier problem. The worker processes aren't responsible for much other than downloading an image, sending it to Rekognition, and storing those results in DynamoDB. However, what happens if, in between a tweet arriving in the queue and our processing of that tweet, a Twitter user deletes the original image or tweet. In this case, our classifier process would fail hard. Look for yourself, and you'll see there are no guards against an HTTP 404 response code from the image fetch.

A hard failure like this will result in the application code skipping the sqs.delete_message function altogether. ...

Get Serverless Design Patterns and Best Practices now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.