Distributed Application Services with BOTO
The full value of Amazonâs SQS messaging service is most evident when it is used to connect the components of a distributed application, bringing together a set of computing resources to cooperate on a task. In this application, we will apply everything we have learned about the Amazonâs Web Services (AWS) so far to create a distributed application that can run arbitrary Linux applications to process data files any way you wish. This application will use S3 for storage, EC2 instances for computer processing power, and SQS messaging to tie the whole system together.
In this application, we will use the open source boto library (version 0.9d) as a starting point. This Python library includes support for the three Amazon services we will use in this example, but most importantly it also includes a framework to create and run service components that process SQS messages. The boto library will allow us to concentrate on the functionality of our application without having to build the underlying plumbing from scratch.
The Boto Services Framework
Mitch Garnaat, the creator of boto, has written an excellent article explaining how his library can be used as the basis for a distributed application that converts video files from one format to another. In the article âMonster Muck Mashup: Mass Video Conversion Using AWS,â he describes the architecture, message format, and processing steps used in the application, as well as offering a prepackaged ...
Get Programming Amazon Web Services now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
