60 Recipes for Apache CloudStack

Chapter 6. Advanced Recipes

In this final chapter, we cover some advanced tools and use cases focused on application deployments and enhancement to your cloud infrastructure. Fluent is a log aggregation framework that you can use in your monitoring environment to store and analyze your logs. In combination with MongoDB or Elasticsearch, it is a very powerful tool. RiakCS is a scalable distributed object store created by Basho and provides an S3-compatible API. Together with a tool like EC2Stack, it can help you build an EC2/S3 clone. Apache Whirr is an automation tool to create clusters of virtual machines that form complete distributed systems, and in theses recipes, we’ll use Whirr to provision a Hadoop cluster. This final chapter rounds up the ecosystem, going from installation, clients, wrappers, configuration management, and software development tools to application deployment, monitoring, and high lever services to your cloud.

6.1. Installing Fluentd to Collect CloudStack Logs and Events

Fluentd is an open source software to collect events and logs in JSON format. It has hundreds of plug-ins that allow you to store the logs/events in your favorite data store like AWS S3, MongoDB, and even Elasticsearch. It is an equivalent to logstash. The source is available on GitHub, but can also be installed via your favorite package manager (e.g., brew, yum, apt, gem). A CloudStack plug-in listens to CloudStack events and stores these events in a chosen storage backend. In this chapter, ...