Chapter 8. Monitoring Flume

The user guide for Flume states:

Monitoring in Flume is still a work in progress. Changes can happen very often. Several Flume components report metrics to the JMX platform MBean server. These metrics can be queried using Jconsole.

While JMX is fine for casual browsing of metric values, the number of eyeballs looking at Jconsole doesn't scale when you have hundreds or even thousands of servers sending data all over the place. What you need is a way to watch everything at once. However, what are the important things to look for? That is a very difficult question, but I'll try and cover several of the items that are important, as we cover monitoring options in this chapter.

Monitoring the agent process

The most obvious type ...

Get Apache Flume: Distributed Log Collection for Hadoop - Second Edition now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.