Hadoop MapReduce v2 Cookbook - Second Edition

Hadoop counters to report custom metrics

Hadoop uses a set of counters to aggregate the metrics for MapReduce computations. Hadoop counters are helpful to understand the behavior of our MapReduce programs and to track the progress of the MapReduce computations. We can define custom counters to track the application-specific metrics in MapReduce computations.

How to do it...

The following steps show you how to define a custom counter to count the number of bad or corrupted records in our log processing application:

1. Define the list of custom counters using enum:

     public static enum LOG_PROCESSOR_COUNTER {
        BAD_RECORDS
       };

2. Increment the counter in your Mapper or Reducer:

   context.getCounter(LOG_PROCESSOR_COUNTER.BAD_RECORDS).increment(1);

3. Add the following ...