Flume configuration file overview

Now that we've downloaded Flume, let's spend some time going over how to configure an agent.

A Flume agent's default configuration provider uses a simple Java property file of key/value pairs that you pass as an argument to the agent upon startup. Since you can configure more than one agent in a single file, you will need to additionally pass an agent identifier (called a name) so it knows which configurations to use. In my examples where I'm only specifying one agent I'm going to use the name agent.

Each agent is configured starting with three parameters:

agent.sources=<list of sources>
agent.channels=<list of channels>
agent.sinks=<list of sinks>

Each source, channel, and sink also has a unique name within the context ...

Get Apache Flume: Distributed Log Collection for Hadoop now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.