Flume agent configuration is stored in a local text file. Please refer to the sample Flume agent configuration file in the code repository of this book. Flume 1.7.0 supports various sources and sinks. Widely used Flume sources (a summary) are as follows:
Source | Description |
Avro source | Listens on Avro port and receives events from external Avro client streams |
Exec source | Runs a given Unix command and expects that process to continuously produce data on standard out |
Spooling directory source | Ingests data from files on disk |
Taildir source | Tails files in near real-time after new lines are detected in the files |
Kafka source | Reads messages from Kafka topics |
Syslog source | Reads syslog data (supports syslog-TCP and syslog-UDP) ... |