Time for action – capturing network traffic in a log file

In the first instance, let's use a simple Flume configuration that will capture the network data to the main Flume log file.

  1. Create the following file as agent1.conf within your Flume working directory:
    agent1.sources = netsource
    agent1.sinks = logsink
    agent1.channels = memorychannel
    
    agent1.sources.netsource.type = netcat
    agent1.sources.netsource.bind = localhost
    agent1.sources.netsource.port = 3000
    
    agent1.sinks.logsink.type = logger
    
    agent1.channels.memorychannel.type = memory
    agent1.channels.memorychannel.capacity = 1000
    agent1.channels.memorychannel.transactionCapacity = 100
    
    agent1.sources.netsource.channels = memorychannel
    agent1.sinks.logsink.channel = memorychannel
  2. Start a Flume agent: ...

Get Hadoop: Data Processing and Modelling now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.