Consuming logs in batch

Another less common approach is to gather logs periodically from the servers, after the logs have rolled. This is very similar to monitoring logs on a shared drive, except that the problems of scale are possibly even worse.

The advantages of this approach include:

  • A forwarder does not need to be installed on each server that is writing its logs to the share

The disadvantages of this approach include:

  • When new logs are dropped, if the files are large, the Splunk process will only read events from one file at a time.
  • When this directory is on an indexer, this is fine, but when a forwarder is trying to distribute events across multiple indexers, only one indexer will receive events at a time.
  • The oldest events in the ...

Get Implementing Splunk 7 - Third Edition now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.