Filtering and processing input
Once we configure the input file, we need to filter the input based on our needs so that we can identify which fields we need, and process them as per the required analysis.
A
filter
plugin will perform the intermediary processing on the input event. We can apply the filter conditionally based on certain fields.
Since our input file is a CSV file, we will use the csv
filter for the same. The csv
filter takes an event field that contains CSV formatted data, parses it, and stores it as individual fields. It can also parse data with any separator other than commas. A typical csv
filter is as follows:
filter { csv { columns => #Array of column names. separator => #String ; default -"," } }
The attribute
columns take the ...
Get Learning ELK Stack now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.