O'Reilly logo

Learning Storm by Anand Nalya, Ankit Jain

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Splitting the server log line

Now, we will create a new Storm topology that will read the data from Kafka using the KafkaSpout spout, process the server logfiles, and store the process data in MySQL for further analysis.

In this section, we will write a bolt, ApacheLogSplitterBolt, which has logic to fetch the IP address, status code, referrer, bytes sent, and other such information from the server log line. We will create a new Maven project for this use case:

  1. Create a new Maven project with com.learningstorm for groupId and stormlogprocessing for artifactId.
  2. Add the following dependencies to the pom.xml file:
        <!-- Dependency for Storm -->
        <dependency>
          <groupId>storm</groupId>
          <artifactId>storm-core</artifactId>
          <version>0.9.0.1</version>

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required