O'Reilly logo

Learning Storm by Anand Nalya, Ankit Jain

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Integration of Storm with Hadoop

The probability that the organizations developing and operating Big Data applications already have a Hadoop cluster deployed is very high. Also, there is a high possibility that they also have real-time stream processing applications deployed to go along with the batch applications running on Hadoop.

It would be great if we can leverage the already deployed YARN cluster to also run Storm topologies. This will reduce the operational cost of maintenance by giving us only one cluster to manage instead of two.

Storm-YARN is a project developed by Yahoo! that enables the deployment of Storm topologies over YARN clusters. It enables the deployment of Storm processes on nodes managed by YARN.

The following diagram illustrates ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required