A fully redundant Hadoop cluster

This cluster runs the exact set of applications as the primary cluster and the data is copied periodically from the main Hadoop cluster. Since this is a one-way copy from the main cluster to the second cluster, we can be 100% sure that the main cluster isn't impacted when we make any changes to this fully redundant cluster.

One important thing to understand is that we are running all other instances of applications in this cluster. Since every application maintains its state in its own predefined location, the application states are not replicated from the main Hadoop cluster to this cluster, which means that the jobs that were created in the main Hadoop cluster are not visible in this cluster. The same applies ...

Get Modern Big Data Processing with Hadoop now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.