Recent developments in YARN architecture
The ResourceManager is a single point of failure and restart because of various reasons: bugs, hardware failure, deliberate downtime for upgrading, and so on.
We already saw how crucial the role of the ResourceManager in YARN architecture is. The ResourceManager has become a single point of failure; if the ResourceManager in a cluster goes down, everything on that cluster will be lost.
So in a recent development of YARN, ResourceManager HA became a high priority. This recent development of YARN not only covers ResourceManager HA, but also provides transparency to users and does not require them to monitor such events explicitly and resubmit the jobs.
Overly complex in MRv1 for the fact that JobTracker has ...
Get YARN Essentials now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
