YARN
YARN started out as part of the MapReduce v2 (MRv2) initiative but is now an independent sub-project within Hadoop (that is, it's at the same level as MapReduce). It grew out of a realization that MapReduce in Hadoop 1 conflated two related but distinct responsibilities: resource management and application execution.
Although it has enabled previously unimagined processing on enormous datasets, the MapReduce model at a conceptual level has an impact on performance and scalability. Implicit in the MapReduce model is that any application can only be composed of a series of largely linear MapReduce jobs, each of which follows a model of one or more maps followed by one or more reduces. This model is a great fit for some applications, but not ...
Get Learning Hadoop 2 now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
