Yarn Hadoop Architecture
YARN stands for “Yet Another Resource Negotiator” . In Hadoop 2.0, The concept of Application Master and Resource Manager was introduced by YARN. Across the cluster of Hadoop, the utilization of resources is monitored by the Resource Manager. YARN is a software rewrite that is capable of decoupling Map Reduce ’s resource management and scheduling capabilities from the data processing component.
Components of YARN
- Container
- Application Master
- Node Manager
- Resource Manager
Resource manager connects with the clients to accept tasks and responsible for allocating the resources to node manager.
Resource Manager is broadly having two machinery i.e Application Manager (for resource allocation)and Scheduler(for scheduling tasks).
Elements like Container, Application Master and Node Manager are present within a data node.
Each node manager will be responsible for the working of a particular machine assigned to them, helps in monitoring the status and updating it to the Resource Manager.
Container is a virtualization of resources managed by application master , Docker is one of technology that works on Container. It uses information from the base machine and run the software.
Application master asks for more resources and work with containers in an efficient way.
Working
So, firstly job is submitted by the client to the resource manager Resource Manager and Application Manager register with each other.
The Application Manager does the negotiation of the Container to the Resource Manager.The Node Manager launches the Container after being notified by the Application Manager.Execution of Application code is done in the Container.
The Application Manager or Resource Manager monitors the status of the application after being contacted by the client. Un-Registration of Application Manager is done with Resource Manager after the process is complete.
