YARN Essentials : A Comprehensive, Hands-On Guide to Install, Administer, and Configure Settings in YARN.

Intro -- YARN Essentials -- Table of Contents -- YARN Essentials -- Credits -- About the Authors -- About the Reviewers -- www.PacktPub.com -- Support files, eBooks, discount offers, and more -- Why subscribe? -- Free access for Packt account holders -- Preface -- What this book covers -- What you need for this book -- Who this book is for -- Conventions -- Reader feedback -- Customer support -- Downloading the example code -- Errata -- Piracy -- Questions -- 1. Need for YARN -- The redesign idea -- Limitations of the classical MapReduce or Hadoop 1.x -- YARN as the modern operating system of Hadoop -- What are the design goals for YARN -- Summary -- 2. YARN Architecture -- Core components of YARN architecture -- ResourceManager -- ApplicationMaster (AM) -- NodeManager (NM) -- YARN scheduler policies -- The FIFO (First In First Out) scheduler -- The fair scheduler -- The capacity scheduler -- Recent developments in YARN architecture -- Summary -- 3. YARN Installation -- Single-node installation -- Prerequisites -- Platform -- Software -- Starting with the installation -- The standalone mode (local mode) -- The pseudo-distributed mode -- The fully-distributed mode -- HistoryServer -- Slave files -- Operating Hadoop and YARN clusters -- Starting Hadoop and YARN clusters -- Stopping Hadoop and YARN clusters -- Web interfaces of the Ecosystem -- Summary -- 4. YARN and Hadoop Ecosystems -- The Hadoop 2 release -- A short introduction to Hadoop 1.x and MRv1 -- MRv1 versus MRv2 -- Understanding where YARN fits into Hadoop -- Old and new MapReduce APIs -- Backward compatibility of MRv2 APIs -- Binary compatibility of org.apache.hadoop.mapred APIs -- Source compatibility of org.apache.hadoop.mapred APIs -- Practical examples of MRv1 and MRv2 -- Preparing the input file(s) -- Running the job -- Result -- Summary -- 5. YARN Administration.

Container allocation -- Container allocation to the application -- Container configurations -- YARN scheduling policies -- The FIFO (First In First Out) scheduler -- The FIFO (First In First Out) scheduler -- The capacity scheduler -- Capacity scheduler configurations -- The fair scheduler -- Fair scheduler configurations -- YARN multitenancy application support -- Administration of YARN -- Administrative tools -- Adding and removing nodes from a YARN cluster -- Administrating YARN jobs -- MapReduce job configurations -- YARN log management -- YARN web user interface -- Summary -- 6. Developing and Running a Simple YARN Application -- Running sample examples on YARN -- Running a sample Pi example -- Monitoring YARN applications with web GUI -- YARN's MapReduce support -- The MapReduce ApplicationMaster -- Example YARN MapReduce settings -- YARN's compatibility with MapReduce applications -- Developing YARN applications -- The YARN application workflow -- Writing the YARN client -- Writing the YARN ApplicationMaster -- Responsibilities of the ApplicationMaster -- Summary -- 7. YARN Frameworks -- Apache Samza -- Writing a Kafka producer -- Writing the hello-samza project -- Starting a grid -- Storm-YARN -- Prerequisites -- Hadoop YARN should be installed -- Apache ZooKeeper should be installed -- Setting up Storm-YARN -- Getting the storm.yaml configuration of the launched Storm cluster -- Building and running Storm-Starter examples -- Apache Spark -- Why run on YARN? -- Apache Tez -- Apache Giraph -- HOYA (HBase on YARN) -- KOYA (Kafka on YARN) -- Summary -- 8. Failures in YARN -- ResourceManager failures -- ApplicationMaster failures -- NodeManager failures -- Container failures -- Hardware Failures -- Summary -- 9. YARN - Alternative Solutions -- Mesos -- Omega -- Corona -- Summary -- 10. YARN - Future and Support.

What YARN means to the big data industry -- Journey - present and future -- Present on-going features -- Future features -- YARN-supported frameworks -- Summary -- Index.

Description based on publisher supplied metadata and other sources.

Electronic reproduction. Ann Arbor, Michigan : ProQuest Ebook Central, 2024. Available via World Wide Web. Access may be limited to ProQuest Ebook Central affiliated libraries.