In Expert Hadoop® Administration, leading Hadoop administrator Sam R. Alapati brings together authoritative knowledge for creating, configuring, securing, managing, and optimizing production Hadoop clusters in any environment. Drawing on his experience with large-scale Hadoop administration, Alapati integrates action-oriented advice with carefully researched explanations of both problems and solutions. He covers an unmatched range of topics and offers an unparalleled collection of realistic examples.

Alapati demystifies complex Hadoop environments, helping you understand exactly what happens behind the scenes when you administer your cluster. You’ll gain unprecedented insight as you walk through building clusters from scratch and configuring high availability, performance, security, encryption, and other key attributes. The high-value administration skills you learn here will be indispensable no matter what Hadoop distribution you use or what Hadoop applications you run.

Table of Contents

I: Introduction to Hadoop- Architecture and Hadoop Clusters

1 Introduction to Hadoop and Its Environment

2 An Introduction to the Architecture of Hadoop

3 Creating and Configuring a Simple Hadoop Cluster

4 Planning for and Creating a Fully Distributed Cluster

II: Hadoop Application Frameworks

5 Running Applications in a Cluster-The MapReduce Framework (and Hive and Pig)

6 Running Applications in a Cluster-The Spark Framework

7 Running Spark Applications

Ill: Managing and Protecting Hadoop Data and High Availability

8 The Role of the NameNode and How HDFS Works

9 HDFS Commands, HDFS Permissions and HDFS Storage

10 Data Protection, File Formats and Accessing HDFS

11 NameNode Operations, High Availability and Federation

IV: Moving Data, Allocating Resources, Scheduling Jobs and Security

12 Moving Data Into and Out of Hadoop

13 Resource Allocation in a Hadoop Cluster

14 Working with Oozie to Manage Job Workflows

15 Securing Hadoop

V: Monitoring, Optimization and Troubleshooting

16 Managing Jobs, Using Hue and Performing Routine Tasks

17 Monitoring, Metrics and Hadoop Logging

18 Tuning the Cluster Resources, Optimizing MapReduce Jobs and Benchmarking

19 Configuring and Tuning Apache Spark on YARN

20 Optimizing Spark Applications

21 Troubleshooting Hadoop- A Sampler

22 Installing VirtualBox and Linux and Cloning the Virtual Machines

About the Author

Sam Alapati is an Oracle Database, Oracle Exadata, Oracle WebLogic Server, Oracle security, UNIX System Administration, and Hadoop expert with over 20 years of experience managing enterprise systems as a senior Oracle DBA and architect. In 2011, Sam received the prestigious Oracle ACE recognition from Oracle Corporation for significant technological contributions to Oracle database technologies.