Master Apache Hadoop Training (Old Version)

This is a online course is to gain comprehensive understanding of Apache Hadoop. The aim is to learn essential concepts in Apache Hadoop right from scratch. The tutorials will help you learn about Introduction to Big data and hadoop, Fundamentals of Hadoop, Programming Model, Code Examples, HDFS, Developing map-reduce application, Introduction to HIVE, PIG and Hadoop Setup.

287

 5h 4m | 29 Videos | 89591 Views |   All Levels

About Master Apache Hadoop

Apache Hadoop is an open source software for storing and processing of large data sets with the help of group of commodity hardware. Apache Hadoop was created in the year 2005 and now it is called as Hadoop. Hadoop will help to analyse both the structured and unstructured data in a fast and reliable manner. The Hadoop consists of five modules Hadoop Common, Hadoop distributed File System, Hadoop YARN and Hadoop MapReduce. Hadoop modules are designed with an assumption that the hardware failures are very common and these problems should be solved automatically by the software. The Hadoop framework is mostly written in Java but some of the codes are in C and Shell Scripts

Benefits of Apache Hadoop

• This is a highly scalable software as it can store large data sets
• It prevents the network overload
• The tasks in Hadoop are independent so it is easy to handle partial failure
• Hadoop has a simple programming model
• Hadoop’s Distributed File System stores large amount of information
• Quick recovery is possible in Hadoop if there is any faults
• Hadoop can be used for a wide range of purposes so it is flexible
• Hadoop can save all the company’s data for later use which saves the cost
• Hadoop processes terabytes of data in just few seconds

Apache Hadoop Training Objectives

At the end of this course you will be able to

• Learn the basics of Hadoop
• Understand the Hadoop distributed File System including its architecture
• Learn how to import and export data in Hadoop
• Know the tools and functions which are needed to work with this software

Pre requisites for taking this course

This course is specially designed for the beginners therefore no prior experience in Hadoop is required for the participants. If you have basic java knowledge then it will be an added advantage for this course. You just need to have a computer with a good internet connection to take up this course.

Target Audience for this course

Students and professionals who wanted to make a career in Big Data Analytics using Hadoop can take up this course. Software developer and IT managers can also go for this course to improve the scope of their career.

Apache Hadoop Training Description

Section 1: Introduction to Big Data and Hadoop

Big data is a collection of large data which is difficult to be processed with the help of traditional computing techniques. Big data involves various tools and techniques. Big data involves the data from various fields like Black box data, social media data, power grid data, transport data and search engine data. Big data includes three types of data – structured data, Semi structured data and Unstructured data. This section contains a brief introduction to Big Data, its advantages, technologies and the challenges faced by Big Data.

Hadoop is an Apache open source framework which is written mainly using Java language and helps to process large datasets across group of computers. Hadoop is not a single entity. It consists of different open source products. Hadoop framework consists of four modules – Hadoop Common, Hadoop YARN, Hadoop Distributed File System and Hadoop MapReduce. This section contains a brief introduction of all the Hadoop modules along with a pictorial representation of the same.

Section 2: Fundamentals of Hadoop

Metrics of Hadoop

Metrics consists of statistical information which is used for monitoring and debugging. There are a wide variety of metrics in Hadoop and these metrics helps to troubleshoot the problems. The topics covered under this section includes the following

• JVM metrics
• rpc context
• rpcdetailed context
• dfs context
• yarn context
• Cluster metrics
• Queue Metrics
• Node manager Metrics
• Metricssystem
• Ugi metrics

Architecture of Hadoop

Hadoop consists of five main elements – Cluster, YARN infrastructure, HDFS Federation, Storage and MapReduce Framework. This section contains in detail the HDFS architecture and the YARN architecture. The topics covered under HDFS architecture includes the following

• Assumptions and goals
• Name nodes and Data nodes
• File System Namespace
• Data replication
• Persistence of File System Metadata
• The communication Protocols
• Robustness
• Data Organization
• Accessibility
• Space Reclamation

Section 3: Programming model

Map Reduce Programming Model Part 1

MapReduce is a framework in Hadoop which is used to write applications for processing large datasets. MapReduce is a program model used for distributed computing which is based on Java language. This framework consists of two main algorithms Map and Reduce. Under this chapter you will learn in detail about Map and Reduce. The topics covered under this section includes the following

• The algorithm
• Inputs and Outputs
• MapReduce – User Interfaces
• Terminology
• Example Scenario
• Example Program
• Compilation and Execution of Process Units Program

Detailed analysis with example

This section gives you an Introduction to reporting and analysis on Hadoop and it includes Indirect batch analysis on Hadoop. There are different type of analysis that can be done with Hadoop. This includes the following

• Analyzing machine and sensor data
• Analyzing big data
• Analyzing twitter data
• Analyzing social media data

Section 4: Code Examples

Code Example in Java word count Part 1

The word count example reads text files and it counts the number of times the words occur. The input and output of word count is text files. This section explains the Word count example Part 1 that is creating your own jar in Hadoop implemented in Java in detail. There are few screenshots given for your easy understanding.

Code Example in Java word count Part 2

This section contains the Word count example Part 2 in Java which is running a Jar on Hadoop with examples.

Code Example in Java group Part 1

This chapter lets you learn how to write MapReduce program in Java along with examples. This contains the following topics

• Input and Output
• Environment setup
• Mapper program
• Running the MapReduce Program

Data Flow Analysis of Map Reduce Part 1

In this chapter an overview of MapReduce is given and the different type of algorithms are explained in detail. The topics covered under this section includes

• M/R algorithm
• MK Means Algorithm
• Large Scale Data Analysis framework
• Scalability and Parallelisation
• Parallelisation approaches
• Data analysis example using MapReduce

Section 5: HDFS

HDFS is designed to save huge amount of data sets and stream data sets at high bandwidth. The architecture of HDFS is explained in detail in this section. The other topics included in this section are

• Introduction
• HDFS Architecture
• Edits viewer
• Image Viewer
• Quotas and HDFS
• HDFS snapshots
• File I/O operations and Replica management

Section 6: Developing Map reduce application

MapRed

We have already seen introduction of the MapReduce and in this chapter we will learn the practical aspects of developing a MapReduce application in Hadoop. The programs written in MapReduce follows a particular pattern. The other topics included in this chapter are listed below

• Introduction to Developing Hadoop Application
• Writing a MapReduce Program
• Using the MapReduce API
• MapReduce Workflow
• MapReduce Job performance
• Different data sources in MapReduce
• Managing multiple MapReduce Jobs
• Using MapReduce streaming

Section 7: Introduction to Hive

Hive is a data warehouse tool which is used to process the structured data type in Hadoop. Hive is a platform which is used to develop SQL scripts in order to reduce MapReduce operations. Hive summarizes Big data and makes the querying and analyzing part easy in Hadoop. Here we will see the following details of Hive

• Features of Hive
• Architecture of Hive
• Units of Hive with its description
• Working of Hive in Hadoop
• Data types in Hive
• Database creation in Hive
• Creating table in Hive
• Built in operators
• Built in Functions
• Views and Indexes in Hive
• Hive Query Language – Select Where, Select Order BY, Select Group By and Select Joins. Syntax, example, a program and output for each Hive QL is given in this section.

Section 8: Introduction to Pig

Pig is a programming language which is used to analyze large data sets. It was initially developed by Yahoo. Pig programming deals with all kinds of data and focuses more on analyzing bulk data sets.

This section begins with an overview of Pig. It also deals with the data structures and relational operators of Pig. Pig consists of two main components – Pig Latin and Runtime environment. These two components are explained in detail in this section. The two execution modes of PIG is also dealt in this section

Section 9: Hadoop Setup

Hadoop can be installed and run in GNU/Linux. Therefore you should have Linux OS before you set up Hadoop. The Hadoop set up includes the following procedures

• Creating a User
• SSH setup and Key generation
• Installing Java with step by step procedure
• Downloading Hadoop
• Select the Hadoop operation modes from one among the three – Local/Standalone mode, Pseudo Distributed mode and Fully distributed mode.
• Installing and setting up Hadoop in Standalone mode with example
• Installing and setting up Hadoop in Pseudo Distributed mode with example
• Verifying Hadoop Installation

This section also describes how to set up and configure a single node cluster Hadoop to perform operations quickly. The This section contains the following topics under it

• Pre requisites – Supported platforms, required software, Installing software
• Download
• Preparation to start the Hadoop cluster

FAQs on Apache Hadoop Training

• What are the career benefits of this course ?

A good knowledge of the basics of Big Data and Hadoop will make it easier to improve your analytic skills and in turn increases your career prospects in the analytics industry. Top companies like Microsoft, IBM, Oracle, HP and SAP have invested a huge amount on data management and analytics. So there is an increased number of opportunities for Big data & Hadoop certified professionals.

• Why this course is considered more important ?

Big data is used to analyze huge amount of data. There is a huge demand for professionals who have a good understanding about the Big data and Hadoop. In this course you will learn all the basics of Big data and Hadoop. This will make it easy for you to take up any advanced level courses in Big data and Hadoop. These courses will make you an expert in this field. Some of the famous companies who use Hadoop technology are Facebook, LinkedIn and Yahoo Search Webmap.

• In case of any doubts whom do I contact ?

We have a customer support team available 24*7 for providing assistance to you. You can chat with us or call us or request a call back to get your query solved.

Testimonials

Nikita Das

This course is an ideal one for individuals who wanted to learn the basics of Big data and Hadoop. The course offers an excellent overview on the components of Hadoop and its commercial distributions. The course covers all the important topics and it is simple and easy to understand. Very comprehensive course.

Thomas

This course is a good introductory course on Hadoop and Big data. The course structure and the flow of the topics are connected in a way to make us easily follow the topic. It not only gives information about the tools but also explains neatly how each has to be used. I felt very contented with this course. I would definitely recommend this course for beginners as well as for professionals. You would love it.

Rathod

This Apache Hadoop Training is very informative. It gives a detailed explanation of all the topics. I learnt a lot from this course about Big data and Hadoop which helped me to improve my skills and move one step ahead in my career. The contents of the course are self explanatory and makes you to easily learn without the help of others.