Hive Tutorial

Hive tutorial is a stepping stone in becoming an expert in querying, summarizing, and analyzing billions or trillions of records with the use of industry-wide popular HiveQL on the Hadoop distributed ﬁle system. This tutorial familiarizes you with the features and scope of the language for better query optimization and processing. With SQL-like dialect, queries can be written using simple DDL, and DML commands to specify or alter the database, table, or views and perform operations on them. This will focus on the various types of queries that can be executed on the Hive, along with the execution plan for MapReduce jobs at the back end.

Why do we need to learn Hive?

As a data analyst, it is important to churn data (clean/unclean) and derive actionable insights from them. Using different file formats like Textfile, Sequencefile, Avro, Parquet, or ORC (Optimised Row Columnar), a variety of data can be processed efficiently.
Hive is a high-level language that summarises data faster and supports user deﬁned functions for manipulating strings, integers, or dates. This SQL abstraction prevents us from writing complex MapReduce jobs.
Ad-hoc querying is easy, and data from external tables can be operated without storing data in HDFS.
Hadoop distributed the File system (HDFS), which manages how data is stored across clusters. Also, the MapReduce computation model helps break jobs into tasks for parallel processing across servers or clusters.

Application of Hive

Being an open-source data warehousing system, Hive finds applications in Big data analysis and data summarization.
Hadoop developers are also using Apache Hive for solving complex analytical problems with Hadoop packages such as RHive, and RHipe. Even Apache Mahout supports Hive queries.
Concepts of Partitioning and bucketing enable data to be stored in logical parts or segments, making query response time faster.

Hive also supports a number of data science applications:

Document Indexing
Text Mining
Google Analytics
Sentiment Analysis
Predictive Modelling
Log Processing
Hypothesis testing

Pre-requisites

In order to learn HiveQL, basic knowledge of SQL, Hadoop architecture, and Unix/Linux shell scripting commands will be helpful. Understanding the logical approach to a problem enables building queries and ETL jobs.

Target Audience

HiveQL tutorial is targeted to cater to the petabytes of data analysis by Big data professionals/engineers and analysts in the ﬁeld of Banking, Retail, Insurance, and many more. This tutorial will help Hadoop developers automate ETL jobs to summarize large data sets on the Hadoop ecosystem. Database architects and administrators also have many concepts to learn from this comprehensive tutorial.