Hive Tutorial

Hive Tutorial and Resources

Hive tutorial is a stepping stone in becoming an expert in querying, summarizing, and analyzing billions or trillions of records with the use of industry-wide popular HiveQL on the Hadoop distributed file system. This tutorial familiarizes you with the features and scope of the language for better query optimization and processing. With SQL-like dialect, queries can be written using simple DDL and DML commands to specify or alter the database, table, or views and perform operations on them. This will focus on the various types of queries that can be executed on the Hive, along with the execution plan for MapReduce jobs at the back end.

Why do we need to learn Hive?

1. As a data analyst, it is important to churn data (clean/unclean) and derive actionable insights from them. Using different file formats like Textfile, Sequencefile, Avro, Parquet, or ORC (Optimised Row Columnar), a variety of data can be processed efficiently.
2. Hive is a high-level language that makes summarising data faster and supports user defined functions for manipulating strings, integers or dates. This SQL abstraction prevents us from writing complex MapReduce jobs.
3. Ad-hoc querying is easy, and data from external tables can be operated on without storing data in HDFS.
4. Hadoop distributed the File system (HDFS), which manages how data is stored across clusters. Also, the MapReduce computation model helps in breaking jobs into tasks for parallel processing across servers or clusters.

Application of Hive

1. Being an open-source data warehousing system, Hive finds applications in Big data analysis and data summarization.
2. Hadoop developers are also using Apache Hive for solving complex analytical problems with Hadoop packages such as RHive, and RHipe. Even Apache Mahout supports Hive queries.
3. Concepts of Partitioning and bucketing enable data to be stored in logical parts or segments, making query response time faster.

Hive also supports a number of data science applications like :

• Document Indexing
• Text Mining
• Google Analytics
• Sentiment Analysis
• Predictive Modelling
• Log Processing
• Hypothesis testing

Pre-requisites

In order to learn HiveQL, basic knowledge of SQL, Hadoop architecture, and Unix/Linux shell scripting commands will be helpful. Understanding the logical approach to a problem enables building queries and ETL jobs.

Target Audience

HiveQL tutorial is targeted to cater to the petabytes of data analysis by Big data professionals/engineers and analysts in the field of Banking, Retail, Insurance, and many more. This tutorial will help Hadoop developers in automating ETL jobs to summarize large data sets on the Hadoop ecosystem. Database architects and administrators also have many concepts to learn from this comprehensive tutorial.