Introduction to Apache PIG interview questions and answers
So you have finally found your dream job in Apache PIG but we are wondering how to crack the 2019 Apache PIG interview and what could be the probable Apache PIG interview questions. Every Apache PIG interview is different and the scope of a job is different too. Keeping this in mind we have designed the most common Apache PIG interview questions and answers to help you get success in your Apache PIG interview.
The following is the list of 2019 Apache PIG Interview questions that are asked mostly asked
1. What are the key differences between MapReduce and Apache Pig?
Answer:
Following are the key differences between Apache Pig and MapReduce due to which Apache Pig came into the picture:
• MapReduce is a low-level data processing model whereas Apache Pig is a high-level data flow platform
• Without writing the complex Java implementations in MapReduce, programmers can achieve the same implementations easily using Pig Latin.
• Apache Pig provides nested data types like bags, tuples, and maps as they are missing from MapReduce.
• Pig support the data operations like filters, joins, ordering, sorting etc. with many built-in operators. Whereas to perform the same function in MapReduce is an immense task.
2. Explain the uses of MapReduce in Pig.
Answer:
Apache Pig programs are written in a query language known as Pig Latin which is similar to the SQL query language. To execute a query, there is a need for an execution engine. And the Pig engine converts the queries into MapReduce jobs and thus MapReduce acts as the execution engine and is needed to run the programs.
3. Explain the uses of Pig.
Answer:
We can use the Pig in three categories, they are:
• ETL data pipeline: It helps to populate our data warehouse. Pig can pipeline the data to an external application, it will wait until it’s finished so that it has received the processed data and continue from there. It is the most common use case for Pig.
• Research on raw data.
• Iterative processing.
4. Compare Apache Pig and SQL.
Answer:
• Apache Pig differs from SQL in its usage for ETL, lazy evaluation, store data at any given point in time in the pipeline, support for pipeline splits and explicit declaration of execution plans. SQL (Structural query language) is oriented around queries which produce a single result. SQL has no inbuilt mechanism for splitting the data processing stream and applying different operators to each sub-stream.
• Apache Pig allows user code to be included at any point in the pipeline whereas if SQL, where to be used data, needs to be imported to the database first and then the process of cleaning and transformation begins.
4.5 (1,856 ratings)
View Course
5. Explain about the different complex data types in Pig.
Answer:
Apache Pig supports three complex data types-
• Maps- These are key, value stores joined together using #.
Example: [‘city’#’pune’,’pin’#411045]
• Tuples- Just similar to the row in a table, where different items are separated by a comma. Tuples can have multiple attributes.
• Bags- An unordered collection of tuples. The Bag allows multiple duplicate tuples.
Example: {(‘Mumbai’,022),(‘New Delhi’,011),(‘Kolkata’,44)}
6. Explain different Execution models available in Pig.
Answer:
Three different execution modes available in Pig they are,
• Interactive mode or Grunt mode.
Interactive mode or grunt mode: Pig’s interactive shell is known as grunt shell. If no file is specified to run in Pig it will start.
• Batch mode or Script mode.
Pig executes the specified commands in the script file.
• Embedded mode
We can embed Pig programs in Java and we can run the programs from Java.
7. Explain about the execution plans (Logical & Physical plan) of a Pig Script
Answer:
Logical and Physical plans are created during the execution of a pig script. Pig scripts are based on interpreter checking. The Logical plan is produced by semantic checking and basic parsing and no data processing takes place during the creation of a logical plan. For each line in the Pig script, the syntax check is performed for operators and a logical plan is created. Whenever an error is encountered within the script, an exception is thrown and the program execution ends, else for each statement in the script has its own logical plan.
A logical plan contains the collection of operators in the script but does not contain the edges between the operators.
After the logical plan is generated, the script execution moves to the physical plan where there is a description of the physical operators, Apache Pig will use, to execute the Pig script. A physical plan is more or less like a series of MapReduce jobs but then the plan does not have any reference on how it will be executed in MapReduce. During the creation of a physical plan, cogroup logical operator is converted into 3 physical operators namely –Local Rearrange, Global Rearrange, and Package. Load and store functions usually get resolved in the physical plan.
8. What are the debugging tools used for Apache Pig scripts?
Answer:
Describe and explain are the important debugging utilities in Apache Pig.
• Explain utility is helpful for Hadoop developers when trying to debug error or optimize PigLatin scripts. explain can be applied to a particular alias in the script or it can be applied to the entire script in the grunt interactive shell. explain utility produces several graphs in text format which can be printed to a file.
• Describe debugging utility is helpful to developers when writing Pig scripts as it shows the schema of a relation in the script. For beginners who are trying to learn Apache Pig can use the describe utility to understand how each operator makes alterations to data. A pig script can have multiple describes.
9. What are some of the Apache Pig use cases you can think of?
Answer:
• Apache Pig big data tool is used in particular for iterative processing, research on raw data and for traditional ETL data pipelines. As Pig can operate in circumstances where the schema is not known, inconsistent or incomplete- it is widely used by researchers who want to make use of the data before it is cleaned and loaded into the data warehouse.
• To build behavior prediction models, for instance, it can be used by a website to track the response of the visitors to various types of ads, images, articles, etc.
10. Highlight the difference between group and Cogroup operators in Pig.
Answer:
Both the operators can work with one or more relations. Group and Cogroup operators are identical. Group operator collects all records with the same key. Cogroup is a combination of group and join, it is a generalization of a group instead of collecting records of one input depends on a key, it collects records of n inputs based on a key. At a time we can Cogroup up to 127 relations.
Recommended Articles
This has been a guide to the List Of Apache PIG Interview questions and answers so that the candidate can crackdown these Apache PIG Interview questions easily. This article consists of all useful Apache PIG Interview questions and answers that will help you in an interview. You may also look at the following articles to learn more –
