Updated March 23, 2023
Introduction of Hadoop Commands
Hadoop commands are mainly used to execute several operations. HDFS operations and supervise the file available in the HDFS cluster. Hadoop HDFS is a distributed file system that provides redundant storage for large-sized files to be stored. It is used to store petabyte files in the terabyte range. HDFS is the primary or main component of this ecosystem that is responsible for storing large data sets of structured or unstructured data across various nodes and thus maintaining the metadata in the form of log files. All the commands are executed by the bin shell scripts.
Here we discussed various HDFS commands which are used for HDFS File Operations.
- copy from local
- move from local
- move to local
- append to file
That command is used to check the Hadoop version.
hdfs dfs version
This Hadoop command is used to make new directories and takes the URI path as parameters
hdfs dfs -mkdir /usr/local/firstdir?
This Hadoop Command is used to displays the list of the contents of a particular directory given by the user. It also contents name, permission, size and owner and last edit date.
hdfs dfs -ls /usr/local/firstdir
This Hadoop Command is used to copies the content from the local file system to the other location within DFS.
hdfs dfs -put source_dir destination_dir
This Hadoop command is the same as put command but here one difference is here like in case this command source directory is restricted to local file reference.
hdfs dfs -copyFromLocal local_src destination_dir
This Hadoop Command fetches all files that match the src dir which is entered by the user in HDFS and generates a copy of them in the local file system.
hdfs dfs -get source_dir local_dir?
This Hadoop Command is using the same as getting command but one difference is that in this the destination is limited to a local file path.
hdfs -dfs -copyToLocal src_dir local_dir
This Hadoop Command displays the content of the file name on the console.
hdfs dfs -cat dir_path
This Hadoop Command moves the file and directory one location to another location within hdfs.
hdfs fs -mv source_dir_filename destination_dir
This Hadoop command copies the file and directory one location to other locations within hdfs.
hdfs fs -cp source_dir_filename destination_dir
It copies content from the local file system to a destination within HDFS but the copy is a success then deletes content from the local file system.
hdfs dfs -move from local local_src destination_dir
12. move to local
This Hadoop command runs as -get commands but one difference is that when the copy operation is a success then delete the file from HDFS location.
move to local source_dir local_dir
It displays 1 KB content on the console of the file.
hdfs dfs -tail file_path
It removes files and directory from the specified path.
hdfs dfs -rm dir_name
This is used to empty the trash.
hdfs dfs -expunge
It used to change the owner of files. We can also use it by -R for recursively.
hdfs dfs -chown owner_name dir_name
This is used to change the group of files. We can also use it by -R for recursively.
hdfs dfs -chgrp group_name dir_name
This displays disk usage for all files available in the present directory with the path given by the user and prints information in bytes format.
hdfs dfs -du dir_name
This Hadoop Command displays free space.
hdfs dfs -df -h
This is used to create a file with a path and includes current time as timestamp and is also the path is exiting if exits then fail to create process.
hdfs dfs -touchz dir_name
It appends one and multiple sources from the local file system to the destination.
hdfs dfs -append to file local_src…. Destination_dir_name
This is used to counts the number of directories and files.
hdfs dfs -count dir_name
It returns checksum information of a particular file.
hdfs dfs -checksum file_name
This is a guide to Hadoop Commands. Here we discuss the introduction, various HDFS Commands in Hadoop that are used for HDFS File Operations. You can also go through our other suggested articles to learn more –