Updated March 23, 2023
Introduction of Hadoop Commands
Hadoop commands are mainly used to execute several operations. HDFS operations and supervise the file available in the HDFS cluster. Hadoop HDFS is a distributed file system that provides redundant storage for large-sized files to be stored. It is used to store petabyte files in the terabyte range. HDFS is the primary or main component of this ecosystem that is responsible for storing large data sets of structured or unstructured data across various nodes and thus maintaining the metadata in the form of log files. All the commands are executed by the bin shell scripts.
HDFS Commands
Here we discussed various HDFS commands which are used for HDFS File Operations.
- version
- mkdir
- ls
- put
- copy from local
- get
- copyTOLocal
- cat
- mv
- cp
- move from local
- move to local
- tail
- rm
- expunge
- chown
- chgrp
- setrep
- du
- df
- touchz
- append to file
- count
- checksum
1. version
That command is used to check the Hadoop version.
Command:
hdfs dfs version
2. mkdir
This Hadoop command is used to make new directories and takes the URI path as parameters
Command:
hdfs dfs -mkdir /usr/local/firstdir?
3. ls
This Hadoop Command is used to displays the list of the contents of a particular directory given by the user. It also contents name, permission, size and owner and last edit date.
Command:
hdfs dfs -ls /usr/local/firstdir
4. put
This Hadoop Command is used to copies the content from the local file system to the other location within DFS.
Command:
hdfs dfs -put source_dir destination_dir
5. copyFromLocal
This Hadoop command is the same as put command but here one difference is here like in case this command source directory is restricted to local file reference.
Command:
hdfs dfs -copyFromLocal local_src destination_dir
6. get
This Hadoop Command fetches all files that match the src dir which is entered by the user in HDFS and generates a copy of them in the local file system.
Command:
hdfs dfs -get source_dir local_dir?
7. copyTOLocal
This Hadoop Command is using the same as getting command but one difference is that in this the destination is limited to a local file path.
Command:
hdfs -dfs -copyToLocal src_dir local_dir
8. cat
This Hadoop Command displays the content of the file name on the console.
Command:
hdfs dfs -cat dir_path
9. mv
This Hadoop Command moves the file and directory one location to another location within hdfs.
Command:
hdfs fs -mv source_dir_filename destination_dir
10. cp
This Hadoop command copies the file and directory one location to other locations within hdfs.
Command:
hdfs fs -cp source_dir_filename destination_dir
11. moveFromLocal
It copies content from the local file system to a destination within HDFS but the copy is a success then deletes content from the local file system.
Command:
hdfs dfs -move from local local_src destination_dir
12. move to local
This Hadoop command runs as -get commands but one difference is that when the copy operation is a success then delete the file from HDFS location.
Commands
move to local source_dir local_dir
13. tail
It displays 1 KB content on the console of the file.
Command:
hdfs dfs -tail file_path
14. rm
It removes files and directory from the specified path.
Command:
hdfs dfs -rm dir_name
15. expunge
This is used to empty the trash.
Command:
hdfs dfs -expunge
16. chown
It used to change the owner of files. We can also use it by -R for recursively.
Command:
hdfs dfs -chown owner_name dir_name
17. chgrp
This is used to change the group of files. We can also use it by -R for recursively.
Command:
hdfs dfs -chgrp group_name dir_name
18. du
This displays disk usage for all files available in the present directory with the path given by the user and prints information in bytes format.
Command:
hdfs dfs -du dir_name
19. df
This Hadoop Command displays free space.
Command:
hdfs dfs -df -h
20. touchz
This is used to create a file with a path and includes current time as timestamp and is also the path is exiting if exits then fail to create process.
Command:
hdfs dfs -touchz dir_name
21. appendToFile
It appends one and multiple sources from the local file system to the destination.
Command:
hdfs dfs -append to file local_src…. Destination_dir_name
22. count
This is used to counts the number of directories and files.
Command:
hdfs dfs -count dir_name
23. checksum
It returns checksum information of a particular file.
Command:
hdfs dfs -checksum file_name
Recommended Articles
This is a guide to Hadoop Commands. Here we discuss the introduction, various HDFS Commands in Hadoop that are used for HDFS File Operations. You can also go through our other suggested articles to learn more –
