EDUCBA

Home Data Science Data Science Tutorials Hadoop Tutorial Hadoop Commands

Hadoop Commands

By Priya PedamkarPriya Pedamkar

Hadoop-Commands

Introduction of Hadoop Commands

Hadoop commands are mainly used to execute several operations. HDFS operations and supervise the file available in the HDFS cluster. Hadoop HDFS is a distributed file system that provides redundant storage for large-sized files to be stored. It is used to store petabyte files in the terabyte range. HDFS is the primary or main component of this ecosystem that is responsible for storing large data sets of structured or unstructured data across various nodes and thus maintaining the metadata in the form of log files. All the commands are executed by the bin shell scripts.

HDFS Commands

Here we discussed various HDFS commands which are used for HDFS File Operations.

Start Your Free Data Science Course

Hadoop, Data Science, Statistics & others

  • version
  • mkdir
  • ls
  • put
  • copy from local
  • get
  • copyTOLocal
  • cat
  • mv
  • cp
  • move from local
  • move to local
  • tail
  • rm
  • expunge
  • chown
  • chgrp
  • setrep
  • du
  • df
  • touchz
  • append to file
  • count
  • checksum

1. version

That command is used to check the Hadoop version.

Command:

hdfs dfs version

2. mkdir

This Hadoop command is used to make new directories and takes the URI path as parameters

Command:

hdfs dfs -mkdir /usr/local/firstdir?

3. ls

This Hadoop Command is used to displays the list of the contents of a particular directory given by the user. It also contents name, permission, size and owner and last edit date.

Command:

hdfs dfs -ls /usr/local/firstdir

4. put

This Hadoop Command is used to copies the content from the local file system to the other location within DFS.

Command:  

hdfs dfs -put  source_dir   destination_dir

5. copyFromLocal

This Hadoop command is the same as put command but here one difference is here like in case this command source directory is restricted to local file reference.

Command: 

hdfs dfs -copyFromLocal  local_src  destination_dir

6. get

This Hadoop Command fetches all files that match the src dir which is entered by the user in HDFS and generates a copy of them in the local file system.

Command: 

hdfs dfs -get  source_dir  local_dir?

7. copyTOLocal

This Hadoop Command is using the same as getting command but one difference is that in this the destination is limited to a local file path.

Command:

hdfs -dfs  -copyToLocal  src_dir  local_dir

8. cat

This Hadoop Command displays the content of the file name on the console.

Command:

hdfs dfs -cat  dir_path

9. mv

This Hadoop Command moves the file and directory one location to another location within hdfs.

Command:

hdfs fs -mv source_dir_filename  destination_dir

10. cp

This Hadoop command copies the file and directory one location to other locations within hdfs.

Command:

hdfs fs -cp source_dir_filename  destination_dir

11. moveFromLocal

It copies content from the local file system to a destination within HDFS but the copy is a success then deletes content from the local file system.

Command:

hdfs dfs -move from local local_src  destination_dir

12. move to local

This Hadoop command runs as -get commands but one difference is that when the copy operation is a success then delete the file from HDFS location.

Commands

move to local source_dir  local_dir

13. tail

It displays 1 KB content on the console of the file.

Command:

hdfs dfs -tail file_path

14. rm

It removes files and directory from the specified path.

Command:

hdfs dfs -rm dir_name

15. expunge

This is used to empty the trash.

Command:

hdfs dfs -expunge

16. chown

It used to change the owner of files. We can also use it by -R for recursively.

Command:

hdfs dfs -chown  owner_name  dir_name

17. chgrp

This is used to change the group of files. We can also use it by -R for recursively.

Command:

hdfs dfs -chgrp  group_name  dir_name

18. du

This displays disk usage for all files available in the present directory with the path given by the user and prints information in bytes format.

Command:

hdfs dfs -du  dir_name

19. df

This Hadoop Command displays free space.

Command:

hdfs dfs -df -h

20. touchz

This is used to create a file with a path and includes current time as timestamp and is also the path is exiting if exits then fail to create process.

Command:

hdfs dfs -touchz dir_name

21. appendToFile

It appends one and multiple sources from the local file system to the destination.

Command:

hdfs dfs -append to file local_src….  Destination_dir_name

22. count

This is used to counts the number of directories and files.

Command:

hdfs dfs -count dir_name

23. checksum

It returns checksum information of a particular file.

Command:

hdfs dfs -checksum file_name

Recommended Articles

This is a guide to Hadoop Commands. Here we discuss the introduction, various HDFS Commands in Hadoop that are used for HDFS File Operations. You can also go through our other suggested articles to learn more –

  1. Hadoop Administrator
  2. Install Hadoop
  3. HBase Commands
  4. Hadoop Tools