EDUCBA

EDUCBA

MENUMENU
  • Free Tutorials
  • Free Courses
  • Certification Courses
  • 600+ Courses All in One Bundle
  • Login

Linux uniq

By Priya PedamkarPriya Pedamkar

Home » Software Development » Software Development Tutorials » Linux Tutorial » Linux uniq

Linux uniq

Introduction to Linux uniq

Linux uniq command-line utility is helpful to filter out the repeated or duplicate records in the file. In other words, it will detect the adjacent duplicate lines and omit/delete the duplicate lines from the inputs data or file.

This utility was written by Richard M. Stallman and David MacKenzie.

Start Your Free Software Development Course

Web development, programming languages, Software testing & others

Syntax:

uniq [ OPTION ] ... [ INPUT [ OUTPUT ] ]

The uniq syntax is simple. As per the above syntax, the “INPUT” refers to the input data or input file. It will contain the repeated lines of duplicate data needs to be filtered out. If the “INPUT” keyword will not specify then “uniq” will read the standard input. The “OUTPUT” refers to the output file in which we can store the filtered output data generated by “uniq” command.

How does Linux uniq Command work?

The Linux uniq command is like filter program and it will use after the sort. The uniq command will get the repeated or duplicate input data or input file. With the help of different filter actions or keywords available in the uniq. We will filter out the adjacent data or duplicate data from the input file and process the end result to the output file.

Note: It will filter out the adjacent duplicate data only. If the number of duplicate records comes in serially then only it will filter out and remove the duplicate records. Consider as a single record. If the same record will come in the input file (without duplicate) but in a different location (except the previous adjacent location) then it will print in the output file. The duplicate data exactly match with each other i.e. it is case sensitive.

Examples to Implement Linux uniq Command

Examples to implement linux uniq command are given below:

1. Uniq Command

The uniq command will help to remove the duplicate data or records from the input file.

Popular Course in this category
Linux Training Program (16 Courses, 3+ Projects)16 Online Courses | 3 Hands-on Projects | 160+ Hours | Verifiable Certificate of Completion | Lifetime Access
4.5 (6,013 ratings)
Course Price

View Course

Related Courses
Kali Linux Training (3 Courses, 3+ Projects)Red Hat Linux Training Program (4 Courses, 1+ Projects)

uniq contents.txt

Explanation: We have the sample text file “contents.txt”. It is having some data in the same file (refer screenshot 1 . Now we are using the uniq command to filter out the duplicate data (refer screenshot 1 ).

Output:

Linux uniq-1.1

Linux uniq-1.2

2. Uniq Count Command

In uniq count command, we will identify how many times the line was repeated in the input file or data with prefix value. To count the number of duplicate records in input file we can use the “-c” option with uniq command.

uniq -c contents.txt

Explanation: In contents.txt file, there are a total of 8 records in it. The non-duplicate records are three. With the help of  “-c” option, we will get the number of duplicate count record information.

Output: 

Linux uniq-2.1

3. Uniq Repeated Command

It will display or print the unique records from the input file. No repeated or duplicate records will display. For repeated records, we can use the “-d” option with uniq command.

uniq -d contents.txt

Explanation: From the input file (contents.txt), we will only identify the repeated unique records.

Output:

Linux uniq-3.1

4. Uniq all-repeated

With the help of all-repeated command, we will able to get all the records from the input file. For all-repeated records, we can use the “-D” option with uniq command.

uniq -D contents.txt

Explanation: From the input file “contents.txt”, we can print all the records with the help of all-repeated functionality in uniq.

Output:

Linux uniq-4.1

5. Uniq Unique Command

The unique command will help to identify the unique records or data from the input file. For unique records, we can use the “-u” option with uniq command.

uniq -u cont.txt

Explanation: In the input file “cont.txt”, we have only one unique word in the file i.e. “thanks”. With the help of “-u” unique option, we are able to identify form the input data or file.

Output:

Linux uniq-5.1

Linux uniq-5.2

6. Uniq Using –f N Command

In this uniq command, it will allow the “N” fields to be skipped while comparing the uniqueness of the lines.

Note: In the input file, it is not mandatory the number of records or input data should start with the numbered sequence.

uniq -f 3 cont.txt

Explanation: In cont.txt input file having the number of input records (refer the screenshot 3 (a)). We are able to print the unique records with the interval of “1” (“N”) [the “N” value is depending on the client requirement]

Output:

Output-6.1

Output-6.2

7. Uniq Using –s N Command

The uniq –s N command is similar to -f N option. But it will skip the “N” characters but not skip the “N” fields.

uniq -s 3 cont.txt

Explanation: In cont.txt input file, we are having multiple lines with special characters and numbers in it (refer screenshot 1). With the help of the  “-s” option will skip the special characters or numbers from the input file. The “N” denotes the number value. The same number of value will help to skip the special character form the input file.

Note: As per the “N” value, the number of character will skip. Then after the uniq command will omit the duplicate data or records form the input file (refer screenshot 2).

Output:

Output-7.1

Output-7.2

8. Uniq Using –w N Command

The uniq –w N command is similar to skipping the characters. With the help of uniq command, we can limit the comparison to the set number of characters (“N” value).

uniq -w 3 input.txt

Explanation: In input.txt file, we are having 3 records starting to form 3 letter words i.e. “How” (refer Screenshot 1). With the help of “-w” option and “N” value (3), we will skip the initial 3 characters of the input file and having character sequence (refer Screenshot 2).

Output:

Output-8.1

Output-8.2

Conclusion

We have seen the uncut concept of “Linux uniq” with the proper example, explanation, and command with different outputs. The uniq command is like a filter program. It will filter out the data as per user requirements. The filter data or output data will further use of shell jobs and other application development tasks.

Recommended Articles

This is a guide to Linux uniq. Here we also discuss the Introduction and how does linux uniq command work? along with different examples and its code implementation. You may also have a look at the following articles to learn more –

  1. Linux Filter Commands
  2. Linux gzip
  3. Linux tee Command
  4. Linux OD

Linux Training Program (16 Courses, 3+ Projects)

16 Online Courses

3 Hands-on Projects

160+ Hours

Verifiable Certificate of Completion

Lifetime Access

Learn More

0 Shares
Share
Tweet
Share
Primary Sidebar
Linux Tutorial
  • Linux Commands
    • CAT Command in Linux
    • PS command in Linux
    • Mkdir Command in Linux
    • rmdir command in Linux
    • mv command in Linux
    • Linux rm Command
    • CHMOD Command in Linux
    • Zip Command in Linux
    • Linux Route Command
    • SCP Command in Linux
    • du Command in Linux
    • Less Command in Linux
    • Copy Command in Linux
    • Head Command in Linux
    • Help Command in Linux
    • Linux History Command
    • hostname Command in Linux
    • GREP Command in Linux
    • Linux Curl Command
    • Linux List Users
    • Linux List Directories
    • Linux Directory Permissions
    • Find Directory Linux
    • Linux User Permission
    • Linux Add User
    • Linux Delete User
    • Linux uniq
    • Linux Filter Commands
    • Linux sort Command
    • Linux Sort by Size
    • Remove dir Linux
    • Linux tr Command
    • Linux Touch Command
    • Linux More Command
    • Linux Cal Command
    • Linux OD
    • Linux Zcat
    • Linux tracepath
    • Linux Sleep
    • Linux bg Command
    • Linux WC
    • Linux Read
    • Linux Write
    • Linux Rename Command
    • Linux Free Command
    • Linux tac
    • Linux Shred
    • Linux wget
    • Linux Date
    • Linux Untar
    • Linux JQ
    • Linux Xargs
    • Linux Mount Command
    • Linux mdadm
    • Linux comm
    • Linux tee Command
    • Tail Command in Linux
    • Linux gzip
    • Linux Directory Structure
    • Linux Alias Command
    • Linux Inode
    • Linux Ping
    • Linux Watch Command
    • Linux Split Command
    • Linux Size
    • Linux Network Stack
    • Linux Diff Command
    • Vim Command in Linux
    • Linux nslookup
  • Basic
    • Introduction to Linux
    • What is Linux
    • Careers in Linux
    • How to Install Linux
    • How to Install Ubuntu
    • How to Install Ubuntu Server
    • How To Install Debian
    • Advantage Of Linux
    • What is Raid in Linux
    • What is Raid 5?
    • Raid 5 vs Raid 6
    • Careers in Linux Administration
    • What is Shell in Linux?
    • Bash Shell in Linux
    • Linux Emulator
    • Linux Operators
    • Linux Apps
    • Linux Keyboard Shortcuts
    • Linux Crontab
    • What is Raid Storage
    • What is Vmware
    • What is RedHat
    • What is Debian
    • Uses of Ubuntu
    • Career In RedHat
    • Linux Distributions
    • Linux From Scratch
    • Linux Format
    • Cheat Sheet Linux
  • Advanced
    • Install VMware
    • Install GRUB
    • Chown Command in Linux
    • Linux Network Command
    • Configuring Linux Network Environment
    • DNS Configuration in Linux
    • Linux Users
    • Create User in Linux
    • DHCP Server in Linux
    • FTP Server in Linux
    • Linux Regular Expression
    • IP Command in Linux
    • Set IP Address in Linux
    • Tar Command in Linux
    • Dynamic Host Configuration Protocol
    • Linux Kill Signals
    • Linux File Permissions
    • Linux File System
    • Linux Create File
    • Linux Concatenate Files
    • Linux Join
    • Linux Process Management
    • Linux Proxy Server
    • Linux System Commands
    • Linux System Logging
    • NFS in Linux
    • Linux LVM
    • Linux Repository
    • Samba Server in Linux
    • Top Commands in Linux
    • Types of Shells in Linux
    • Shell Script Types For Linux
    • GCC Command in Linux
    • Monolithic Kernel
    • YARN Package Manager
    • Linux Sed Replace
    • Linux List Groups
    • Linux Container
    • Linux Find File by Name
    • Create Folder in Linux
    • Linux Move Folder
    • Linux Error Codes
    • Linux Automount
    • Linux HugePages
  • Kali Linux
    • Install Kali Linux
    • Kali Linux Uses
    • Kali Linux Commands
    • Kali Linux Repository
    • Kali Linux Packages
    • Kali Linux root password
    • Kali Linux Alternative
  • Interview Questions
    • Linux Interview Questions
    • Linux System Administration Interview
    • Red Hat Interview Questions

Related Courses

Kali Linux Training

Linux Training Course

Red Hat Training Course

Footer
About Us
  • Blog
  • Who is EDUCBA?
  • Sign Up
  • Corporate Training
  • Certificate from Top Institutions
  • Contact Us
  • Verifiable Certificate
  • Reviews
  • Terms and Conditions
  • Privacy Policy
  •  
Apps
  • iPhone & iPad
  • Android
Resources
  • Free Courses
  • Java Tutorials
  • Python Tutorials
  • All Tutorials
Certification Courses
  • All Courses
  • Software Development Course - All in One Bundle
  • Become a Python Developer
  • Java Course
  • Become a Selenium Automation Tester
  • Become an IoT Developer
  • ASP.NET Course
  • VB.NET Course
  • PHP Course

© 2020 - EDUCBA. ALL RIGHTS RESERVED. THE CERTIFICATION NAMES ARE THE TRADEMARKS OF THEIR RESPECTIVE OWNERS.

EDUCBA Login

Forgot Password?

EDUCBA
Free Software Development Course

Web development, programming languages, Software testing & others

*Please provide your correct email id. Login details for this Free course will be emailed to you
Book Your One Instructor : One Learner Free Class

Let’s Get Started

This website or its third-party tools use cookies, which are necessary to its functioning and required to achieve the purposes illustrated in the cookie policy. By closing this banner, scrolling this page, clicking a link or continuing to browse otherwise, you agree to our Privacy Policy

EDUCBA

*Please provide your correct email id. Login details for this Free course will be emailed to you
EDUCBA
Free Software Development Course

Web development, programming languages, Software testing & others

*Please provide your correct email id. Login details for this Free course will be emailed to you

Special Offer - Linux Training Program (16 Courses, 3+ Projects) Learn More