35.20 Quick Reference: uniq

uniq is used to remove duplicate adjacent lines from a sorted file, sending one copy of each line to standard output or to a second file, if one is specified on the command line.

NOTE: Be warned:

% uniq file1 file2

will not print the unique lines from both file1 and file2 to standard output! It will replace the contents of file2 with the unique lines from file1 .

uniq is often used as a filter (1.30 ) . See also comm (28.12 ) , sort (36.1 ) , and especially sort -u (36.6 ) .

options

-c       Print each line once, counting instances of each.
-d       Print duplicate lines once, but no unique lines.
-u       Print only unique lines (no copy of duplicate entries is kept).
-n
       Ignore first n
 fields of a line. Fields are separated by spaces
or by tabs.
+n
       Ignore first n
 characters of a field.

You can specify only one of the options -c , -d , or -u .

Examples

To send only one copy of each line from list to output file list.new :

uniq list list.new

To show which names appear more than once:

sort names | uniq -d

To show which lines appear exactly three times:

grep
 

sort names | uniq -c | 
grep " 3 "