28.17. Dealing with Too Many Arguments
Historically, one of the more annoying
things about the design of many UNIX tools was their inability to
handle large numbers of arguments. For example, if you wanted to
print several hundred files using lpr, you either
had to pass them a few at a time, perhaps using wildcards on the
command line to split the list up into shorter groups, or call
lpr once per file, perhaps using
find or a loop. One other method, which is still
useful today, involves the use of xargs.
xargs is one of those Unix utilities that seems
pretty useless when you first hear about it -- but turns into one
of the handiest tools you can have.
Go to http://examples.oreilly.com/upt3 for more information on: xargs
If your system doesn't already have
xargs, be sure to install it from the web site.
xargs reads a group of arguments from its standard
input, then runs a Unix command with that group of arguments. It
keeps reading arguments and running the command until it runs out of
arguments. The shell's backquotes (Section 28.14) do the
same kind of thing, but they give all the arguments to the command at
once. This can give you a Too many arguments
Here are some examples:
If you want to print most of the files in a large directory, put the
output of ls into a file. Edit the file to leave
just the filenames you want printed. Give the file to
xargs' standard input:
< Section 43.1
% ls > allfiles.tmp
% vi allfiles.tmp
% xargs lpr < allfiles.tmp
What did that do? With lines like these in
% cat allfiles.tmp
xargs ran one or more lpr
commands, each with a group of arguments, until it had read every
word in the file:
lpr afile application ...
lpr ... yoyotest zapme
This has another advantage for lpr: each print job
is fairly short, so you can delete one from the print queue without
losing all of them.
The standard output of xargs is the standard
output of the commands it runs. So, if you'd created
allfiles.tmp above, but you wanted to format the
files with pr (Section 45.6) first, you could type:
% xargs pr < allfiles.tmp | lpr
Then xargs would run all of these
pr commands. The shell would pipe their standard
outputs to a single
pr afile application ...
In the next example, find (Section 9.1) gets a list of all files in the directory
tree. Next, we use xargs to read those filenames
and run grep -l (Section 33.6) to find which files contain the word
"WARNING". Next, we pipe that to a
setup with pr and lpr, like the
one in the previous example:
% find . -type f -print | xargs grep -l WARNING | xargs pr | lpr
"Huh?" you might say. Just take
that step by step. The output of find is a list of
filenames, like ./afile ./bfile ... ./adir/zfile
and so on. The first xargs gives those filenames
to one or more grep -l commands:
grep -l WARNING ./afile ./bfile ...
grep -l WARNING ./adir/zfile ...
The standard output of all those greps is a
(shortened) list of filenames that match. That's
piped to another xargs -- it runs
pr commands with the filenames that
Unix is weird and wonderful!
Sometimes you don't
want xargs to run its command with as many
arguments as it can fit on the command line. The
option sets the maximum number of arguments xargs
will give to each command. Another handy
option, -p, prompts you
before running each command.
Here's a directory full of files with errors (whose
names end with .bad) and corrected versions
(named .fixed). I use ls to
give the list of files to xargs; it reads two
filenames at once, then asks whether I want to run diff
-c to compare those two files. It keeps
prompting me and running diff -c until it runs out
of file pairs:
% ls | xargs -p -n2 diff -c
diff -c chap1.bad chap1.fixed ?...y
...Output of diff command for chap1...
diff -c chap2.bad chap2.fixed ?...n
diff -c chap3.bad chap3.fixed ?...y
...Output of diff command for chap3...
--JP and SJC
Copyright © 2003 O'Reilly & Associates. All rights reserved.