Saving Your Output (Running Linux)

4.9. Saving Your Output

System administrators (and other human beings too) see a lot of critical messages fly by on the computer screen. It's often important to save these messages so you can scrutinize them later, or (all too often) send them to a friend who can figure out what went wrong. So, in this section, we'll explain a little bit about redirection, a powerful feature provided by Unix shells. If you come from MS-DOS, you have probably seen a similar, but more limited, type of redirection.

If you put a greater-than sign (>) and a filename after any command, the output of the command will be sent to that file. For instance, to capture the output of ls, you can enter:

$ ls /usr/bin > ~/Binaries

A listing of /usr/bin will be stored in your home directory in a file named Binaries. If Binaries had already existed, the > would wipe out what was there and replace it with the output of the ls command. Overwriting a current file is a common user error. If your shell is csh or tcsh, you can prevent overwriting with the command:


$ set noclobber

And in bash you can achieve the same effect by entering:

$ noclobber=1           It doesn't have to be 1; any value will have the same effect.

Another (and perhaps more useful) way to prevent overwriting is to append new output. For instance, having saved a listing of /usr/bin, suppose we now want to add the contents of /bin to that file. We can append it to the end of the Binaries file by specifying two greater-than signs:

$ ls /bin >> ~/Binaries

You will find the technique of output redirection very useful when you are running a utility many times and saving the output for troubleshooting.

Most Unix programs have two output streams. One is called the standard output, and the other is the standard error. If you're a C programmer you'll recognize these: the standard error is the file named stderr to which you print messages.

The > character does not redirect the standard error. It's useful when you want to save legitimate output without mucking up a file with error messages. But what if the error messages are what you want to save? This is quite common during troubleshooting. The solution is to use a greater-than sign followed by an ampersand. (This construct works in almost every modern UNIX shell.) It redirects both the standard output and the standard error. For instance:

$ gcc invinitjig.c >& error-msg

This command saves all the messages from the gcc compiler in a file named error-msg. (Of course, the object code is not saved there. It's stored in invinitjig.o as always.) On the Bourne shell and bash you can also say it slightly differently:

$ gcc invinitjig.c &> error-msg

Now let's get really fancy. Suppose you want to save the error messages but not the regular output--the standard error but not the standard output. In the Bourne-compatible shells you can do this by entering the following:

$ gcc invinitjig.c 2> error-msg

The shell arbitrarily assigns the number 1 to the standard output and the number 2 to the standard error. So the above command saves only the standard error.

Finally, suppose you want to throw away the standard output--keep it from appearing on your screen. The solution is to redirect it to a special file called /dev/null. (You've heard people say things like "Send your criticisms to /dev/null"? Well, this is where the phrase came from.) The /dev directory is where Unix systems store special files that refer to terminals, tape drives, and other devices. But /dev/null is unique; it's a place you can send things so that they disappear into a black hole. For example, the following command saves the standard error and throws away the standard output:

$ gcc invinitjig.c 2>error-msg >/dev/null

So now you should be able to isolate exactly the output you want.

In case you've wondered whether the less-than sign (<) means anything to the shell: yes, it does. It causes commands to take their input from a file. But most commands allow you to specify input files on their command lines anyway, so this "input redirection" is rarely necessary.

Sometimes you want one utility to operate on the output of another utility. For instance, you can use the sort command to put the output of other commands into a more useful order. A crude way to do this would be to save output from one command in a file, and then run sort on it. For instance:

$ du > du_output 
$ sort -n du_output

Unix provides a much more succinct and efficient way to do this using a pipe. Just place a vertical bar between the first command and the second:

$ du | sort -n

The shell sends all the input from the du program to the sort program.

In the previous example, du stands for "disk usage" and shows how many blocks each file occupies under the current directory. Normally, its output is in a somewhat random order:

$ du 
10        ./zoneinfo/Australia 
13        ./zoneinfo/US 
9         ./zoneinfo/Canada 
4         ./zoneinfo/Mexico 
5         ./zoneinfo/Brazil 
3         ./zoneinfo/Chile 
20        ./zoneinfo/SystemV 
118       ./zoneinfo 
298       ./ghostscript/doc 
183       ./ghostscript/examples 
3289      ./ghostscript/fonts 
        . 
        . 
        .

So we have decided to run it through sort with the -n and -r options. The -n option means "sort in numerical order" instead of the default ASCII sort, and the -r option means "reverse the usual order" so that the highest number appears first. The result is output that quickly shows you which directories and files hog the most space:

$ du | sort -rn 
34368     . 
16005     ./emacs 
16003     ./emacs/20.4 
13326     ./emacs/20.4/lisp 
4039      ./ghostscript 
3289      ./ghostscript/fonts 
        . 
        . 
        .

Since there are so many files, we had better use a second pipe to send output through the more command (one of the more common uses of pipes):

$ du | sort -rn | more 
34368     . 
16005     ./emacs 
16003     ./emacs/20.4 
13326     ./emacs/20.4/lisp 
4039      ./ghostscript 
3289      ./ghostscript/fonts 
        . 
        . 
        .