Redirecting I/O (Learning the Unix Operating System, 5th Edition)

Many Unix programs read input (such as a file) and write output. In this chapter, we discuss Unix programs that handle their input and output in a standard way. This lets them work with each other.

This chapter generally doesn't apply to full-screen programs, such as the Pico editor, that take control of your whole terminal window. (The pager programs, less, more, and pg, do work together in this way.) It also doesn't apply to graphical programs, such as StarOffice or Netscape, that open their own windows on your screen.

5.1. Standard Input and Standard Output

What happens if you don't give a filename argument on a command line? Most programs will take their input from your keyboard instead (after you press the first RETURN to start the program running, that is). Your terminal keyboard is the program's standard input.

As a program runs, the results are usually displayed on your terminal screen. The terminal screen is the program's standard output.

So, by default, each of these programs takes its input from the standard input and sends the results to the standard output.

These two default cases of input/output (I/O) can be varied. This is called I/O redirection.

If a program doesn't normally read from files, but reads from its standard input, you can give a filename by using the < (less-than symbol) operator. For example, the mail program (see Section 6.5.2 in Chapter 6) normally reads the message to send from your keyboard. Here's how to use the input redirection operator to mail the contents of the file to_do to bigboss@corp.xyz:

$ mail bigboss@corp.xyz < to_do
$

If a program writes to its standard output, which is normally the screen, you can make it write to a file instead by using the greater-than symbol (>) operator. The pipe operator (|) sends the standard output of one program to the standard input of another program. Input/output redirection is one of the most powerful and flexible Unix features, We'll take a closer look at it soon.

5.1.1. Putting Text in a File

Instead of always letting a program's output come to the screen, you can redirect output into a file. This is useful when you'd like to save program output or when you put files together to make a bigger file.

5.1.1.1. cat

cat, which is short for "concatenate," reads files and outputs their contents one after another, without stopping.

To display files on the standard output (your screen), use:

cat file(s)

For example, let's display the contents of the file /etc/passwd. This system file describes users' accounts. (Your system may have a more complete list somewhere else.)

$ cat /etc/passwd
root:x&k8KP30f;(:0:0:Root:/:
daemon:*:1:1:Admin:/:
	.
	.
	.
john::128:50:John Doe:/usr/john:/bin/sh
$

You cannot go back to view the previous screens, as you can when you use a pager program such as less (unless you're using a terminal window with a scrollbar, that is). cat is mainly used with redirection, as we'll see in a moment.

By the way: if you enter cat without a filename, it tries to read from the keyboard (as we mention earlier). You can get out by pressing RETURN followed by a single CTRL-D.

5.1.1.2. The > operator

When you add "> filename" to the end of a command line, the program's output is diverted from the standard output to the named file. The > symbol is called the output redirection operator.

WARNING: When you use the > operator, be careful not to accidentally overwrite a file's contents. Your system may let you redirect output to an existing file. If so, the old file will be deleted (or, in Unix lingo, "clobbered"). Be careful not to overwrite a much needed file!

Many shells can protect you from this risk. In the C shell, use the command set noclobber. The Korn shell and bash command is set -o noclobber. Enter the command at a shell prompt or put it in your shell's startup file. After that, the shell does not allow you to redirect onto an existing file and overwrite its contents.

This doesn't protect against overwriting by Unix programs such as cp; it works only with the > redirection operator. For more protection, you can set Unix file access permissions.

For example, let's use cat with this operator. The file contents that you'd normally see on the screen (from the standard output) are diverted into another file, which we'll then read using cat (without any redirection!):

$ cat /etc/passwd > password
$ cat password
root:x&k8KP30f;(:0:0:Root:/:
daemon:*:1:1:Admin:/:
	.
	.
	.
john::128:50:John Doe:/usr/john:/bin/sh
$

An earlier example (in Section 5.1.1.1) showed how cat /etc/passwd displays the file /etc/passwd on the screen. The example here adds the > operator; so the output of cat goes to a file called password in the working directory. Displaying the file password shows that its contents are the same as the file /etc/passwd (the effect is the same as the copy command cp /etc/passwd password).

You can use the > redirection operator with any program that sends text to its standard output--not just with cat. For example:

$ who > users
$ date > today
$ ls
password   today   users   ...

We've sent the output of who to a file called users and the output of date to the file named today. Listing the directory shows the two new files. Let's look at the output from the who and date programs by reading these two files with cat:

$ cat users
tim     tty1    Aug 12  07:30
john    tty4    Aug 12  08:26
$ cat today
Tue Aug 12 08:36:09 EDT 2001
$

You can also use the cat program and the > operator to make a small text file. We told you earlier to type CTRL-D if you accidentally enter cat without a filename. This is because the cat program alone takes whatever you type on the keyboard as input. Thus, the command:

cat > filename

takes input from the keyboard and redirects it to a file. Try the following example:

$ cat > to_do
Finish report by noon
Lunch with Xannie
Swim at 5:30
^D
$

cat takes the text that you typed as input (in this example, the three lines that begin with Finish, Lunch, and Swim), and the > operator redirects it to a file called to_do. Type CTRL-D once, on a new line by itself, to signal the end of the text. You should get a shell prompt.

You can also create a bigger file from smaller files with the cat command and the > operator. The form:

cat file1 file2 > newfile

creates a file newfile, consisting of file1 followed by file2.

$ cat today to_do > diary
$ cat diary
Tue Aug 12 08:36:09 EDT 2001
Finish report by noon
Lunch with Xannie
Swim at 5:30
$

WARNING: You can't use redirection to add a file to itself, along with other files. For example, you might hope that the following command would merge today's to-do list with tomorrow's. This won't work!
$ cat to_do to_do.tomorrow > to_do.tomorrow
cat: to_do.tomorrow: input file is output file
cat warns you, but it's actually already too late. When you redirect a program's output to a file, Unix empties (clobbers) the file before the program starts running. The right way to do this is by using a temporary file (as you'll see in a later example) or simply by using a text editor program.

5.1.1.3. The >> operator

You can add more text to the end of an existing file, instead of replacing its contents, by using the >> (append redirection) operator. Use it as you would the > (output redirection) operator. So:

cat file2 >> file1

appends the contents of file2 to the end of file1. For an example, let's append the contents of the file users, and also the current date and time, to the file diary. Then we display the file:

$ cat users >> diary
$ date >> diary
$ cat diary
Tue Aug 12 08:36:09 EDT 2001
Finish report by noon
Lunch with Xannie
Swim at 5:30
tim     tty1    Aug 12  07:30
john    tty4    Aug 12  08:26
Tue Aug 12 09:07:24 EDT 2001
$

Unix doesn't have a redirection operator that adds text to the beginning of a file. You can do this by storing the new text in a temporary file, then by using a text editor program to read the temporary file into the start of the file you want to edit. You also can do the job with a temporary file and redirection. Maybe you'd like each day's entry to go at the beginning of your diary file. Simply rename diary to something like temp. Make a new diary file with today's entries, then append temp (with its old contents) to the new diary. For example:[14]

[14] This example could be shortened by combining the two cat commands into one, giving both filenames as arguments to a single cat command. That wouldn't work, though, if you were making a real diary with a command other than cat users.

$ mv diary temp
$ date > diary
$ cat users >> diary
$ cat temp >> diary
$ rm temp

Chapter 5. Redirecting I/O

Contents:

5.1. Standard Input and Standard Output

5.1.1. Putting Text in a File

5.1.1.1. cat

5.1.1.2. The > operator

5.1.1.3. The >> operator