Shell Scripts On-the-Fly from Standard Input (Unix Power Tools, 3rd Edition)

36.18. Shell Scripts On-the-Fly from Standard Input

The shell can read commands from its standard input or from a file. To run a series of commands that can change, you may want to use one program to create the command lines automatically -- and pipe that program's output to a shell, which will run those "automatic" commands.

Here's an example.[113] You want to copy files from a subdirectory and all its subdirectories into a single directory. The filenames in the destination directory can't conflict; no two files can have the same name. An easy way to name the copies is to replace each slash (/) in the file's relative pathname with a minus sign (-).[114] For instance, the file named lib/glob/aprog.c would be copied to a file named lib-glob-aprog.c. You can use sed (Section 34.2) to convert the filenames and output cp commands like these:

[113]This isn't recommended for systems with a 14-character filename limit. You may also want to watch out on Darwin, which, although it has the typical UNIX filename limits, only displays 31 characters on the Finder Desktop (munging the last few chars or inserting...to provide a unique <32-character filename).

[114]A replacement like CTRL-a would make unique filenames (legal, but also harder to type).

cp from/lib/glob/aprog.c to/lib-glob-aprog.c
cp from/lib/glob/aprog.h to/lib-glob-aprog.h
    ...

However, an even better solution can be developed using nawk (Section 20.11). The following example uses find (Section 9.1) to make a list of pathnames, one per line, in and below the copyfrom directory. Next it runs nawk to create the destination file pathnames (like to/lib-glob-aprog.c) and write the completed command lines to the standard output. The shell reads the command lines from its standard input, through the pipe.

This example is in a script file because it's a little long to type at a prompt. But you can type commands like these at a prompt, too, if you want to:

#!/bin/sh
find copyfrom -type f -print |
awk '{
    out = $0
    gsub("/", "-", out)
    sub("^copyfrom-", "copyto/", out)
    print "cp", $0, out
}' |
sh

If you change the last line to sh -v, the shell's verbose option (Section 37.1) will show each command line before executing it. If the last line has sh -e, the shell will quit immediately after any command returns a nonzero exit status (Section 35.12) -- that might happen, for instance, if the disk fills up and cp can't make the copy. Finally, you may need to use nawk rather than awk, depending on your system.

-- JP