Using ex Scripts (Learning the vi Editor, Sixth Edition)

7.4. Using ex Scripts

Certain ex commands you use only within vi, such as maps, abbreviations, and so on. If you store these commands in your .exrc file, the commands will automatically be executed when you invoke vi. Any file that contains commands to execute is called a script.

The commands in a typical .exrc script are of no use outside vi. However, you can save other ex commands in a script, and then execute the script on a file or on multiple files. Mostly you'll use substitute commands in these external scripts.

For a writer, a useful application of ex scripts is to ensure consistency of terminology—or even of spelling—across a document set. For example, let's assume that you've run the UNIX spell command on two files and that the command has printed out the following list of misspellings:

$ spell sect1 sect2
chmod
ditroff
myfile
thier
writeable

As is often the case, spell has flagged a few technical terms and special cases it doesn't recognize, but it has also identified two genuine spelling errors.

Because we checked two files at once, we don't know which files the errors occurred in or where they are in the files. Although there are ways to find this out, and the job wouldn't be too hard for only two errors in two files, you can easily imagine how time-consuming the job could grow to be for a poor speller or for a typist proofing many files at once.

To make the job easier, you could write an ex script containing the following commands:

%s/thier/their/g
%s/writeable/writable/g
wq

Assume you've saved these lines in a file named exscript. The script could be executed from within vi with the command:

:so exscript

or the script can be applied to a file right from the command line. Then you could edit the files sect1 and sect2 as follows:

$ ex - sect1 < exscript
$ ex - sect2 < exscript

The minus sign following the invocation of ex tells it to suppress the normal terminal messages.[33]

[33]According to the POSIX standard, ex should use -s instead of - as shown here. Typically, for backwards compatibility, both versions are accepted.

If the script were longer than the one in our simple example, we would already have saved a fair amount of time. However, you might wonder if there isn't some way to avoid repeating the process for each file to be edited. Sure enough, we can write a shell script that includes, but generalizes, the invocation of ex, so that it can be used on any number of files.

7.4.1. Looping in a Shell Script

You may know that the shell is a programming language as well as a command-line interpreter. To invoke ex on a number of files, we use a simple type of shell script command called the for loop. A for loop allows you to apply a sequence of commands for each argument given to the script. (The for loop is probably the single most useful piece of shell programming for beginners. You'll want to remember it even if you don't write any other shell programs.)

Here's the syntax of a for loop:

for variable in list
  do
  	  command(s)
  done

For example:

for file in $*
  do
	  ex - $file < exscript
  done

(The command doesn't need to be indented; we indented it for clarity.) After we create this shell script, we save it in a file called correct and make it executable with the chmod command. (If you aren't familiar with the chmod command and the procedures for adding a command to your UNIX search path, see Learning the UNIX Operating System, published by O'Reilly & Associates.) Now type:

$ correct sect1 sect2

The for loop in correct will assign each argument (each file in the list specified by $*, which stands for all arguments) to the variable file and execute the ex script on the contents of that variable.

It may be easier to grasp how the for loop works with an example whose output is more visible. Let's look at a script to rename files:

for file in $*
do
	  mv $file $file.x
done

Assuming this script is in an executable file called move, here's what we can do:

$ ls
ch01 ch02 ch03 move
$ move ch??
$ ls
ch01.x ch02.x ch03.x move

With creativity, you could rewrite the script to rename the files more specifically:

for nn in $*
do
  	  mv ch$nn sect$nn
done

With the script written this way, you'd specify numbers instead of filenames on the command line:

$ ls
ch01 ch02 ch03 move
$ move 01 02 03
$ ls
sect01 sect02 sect03 move

The for loop need not take $* (all arguments) as the list of values to be substituted. You can specify an explicit list as well. For example:

for variable in a b c d

will assign variable to a, b, c, and d in turn. Or you can substitute the output of a command. For example:

for variable in `grep -l "Alcuin" *`

will assign variable in turn to the name of each file in which grep finds the string Alcuin.

If no list is specified:

for variable

the variable will be assigned to each command-line argument in turn, much as it was in our initial example. This is actually not equivalent to:

for variable in $*

but to:

for variable in "$@"

which has a slightly different meaning. The symbol $* expands to $1, $2, $3, etc., but the four-character sequence "$@" expands to "$1", "$2", "$3", etc. Quotation marks prevent further interpretation of special characters.

Let's return to our main point and our original script:

for file in $*
do
	  ex - $file < exscript
done

It may seem a little inelegant to have to use two scripts—the shell script and the ex script. And in fact, the shell does provide a way to include an editing script inside a shell script.

7.4.2. Here Documents

In a shell script, the operator << means to take the following lines, up to a specified string, as input to a command. (This is often called a here document.) Using this syntax, we could include our editing commands in correct like this:

for file in $*
do
ex - $file << end-of-script
g/thier/s//their/g
g/writeable/s//writable/g
wq
end-of-script
done

The string end-of-script is entirely arbitrary—it just needs to be a string that won't otherwise appear in the input and can be used by the shell to recognize when the here document is finished. By convention, many users specify the end of a here document with the string EOF, or E_O_F, to indicate the end of the file.

There are advantages and disadvantages to each approach shown. If you want to make a one-time series of edits and don't mind rewriting the script each time, the here document provides an effective way to do the job.

However, it's more flexible to write the editing commands in a separate file from the shell script. For example, you could establish the convention that you will always put editing commands in a file called exscript. Then you only need to write the correct script once. You can store it away in your personal "tools" directory (which you've added to your search path) and use it whenever you like.

7.4.3. Sorting Text Blocks: A Sample ex Script

Suppose you want to alphabetize a file of troff-encoded glossary definitions. Each term begins with an .IP macro. In addition, each entry is surrounded by the .KS/.KE macro pair. (This ensures that the term and its definition will print as a block and will not be split across a new page.) The glossary file looks something like this:

.KS
.IP "TTY_ARGV" 2n
The command, specified as an argument vector,
that the TTY subwindow executes.
.KE
.KS
.IP "ICON_IMAGE" 2n
Sets or gets the remote image for icon's image.
.KE
.KS
.IP "XV_LABEL" 2n
Specifies a frame's header or an icon's label.
.KE
.KS
.IP "SERVER_SYNC" 2n
Synchronizes with the server once.
Does not set synchronous mode.
.KE

You can alphabetize a file by running the lines through the UNIX sort command, but you don't really want to sort every line. You want to sort only the glossary terms, moving each definition—untouched—along with its corresponding term. As it turns out, you can treat each text block as a unit by joining the block into one line. Here's the first version of your ex script:

g/^\.KS/,/^\.KE/j
%!sort

Each glossary entry is found between a .KS and .KE macro. j is the ex command to join a line (the equivalent in vi is J). So, the first command joins every glossary entry into one "line." The second command then sorts the file, producing lines like this:

.KS .IP "ICON_IMAGE" 2n Sets or gets ... image.   .KE
.KS .IP "SERVER_SYNC" 2n Synchronizes with ... mode.   .KE
.KS .IP "TTY_ARGV" 2n The command, ... executes.   .KE
.KS .IP "XV_LABEL" 2n Specifies a ... icon's label.   .KE

The lines are now sorted by glossary entry; unfortunately, each line also has macros and text mixed in (we've used ellipses [...] to show omitted text). Somehow, you need to insert newlines to "un-join" the lines. You can do this by modifying your ex script: mark the joining points of the text blocks before you join them, and then replace the markers with newlines. Here's the expanded ex script:

g/^\.KS/,/^\.KE/-1s/$/@@/
g/^\.KS/,/^\.KE/j
%!sort
%s/@@ /^M/g

The first three commands produce lines like this:

.KS@@ .IP "ICON_IMAGE" 2nn@@ Sets or gets ... image. @@ .KE
.KS@@ .IP "SERVER_SYNC" 2nn@@ Synchronizes with ... mode. @@ .KE
.KS@@ .IP "TTY_ARGV" 2nn@@ The ... vector, @@ that ... .@@ .KE
.KS@@ .IP "XV_LABEL" 2nn@@ Specifies a ... icon's label. @@ .KE

Note the extra space following the @@. The spaces result from the j command, because it converts each newline into a space.

The first command marks the original line breaks with @@. You don't need to mark the end of the block (after the .KE), so the first command uses a -1 to move back up one line at the end of each block. The fourth command restores the line breaks by replacing the markers (plus the extra space) with newlines. Now your file is sorted by blocks.

7.4.4. Comments in ex Scripts

You may want to reuse such a script, adapting it to a new situation. With a complex script like this, it is wise to add comments so that it's easier for someone else (or even yourself!) to reconstruct how it works. In ex scripts, anything following a double quote is ignored during execution, so a double quote can mark the beginning of a comment. Comments can go on their own line. They can also go at the end of any command that doesn't interpret a quote as part of the command. (For example, a quote has meaning to map commands and shell escapes, so you can't end such lines with a comment.)

Besides using comments, you can specify a command by its full name, something that would ordinarily be too time consuming from within vi. Finally, if you add spaces, the ex script above becomes this more readable one:

" Mark lines between each KS/KE block
global /^\.KS/,/^\.KE/-1 s /$/@@/
" Now join the blocks into one line
global /^\.KS/,/^\.KE/ join
" Sort each block--now really one line each
%!sort
" Restore the joined lines to original blocks
% s /@@ /^M/g

Surprisingly, the substitute command does not work in ex, even though the full names for the other commands do.

7.4.5. Beyond ex

If this discussion has whetted your appetite for even more editing power, you should be aware that UNIX provides editors even more powerful than ex: the sed stream editor and the awk data manipulation language. There is also the extremely popular Perl programming language. For information on these programs, see the O'Reilly books sed & awk, Learning Perl, and Programming Perl.