We can design this program for use as a filter
that prints all lines, regardless of whether a change has been made.
We'll call it awkro.
awk '# awkro - expand acronyms
# load acronyms file into array "acro"
FILENAME == "acronyms" {
split($0, entry, "\t")
acro[entry[1]] = entry[2]
next
}
# process any input line containing caps
/[A-Z][A-Z]+/ {
# see if any field is an acronym
for (i = 1; i <= NF; i++)
if ( $i in acro ) {
# if it matches, add description
$i = acro[$i] " (" $i ")"
}
}
{
# print all lines
print $0
}' acronyms $*
Let's first see it in action. Here's a sample input file.
$ cat sample
The USGCRP is a comprehensive
research effort that includes applied
as well as basic research.
The NASA program Mission to Planet Earth
represents the principal space-based component
of the USGCRP and includes new initiatives
such as EOS and Earthprobes.
And here is the file acronyms:
$ cat acronyms
USGCRP U.S. Global Change Research Program
NASA National Aeronautic and Space Administration
EOS Earth Observing System
Now we run the program on the sample file.
$ awkro sample
The U.S. Global Change Research Program (USGCRP) is a comprehensive
research effort that includes applied
as well as basic research.
The National Aeronautic and Space Administration (NASA) program
Mission to Planet Earth
represents the principal space-based component
of the U.S. Global Change Research Program (USGCRP) and includes new
initiatives
such as Earth Observing System (EOS) and Earthprobes.
We'll look at this program in two parts. The first part reads records
from the acronyms file.
# load acronyms file into array "acro"
FILENAME == "acronyms" {
split($0, entry, "\t")
acro[entry[1]] = entry[2]
next
}
The two fields from these records are loaded into an array using the
first field as the subscript and assigning the second field to an
element of the array. In other words, the acronym itself is the index
to its description.
Note that we did not change the field separator, but instead used the
split() function to create the array
entry. This array is then used in creating an
array named acro.
Here is the second half of the program:
# process any input line containing caps
/[A-Z][A-Z]+/ {
# see if any field is an acronym
for (i = 1; i <= NF; i++)
if ( $i in acro ) {
acronym =$i
# if it matches, add description
$i = acro[$i] " (" $i ")"
}
}
{
# print all lines
print $0
}
Only lines that contain more than one consecutive capital letter are
processed by the first of the two actions shown here. This action
loops through each field of the record. At the heart of this section
is the conditional statement that tests if the current field
($i) is a subscript of the array
(acro). If the field is a subscript, we replace
the original value of the field with the array element and the
original value in parentheses. (Fields can be assigned new values,
just like regular variables.) Note that the insertion of the
description of the acronym results in lines that may be too long. See
the next chapter for a discussion of the
length() function, which can be used to
determine the length of a string so you can divide it up if it is too
long.
Now we're going to change the program so it makes a replacement only
the first time an acronym appears. After we've found it, we don't
want to search for that acronym any more. This is easy to do; we
simply delete that acronym from the array.
if ( $i in acro ) {
# if it matches, add description
$i = acro[$i] " (" $i ")"
# only expand the acronym once
delete acro[acronym]
}
There are other changes that would be good to make. In running the
awkro program, we
soon discovered that it failed to
match the acronym if it was followed by a punctuation mark. Our
initial solution was not to handle it in awk at all. Instead, we used
two sed scripts, one before processing:
sed 's/\([^.,;:!][^.,;:!]*\)\([.,;:!]\)/\1 @@@\2/g'
and one after:
sed 's/ @@@\([.,;:!]\)/\1/g'
A sed script, run prior to invoking awk, could simply insert a
space before any punctuation mark, causing it to be interpreted as a
separate field. A string of garbage characters
(@@@) was also added so we'd be able to easily
identify and restore the punctuation mark. (The complicated
expression used in the first sed command makes sure that we catch the
case of more than one punctuation mark on a line.)
This kind of solution, using another tool in the UNIX toolbox,
demonstrates that not everything needs to be done as an awk procedure.
Awk is all the more valuable because it is situated in the UNIX
environment.
However, with POSIX awk, we can implement a different solution, one
that uses a regular expression to match the acronym. Such a solution
can be implemented with the match() and
sub() functions described in the next
chapter.
8.5.1. Multidimensional Arrays
Awk supports linear arrays in which the index to each element of the
array is a single subscript. If you imagine a linear array as a row
of numbers, a two-dimensional array represents rows and columns of
numbers. You might refer to the element in the second column of the
third row as "array[3, 2]." Two- and three-dimensional
arrays are examples of multidimensional arrays. Awk does not support
multidimensional arrays but instead offers a syntax for subscripts
that simulate a reference to a multidimensional array.
For instance, you could write the following expression:
file_array[NR, i] = $i
where each field of an input record is indexed by its record number
and field number. Thus, the following reference:
file_array[2, 4]
would produce the value of the fourth field of the second record.
This syntax does not create a multidimensional array. It is
converted into a string that uniquely identifies the element in a
linear array. The components of a multidimensional subscript are
interpreted as individual strings ("2" and "4," for instance) and
concatenated together separated by the value of the system variable
SUBSEP. The subscript-component separator is
defined as
"\034"
by default,
an unprintable character rarely
found in ASCII text. Thus, awk maintains a one-dimensional array and
the subscript for our previous example would actually
be "2\0344" (the
concatenation of "2," the value of
SUBSEP, and "4"). The main
consequence of this simulation of multidimensional arrays is that the
larger the array, the slower it is to access individual elements.
However, you should time this, using your own application, with
different awk implementations (see Chapter 11, "A Flock of awks").
Here is a sample awk script named bitmap.awk that
shows how to load and output the elements of a multidimensional
array. This array represents a two-dimensional bitmap that is 12
characters in width and height.
BEGIN { FS = "," # comma-separated fields
# assign width and height of bitmap
WIDTH = 12
HEIGHT = 12
# loop to load entire array with "O"
for (i = 1; i <= WIDTH; ++i)
for (j = 1; j <= HEIGHT; ++j)
bitmap[i, j] = "O"
}
# read input of the form x,y.
{
# assign "X" to that element of array
bitmap[$1, $2] = "X"
}
# at end output multidimensional array
END {
for (i = 1; i <= WIDTH; ++i){
for (j = 1; j <= HEIGHT; ++j)
printf("%s", bitmap[i, j] )
# after each row, print newline
printf("\n")
}
}
Before any input is read, the bitmap array is
loaded with O's. This array has 144 elements. The input to this
program is a series of coordinates, one per line.
$ cat bitmap.test
1,1
2,2
3,3
4,4
5,5
6,6
7,7
8,8
9,9
10,10
11,11
12,12
1,12
2,11
3,10
4,9
5,8
6,7
7,6
8,5
9,4
10,3
11,2
12,1
For each coordinate, the program will put an "X" in place of an "O" as
that element of the array. At the end of the script, the same kind of
loop that loaded the array, now outputs it. The following example
reads the input from the file bitmap.test.
$ awk -f bitmap.awk bitmap.test
XOOOOOOOOOOX
OXOOOOOOOOXO
OOXOOOOOOXOO
OOOXOOOOXOOO
OOOOXOOXOOOO
OOOOOXXOOOOO
OOOOOXXOOOOO
OOOOXOOXOOOO
OOOXOOOOXOOO
OOXOOOOOOXOO
OXOOOOOOOOXO
XOOOOOOOOOOX
The multidimensional array syntax is also supported in testing for
array membership. The subscripts must be placed inside parentheses.
if ((i, j) in array)
This tests whether the subscript i,j (actually,
i SUBSEP j) exists in the specified array.
Looping over a multidimensional array is the same as with
one-dimensional arrays.
for (item in array)
You must use the split() function to access
individual subscript components. Thus:
split(item, subscr, SUBSEP)
creates the array subscr from the subscript
item.
Note that we needed to use the loop-within-a-loop to output the
two-dimensional bitmap array in the previous example because
we needed to maintain rows and columns.