9.2.1. Substrings

pos = index("Mississippi", "is")

The value of pos is 2. If the substring is not found, the index() function returns 0.

phone = substr("707-555-1111", 5)

You can also supply a third argument which is the number of characters to return. The next example returns just the area code:

area_code = substr("707-555-1111", 1, 3)

The two functions can be and often are used together, as in the next example. This example capitalizes the first letter of the first word for each input record.

awk '# caps - capitalize 1st letter of 1st word
# initialize strings
BEGIN { upper = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
        lower = "abcdefghijklmnopqrstuvwxyz" 
}

# for each input line
{
# get first character of first word
	FIRSTCHAR = substr($1, 1, 1)
# get position of FIRSTCHAR in lowercase array; if 0, ignore
	if (CHAR = index(lower, FIRSTCHAR)) 
		# change $1, using position to retrieve
		# uppercase character 
		$1 = substr(upper, CHAR, 1) substr($1, 2)
# print record
	print $0
}'

This script creates two variables, upper and lower, consisting of uppercase and lowercase letters. Any character that we find in lower can be found at the same position in upper. The first statement of the main procedure extracts a single character, the first one, from the first field. The conditional statement tests to see if that character can be found in lower using the index() function. If CHAR is not 0, then CHAR can be used to extract the uppercase character from upper. There are two substr() function calls: the first one retrieves the capitalized letter and the second call gets the rest of the first field, extracting all characters, beginning with the second character. The values returned by both substr() functions are concatenated and assigned to $1. Making an assignment to a field as we do here is a new twist, but it has the added benefit that the record can be output normally. (If the assignment was made to a variable, you'd have to output the variable and then output the record's remaining fields.) The print statement prints the changed record. Let's see it in action:

$ caps
root user
Root user
dale
Dale
Tom
Tom

In a little bit, we'll see how to revise this program to change all characters in a string from lower- to uppercase or vice versa.

9.2.2. String Length

The length() function is often used to find the length of the current input record, in order to determine if we need to break the line.

One way to handle the line break, perhaps more efficiently, is to use the length() function to get the length of each field. By accumulating those lengths, we could specify a line break when a new field causes the total to exceed a certain number.

9.2.3. Substitution Functions

Both functions take at least two arguments. The first is a regular expression (surrounded by slashes) that matches a pattern and the second argument is a string that replaces what the pattern matches. The regular expression can be supplied by a variable, in which case the slashes are omitted. An optional third argument specifies the string that is the target of the substitution. If there is no third argument, the substitution is made for the current input record ($0).

The substitution functions change the specified string directly. You might expect, given the way functions work, that the function returns the new string created when the substitution is made. The substitution functions actually return the number of substitutions made. sub() will always return 1 if successful; both return 0 if not successful. Thus, you can test the result to see if a substitution was made.

For example, the following example uses gsub() to replace all occurrences of "UNIX" with "POSIX".

if (gsub(/UNIX/, "POSIX"))
	print

The conditional statement tests the return value of gsub() such that the current input line is printed only if a change is made.

As with sed, if an "&" appears in the substitution string, it will be replaced by the string matched by the regular expression. Use "\&" to output an ampersand. (Remember that to get a literal "\" into a string, you have to type two of them.) Also, note that awk does not "remember" the previous regular expression, as does sed, so you cannot use the syntax "//" to refer to the last regular expression.

The following example surrounds any occurrence of "UNIX" with the troff font-change escape sequences.

gsub(/UNIX/, "\\fB&\\fR")

If the input is "the UNIX operating system", the output is "the \fBUNIX\fR operating system".

9.2.4. Converting Case

$ cat test
Hello, World!
Good-bye CRUEL world!
1, 2, 3, and away we GO!
$ awk '{ printf("<%s>, <%s>\n", tolower($0), toupper($0)) }' test
<hello, world!>, <HELLO, WORLD!>
<good-bye cruel world!>, <GOOD-BYE CRUEL WORLD!>
<1, 2, 3, and away we go!>, <1, 2, 3, AND AWAY WE GO!>

Note that nonalphabetic characters are left unchanged.

9.2.5. The match() Function

The match() function returns the starting position of the substring that was matched by the regular expression. You might consider it a close relation to the index() function. In the following example, the regular expression matches any sequence of capital letters in the string "the UNIX operating system".

match("the UNIX operating system", /[A-Z]+/)

The value returned by this function is 5, the character position of "U," the first capital letter in the string.

The match() function also sets two system variables: RSTART and RLENGTH. RSTART contains the same value returned by the function, the starting position of the substring. RLENGTH contains the length of the string in characters (not the ending position of the substring). When the pattern does not match, RSTART is set to 0 and RLENGTH is set to -1. In the previous example, RSTART is equal to 5 and RLENGTH is equal to 4. (Adding them together gives you the position of the first character after the match.)

$ cat test
Every NOW and then, a WORD I type appears in CAPS.
$ lower test
every now and then, a word i type appears in caps.

Note that you could change the regular expression to avoid matching individual capital letters by matching a sequence of two or more uppercase characters, by using: "/[A-Z][A-Z]+/." This would also require revising the way the lowercase conversion was made using gsub(), since it matches a single character on the line.

In our discussion of the sed substitution command, you saw how to save and recall a portion of a string matched by a pattern, using \( and \) to surround the pattern to be saved and \n to recall the saved string in the replacement pattern. Unfortunately, awk's standard substitution functions offer no equivalent syntax. The match() function can solve many such problems, though.

For instance, if you match a string using the match() function, you can single out characters or a substring at the head or tail of the string. Given the values of RSTART and RLENGTH, you can use the substr() function to extract the characters. In the following example, we replace the second of two colons with a semicolon. We can't use gsub() to make the replacement because "/:/" matches the first colon and "/:[^:]*:/" matches the whole string of characters. We can use match() to match the string of characters and to extract the last character of the string.

# replace 2nd colon with semicolon using match, substr
if (match($1, /:[^:]*:/)) {
	before = substr($1, 1, (RSTART + RLENGTH - 2))
	after = substr($1, (RSTART + RLENGTH))
	$1 = before ";" after
}

The match() function is placed within a conditional statement that tests that a match was found. If there is a match, we use the substr() function to extract the substring before the second colon as well as the substring after it. Then we concatenate before, the literal ";", and after, assigning it to $1.