A Korn Shell Debugger (Learning the Korn Shell, 2nd Edition)

9.2.3. Debugger Functions

The function _steptrap is the entry point into the debugger; it is defined in the file kshdb.fns, listed in its entirety at the end of this chapter. Here is _steptrap:

# Here before each statement in script being debugged.
# Handle single-step and breakpoints.
function _steptrap {
    _curline=$1                       # arg is no. of line that just ran
    (( $_trace )) && _msg "$PS4 line $_curline: ${_lines[$_curline]}"
    if (( $_steps >= 0 )); then       # if in step mode
        let _steps="$_steps - 1"      # decrement counter
    fi

    # first check: if line num breakpoint reached
    if _at_linenumbp; then
        _msg "Reached line breakpoint at line $_curline"
        _cmdloop                      # breakpoint, enter debugger

    # second check: if string breakpoint reached
    elif _at_stringbp; then
        _msg "Reached string breakpoint at line $_curline"
        _cmdloop                      # breakpoint, enter debugger

    # if neither, check whether break condition exists and is true
    elif [[ -n $_brcond ]] && eval $_brcond; then
        _msg "Break condition '$_brcond' true at line $_curline"
        _cmdloop                      # break condition, enter debugger

    # finally, check if step mode and number of steps is up
    elif (( _steps == 0 )); then      # if step mode and time to stop
        _msg "Stopped at line $_curline"
        _cmdloop                      # enter debugger
    fi
}

_steptrap starts by setting _curline to the number of the guinea pig line that just ran. If execution tracing is turned on, it prints the PS4 execution trace prompt (a la xtrace mode), the line number, and the line of code itself.

Then it does one of two things: enter the debugger, the heart of which is the function _cmdloop, or just return so that the shell can execute the next statement. It chooses the former if a breakpoint or break condition (see below) has been reached, or if the user stepped into this statement.

9.2.3.1. Commands

We'll explain shortly how _steptrap determines these things; now we'll look at _cmdloop. It's a typical command loop, resembling a combination of the case statements we saw in Chapter 5 and the calculator loop we saw in Chapter 8.

# Debugger command loop.
# Here at start of debugger session, when breakpoint reached,
# after single-step.  Optionally here inside watchpoint.
function _cmdloop {
    typeset cmd args

    while read -s cmd"?kshdb> " args; do
        case $cmd in
        \#bp ) _setbp $args ;;       # set breakpoint at line num or string.
        \#bc ) _setbc $args ;;       # set break condition.
        \#cb ) _clearbp ;;           # clear all breakpoints.
        \#g  ) return ;;             # start/resume execution
        \#s  ) let _steps=${args:-1} # single-step N times (default 1)
               return ;;
        \#wp ) _setwp $args ;;       # set a watchpoint
        \#cw ) _clearwp $args ;;     # clear one or more watchpoints

        \#x  ) _xtrace ;;            # toggle execution trace
        \#\? | \#h ) _menu ;;        # print command menu
        \#q  ) exit ;;               # quit
        \#*  ) _msg "Invalid command: $cmd" ;;
        *  ) eval $cmd $args ;;      # otherwise, run shell command
        esac
    done

At each iteration, _cmdloop prints a prompt, reads a command, and processes it. We use read -s so that the user can take advantage of command-line editing within kshdb. All kshdb commands start with # to prevent confusion with shell commands. Anything that isn't a kshdb command (and doesn't start with #) is passed off to the shell for execution. Using # as the command character prevents a mistyped command from having any ill effect when the last case catches it and runs it through eval. Table 9-5 summarizes the debugger commands.

Table 9-5. kshdb commands

Command	Action
`#bp` `N`	Set breakpoint at line N.
`#bp` `str`	Set breakpoint at next line containing str.
`#bp`	List breakpoints and break condition.
`#bc` `str`	Set break condition to str.
`#bc`	Clear break condition.
`#cb`	Clear all breakpoints.
`#g`	Start or resume execution (go).
`#s` [`N`]	Step through N statements (default 1).
`#wp` [`-c`] `var` `get`	Set a watchpoint on variable var when the value is retrieved. With -c, enter the command loop from within the watchpoint.
`#wp` [`-c`] `var` `set`	Set a watchpoint on variable var when the value is assigned. With -c, enter the command loop from within the watchpoint.
`#wp` [`-c`] `var` `unset`	Set a watchpoint on variable var when the variable is unset. With -c, enter the command loop from within the watchpoint.
`#cw` `var discipline`	Clear the given watchpoint.
`#cw`	Clear all watchpoints.
`#x`	Toggle execution tracing.
`#h`, `#?`	Print a help menu.
`#q`	Quit.

Before we look at the individual commands, it is important that you understand how control passes through _steptrap, the command loop, and the guinea pig.

_steptrap runs before every statement in the guinea pig as a result of the trap ... DEBUG statement in the preamble. If a breakpoint has been reached or the user previously typed in a step command (#s), _steptrap calls the command loop. In doing so, it effectively interrupts the shell that is running the guinea pig to hand control over to the user.[131]

[131] In fact, low-level systems programmers can think of the entire trap mechanism as quite similar to an interrupt-handling scheme.

The user can invoke debugger commands as well as shell commands that run in the same shell as the guinea pig. This means that you can use shell commands to check values of variables, signal traps, and any other information local to the script being debugged.

The command loop runs, and the user stays in control, until the user types #g, #s, or #q. Let's look in detail at what happens in each of these cases.

#g has the effect of running the guinea pig uninterrupted until it finishes or hits a breakpoint. But actually, it simply exits the command loop and returns to _steptrap, which exits as well. The shell takes control back; it runs the next statement in the guinea pig script and calls _steptrap again. Assuming that there is no breakpoint, this time _steptrap just exits again, and the process repeats until there is a breakpoint or the guinea pig is done.

9.2.3.2. Stepping

When the user types #s, the command loop code sets the variable _steps to the number of steps the user wants to execute, i.e., to the argument given. Assume at first that the user omits the argument, meaning that _steps is set to 1. Then the command loop exits and returns control to _steptrap, which (as above) exits and hands control back to the shell. The shell runs the next statement and returns to _steptrap, which sees that _steps is 1 and decrements it to 0. Then the third elif conditional sees that _steps is 0, so it prints a "stopped" message and calls the command loop.

Now assume that the user supplies an argument to #s, say 3. _steps is set to 3. Then the following happens:

After the next statement runs, _steptrap is called again. It enters the first if clause, since _steps is greater than 0. _steptrap decrements _steps to 2 and exits, returning control to the shell.
This process repeats, another step in the guinea pig is run, and _steps becomes 1.
A third statement is run and we're back in _steptrap. _steps is decremented to 0, the third elif clause is run, and _steptrap breaks out to the command loop again.

The overall effect is that three steps run and then the debugger takes over again.

Finally, the #q command exits. The EXIT trap then calls the function _cleanup, which just erases the temp file and exits the entire program.

All other debugger commands (#bp, #bc, #cb, #wp, #cw, #x, and shell commands) cause the shell to stay in the command loop, meaning that the user prolongs the interruption of the shell.

9.2.3.3. Breakpoints

Now we'll examine the breakpoint-related commands and the breakpoint mechanism in general. The #bp command calls the function _setbp, which can set two kinds of breakpoints, depending on the type of argument given. If it is a number, it's treated as a line number; otherwise, it's interpreted as a string that the breakpoint line should contain.

For example, the command #bp 15 sets a breakpoint at line 15, and #bp grep sets a breakpoint at the next line that contains the string grep -- whatever number that turns out to be. Although you can always look at a numbered listing of a file,[132] string arguments to #bp can make that unnecessary.

[132] pr -n filename prints a numbered listing to standard output on System V-derived versions of Unix. Some very old BSD-derived systems don't support it. If this doesn't work on your system, try cat -n filename, or if that doesn't work, create a shell script with the single line awk '{ printf("%d\t%s\n", NR, $0 }' $1

Here is the code for _setbp:

# Set breakpoint(s) at given line numbers or strings
# by appending patterns to breakpoint variables
function _setbp {
    if [[ -z $1 ]]; then
        _listbp
    elif [[ $1 == +([0-9]) ]]; then  # number, set bp at that line
        _linebp="${_linebp}$1|"
        _msg "Breakpoint at line " $1
    else                             # string, set bp at next line w/string
        _stringbp="${_stringbp}$@|"
        _msg "Breakpoint at next line containing '$@'."
    fi
}

_setbp sets the breakpoints by storing them in the variables _linebp (line number breakpoints) and _stringbp (string breakpoints). Both have breakpoints separated by pipe character delimiters, for reasons that will become clear shortly. This implies that breakpoints are cumulative; setting new breakpoints does not erase the old ones.

The only way to remove breakpoints is with the command #cb, which (in function _clearbp) clears all of them at once by simply resetting the two variables to null. If you don't remember what breakpoints you have set, the command #bp without arguments lists them.

The functions _at_linenumbp and _at_stringbp are called by _steptrap after every statement; they check whether the shell has arrived at a line number or string breakpoint, respectively.

Here is _at_linenumbp:

# See if next line no. is a breakpoint.
function _at_linenumbp {
    [[ $_curline == @(${_linebp%\|}) ]]
}

_at_linenumbp takes advantage of the pipe character as the separator between line numbers: it constructs a regular expression of the form @(N1|N2|...) by taking the list of line numbers _linebp, removing the trailing |, and surrounding it with @( and ). For example, if $_linebp is 3|15|19|, the resulting expression is @(3|15|19).

If the current line is any of these numbers, the conditional becomes true, and _at_linenumbp also returns a "true" (0) exit status.

The check for a string breakpoint works on the same principle, but it's slightly more complicated; here is _at_stringbp:

# Search string breakpoints to see if next line in script matches.
function _at_stringbp {
    [[ -n $_stringbp && ${_lines[$_curline]} == *@(${_stringbp%\|})* ]]
}

The conditional first checks if $_stringbp is non-null (meaning that string breakpoints have been defined). If not, the conditional evaluates to false, but if so, its value depends on the pattern match after the && -- which tests the current line to see if it contains any of the breakpoint strings.

The expression on the right side of the double equal sign is similar to the one in _at_linenumbp above, except that it has * before and after it. This gives expressions of the form *@(S1|S2|...)*, where the Ss are the string breakpoints. This expression matches any line that contains any one of the possibilities in the parentheses.

The left side of the double equal sign is the text of the current line in the guinea pig. So, if this text matches the regular expression, we've reached a string breakpoint; accordingly, the conditional expression and _at_stringbp return exit status 0.

_steptrap tests each condition separately, so that it can tell you which kind of breakpoint stopped execution. In both cases, it calls the main command loop.

9.2.3.4. Break conditions

kshdb has another feature related to breakpoints: the break condition. This is a string that the user can specify that is evaluated as a command; if it is true (i.e., returns exit status 0), the debugger enters the command loop. Since the break condition can be any line of shell code, there's lots of flexibility in what can be tested. For example, you can break when a variable reaches a certain value (e.g., (( $x < 0 ))) or when a particular piece of text has been written to a file (grep string file). You will probably think of all kinds of uses for this feature.[133] To set a break condition, type #bc string. To remove it, type #bc without arguments -- this installs the null string, which is ignored. _steptrap evaluates the break condition $_brcond only if it's non-null. If the break condition evaluates to 0, the if clause is true and, once again, _steptrap calls the command loop.

[133] Bear in mind that if your break condition produces any standard output (or standard error), you will see it before every statement. Also, make sure your break condition doesn't take a long time to run; otherwise your script will run very, very slowly.

9.2.3.5. Execution tracing

The next feature is execution tracing, available through the #x command. This feature is meant to overcome the fact that a kshdb user can't use set -o xtrace while debugging (by entering it as a shell command), because its scope is limited to the _cmdloop function.[134]

[134] Actually, by entering typeset -ft funcname, the user can enable tracing on a per-function basis, but it's probably better to have it all under the debugger's control.

The function _xtrace toggles execution tracing by simply assigning to the variable _trace the logical "not" of its current value, so that it alternates between 0 (off) and 1 (on). The preamble initializes it to 0.

9.2.3.6. Watchpoints

kshdb takes advantage of the shell's discipline functions to provide watchpoints. You can set a watchpoint on any variable when the variable's value is retrieved or changed, or when the variable is unset. Optionally, the watchpoint can be set up to drop into the command loop as well. You do this with the #wp command, which in turn calls _setwp:

# Set a watchpoint on a variable
# usage: _setwp [-c] var discipline
# $1 = variable
# $2 = get|set|unset
typeset -A _watchpoints
function _setwp {
    typeset funcdef do_cmdloop=0
    if [[ $1 == -c ]]; then
        do_cmdloop=1
        shift
    fi

    funcdef="function $1.$2 { "

    case $2 in
    get)    funcdef+="_msg $1 \(\$$1\) retrieved, line \$_curline"
            ;;
    set)    funcdef+="_msg $1 set to "'${.sh.value}'", line \$_curline"
            ;;
    unset)  funcdef+="_msg $1 cleared at line \$_curline"
            funcdef+=$'\nunset '"$1"
            ;;
    *)      _msg invalid watchpoint function $2
            return 1
            ;;
    esac

    if ((do_cmdloop)); then
        funcdef+=$'\n_cmdloop'
    fi
    funcdef+=$'\n}'

    eval "$funcdef"

    _watchpoints[$1.$2]=1
}

This function illustrates several interesting techniques. The first thing it does is declare some local variables and check if it was invoked with the -c option. This indicates that the watchpoint should enter the command loop.

The general idea is to build up the text of the appropriate discipline function in the variable funcdef. The initial value is the function keyword, the discipline function name, and the opening left curly brace. The space following the brace is important, so that the shell will correctly recognize it as a keyword.

Then, for each kind of discipline function, the case construct appends the appropriate function body to the funcdef string. The code uses judiciously placed backslashes to get the correct mixture of immediate and delayed shell variable evaluation. Consider the get case: for the \(, the backslash stays intact for use as a quoting character inside the body of the discipline function. For \$$1, the quoting happens as follows: the \$ becomes a $ inside the function, while the $1 is evaluated immediately inside the double quoted string.

In the case that the -c option was supplied, it uses the $'...' notation to append a newline and a call to _cmdloop to the function body, and then at the end appends another newline and closing right brace. Finally, by using eval, it installs the newly created function.

For example, if -c was used, the text of the generated get function for the variable count ends up looking like this:

function count.get { _msg count \($count\) retrieved, line $_curline
_cmdloop
}

At the end of _setwp, _watchpoints[$1.$2] is set to 1. This creates an entry in the associative array _watchpoints indexed by discipline function name. This conveniently stores the names of all watchpoints for when we want to clear them.

Watchpoints are cleared with the #cw command, which in turn runs the _clearwp function. Here it is:

# Clear watchpoints:
# no args: clear all
# two args: same as for setting: var get|set|unset
function _clearwp {
    if [ $# = 0 ]; then
        typeset _i
        for _i in ${!_watchpoints[*]}; do
            unset -f $_i
            unset _watchpoints[$_i]
        done
    elif [ $# = 2 ]; then
        case $2 in
        get | set | unset)
            unset -f $1.$2
            unset _watchpoints[$1.$2]
            ;;
        *)  _msg $2: invalid watchpoint
            ;;
        esac
    fi
}

When invoked with no arguments, _clearwp clears all the watchpoints, by looping over all the subscripts in the _watchpoints associative array. Otherwise, if invoked with two arguments, the variable name and discipline function, it unsets the function using unset -f. In either case, the entry in _watchpoints is also unset.

9.2.3.7. Limitations

kshdb was not designed to push the state of the debugger art forward or to have an overabundance of features. It has the most useful basic features; its implementation is compact and (we hope) comprehensible. But it does have some important limitations. The ones we know of are described in the list that follows:

String breakpoints cannot begin with digits or contain pipe characters (|) unless they are properly escaped.

You can only set breakpoints -- whether line number or string -- on lines in the guinea pig that contain what the shell's documentation calls simple commands, i.e., actual Unix commands, shell built-ins, function calls, or aliases. If you set a breakpoint on a line that contains only whitespace or a comment, the shell always skips over that breakpoint. More importantly, control keywords like while, if, for, do, done, and even conditionals ([[...]] and ((...))) won't work either, unless a simple command is on the same line.

kshdb does not "step down" into shell scripts that are called from the guinea pig. To do this, you have to edit your guinea pig and change a call to scriptname to kshdb scriptname.

Similarly, subshells are treated as one gigantic statement; you cannot step down into them at all.

The guinea pig should not trap on the fake signals DEBUG or EXIT; otherwise the debugger won't work.

Variables that are typeset (see Chapter 4) are not accessible in break conditions. However, you can use the shell command print to check their values.
Command error handling is weak. For example, a non-numeric argument to #s will cause it to bomb.
Watchpoints that invoke the command loop are fragile. For ksh93m under GNU/Linux, trying to unset a watchpoint when in the command loop invoked from the watchpoint causes the shell to core dump. But this does not happen on all platforms, and this will eventually be fixed.

Many of these are not insurmountable; see the exercises.

9.2.4. A Sample kshdb Session

Now we'll show a transcript of an actual session with kshdb, in which the guinea pig is (a slightly modified version of) the solution to Task 6-3. For convenience, here is a numbered listing of the script, which we'll call lscol.

 1    set -A filenames $(ls $1)
 2    typeset -L14 fname
 3    let numfiles=${#filenames[*]}
 4    let numcols=5
 5
 6    for ((count = 0; $count < $numfiles ; )); do
 7        fname=${filenames[count]}
 8        print -n "$fname  "
 9        let count++
10        if (( count % numcols == 0 )); then
11            print           # newline
12        fi
13    done
14    
15    if (( count % numcols != 0 )); then
16        print
17    fi

Here is the kshdb session transcript:

$ kshdb lscol book
Korn Shell Debugger version 2.0 for ksh Version M 1993-12-28 m
Stopped at line 1
kshdb> #bp 4
Breakpoint at line  4
kshdb> #g
Reached line breakpoint at line 4
kshdb> #s
Stopped at line 6
kshdb> print $numcols
5
kshdb> #bc (( count == 10 ))
Break when true: (( count == 10 ))
kshdb> #g
appa.xml        appb.xml        appc.xml        appd.xml        appf.xml
book.xml        ch00.xml        ch01.xml        ch02.xml        ch03.xml
Break condition '(( count == 10 ))' true at line 10
kshdb> #bc
Break condition cleared.
kshdb> #bp newline
Breakpoint at next line containing 'newline'.
kshdb> #g
Reached string breakpoint at line 11
kshdb> print $count
10
kshdb> let count=9
kshdb> #g

ch03.xml        Reached string breakpoint at line 11
kshdb> #bp
Breakpoints at lines:
4
Breakpoints at strings:
newline
Break on condition:

kshdb> #g

ch04.xml        ch05.xml        ch06.xml        ch07.xml        ch08.xml
Reached string breakpoint at line 11
kshdb> #g

ch09.xml        ch10.xml        colo1.xml       copy.xml
$

First, notice that we gave the guinea pig script the argument book, meaning that we want to list the files in that directory. We begin by setting a simple breakpoint at line 4 and starting the script. It stops before executing line 4 (let numcols=5). We issue the #s command to single step through the command (i.e., to actually execute it). Then we issue a shell print command to show that the variable numcols is indeed set correctly.

Next, we set a break condition, telling the debugger to kick in when count is 10, and we resume execution. Sure enough, the guinea pig prints 10 filenames and stops at line 10, right after count is incremented. We clear the break condition by typing #bc without an argument, since otherwise the shell would stop after every statement until the condition becomes false.

The next command shows how the string breakpoint mechanism works. We tell the debugger to break when it hits a line that contains the string newline. This string is in a comment on line 11. Notice that it doesn't matter that the string is in a comment -- just that the line it's on contains an actual command. We resume execution, and the debugger hits the breakpoint at line 11.

After that, we show how we can use the debugger to change the guinea pig's state while running. We see that $count is still greater than 10; we change it to 9. In the next iteration of the while loop, the script accesses the same filename that it just did (ch03.xml), increments count back to 10, and hits the string breakpoint again. Finally, we list breakpoints and step through to the end, at which point it exits.

9.2. A Korn Shell Debugger

9.2.1. Structure of the Debugger

9.2.1.1. The driver script

9.2.1.2. exec

9.2.2. The Preamble