Debugging Tools (CGI Programming with Perl)

15.3. Debugging Tools

We've looked at what can cause common errors, but not everything is a common problem. If you are having problems and none of the earlier solutions helps, then you need to do some investigative work. In this section, we'll look at some tools to help you uncover the source of the problem. Briefly, here is an outline of the steps you can take:

Check the syntax of your scripts with the -c flag.
Check the web server's error logs.
Run your script from the command line.
Test the value of variables by dumping them to the browser.
Use an interactive debugger.

Let's review each in more detail.

15.3.1. Check Syntax

We mentioned this within one of the sections above, but it bears repeating again in its own section: if your code does not parse or compile, then it will never run correctly. So get in the habit of testing your scripts with the -c flag from the command line before you test them in the browser, and while you're add it, have it check for warnings too with the -w flag. Remember, if you use taint mode (and you are using taint mode with all of your scripts, right?), you also need to pass the -T flag to avoid the following error:

$ perl -wc myScript.cgi
Too late for "-T" option.

Therefore, use the -wcT combination:

perl -wcT calendar.cgi

This will either return:

Syntax OK

or a list of problems. Of course you should only use the -c flag from the command line and not add it to the pound-bang line in your scripts.

15.3.2. Check Error Logs

Typically, errors are printed to STDERR, and on some web servers anything that is printed to STDERR while a CGI script is running ends up in your server's error logs. Thus, you can often find all sorts of useful clues by scanning these logs when you have problems. Possible locations of this file with Apache are /usr/local/apache/logs/error_log or /usr/var/logs/httpd/error_log. Errors are appended to the bottom; you may want to watch the log as you test your CGI script. If you use the tail command with a -f option:

$ tail -f /usr/local/apache/logs/error_log

it will print new lines as they are written to the file.

15.3.3. Running Scripts from the Command Line

Once your scripts pass a syntax check, the next step is to try to run them from the command line. Remember that because CGI scripts receive much of their data from environment variables, you can set these manually yourself before you run your script:

$ export HTTP_COOKIE="user_id=abc123"
$ export QUERY_STRING="month=jan&year=2001"
$ export REQUEST_METHOD="GET"
$ ./calendar.cgi

You will see the full output of your script including any headers you print. This can be quite useful if you suspect your problem has to do with the headers you are sending.

If you are using Version 2.56 or previous of CGI.pm, it makes accepting form parameters much easier, by prompting for them when you run your script:

(offline mode: enter name=value pairs on standard input)

You can then enter parameters as name-value pairs separated by an equals sign. CGI.pm ignores whitespace and allows you to use quotes:

(offline mode: enter name=value pairs on standard input)
month = jan
year=2001

When you are finished, press the end-of-file character on your system (use Ctrl-D on Unix or Mac; use Ctrl-Z on Windows).

As of 2.57, CGI.pm no longer automatically prompts for values. Instead, you can pass parameters as arguments to your script (this works for previous versions, too):

$ ./calendar.cgi month=jan year=2001

If you prefer to have CGI.pm prompt you for input instead, you can still enable this in later versions by using the -debug argument with CGI.pm:

use CGI qw( -debug );

If you are working with a complex form, and it is too much work to manually enter parameters, then you can capture the parameters to a file to use offline by adding a few lines to the top of your script:

#!/usr/bin/perl -wT

use strict;
use CGI;

my $q = new CGI;

## BEGIN INSERTED CODE
open FILE, "> /tmp/query1" or die $!;
$q->save( \*FILE );
print $q->header( "text/plain" ), "File saved\n";
## END INSERTED CODE
.
.

Now you should have a file saved to /tmp/query1 which you can use from the command line. Remove the inserted code first (or comment it out for future use), then you can use the query file like this:

$ ./catalog.cgi < /tmp/query1

15.3.4. Dumping Variables

If you script runs correctly but it does not do what you expect, then you need to break it down into chunks to determine where it is failing. The simplest way to do this is to include a handful of print statements:

sub fetch_results {
print "Entering fetch_results( @_ )\n"; #DEBUG#
    .
    .

You may want to outdent these commands or place comments at the end so that it is easy to find and remove them when you are done.

If you are working with a complex Perl data structure, you can print it quite easily by using the Data::Dumper module. Simply add code like the following:

.
    .
use Data::Dumper;        #DEBUG#
print Dumper( $result ); #DEBUG#
    return $result;
}

The Dumper function will serialize your data structure into neatly indented Perl source code. If you want to look at this within an HTML page, be sure to enclose it within <PRE> tags or view the page source.

If you are outputting complex HTML, you may need to view the source in order to see whether your statements printed. It is often much easier to open a separate filehandle to your own log file and print your debugging commands there. In fact, you may want to develop your own module that provides a way to send debugging output to a common debug log file as well as a simple way to turn debugging mode on and off.

15.3.5. Debuggers

All the previous strategies help isolate bugs, but the best solution by far is to use debuggers. Debuggers allow you to interact with your program as it runs. You can monitor the program flow, watch the value of variables, and more.

15.3.5.1. The Perl debugger

If you invoke perl with the -d flag, you will end up in an interactive session. Unfortunately, this means that you can use the debugger only from the command line. This is not the traditional environment for CGI scripts, but it is not difficult to mimic the CGI environment, as we saw earlier. The best way to do this is to save a CGI object to a query file, initialize any additional environment variables you might need, such as cookies, and then run your CGI script like this:

$perl -dT calendar.cgi </tmp/query1

Loading DB routines from perl5db.pl version 1
Emacs support available.

Enter h or `h h' for help.

main::(Dev:Pseudo:7):	my $q = new CGI;
  DB<1>

The debugger can be intimidating at first, but it is very powerful. To help you get going, Table 15-1 shows a brief summary of all the basic commands you need to know to debug a script. You can debug all of your CGI scripts with just these commands, although there are many more features actually available. Practice walking through scripts that you know work in order to learn how to move around within the debugger. The debugger will not change your files, so you cannot damage a working script by typing a wrong command.

Complete documentation for the Perl debugger is available in the perldebug manpage, and a quick reference for the complete set of commands is available by typing h within the debugger.

Table 15-1. Basic Perl Debugger Commands

Command	Description
`s`	Step; Perl executes the line listed above the prompt, stepping into any subroutines; note that a line with multiple commands may take a few steps to evaluate.
`n`	Next; Perl executes the line listed above the prompt, stepping over any subroutines (they still run; Perl waits for them to finish before continuing).
`c`	Continue to the end of the program or the next break point, whichever comes first.
`c 123`	Continue up to line 123; line 123 must contain a command (it cannot be a comment, blank line, the second half of a command, etc.).
`b`	Set a breakpoint at current line; breakpoints halt execution caused by `c`.
`b 123`	Set a breakpoint at line 123; line 123 must contain a command (it cannot be a comment, blank line, the second half of a command, etc.).
`b my_sub`	Set a breakpoint at the first executable line of the `my_sub` sub.
`d`	Delete a breakpoint from the current line; takes same arguments as `b`.
`D`	Deletes all breakpoints.
`x $var`	Display the value of `$var` in list and scalar contexts; note that it will recurse down complex, nested data structures.
`r`	Return from the current sub; Perl finishes executing the current subroutine, displays the result, and continues at the next line after the sub.
`l`	List the next 10 lines of your script; this command can be used successively.
`l 123`	List line 123 of your script.
`l 200-300`	List lines 200 through 300 of your script.
`l my_sub`	List the first 10 lines of the `my_sub` sub.
`q`	Quit.
`R`	Restart the script in the debugger.

15.3.5.2. ptkdb

Another option is ptkdb (see Figure 15-1), the Perl/Tk debugger, which is available on CPAN as Devel-ptkdb. It allows you to debug your scripts with a graphical interface. It also allows you to debug your CGI interactively as they are running.

Figure 15-1. Debugging a CGI script with ptkdb

In order to use ptkdb, you need two things. First, you need access to an X Window server;[21] the X Window System is included with most Unix and compatible systems; commercial versions are available for other operating systems as well. Second, the web server must have Tk.pm module, available on CPAN, which requires Tk. Tk is a graphics toolkit that is typically distributed with the Tcl scripting language. You can obtain Tcl/Tk from http://www.scriptics.com/. For more information on using Perl with Tk via Tk.pm, refer to Learning Perl/Tk by Nancy Walsh (O'Reilly & Associates, Inc.).

[21]In the X Window System, you run an X Window server locally, which displays programs that you may execute remotely. The use of "server" in this context is sometimes confusing, since you typically use a client to interact with remote systems.

In order to debug a CGI script with ptkdb, begin your CGI scripts as follows:

#! /usr/bin/perl -d:ptkdb

sub BEGIN {
    $ENV{DISPLAY} = "your.machine.hostname:0.0" ;
}

You should replace your.machine.hostname with the hostname or IP address of your machine. You can use localhost if you are running an X Window session on the web server.

You also need to allow the web server to display programs on your X Window server. On Unix and compatible systems, you do so by adding the registering the hostname or IP address of the webserver with the xhost command:

$ xhost www.webserver.hostname
www.webserver.hostname being added to access control list

You can then access your CGI script via a browser, which should open a debugging window on your system. Note that your web browser may time out if you spend much time interacting with the debugger without your script producing output.

15.3.5.3. ActiveState Perl debugger

The final option is available only to Win32 users. ActiveState distributes a graphical Perl debugger with their Perl Development Kit (PDK), shown in Figure 15-2.

Figure 15-2. Debugging a CGI script with the ActiveState Perl debugger

Once installed, using the -d flag with perl invokes this debugger instead of the standard Perl debugger. It can also be invoked when running CGI scripts if you are logged into the web server.

You can obtain the PDK and corresponding documentation from ActiveState's web site at http://www.activestate.com/. The PDK is a commercial product, but as of the time this book was written, ActiveState offers a free seven-day trial.