6.8.2.4.1. Writing configuration files

If your configuration file contains only a few variables, it doesn't matter how you write the file. In practice, however, configuration files often grow as a project develops. This is especially true for projects that generate HTML files, since they tend to demand many easily configurable settings, such as the location of headers, footers, templates, colors, and so on.

$cgi_dir  = '/home/httpd/perl';
$cgi_url  = '/perl';
$docs_dir = '/home/httpd/docs';
$docs_url = '/';
$img_dir  = '/home/httpd/docs/images';
$img_url  = '/images';
# ... many more config params here ...
$color_hint   = '#777777';
$color_warn   = '#990066';
$color_normal = '#000000';

The use strict; pragma demands that all variables be declared. When using these variables in a mod_perl script, we must declare them with use vars in the script, so we start the script with:

use strict;
use vars qw($cgi_dir $cgi_url $docs_dir $docs_url 
            # ... many more config params here ....
            $color_hint  $color_warn $color_normal
           );

It is a nightmare to maintain such a script, especially if not all features have been coded yet—we have to keep adding and removing variable names. Since we're writing clean code, we also start the configuration file with use strict;, so we have to list the variables with use vars here as well—a second list of variables to maintain. Then, as we write many different scripts, we may get name collisions between configuration files.

package Book::Config0;

A script that uses this package will start with this code:

use strict;
use Book::Config0 qw($cgi_dir $cgi_url $docs_dir $docs_url 
                     # ... many more config params here ....
                     $color_hint  $color_warn $color_normal
                  );
use vars          qw($cgi_dir $cgi_url $docs_dir $docs_url 
                     # ... many more config params here ....
                     $color_hint  $color_warn $color_normal
                  );

Whoa! We now have to update at least three variable lists when we make a change in naming of the configuration variables. And we have only one script using the configuration file, whereas a real-life application often contains many different scripts.

There are a number of techniques we can use to get rid of these problems. First, variables can be grouped in named groups called tags. The tags are later used as arguments to the import( ) or use( ) calls. You are probably familiar with:

use CGI qw(:standard :html);

We can implement this quite easily, with the help of export_ok_tags( ) from Exporter. For example:

BEGIN {
  use Exporter ( );
  use vars qw( @ISA @EXPORT @EXPORT_OK %EXPORT_TAGS );
  @ISA         = qw(Exporter);
  @EXPORT      = ( );
  @EXPORT_OK   = ( );

  %EXPORT_TAGS = (
      vars => [qw($firstname $surname)],
      subs => [qw(reread_conf untaint_path)],
  );
  Exporter::export_ok_tags('vars');
  Exporter::export_ok_tags('subs');
}

In the script using this configuration, we write:

use Book::Config0 qw(:subs :vars);

use Book::Config0 ( );

You can also go even further and group tags into other named groups. For example, the :all tag from CGI.pm is a group tag of all other groups. It requires a little more effort to implement, but you can always save time by looking at the solution in CGI.pm's code. It's just a matter of an extra code to expand all the groups recursively.

Good Perl style suggests keeping a comma at the end of each list. This makes it easy to add new items at the end of a list.

Our script now looks like this:

use strict;
use Book::Config1 qw(%c);
use vars          qw(%c);
print "Content-type: text/plain\n\n";
print "My url docs root: $c{url}{docs}\n";

The whole mess is gone. Now there is only one variable to worry about.

The one small downside to this approach is auto-vivification. For example, if we write $c{url}{doc} by mistake, Perl will silently create this element for us with the value undef. When we use strict;, Perl will tell us about any misspelling of this kind for a simple scalar, but this check is not performed for hash elements. This puts the onus of responsibility back on us, since we must take greater care.

Our script is modified to use fully qualified names for the configuration variables it uses:

use strict;
use Book::Config2 ( );
print "Content-type: text/plain\n\n";
print "My url docs root: $Book::Config2::c{url}{docs}\n";

To save typing and spare the need to use fully qualified variable names, we'll use a magical Perl feature to alias the configuration variable to a script's variable:

use strict;
use Book::Config2 ( );
use vars qw(%c);
*c = \%Book::Config2::c;
print "Content-type: text/plain\n\n";
print "My url docs root: $c{url}{docs}\n";

We've aliased the *c glob with a reference to the configuration hash. From now on, %Book::Config2::c and %c refer to the same hash for all practical purposes.

One last point: often, redundancy is introduced in configuration variables. Consider:

$cgi_dir  = '/home/httpd/perl';
$docs_dir = '/home/httpd/docs';
$img_dir  = '/home/httpd/docs/images';

It's obvious that the base path /home/httpd should be moved to a separate variable, so only that variable needs to be changed if the application is moved to another location on the filesystem.

$base     = '/home/httpd';
$cgi_dir  = "$base/perl";
$docs_dir = "$base/docs";
$img_dir  = "$docs_dir/images";

This cannot be done with a hash, since we cannot refer to its values before the definition is completed. That is, this will not work:

%c = (
   base => '/home/httpd',
   dir => {
       cgi  => "$c{base}/perl",
       docs => "$c{base}/docs",
       img  => "$c{base}{docs}/images",
      },
  );

But nothing stops us from adding additional variables that are lexically scoped with my( ). The following code is correct:

my $base = '/home/httpd';
%c = (
   dir => {
       cgi  => "$base/perl",
       docs => "$base/docs",
       img  => "$base/docs/images",
      },
  );

6.8.2.4.2. Reloading configuration files

use CGI ( );
use strict;

my $firstname = "Jarkko";
my $surname = "Hietaniemi";
my $q = CGI->new;

print $q->header(-type=>'text/html');
print $q->p("$firstname $surname holds the patch pumpkin" .
             "for this Perl release.");

The script is very simple: it initializes the CGI object, prints the proper HTTP header, and tells the world who the current patch pumpkin is. The name of the patch pumpkin is a hardcoded value.

We don't want to modify the script every time the patch pumpkin changes, so we put the $firstname and $surname variables into a configuration file:

$firstname = "Jarkko";
$surname = "Hietaniemi";
1;

Note that there is no package declaration in the above file, so the code will be evaluated in the caller's package or in the main:: package if none was declared. This means that the variables $firstname and $surname will override (or initialize) the variables with the same names in the caller's namespace. This works for global variables only—you cannot update variables defined lexically (with my( )) using this technique.

Let's say we have started the server and everything is working properly. After a while, we decide to modify the configuration. How do we let our running server know that the configuration was modified without restarting it? Remember, we are in production, and a server restart can be quite expensive. One of the simplest solutions is to poll the file's modification time by calling stat( ) before the script starts to do real work. If we see that the file was updated, we can force a reconfiguration of the variables located in this file. We will call the function that reloads the configuration reread_conf( ) and have it accept the relative path to the configuration file as its single argument.

Apache::Registry executes a chdir( ) to the script's directory before it starts the script's execution. So if your CGI script is invoked under the Apache::Registry handler, you can put the configuration file in the same directory as the script. Alternatively, you can put the file in a directory below that and use a path relative to the script directory. However, you have to make sure that the file will be found, somehow. Be aware that do( )searches the libraries in the directories in @INC.

use vars qw(%MODIFIED);
sub reread_conf {
    my $file = shift;
    return unless defined $file;
    return unless -e $file and -r _;
    my $mod = -M _;
    unless (exists $MODIFIED{$file} and $MODIFIED{$file} =  = $mod) {
        unless (my $result = do $file) {
            warn "couldn't parse $file: $@" if $@;
            warn "couldn't read $file: $!" unless defined $result;
            warn "couldn't run $file"      unless         $result;
        }
        $MODIFIED{$file} = $mod; # Update the MODIFICATION times
    }
}

Notice that we use the = = comparison operator when checking the file's modification timestamp, because all we want to know is whether the file was changed or not.

When the require( ), use( ), and do( ) operators successfully return, the file that was passed as an argument is inserted into %INC. The hash element key is the name of the file, and the element's value is the file's path. When Perl sees require( ) or use( ) in the code, it first tests %INC to see whether the file is already there and thus loaded. If the test returns true, Perl saves the overhead of code rereading and recompiling; however, calling do( ) will load or reload the file regardless of whether it has been previously loaded.

We use do( ), not require( ), to reload the code in this file because although do( ) behaves almost identically to require( ), it reloads the file unconditionally. If do( ) cannot read the file, it returns undef and sets $! to report the error. If do( ) can read the file but cannot compile it, it returns undef and sets an error message in $@. If the file is successfully compiled, do( ) returns the value of the last expression evaluated.

The configuration file can be broken if someone has incorrectly modified it. Since we don't want the whole service using that file to be broken that easily, we trap the possible failure to do( ) the file and ignore the changes by resetting the modification time. If do( ) fails to load the file, it might be a good idea to send an email about the problem to the system administrator.

However, since do( ) updates %INC like require( ) does, if you are using Apache::StatINC it will attempt to reload this file before the reread_conf( ) call. If the file doesn't compile, the request will be aborted. Apache::StatINC shouldn't be used in production anyway (because it slows things down by stat( ) ing all the files listed in %INC), so this shouldn't be a problem.

Note that we assume that the entire purpose of this function is to reload the configuration if it was changed. This is fail-safe, because if something goes wrong we just return without modifying the server configuration. The script should not be used to initialize the variables on its first invocation. To do that, you would need to replace each occurrence of return( ) and warn( ) with die( ).

We've used the above approach with a huge configuration file that was loaded only at server startup and another little configuration file that included only a few variables that could be updated by hand or through the web interface. Those variables were initialized in the main configuration file. If the webmaster breaks the syntax of this dynamic file while updating it by hand, it won't affect the main (write-protected) configuration file and won't stop the proper execution of the programs. In the next section, we will see a simple web interface that allows us to modify the configuration file without the risk of breaking it.

You should be using (stat $file)[9] instead of -M $file if you are modifying the $^T variable. This is because -M returns the modification time relative to the Perl interpreter startup time, set in $^T. In some scripts, it can be useful to reset $^T to the time of the script invocation with "local $^T = time( )". That way, -M and other -X file status tests are performed relative to the script invocation time, not the time the process was started.

6.8.2.4.3. Dynamically updating configuration files

We want to make the variables name, release, and comments dynamically configurable. We'll need a web interface with an input form that allows modifications to these variables. We'll also need to update the configuration file and propagate the changes to all the currently running processes.

Let's look at the main stages of the implementation:

1. Create a form with preset current values of the variables.

2. Let the administrator modify the variables and submit the changes.

3. Validate the submitted information (numeric fields should hold numbers within a given range, etc.).

4. Update the configuration file.

5. Update the modified value in the current process's memory.

6. Display the form as before with the (possibly changed) current values.

The only part that seems hard to implement is a configuration file update, for a couple of reasons. If updating the file breaks it, the whole service won't work. If the file is very big and includes comments and complex data structures, parsing the file can be quite a challenge.

So let's simplify the task. If all we want is to update a few variables, why don't we create a tiny configuration file containing just those variables? It can be modified through the web interface and overwritten each time there is something to be changed, so that we don't have to parse the file before updating it. If the main configuration file is changed, we don't care, because we don't depend on it any more.

The dynamically updated variables will be duplicated in the main file and the dynamic file. We do this to simplify maintenance. When a new release is installed, the dynamic configuration file won't exist—it will be created only after the first update. As we just saw, the only change in the main code is to add a snippet to load this file if it exists and was changed.

For example, on July 19 2002, Perl 5.8.0 was released. On that date, Jarkko Hietaniemi exclaimed:

The pumpking is dead! Long live the pumpking!

Hugo van der Sanden is the new pumpking for Perl 5.10. Therefore, we run manage_conf.pl and update the data. Once updated, the script overwrites the previous config.pl file with the following content:

$c{release}  =  '5.10';

$c{name}  =  'Hugo van der Sanden';

$c{comments}  =  'Perl rules the world!';

1;