Chapter 9. Log Files

If this weren't a book on system administration, an entire chapter on log files would seem peculiar. But system administrators have a very special relationship with log files. Like Doctor Doolittle, who could talk to the animals, system administrators are expected to be able to communicate with a large menagerie of software and hardware. Much of this communication takes place through log files, so we become log file linguists. Perl can be a big help in this process.

It is impossible to touch on all of the different kinds of processing and analysis you can do with logs. Entire books have been devoted to just statistical analysis of this sort of data. However, this chapter should give you some general approaches to the topic and Perl tools to whet your appetite for more.

9.1. Text Logs

Logs come in different flavors, so we need several approaches for dealing with them. The most common type of log file is one composed entirely of lines of text. Popular server packages like Apache (web), INN (Usenet news), and Sendmail (email) spew log text in voluminous quantities. Most logs on Unix machines look similar because they are created by a centralized logging facility known as syslog. For our purposes, we can treat files created by syslog like any other text file.

Here's a simple Perl program to scan for the word "error" in a text-based log file:

open(LOG,"logfile") or die "Unable to open logfile:$!\n";
while(<LOG>){
	print if /\berror\b/i;
}
close(LOG);

Perl-savvy readers are probably itching to turn it into a one-liner. For those folks:

perl -ne 'print if /\berror\b/i' logfile

Chapter 9. Log Files

Contents:

9.1. Text Logs