 |
» |
|
|
|
Log forwarding is a feature of the standard UNIX syslogd. In addition to logging messages to the local host's log files, syslogd can forward log messages to one or more remote
systems. These systems are referred to as log sinks or log consolidation
servers. Log consolidation offers benefits such as the
following: Easier log file analysis
- The centralized log provides a single location for the administrator
to perform log file analysis. It offers single view of events that
impact multiple systems. Increased security - A
security breach might compromise the local logs but not the centralized
copy. The log consolidation system can be hardened in ways that are
likely to be inappropriate for log forwarding clients. Simplified archiving of
logs - It is sometimes simpler to archive a set of centralized logs
rather than per-system logs.
There are several disadvantages of using the standard syslogd on a log consolidation server:
syslogd supports forwarding using
UDP only. The Universal Datagram Protocol (UDP) is a "connectionless"
protocol and does not offer flow control or guaranteed delivery of
messages. As such, it is possible for forwarded log messages to be
lost. The filtering features of syslogd are quite simple: you can filter only on a message’s facility
and priority. A log consolidation system represents a single point
of failure. If the system is unavailable, the messages forwarded from
clients are lost. Note that the messages still exist on the individual
client systems. They are lost only from the consolidated log.
Improved Log Consolidation | |
The Distributed Systems Administration Utilities
(DSAU) use syslog-ng, or syslog “Next Generation,” to address the weaknesses of the
traditional syslogd mentioned above. syslog-ng is an open source syslogd replacement. It performs all the functions of the
standard syslogd in addition to providing features
such as the following: Improved filtering functionality. In addition to syslog's facility/priority level filtering, syslog-ng can perform regular expression filtering against
the program name, hostname, text of the message itself, the sender's
IP address, and so on.
Log rotation based on output filenames - Log output
filenames can be based on templates names which support macro expansion.
For example, if the output filename template contains the month macro,
a new filename will created each month. Launching programs - A message can trigger a program
to be launched, sending the message to its standard input. Log forwarding for arbitrary text-based logs - In
conjunction with DSAU's clog_tail tool, syslog-ng can be used to forward and consolidate arbitrary
text-based application log files such as Serviceguard’s package
log files.
syslog Co-existence | |
The Distributed Systems Administration Utilities
configures syslog-ng to co-exist and work alongside
the standard syslogd. syslogd continues
to handle all the local logging for the system. syslog-ng is used when forwarding messages to a log consolidation system and
is used on the log consolidator to receive and filter messages.
The following diagrams illustrate the relationship between syslogd and syslog-ng. Figure 3-1 depicts the configuration
on a syslog-ng client system that is forwarding
logs to a remote log consolidation server.
The grey area represents standard syslogd operation. Applications such as Serviceguard’s cmcld daemon call syslog (see syslog(3C)) to send messages to syslogd. syslog writes messages to the local system’s /var/adm/syslog/syslog.log and related
files. Applications also frequently have application-specific log
files. In this example, Serviceguard maintains a log of package operations
in /etc/cmcluster/package-name/package-name.log. The clog_tail daemon of DSAU, labeled
“Log reader” in the diagram, monitors text-based logs
and sends new log lines to syslog-ng for processing.
In a Serviceguard cluster, clog_tail defaults to
monitoring all the package logs. The log_reader sends all new log
messages to a named pipe (log_consolidation_fifo), which is one of the log sources for syslog-ng. The syslog-ng reads any new data
from the named pipe and forwards it to the log consolidation server. The local syslogd, in addition to writing log messages to the local /var/adm/syslog/syslog.log, is configured to additionally forward all messages to the local
instance of syslog-ng. syslog-ng in turn, forwards these messages to the log consolidator. The administrator
can choose to use UDP, TCP, or TCP with ssh when
forwarding messages.
Figure 3-2 illustrates
the configuration on the log consolidation server.
The syslog-ng server reads the incoming log data from the UDP or TCP connected
clients. Note: gray arrows indicate a read operation; black arrows,
a write. The grey area is identical
to the client configuration in Figure 3-1: “syslog-ng Log-Forwarding Configuration”. In terms of the local system, syslog-ng acts as a client and processes locally forwarded syslog messages
and clog_tail messages. The syslog-ng server processes all messages and filters them into the appropriate
consolidated log files. In this specific example, the administrator
has created a filesystem named “/clog”
to house the consolidated logs. /clog/syslog/ would contain the consolidated syslog-related
file. /clog/packages would contain consolidated
package logs for a Serviceguard cluster.
|
Printable version
