Security Monitoring (TCP/IP Network Administration, 3rd Edition)

12.4. Security Monitoring

A key element of effective network security is security monitoring. Good security is an ongoing process, and following the security guidelines discussed above is just the beginning. You must also monitor the systems to detect unauthorized user activity and to locate and close security holes. Over time, a system will change -- active accounts become inactive and file permissions are changed. You need to detect and fix these problems as they arise.

12.4.1. Know Your System

Network security is monitored by examining the files and logs of individual systems on the network. To detect unusual activity on a system, you must know what activity is normal. What processes are normally running? Who is usually logged in? Who commonly logs in after hours? You need to know this, and more, about your system in order to develop a "feel" for how things should be. Some common Unix commands -- ps and who -- can help you learn what normal activity is for your system.

The ps command displays the status of currently running processes. Run ps regularly to gain a clear picture of what processes run on the system at different times of the day and who runs them. The Linux ps -au command and the Solaris ps -ef command display the user and the command that initiated each process. This should be sufficient information to learn who runs what and when they run it. If you notice something unusual, investigate it. Make sure you understand how your system is being used.

The who command provides information about who is currently logged into your system. It displays who is logged in, what device they are using, when they logged in and, if applicable, what remote host they logged in from. (The w command, a variation of who available on some systems, also displays the currently active process started by each user.) The who command helps you learn who is usually logged in as well as what remote hosts they normally log in from. Investigate any variations from the norm.

If any of these routine checks gives you reason to suspect a security problem, examine the system for unusual or modified files, for files that you know should be there but aren't, and for unusual login activity. This close examination of the system can also be made using everyday Unix commands. Not every command or file we discuss will be available on every system. But every system will have some tools that help you keep a close eye on how your system is being used.

12.4.2. Looking for Trouble

Intruders often leave behind files or shell scripts to help them re-enter the system or gain root access. Use the ls -a | grep '^\'. command to check for files with names that begin with a dot (.). Intruders particularly favor names such as .mail, .xx, ... (dot, dot, dot), .. (dot, dot, space), or ..^G (dot, dot, Ctl-G).

If any files with names like these are found, suspect a break-in. (Remember that one directory named . and one directory named .. are in every directory except the root directory.) Examine the contents of any suspicious files and follow your normal incident-reporting procedures.

You should also examine certain key files if you suspect a security problem:

/etc/inetd.conf and /etc/xinetd.conf: Check the names of the programs started from the /etc/inetd.conf file or the /etc/xinetd.conf file if your system uses xinetd. In particular, make sure that it does not start any shell programs (e.g., /bin/csh). Also check the programs that are started by inetd or by xinetd to make sure the programs have not been modified. /etc/inetd.conf and /etc/xinetd.conf should not be world-writable.
r command security files: Check /etc/hosts.equiv, /etc/hosts.lpd, and the .rhosts file in each user's home directory to make sure they have not been improperly modified. In particular, look for any plus sign (+) entries and any entries for hosts outside of your local trusted network. These files should not be world-writable. Better yet, remove the r commands from your system and make sure no one reinstalls them.
/etc/passwd: Make sure that the /etc/passwd file has not been modified. Look for new usernames and changes to the UID or GID of any account. /etc/passwd should not be world-writable.
Files run by cron or at: Check all of the files run by cron or at, looking for new files or unexplained changes. Sometimes intruders use procedures run by cron or at to readmit themselves to the system, even after they have been kicked off.
Executable files: Check all executable files, binaries, and shell files to make sure they have not been modified by the intruder. Executable files should not be world-writable.

If you find or even suspect a problem, follow your reporting procedure and let people know about the problem. This is particularly important if you are connected to a local area network. A problem on your system could spread to other systems on the network.

12.4.2.1. Checking files

The find command is a powerful tool for detecting potential filesystem security problems because it can search the entire filesystem for files based on file permissions. Intruders often leave behind setuid programs to grant themselves root access. The following command searches for these files recursively, starting from the root directory:

# find / -user root -perm -4000 -print

This find command starts searching at the root (/) for files owned by the user root (-user root) that have the setuid permission bit set (-perm -4000). All matches found are displayed at the terminal (-print). If any filenames are displayed by find, closely examine the individual files to make sure that these permissions are correct. As a general rule, shell scripts should not have setuid permission.

You can use the find command to check for other problems that might open security holes for intruders. The other common problems that find checks for are world-writable files (-perm -2), setgid files (-perm -2000), and unowned files (-nouser -o -nogroup). World-writable and setgid files should be checked to make sure that these permissions are appropriate. As a general rule, files with names beginning with a dot (.) should not be world-writable, and setgid permission, like setuid, should be avoided for shell scripts.

The process of scanning the filesystem can be automated with the Tripwire program. A commercially supported version of Tripwire is available from http://www.tripwiresecurity.com, and an open source version for Linux is available from http://www.tripwire.org. This package not only scans the filesystem for problems, it computes digital signatures to ensure that if any files are changed, the changes will be detected.

12.4.2.2. Checking login activity

Strange login activity (at odd times of the day or from unfamiliar locations) can indicate attempts by intruders to gain access to your system. We have already used the who command to check who is currently logged into the system. To check who has logged into the system in the past, use the last command.

The last command displays the contents of the wtmp file.[135] It is useful for learning normal login patterns and detecting abnormal login activity. The wtmp file keeps a historical record of who logged into the system, when they logged in, what remote site they logged in from, and when they logged out.

[135]This file is frequently stored in /usr/adm, /var/log, or /etc.

Figure 12-3 shows a single line of last command output. The figure highlights the fields that show the user who logged in, the device, the remote location from which the login originated (if applicable), the day, the date, the time logged in, the time logged out (if applicable), and the elapsed time.

Figure 12-3. Last command output

Simply typing last produces a large amount of output because every login stored in wtmp is displayed. To limit the output, specify a username or tty device on the command line. This limits the display to entries for the specified username or terminal. It is also useful to use grep to search last's output for certain conditions. For example, the command below checks for logins that occur on Saturday or Sunday:

% last | grep 'S[au]' | more 
craig     console     :0            Sun Dec 15 10:33   still logged in 
reboot    system boot               Sat Dec 14 18:12 
root      console                   Sat Dec 14 18:14 
craig     pts/5       jerboas       Sat Dec 14 17:11 - 17:43  (00:32) 
craig     pts/2       172.16.12.24  Sun Dec  8 21:47 - 21:52  (00:05) 
       . 
       . 
--More--

The next example searches for root logins not originating from the console. If you don't know who made the two logins reported in this example, be suspicious:

% last root | grep -v console 
root   pts/5   rodent.wrotethebook.com   Tue Oct 29 13:12 - down  (00:03)
root   ftp     crab.wrotethebook.com     Tue Sep 10 16:37 - 16:38  (00:00)

The last command is a major source of information about previous login activity. User logins at odd times or from odd places are suspicious. Remote root logins should always be discouraged. Use last to check for these problems.

Report any security problems that you detect, or even suspect. Don't be embarrassed to report a problem because it might turn out to be a false alarm. Don't keep quiet because you might get "blamed" for the security breach. Your silence will only help the intruder.

12.4.3. Automated Monitoring

Manually monitoring your system is time consuming and prone to errors and omissions. Fortunately, several automated monitoring tools are available. At this writing, the web site http://www.insecure.com lists the monitoring tools that are currently most popular. Tripwire (mentioned earlier) is one of them. Some other currently popular tools are:

Nessus: Nessus is a network-based security scanner that uses a client/server architecture. Nessus scans target systems for a wide range of known security problems.
SATAN: Security Auditing Tool for Analyzing Networks is the first network-based security scanner that became widely distributed. Somewhat outdated, it is still popular and can detect a wide range of known security problems. SATAN has spawned some children, SAINT and SARA, that are also popular.
SAINT: System Administrator's Integrated Network Tool scans systems for a wide range of known security problems. SAINT is based on SATAN.
SARA: Security Auditor's Research Assistant is the third-generation security scanner based on SATAN and SAINT. SARA detects a wide range of known security problems.
Whisker: Whisker is a security scanner that is particularly effective at detecting certain CGI script problems that threaten web site security.
ISS: Internet Security Scanner is a commercial security scanner for those who prefer a commercial product.
Cybercop: Cybercop is another commercial security scanner for those who prefer commercial products.
Snort: Snort provides a rule-based system for logging packets. Snort attempts to detect intrusions and report them to the administrator in real time.
PortSentry: PortSentry detects port scans and can, in real time, block the system initiating the scan. Port scans often precede a full-blown security attack.

The biggest problem with security scanners and intrusion detection tools is that they rapidly become outdated. New attacks emerge that the tools are not equipped to detect. For this reason, this book does not spend time describing the details of any specific scanner. These are the currently popular scanners. By the time you read this, new security tools or new versions of these tools may have taken their place. Use this list as a starting point to search the Web for the latest security tools.

Well-informed users and administrators, good password security, and good system monitoring are the foundation of network security. But more is needed. That "more" is some technique for controlling access to the systems connected to the network, or for controlling access to the data the network carries. In the remainder of this chapter, we look at various security techniques that control access.