Modular Authentication for Unix (Building Internet Firewalls, 2nd Edition)

21.4. Modular Authentication for Unix

Traditionally, programs wishing to authenticate a user (such as the login program or the ftpd daemon) have had to know how to authenticate a user; they have had to understand and implement whatever authentication method or methods were to be used. In a Unix system, this means that these programs have to do all of the following to authenticate a user:

Prompt the user for a login name.
Look up that login name and obtain its encrypted password.
Prompt the user for a password.
Use the user-provided password and the first two characters from the encrypted password to encrypt a known string (eight bytes of nulls).
Check to see if the result of this encryption matches the encrypted password for the user.

If you want to add a second authentication mechanism (for example, the S/Key mechanism, which we discussed earlier), you have to modify all of these programs to understand this second mechanism as well as, or instead of, the standard Unix password mechanism. And if you later want to add a third authentication mechanism (for example, support for the SecurID cards), you have to modify the programs yet again; and so it would go for each additional authentication mechanism. Each time you modify these programs, you're making them bigger and more complex and increase the chances that you've introduced some kind of bug that's going to result in a security problem. (This is a serious risk because these are very security-critical programs -- they control access to your system.)

odular authentication takes a different approach. With it, you modify all the authenticating programs (e.g., login, ftpd) once, to make them talk to an authentication service instead of doing the authentication themselves. All of the details of the authentication mechanism -- for example, what to prompt the user with, how to validate the user's response, etc. -- are then handled by the authentication service. When you want to add or modify authentication methods, you do so by changing the authentication service (which is modular and designed to accommodate such changes), not by changing the individual authenticating programs.

21.4.1. The TIS FWTK Authentication Server

The authentication server in TIS FWTK is a modular solution for authenticating users coming in from the Internet. The server implements a variety of authentication mechanisms, such as standard reusable passwords (not recommended), S/Key, Security Dynamics SecurID cards, and Digital Pathways SNK-004 cards. In addition, the server is modular and extensible, and is designed so that new authentication mechanisms can easily be integrated.

A single authentication server can handle any number of client machines and programs, and any number of different authentication methods; different users within the same server can use different authentication methods. For example, some might use S/Key while some might use the Digital Pathways SNK-004 cards.

When a client program (such as login or ftpd) wishes to authenticate someone using the TIS FWTK authentication server, it has to go through the following steps:

Prompt the user for a login name.
Contact the authentication server and tell it who is trying to log in.
Receive a response from the authentication server that tells it what to prompt the user with.
Display the prompt specified by the authentication server.
Collect the user's response and send it to the authentication server.
Receive either an OK or an error message from the authentication server.
Allow the user access (if OK) or display the error message.

This whole process is carried out with a single TCP connection between the client and the authentication server, so that the server knows it's talking to the same client and the client knows it's talking to the same server throughout the authentication process.

The authentication server consults its databases to determine how to authenticate that user and determines the appropriate prompt for the authentication mechanism for that user. For example:

If traditional passwords are being used as the authentication method, the prompt will be a simple "Password:" prompt.
If the authentication method is S/Key, the prompt will be the number of the key the user is to respond with.
If the authentication method is the Digital Pathways SNK-004 card, the prompt will be a randomly generated challenge number.

Figure 21-2 shows how the TIS FWTK authentication server works.

Figure 21-2. How the TIS FWTK authentication server works

TIS FWTK includes a number of programs (such as ftpd) that, in addition to other modifications and enhancements for security, have already been modified to use the authentication server. Converting an existing program to use the authentication server, rather than traditional Unix passwords, is pretty straightforward. It typically involves only 20 or so lines of C code, examples of which are given in the toolkit.

The toolkit also includes some programs to support binary-only systems where you don't have the source to modify. For example, for systems in which you don't have the source code to the login program available for modification, the toolkit includes a program you can use as the user's shell (which is specified for each user in the /etc/passwd file) instead of one of the normal shells (e.g., /bin/csh or /bin/sh) This replacement shell authenticates the user with the authentication server, and if the user passes, starts his or her real shell.

21.4.1.1. Problems with the authentication server

The major problem in running an authentication server is getting secure communication between the client and the server. An attacker who can convincingly pretend to be the authentication server can authenticate as anybody.

Some configurations may have additional problems; for example, using shell replacement can produce problems because not all programs deal well with situations in which a user's shell environment variable and the entry for that user in the /etc/passwd file do not match.

21.4.2. Pluggable Authentication Modules (PAM)

PAM is a more recent system for providing modular authentication, which uses an even more flexible mechanism than the TIS FWTK authentication server. The basic principle is much the same; programs that need to do authentication are modified to use the PAM system, after which you can completely control what kind of authentication they use through PAM, without making further changes to the individual programs. PAM differs from the authentication server in that it is a local service; PAM operates by dynamically loading authentication modules into applications that have been configured to use them. Authentication modules execute with the same privileges as the application that loaded them.

A program can ask PAM to authenticate a user or to change a password. The password change feature saves users from having to learn a new password change mechanism for each new authentication mechanism.

An administrator can configure PAM to do extra work in addition to authenticating the user. For instance, it can grant or deny access based on criteria other than whether or not the user authenticated properly. This allows you to control what time of day people can use a service, or how many simultaneous users there can be, or whether or not they're accessible from the machine's console. PAM can also set up the environment that the service will execute in, by setting environment variables or by running programs (for instance, it can use chroot to limit the parts of the filesystem that the service has access to).

Authentication modules can be stacked up, either to allow for fail-over (if you can't authenticate one way, try another) or to enforce complicated rules (allow George to log in only at the console and only between 3:00 and 5:00 in the morning, but allow everybody else to log in from the local network with reusable passwords, or from the Internet with a one-time password). It's also easy to add new authentication modules, but you should use new modules with caution. When you trust a piece of code to authenticate users, you are trusting it with the security of your system; that's not something you should do lightly.

PAM is an extraordinarily flexible system, and as a natural consequence, it can be hard to configure. Configuration errors will often result in an unusable computer, but it's also easy to accidentally produce a configuration that has unexpected insecurities. The module-stacking features, for instance, provide any number of ways to accidentally allow access that you intended to deny. Effectively, they give you an entire programming language with obscure and varying syntax, and an error as simple as using the keyword "required" instead of "requisite" can have unfortunate consequences.

Like any powerful tool, PAM should be used with caution:

Because relatively minor errors have the potential to disable all access to the machine, you should know how to boot your machine into single-user mode before you start to configure PAM. Access to single-user mode is not controlled by PAM, so it will give you a way to re-enable access to the rest of the machine.

Because all of PAM's files are security-critical (an intruder can easily install back doors by modifying any part of PAM), you should use it in conjunction with a checksumming system that will detect unauthorized changes (see Chapter 11, "Unix and Linux Bastion Hosts", for more information on such systems).
Because PAM configuration is tricky, you should test your configurations before relying on them. When you are deciding how to configure a service, make a list of the important cases when it should allow access and when it should deny access. Once you have written the configuration, test each of these cases to make sure there are no surprises.

In most installations, PAM will allow you to use one single configuration file that contains the configuration for all services, or a directory with a separate file for each service. You should use a separate file for each service; it will help keep configuration errors for one service from propagating to others.

No matter how you set up the configuration files, you will need to specify a default behavior for the service called "other", which will be used if there is no specific information for a PAM-enabled service. You should set this up so that it denies access and logs this fact; this way, if you accidentally install a PAM-enabled service without installing configuration information for it, you will not grant access to it. Of course, if you accidentally remove the configuration for an existing service, you will turn off all access to it. This is unpleasant, but not as unpleasant as a compromised machine.