10.5.1 At What Level Do You Encrypt?

Encryption over the Internet can take place at various levels; the most common are the application, link, and network levels.

Application-level encryption requires support in all the applications (both clients and servers) you want to use. This might be an effective approach if you have one or two applications that you use heavily (or are particularly concerned about) across the Internet among a small number of machines, because you can install custom, encrypting versions of those clients and servers on those machines. For general use, however, it doesn't scale up well; you may not even have source code available for all the applications you wish to operate encrypted. Some applications use PGP to provide application-level encryption. Senders use PGP to encrypt their outgoing mail before sending it across the Internet, and recipients use PGP to decrypt it. ( PGP is also frequently used independently of applications, which is less convenient for the user, but extremely flexible.)

Application-level encryption is done at too high a level to be useful as a blanket protection for a network link; you have to spend too much time and effort integrating support for it into too many different clients, servers, and procedures. However, application-level encryption may be useful if you only need a single application to work securely.

Link-level encryption protects only a single network link. For example, encryption in modems at either end of a leased line will protect your data as it traverses that leased line, but not elsewhere, as it traverses other lines, or passes through routers or other intermediate hosts. Link-level encryption is done at too low a level to be widely useful. Usually, you don't control all the links in the untrusted network between the source and the destination, so you can't really ensure that the link-level encryption is being done properly (or at all) at each of those points.

Network-level encryption seems to be a workable middle ground between application-level encryption and link-level encryption. With network-level encryption, all network traffic between two trusted sites is encrypted at one end, sent across the untrusted intermediate network, and then decrypted at the other end. The encryption and decryption is done by routers or other network devices at the perimeter of each trusted site. A firewall, which all traffic must pass through anyway, is thus a natural place to do network-level encryption. To machines within either trusted site, the traffic is unencrypted; this means that those machines don't require any custom applications or configuration to use or benefit from the encryption. To machines outside the trusted sites, the packets are encrypted, and thus are private (unintelligeable to those without the keys) and authenticated (could only have been sent by a key-holder).

10.5.2 What Do You Encrypt?

When you are performing network-level encryption, how much of the packet do you want to encrypt?

• Only the TCP , UDP , or ICMP data segments (leaving the TCP , UDP , or ICMP headers, as well as the IP headers, unencrypted)?

• The IP data segment (including the whole TCP , UDP , or ICMP packet)?

• The whole IP packet?

If you encrypt only the TCP , UDP , or ICMP data segments, you're protecting the data itself from compromise by an attacker, and you're making life easy for your packet filtering system (it can still see all the headers it needs to for normal packet filtering). However, a snooper can still see which machines are using which protocols to communicate with which other machines.

If you encrypt the IP data segment (which means that the whole UDP , TCP , or ICMP packet, headers and all, is encrypted), you prevent a snooper from seeing what protocols you're using (they can still see what hosts are talking to what other hosts), but you may make life harder for your own packet filtering system. Unless the packet filtering system is between the encryption unit and the internal network, it can no longer see the TCP , UDP , or ICMP headers, because they're encrypted as part of the IP data segment. If your packet filtering system is outside the encryption unit (between the encryption unit and the untrusted network), it would have to make all of its decisions strictly on the basis of IP source and destination addresses, which is very rarely enough.

If you encrypt the whole IP packet, you prevent an attacker from seeing anything, but you may also prevent your packet filtering system from seeing anything either, depending on where it's located relative to the encryption unit. To fully encrypt an IP packet between two sites, you have to provide some sort of encapsulation "tunnel" - e.g., a simple TCP connection - between the encryption units at the two sites to send the encrypted packets through. The reason the tunnel is necessary is because the IP headers are no longer there for the intermediate routers to look at. All an attacker can see is that the two encryption units are talking to each other. Some commercial routers, such as Morning Star Express routers (as well as UNIX machines running Morning Star PPP software) are capable of creating such "virtual private networks." Morning Star does it by running encrypted PPP over a TCP connection between two of their boxes; so you have your original packet encapsulated in an encrypted PPP packet, which is itself encapsulated in a TCP packet from one Morning Star box to the other.

Most sites using network-level encryption don't mind if attackers can determine which machines are talking to each other, or even what protocols they're using (this attack is commonly called traffic analysis ). Such sites generally encrypt only the TCP , UDP , or ICMP data segments of packets, for their own convenience in processing the packets. Sites that encrypt the whole TCP , UDP , or ICMP packet (the IP data segment) generally do it for performance reasons, rather than for security reasons: it's faster for a router to find the start of the IP data segment than for it to find the start of the IP data segment and, within that, the start of the TCP , UDP , or ICMP data segment.

10.5.3 Where Do You Encrypt?

If you're going to set up network-level encryption, the question of where you do the encryption and decryption relative to your packet filtering is an important one. If you do the encryption and decryption inside the packet filtering perimeter (i.e., on your internal net), then the filters just have to allow the encrypted packets in and out. This is especially easy if you're doing tunneling, because all the tunneled packets will be addressed to the same remote address and port number at the other end of the tunnel (the decryption unit). On the other hand, doing the encryption and decryption inside your filtering perimeter means that packets arriving encrypted are not subject to the scrutiny of the packet filters. This leaves you vulnerable to attack from the other site if that site has been compromised.

If you do the encryption and decryption outside the packet filtering perimeter (i.e., on your perimeter net or in your exterior router), then the packets coming in from the other site can be subjected to the full scrutiny of your packet filtering system. On the other hand, they can also be subjected to the full scrutiny of anyone who has broken into a machine on your perimeter net, such as your bastion host.

Encryption is starting to appear as a feature in some commercial router products, and several vendors have announced their intentions to add encryption. This trend will probably continue over the next few years. As yet, though, there are few or no standards for IP network-level encryption, so few of the products interoperate with each other. They all assume that you have another one of their products at the other end. We hope that his will change, and that standards will emerge as encryption becomes more common.

Figure 10.4 shows a simplified view of network-level encryption scheme.

Figure 10.4: Network-level encryption

10.5.4 Key Distribution

As with any encryption system, key distribution for network-level encryption can be a very sticky problem. All of the discussion above assumes that the two ends share a key so that each knows how to encrypt and decrypt data sent to and from the other. Most of the systems in use today rely on "out of band" key distribution: their manufacturers say "key distribution is your problem, not ours." Customers have to manually establish keys (by transferring them by voice over the phone, or on floppy disk, or through some other secure, private mechanism) on each participating system. This means that this network-level encryption can work well for sites you exchange information with frequently (e.g., partners, key clients, or other branches of your own organization). But it doesn't work well for ad hoc or transient connections, because of the setup time and effort involved. Some systems use public key technology or key distribution systems, and can deal more quickly and effectively with ad hoc or transient connections.