Chapter 8. Packet Filtering

What Can You Do with Packet Filtering?
Configuring a Packet Filtering Router
What Does the Router Do with Packets?
Packet Filtering Tips and Tricks
Conventions for Packet Filtering Rules
Filtering by Address
Filtering by Service
Choosing a Packet Filtering Router
Packet Filtering Implementations for General-Purpose Computers
Where to Do Packet Filtering
What Rules Should You Use?
Putting It All Together

Packet filtering is a network security mechanism that works by controlling what data can flow to and from a network. The basic device that interconnects IP networks is called a router. A router may be a dedicated piece of hardware that has no other purpose, or it may be a piece of software that runs on a general-purpose computer running Unix, Windows NT, or another operating system (MS-DOS, Windows 95/98, Macintosh, or other). Packets traversing an internetwork (a network of networks) travel from router to router until they reach their destination. The Internet itself is sort of the granddaddy of internetworks -- the ultimate "network of networks".

A router has to make a routing decision about each packet it receives; it has to decide how to send that packet on towards its ultimate destination. In general, a packet carries no information to help the router in this decision, other than the IP address of the packet's ultimate destination. The packet tells the router where it wants to go but not how to get there. Routers communicate with each other using routing protocols such as the Routing Information Protocol (RIP) and Open Shortest Path First (OSPF) to build routing tables in memory to determine how to get the packets to their destinations. When routing a packet, a router compares the packet's destination address to entries in the routing table and sends the packet onward as directed by the routing table. Often, there won't be a specific route for a particular destination, and the router will use a default route; generally, such a route directs the packet towards smarter or better-connected routers. (The default routes at most sites point towards the Internet.)

In determining how to forward a packet towards its destination, a normal router looks only at a normal packet's destination address and asks only "How can I forward this packet?" A packet filtering router also considers the question "Should I forward this packet?" The packet filtering router answers that question according to the security policy programmed into the router via the packet filtering rules.

Some machines do packet filtering without doing routing; that is, they may accept or reject packets destined for them before they do further processing.

TIP: Some unusual packets do contain routing information about how they are to reach their destination, using the "source route" IP option. These packets, called source-routed packets, are discussed in Section 4.2.2, "IP Options", in Chapter 4, "Packets and Protocols ".

8.1. What Can You Do with Packet Filtering?

If you put enough work into it, you can do anything you want to with packet filtering; all of the information that crosses the Internet has to go into a packet at some point, after all. But some things are very much easier to do than others. For instance, operations that require detailed protocol knowledge or prolonged tracking of past events are easier to do in proxy systems. Operations that are simple but need to be done fast and on individual packets are easier to do in packet filtering systems.

The main advantage of packet filtering is leverage: it allows you to provide, in a single place, particular protections for an entire network. Consider the Telnet service as an example. If you disallow Telnet by turning off the Telnet server on all your hosts, you still have to worry about someone in your organization installing a new machine (or reinstalling an old one) with the Telnet server turned on. On the other hand, if Telnet is not allowed by your filtering router, such a new machine would be protected right from the start, regardless of whether or not its Telnet server was actually running. This is an example of the kind of "fail safe" stance we discussed in Chapter 3, "Security Strategies".

Routers also present a useful choke point (also discussed in Chapter 3, "Security Strategies") for all of the traffic entering or leaving a network. Even if you have multiple routers for redundancy, you probably have far fewer routers, under much tighter control, than you have host machines.

Certain protections can be provided only by filtering routers, and then only if they are deployed in particular locations in your network. For example, it's a good idea to reject all external packets that have internal source addresses -- that is, packets that claim to be coming from internal machines but that are actually coming in from the outside -- because such packets are usually part of address-spoofing attacks. In such attacks, an attacker is pretending to be coming from an internal machine. You should also reject all internal packets that have external source addresses; once again, they are usually part of address-spoofing attacks. Decision-making of this kind can be done only in a filtering router at the perimeter of your network. Only a filtering router in that location (which is, by definition, the boundary between "inside" and "outside") is able to recognize such a packet, by looking at the source address and whether the packet came from the inside (the internal network connection) or the outside (the external network connection). Figure 8-1 illustrates this type of source address forgery.

Figure 8-1. Source address forgery

Filtering routers are also good at detecting and filtering out illegal packets. Many denial of service attacks depend on sending misformatted packets of one sort or another. Routers in general have very reliable TCP/IP implementations (so they are not vulnerable to these attacks) and are well placed to prevent these attacks. General-purpose computers being used as packet filters are more likely to be vulnerable to these attacks, but at least it is easier to fix them than it is to fix all your internal machines.

8.1.1. Basic Packet Filtering

The most straightforward kind of packet filtering lets you control (allow or disallow) data transfer based on:

The address the data is (supposedly) coming from
The address the data is going to
The session and application ports being used to transfer the data

Basic packet filtering systems don't do anything based on the data itself; they don't make content-based decisions. Straightforward packet filtering will let you say:

Don't let anybody use the port used by Telnet (an application protocol) to log in from the outside.

or:

Let everybody send us data over the port used for electronic mail by SMTP (another application protocol).

or even:

That machine can send us data over the port used for news by NNTP (yet another application protocol), but no other machines can do so.

However, it won't let you say:

This user can Telnet in from outside, but no other users can do so.

because "user" isn't something a basic packet filtering system can identify. And it won't let you say:

You can transfer these files but not those files.

because "file" also isn't something a basic packet filtering system can identify. It won't even let you say:

Only allow people to send us electronic mail over the port used by SMTP.

because a basic packet filtering system looks only at the port being used; it can't tell whether the data is good data, conforming to the protocol that's supposed to use that port, or whether somebody is using the port for some other purpose.

ore advanced packet filtering systems will let you look further into the data of a packet. Instead of paying attention only to headers for lower-level protocols, they also understand the data structures used by higher-level protocols, so they can make more detailed decisions.

8.1.2. Stateful or Dynamic Packet Filtering

Slightly more advanced packet filtering systems offer state tracking and/or protocol checking (for well-known protocols). State tracking allows you to make rules like the following:

Let incoming UDP packets through only if they are responses to outgoing UDP packets you have seen.

or:

Accept TCP packets with SYN set only as part of TCP connection initiation.

This is called stateful packet filtering because the packet filter has to keep track of the state of transactions. It is also called dynamic packet filtering because the behavior of the system changes depending on the traffic it sees. For instance, if it's using the preceding rule, you can't look at an incoming UDP packet and say that it will always be accepted or rejected.

Different systems keep track of different levels of state information. Some people are willing to call something a stateful packet filtering system if it enforces TCP state rules (which control the flags used during startup and teardown of TCP sessions), even if the packet filtering system provides no further stateful features. While TCP state enforcement is nice to have (it helps to prevent some forms of port scanning and denial of service), it does not allow you to support additional protocols, and we do not consider it stateful packet filtering.

Figure 8-2 illustrates dynamic packet filtering at the UDP layer.

Figure 8-2. Dynamic packet filtering at the UDP layer

State tracking provides the ability to do things that you can't do otherwise, but it also adds complications. First, the router has to keep track of the state; this increases the load on the router, opens it to a number of denial of service attacks, and means that if the router reboots, packets may be denied when they should have been accepted. If a packet may go through redundant routers, they all need to have the same state information. There are protocols for exchanging this information, but it's still a tricky business. If you have redundant routers only for emergency failover, and most traffic consistently uses the same router, it's not a problem. If you are using redundant routers simultaneously, the state information needs to be transferred between them almost continuously, or the response packet may come through before the state is updated.

Second, the router has to keep track of state without any guarantee that there's ever going to be a response packet. Not all UDP packets have responses. At some point, the router's going to have to give up and get rid of the rule that will allow the response. If the router gives up early, it will deny packets that should have been accepted, causing delays and unneeded network traffic. If the router keeps the rule too long, the load on the router will be unneccessarily high, and there's an increased chance that packets will be accepted when they should have been denied. Some protocol specifications provide guidelines, but those are not necessarily useful. For instance, DNS replies are supposed to arrive within 5 seconds, but reply times for name service queries across the Internet can be as high as 15 seconds; implementing to the protocol specification will almost always deny a response that you wanted to accept.

This sort of filtering is also vulnerable to address forging; it is validating that packets are responses based on their source addresses, so an attacker who intercepts an outgoing packet can forge the appropriate source address and return an acceptable "reply" (or, depending on the implementation, a whole bunch of packets all of which will be accepted as replies). Nonetheless, this provides a reasonable degree of security for some UDP-based protocols that would otherwise be extremely difficult to protect.

8.1.3. Protocol Checking

Protocol checking allows you to make rules like:

Let in packets bound for the DNS port, but only if they are formatted like DNS packets.

Protocol checking therefore helps you avoid situations where somebody has set up an unsafe service on a port that is allowed through because it normally belongs to a safe service. It can also help avoid some attacks that involve sending misformatted packets to perfectly genuine servers. Protocol checking is normally fairly rudimentary and still can be circumvented by a determined insider. It also gives you no guarantee that the data is good, so it will catch only a fairly small number of attacks that involve sending hostile data to genuine servers. However, it still provides a useful degree of sanity checking.

The most advanced packet filtering systems will allow you to specify all sorts of data-specific rules for well-known protocols. For instance, you can say:

Disconnect any FTP connection where the remote username is "anonymous".

or:

Do not allow HTTP transfers to these sites.

In order to do this, these packet filters have to have a deep understanding of the application protocol. In general, they can provide this level of control only for a few popular protocols, and there is a significant cost to provide it, since they have to process larger amounts of data. Furthermore, it is often possible to circumvent this sort of control. For instance, there are numerous ways of getting to a site via HTTP without having the site's name appear in the outgoing HTTP request, including using an IP address instead of a hostname and using an anonymizing site set up to provide this sort of service.

Stateful packet filters may also look at protocol-specific details to make state changes. Some protocols contain information about what ports transactions will use. For instance, the file transfer protocol FTP often uses a connection that is started from the server to the client, and the two ends negotiate the port number that will be used for this connection. A stateful packet filter that understands the FTP protocol can watch this negotiation and allow the new connection to be made, without allowing other connections of the same sort.