FTP is the de facto standard for file transfer on the Internet. In addition, there are some specialized protocols used for applications where FTP is not suitable. TFTP is used by dedicated devices to transfer configuration files. FSP is a UDP -based protocol used when TCP -based connections won't work or aren't allowed. UUCP is used for batch transfers, particularly across phone lines.
FTP is used to transfer files from one machine to another. You can use FTP to transfer any type of file, including executable binaries, graphics images, ASCII text, PostScript, sound and video files, and more. There are two types of FTP access: user FTP and anonymous FTP . User FTP requires an account on the server and lets users access any files they could access if they were logged in. Anonymous FTP is for people who don't have an account and is used to provide access to specific files to the world at large.
Anonymous FTP is by far the most common use of FTP on the Internet. Anonymous FTP servers are the standard mechanism for distributing programs, information, and other files that sites wish to make available to the Internet at large. If a site provides an anonymous FTP server, anyone on the Internet can initiate an FTP connection to the site, tell the FTP server that their login name is "anonymous", and access whatever files the server's owners have chosen to make available in a restricted area.
FTP uses two separate TCP connections: one to carry commands and results between the client and the server (commonly called the command channel ), and the other to carry any actual files and directory listings transferred (the data channel ). On the server end, the command channel uses well-known port 21, and the data channel normally uses well-known port 20. The client uses ports above 1023 for both the command and data channels.
To start an FTP session, a client first allocates two TCP ports for itself, each of them with a port number above 1024. It uses the first to open the command channel connection to the server and then issues FTP 's PORT command to tell the server the number of the second port, which the client wants to use for the data channel. The server then opens the data channel connection. This data channel connection is backwards from most protocols, which open connections from the client to the server. This backwards open complicates things for sites that are attempting to do "start-of-connection" packet filtering to ensure that all TCP connections are initiated from the inside, because external FTP servers will attempt to initiate data connections to internal clients, in response to command connections opened from those internal clients. Furthermore, these connections will be going to ports known to be in an unsafe range. Figure 8.6 shows this kind of FTP connection.
Most FTP servers (particularly those used at major anonymous FTP sites on the Internet) and many FTP clients support an alternative mode that allows the client to open both the command and the data channels to the server. This mode is called "passive mode" or "PASV mode" (after the FTP command the client sends to the server to initiate this mode). If both your FTP client and the FTP server you're trying to connect to support passive mode, you can use it instead of the regular mode. This way, you can avoid start-of-connection filtering problems, because all connections will be opened from the inside, by the client.
To use passive mode, an FTP client allocates two TCP ports for its own use, and uses the first port to contact the FTP server, just as when using normal mode. However, instead of issuing the PORT command to tell the server the client's second port, the client issues the PASV command. This causes the server to allocate a second port of its own for the data channel (for architectural reasons, servers use random ports above 1023 for this, not port 20 as in normal mode; you couldn't have two servers on the same machine simultaneously listening for incoming PASV -mode data connections on port 20), and tell the client the number of that port. The client then opens the data connection from its port to the data port the server has just told it about. Figure 8.7 shows a passive-mode FTP connection.
Not all FTP clients support passive mode. If a given client does support passive mode, it will usually mention this as a feature in the documentation or description. Some clients support both normal and passive modes and provide the user some way to specify which mode to use. If you're having trouble finding passive-mode clients, it's useful to know that the built-in FTP clients in many WWW browsers (Netscape Navigator, for example) use passive mode. Chances are, your users will want to have these browsers anyway for WWW access, and you can show them how to use the browsers as FTP clients as well.
Because passive mode has only recently become popular, many times even servers that support passive mode have problems with it. You may find that combinations of servers and clients that work well with normal-mode transfers hang periodically when you do passive-mode transfers.
If your FTP client (or one of the FTP servers you wish to communicate with) does not support passive mode, and you still want to allow FTP via packet filtering (rather than via proxy), you'll have to put a special-case exception in your packet filtering rules to allow the server to open the data channel back in to the client. If you do so, you will still be vulnerable to attackers launching a connection from port 20 on the attacker's end (nominally the FTP data channel, but you have no way to guarantee that on a machine you don't control) to a port above 1023 on your end (such as an X server, for example). Therefore, you should restrict this special-case exception as much as possible, e.g., by tying it to the address of the particular client or server that doesn't support passive mode. (Even an exception for a single server makes you vulnerable to forged connections from that server).
Some dynamic packet filtering implementations (such as FireWall-1 from CheckPoint Software) monitor the commands sent over the FTP command channel and notice the PORT command the client sends to the server; This command tells the server on which port the client is listening for the server to open the data channel. These implementations also put in place a temporary (time-limited) exception in the packet filtering rules to allow the server to open the data channel back to the client.
Because of the problems with passive mode, and because of complications introduced in name service (the "double-reverse lookups" discussed in the "Domain Name Service ( DNS )" section below), proxying is a particularly attractive solution for outbound FTP . Using a normal-mode proxied client allows you to talk reliably to external servers without having to allow incoming TCP connections for the data channel to any host except the bastion host doing the proxying. For this reason, you may choose to proxy FTP even if you allow most other protocols directly through the firewall via packet filtering. Both modified-client and modified-procedure proxies are available for FTP .
The SOCKS package includes an FTP client for UNIX that has been modified to use SOCKS . Because of the multiple simultaneous TCP connections involved in FTP , modifying other FTP clients yourself requires some work (more than modifying clients for straightforward single-connection protocols like SMTP and POP ).
The TIS FWTK provides a proxy FTP server that operates with modified clients or modified user procedures. It provides additional logging, operation denial, and user authentication features, giving you finer control than you can achieve with packet filters or SOCKS proxying.
If you want to use modified clients with the TIS FWTK FTP proxy server, you will need to do all of the modification yourself; they do not provide a modified client or even a client library. Using modified clients with the TIS FWTK FTP proxy server will also prevent you from running a standard FTP server on the machine you're using as the proxy server. Some versions of the FWTK have had an FTP server that could act as a proxy server and a regular FTP server, but there have been some problems with it, and it's not clear that it will continue to be included with newer releases of the toolkit.
Some FTP clients are not sufficiently flexible to be used with modified user procedures involving the TIS FWTK FTP proxy server. The custom procedure users have to follow involve opening an FTP connection to the machine where the proxy server is running, and then logging into the FTP proxy server as firstname.lastname@example.org , specifying the name of the host they really want to connect to as part of the login. Some FTP clients have "anonymous" simply hardcoded in, or limit the length of the login field to something too short to contain "anonymous@" plus a reasonably long hostname.
Any commercial proxying packages will almost certainly support outbound FTP proxying, because FTP is such a commonly used protocol on the Internet.
Many sites use both proxy and packet filtering solutions for FTP . You can sometimes reduce the number of modified clients you need by using proxying to support normal-mode connections, and packet filtering to support passive-mode connections. You can also use a combined solution for added security by using a proxy FTP server that uses passive mode to make external connections, regardless of the mode it uses to talk to the internal hosts; this converts all connections that cross the firewall to passive mode and allows you to tighten the packet filters that protect the host doing the proxying. On the other hand, it keeps you from using servers that don't support passive mode.
With anonymous FTP , a user logs in to the FTP server as "anonymous". The user is then asked for a password and is expected to enter his or her full email address in response. At most sites, this request is not enforced, however, and users can enter whatever they want, as long as it looks like an email address; even if the information is entered, it's usually just logged, not verified in any way beyond a superficial plausibility check (i.e., does it contain an at (@) sign?). Many standard FTP servers, like the ones shipped with most versions of UNIX , don't even log the information.
If you are providing anonymous FTP service, the challenge is to ensure that the anonymous FTP server makes available only the information that you want made available and that it doesn't give an outsider access to other, supposedly private, information on the machine.
In setting up anonymous FTP , one precaution you can take is to limit what other information is available on the machine that's providing anonymous FTP service. In this way, even if attackers get "outside" the anonymous FTP area on the machine, there is nothing of interest to them elsewhere on the machine (or reachable from the machine via NFS or some other mechanism).
Many FTP servers perform a chroot to the anonymous FTP area before the FTP server starts processing commands from an anonymous user. To support both anonymous and user FTP , however, FTP servers need access to all files. This means that chroot , which is normally regarded as extremely safe, doesn't guarantee as much for an FTP server, because the server is not always running in the chroot 'd environment.
To deal with this problem, you can modify inetd 's configuration so that instead of starting the FTP server directly, it chroot s (using something like the chrootuid program described in Appendix B ) and then starts the FTP server. Normally, FTP runs with limited access only for anonymous users, and nonanonymous users have their normal access permissions. Doing the chroot before starting up the FTP server means that the nonanonymous users will also be limited; if you don't have any nonanonymous users of your FTP server (and you probably shouldn't), this is irrelevant.
The details of setting up an anonymous FTP system vary depending on the operating system and the particular FTP daemon code in use. Start with the instructions (if any) in the manual pages for your FTP daemon; this should get you through most of the vendor-specific steps. Then, once you've performed all the steps there, obtain and follow CERT Advisory 93:10 (for information on obtaining CERT advisories, see Appendix A ), which addresses setting up anonymous FTP servers to close the holes left by most of the vendor instructions.
Unfortunately, one of the most common ways that anonymous users get access to files they shouldn't be able to see is that an internal user innocently puts the files up for anonymous FTP , on the assumption that this is somehow safe. Usually, the internal user is relying on security through obscurity; he assumes that nobody will notice the files. This does not work well. People do notice, especially if the names of the new files are meaningful. At popular FTP sites, curious people poke around randomly, and they notice new files almost immediately and may transfer them out of pure curiosity. On less-visited FTP sites, files may remain unnoticed until an Archie server indexes them. Unless you have explicitly arranged to have your FTP site skipped, you should assume that it is being indexed; Archie is quite talented at finding sites.
It's best to avoid putting files up for anonymous FTP if you don't want the entire world to read them. Use other methods of file transfer if possible. If not, you may want to use a modified FTP server, like the wuarchive server, which allows semianonymous access that requires an anonymous user to provide an additional password to get access to certain directories. You can also put up files in directories that have execute permission but not read permission. Doing so will let people who know the names transfer the files, but won't let people look to see what files exist.
Whatever method you choose, be sure that everybody at your site who can put files in the anonymous FTP directories knows not to put confidential files where they're world-readable. An easy way to do this is to prevent your internal users from writing to the anonymous FTP directories and to require them to ask a system administrator to make a file available.
Many Internet sites - both major and minor - that provide anonymous FTP run the wuarchive version of the UNIX FTP server, developed at Washington University (the "wu" in the name) in St. Louis. This server provides a number of features that are especially useful for anonymous FTP servers. These include the following features and many others:
For information about obtaining the wuarchive FTP daemon, see Appendix B . Be sure that you are installing the genuine, most recent version. This program has in the past been distributed by attackers with added trap doors, and older versions may have security problems that have since been fixed.
Be aware that one cost of the additional power and complexity offered by wuarchive is a bigger potential for security problems. The bigger and more complex a program is, the more likely it is to contain bugs. If the program is security-critical (as the FTP server is), many of those bugs are likely to have security impacts. Some features of the wuarchive server may also interact badly with some clients (in particular, some clients do not deal well with the displayed messages and hang). There are workarounds available, but if a significant percentage of your users have these clients, you may want to avoid wuarchive .
Instead of using the wuarchive version of the FTP server, you might consider using the FTP server from the TIS Internet Firewall Toolkit. The TIS FWTK FTP daemon has an emphasis different from that of the wuarchive daemon. The wuarchive version adds code in order to add features that support anonymous FTP . The TIS FWTK version, by contrast, concentrates on simplifying the server and making it more secure, stripping it down to the bare minimum capabilities necessary to provide anonymous FTP service. However, the maintainers of the FWTK have been considering removing it from the FWTK release, because despite all the work they've done on it, they're still not convinced that it's as secure as they'd like it to be (though it is almost certainly more secure than any alternative). It's possible that it will have been removed from the release by the time you read this.
You'll have to decide for yourself whether you really need the enhanced features provided by the wuarchive server (and are willing to accept the potential security implications of those features), or whether you'd rather have the enhanced security provided by the TIS FWTK server (assuming it's still available), at a cost of fewer features. Many users are accustomed to using wuarchive FTP and have come to depend on its features; it is probably not reasonable to attempt to run a major anonymous FTP site with the TIS FWTK server. For a site that provides only occasional anonymous FTP , however, the FWTK server is probably a more secure and reliable option.
Regardless of which FTP daemon you use, you'll have to confront a particularly troublesome issue raised by anonymous FTP servers: writable directories in the anonymous FTP area. Sites frequently provide writable space in this area, so that outsiders can upload files to them as well as download files from them; for example, a software company's customers might be asked to upload crash dump files (which are too big to conveniently send through email) so the company can do a crash analysis on them.
Writable areas can be very useful, but they have a dark side. Such writable directories will (notice that we didn't say may ) be found and used by "the underground" on the Internet as storage space and distribution areas for illicit material; generally this means pirated software packages and pornographic image files.
The folks who do this are amazingly well organized and hard to track down. They have their own communication mechanisms for telling each other about new sites - places they've found to store their stuff - without revealing who they are. When they find a new site, they typically create a hidden subdirectory in which to store their files and images. They give the subdirectory an innocuous name such as ".. " (that's "dot dot space space"). When casually looking around an anonymous FTP area, you probably won't notice a name like this. It's particularly easy to miss because file and directory names beginning with "." are ignored by the UNIX ls command, unless you give the command a special argument or run it as root.
On some sites in which intruders play this game, you can see a barter economy in operation. Someone leaves a note saying they're seeking a certain package or file, and listing what they have to offer in exchange. A short time later, someone else comes along, uploads the requested files, and leaves another note telling the original poster what they want in return.
What's wrong with this misuse of your anonymous FTP areas? There are several problems:
How can you protect your anonymous FTP areas from such misuse? The first question to ask yourself is this: do you really need to provide writable space in your anonymous FTP area? Often there are other acceptable ways (electronic mail, for example) for folks to send files to you. If you decide that you must provide writable space in your anonymous FTP area, there are a number of ways to limit your vulnerability, as we describe below.Making your incoming directory write-only
The most obvious approach, if you're using a UNIX machine as your bastion host, is to make your "incoming" directory write-only (directory permissions 773 or 733 - that is, "rwxrwx-wx" or "rwx-wx-wx"). Make sure that the directory is owned by some user other than "ftp" (or whatever your anonymous FTP server runs as when doing anonymous FTP ). If the mode is 773 instead of 733, then also make sure that the group of the directory is something other than the default group of the "ftp" login.
The problem with this approach is that all you're doing is keeping people from being able to see what's in the top-level directory. They can still see what's in subdirectories, and they can still access files and directories they create in the top-level directory if they communicate exact filenames among themselves. (Unfortunately, they can, via their mailing lists and other communications channels.)Disabling the creation of directories and certain files
Another approach is to disable the creation of directories and files with funny names (for example, files that begin with ".") in your anonymous FTP server. Depending on your server, you may be able to do this with a configuration file (for example, the wuarchive server lets you restrict anonymous users from deleting, overwriting, or creating certain types of files), or you may have to modify the server source code. (This is a nontrivial modification, which requires a reasonably competent C programmer.)
This approach doesn't keep people from uploading stuff to the writable directory you provide; it simply makes it more difficult for them to hide that stuff so that it escapes your notice. If you do this, you will still need to look at the writable area every day (and look at the content of files, not just the names) to be sure everything is something that belongs there.Uploading by prearrangement
Another approach is used frequently by sites that want people to be able to upload files, but only by prearrangement. These sites take a page from the underground's own book by creating hidden writable subdirectories that you can only access if you know they're there. The attackers can't see them; they're unaware that there's a "there" there for their wares.
Here's what you do, assuming you're using a UNIX system for your FTP server:
unix# cd ~ftp/pub unix# mkdir incoming unix# cd incoming unix# mkdir b2b_Free unix# chmod a+w b2b_Free # or "chmod 0777 b2b_Free" unix# cd .. unix# chmod a=x incoming # or "chmod 0111 incoming"
Users can now upload files to the writable directory only if they know (presumably because you've told them) its secret, password-like name. You can create as many of these secret subdirectories as necessary, and you can change or delete them as often as necessary. If your site creates a top-level index of your FTP area (such as a file containing the output of "ls -lR ~ftp"), you need to make sure that the hidden directories don't appear in the index. The easiest way to do that is to run the indexing command as the "ftp" user, so that it has the same permissions that someone using anonymous FTP would have.
Beware that some FTP clients with graphical user interfaces will only let a user access a directory that the FTP client can see; they don't provide a way for the user to jump blindly to a directory that doesn't appear in a directory listing. Such clients won't work with this scheme because, by design, the client can't see the names of the subdirectories containing the actual data. This is not usually a problem for people coming in from UNIX machines, and there are publicly available clients for most platforms that do not have this problem, so you may be able to work around this limitation.Removing the files
There is one other approach you might take, particularly if you find that your anonymous FTP area is already being abused and you're curious to see what people are uploading there. Basically, you run a short shell script once a minute as a cron job that moves files from the writable incoming directory to another directory outside the anonymous FTP area. This will ensure that the intruders won't be able to see what's been uploaded. You may need to rename files when you move them to avoid overwriting files with the same name. Because the files aren't there to look at, it's easy for people to unintentionally create name conflicts (particularly if they're sending you crash dumps, which probably all start out having the same name).
Make sure that the new directory is on the same filesystem, so the operating system doesn't have to copy the data. Because of the way that the UNIX filesystem works, this approach works even if the file is still being written when the "move" takes place, as long as the directory you're moving it to is on the same filesystem as the original directory ("moving" a file in such a case doesn't actually move the data; it merely renames the file).
This doesn't avoid denial of service attacks; people can still fill up your disk space. In fact, they may retry downloads multiple times (because the files keep mysteriously disappearing) and unintentionally fill up your disks.
TFTP is a simplified file transfer protocol; it is simpler than FTP , and is designed to be implemented in ROM for booting diskless systems like X terminals, diskless workstations, and routers. There is no authentication with TFTP ; a TFTP client simply connects to the server and asks for a file, without saying who the file is for. If the file is one that the server can access, the server gives the client the file. For this reason, you need to be very careful about what your TFTP server (if you have one) can access, and what clients can access the server.
Generally, there's no reason at all to allow TFTP across your firewall, even if you use it internally. You do not want to boot diskless systems across the Internet, and people do not transfer files with TFTP .
TFTP is a UDP -based protocol. Servers listen on port 69 for the initial client-to-server packet to establish the TFTP session, then use a port above 1023 for all further packets during that session. Clients use ports above 1023.
TFTP does not lend itself well to proxying. Because TFTP clients are often implemented in hardware, with no users involved, neither modified clients nor modified user procedures are generally implementable. A transparent proxy could easily support TFTP , providing the same extremely minimal amount of security achievable if you allow TFTP through packet filters.
FSP is a file transfer protocol that was designed to circumvent FTP restrictions. Very few sites provide supported FSP service; most FSP servers are set up by users or attackers, without the knowledge or consent of a machine's managers. FSP activity is often a clue that attackers have compromised a site; attackers often use FSP to move their files from site to site. FSP activity can be difficult to detect, however, because, as we discuss below, there are no standard port numbers that FSP clients and servers use.
FSP is intentionally difficult to suppress with packet filtering. Filtering out all UDP services except those you need, and restricting those to specific hosts, will pretty well get rid of it except on the hosts you're allowing other UDP services to. If you want to allow FSP, that's extremely simple; allowing UDP packets to and from a host, at any port number, will allow it to be an FSP server and/or client.
FSP is a UDP -based protocol. There is no "standard" port number that the server runs on; anyone running a server chooses the port that server will use. If they have root access, they often choose port 21, the UDP equivalent of the TCP port used by FTP ; if they don't have root access, they have to use a port above 1023, and there's no predicting what it will be. Clients generally use ports above 1023.
Using generic proxies to proxy a UDP -based application with no standard port number is as difficult as trying to suppress it with packet filters. No standard proxy package supports FSP . A dedicated FSP proxy would be possible to write.
UUCP was designed for transferring files, including electronic mail and news, between UNIX machines; it is the original UNIX networking system. UUCP is primarily used as a dial-up protocol (as the sole protocol over a modem connection), but some sites, particularly those with intermittent dial-up Internet service, use UUCP over TCP to transfer mail and news from their service provider. UUCP allows the service provider to collect and hold all of the site's mail and news; when the site brings up its Internet connection and contacts the service provider using UUCP , the service provider transfers all of the accumulated mail and news at once.
UUCP over TCP is also used by some sites that have much faster permanent Internet connections (56 Kb/s lines, for example) to transfer Usenet news, because UUCP over TCP is more bandwidth-efficient than NNTP for this purpose. (It has recently become difficult or impossible to keep up with the news in every group over a 56 Kb/s line using NNTP, for example; there isn't enough bandwidth. As the volume of Usenet news continues to grow, this problem is going to get worse.)
One of the nice features of UUCP is that, once a connection between two sites is established, that connection is used to transfer all the pending data between the two sites, regardless of which site initiated the connection. This means that you could allow only outgoing UUCP connections, and poll your service provider on a regular basis (hourly, for example) to both send your outgoing messages and collect your incoming messages.
Most sites do not use UUCP in any form any more. If you don't have any special needs, you can simply not allow it across your firewall.
Because UUCP uses reusable passwords, someone who snoops on one of your UUCP sessions will later be able to connect to your service provider and impersonate you, or connect to you and impersonate your service provider. The passwords are also generally stored unencrypted in the configuration files on the calling machine, so it may be possible to do this without even snooping on a connection first. You may be able to defend against this with various mechanisms that depend on what version of UUCP you and your service provider are using.
UUCP is a store-and-forward protocol, and as such is not used with general-purpose proxies; instead, it's configured to effectively do its own proxying. It is a straightforward single-connection protocol used with a small number of clients, and thus is well-suited to both modified-client and modified-procedure proxying. On the other hand, it is rare for people to want to proxy it, so there are no widely available proxies specifically written for it. Generic proxy servers, such as the plug-gw program in the TIS FWTK , should work if you can decide which real server to connect to based on which client the connection request is coming from (in other words, if all connections are to the same server, or if a given client host always connects to the same server).