Chapter 23
Internet News

Contents:
Some INN Internals
Newsreaders and INN
Installing INN
Configuring INN: the Basic Setup
INN Configuration Files
Running INN
Managing INN: The ctlinnd Command

The Internet News daemon (INN) is arguably the most popular Netnews server in use today. INN is extremely flexible and is suitable for all but the smallest news sites.[1] INN scales well and is suited to large news server configurations.

[1] Very small news sites should consider a caching NNTP server program like leafnode, which is available at http://wpxx02.toxi.uni-wuerzburg.de/~krasel/leafnode.html.

The INN server comprises a number of components, each with their own configuration files that we will discuss in turn. Configuration of INN can be a little involved, but we'll describe each of the stages in this chapter and arm you with enough information to make sense of the INN manual pages and documentation and build configurations for just about any application.

Some INN Internals

INN's core program is the innd daemon. innd's task is to handle all incoming articles, storing them locally, and to pass them on to any outgoing newsfeeds if required. It is started at boot time and runs continually as a background process. Running as a daemon improves performance because it has to read its status files only once when starting. Depending on the volume of your news feed, certain files such as history (which contain a list of all recently processed articles) may range from a few megabytes to tens of megabytes.

Another important feature of INN is that there is always only one instance of innd running at any time. This is also very beneficial to performance, because the daemon can process all articles without having to worry about synchronizing its internal states with other copies of innd rummaging around the news spool at the same time. However, this choice affects the overall design of the news system. Because it is so important that incoming news is processed as quickly as possible, it is unacceptable that the server be tied up with such mundane tasks as serving newsreaders accessing the news spool via NNTP, or decompressing newsbatches arriving via UUCP. Therefore, these tasks have been broken out of the main server and implemented in separate support programs. Figure 23.1 attempts to illustrate the relationships between innd, the other local tasks, and remote news servers and newsreaders.

Today, NNTP is the most common means of transporting news articles around, and innd doesn't directly support anything else. This means that innd listens on a TCP socket (port 119) for connections and accepts news articles using the "ihave" protocol.

Articles arriving by transports other than NNTP are supported indirectly by having another process accept the articles and forward them to innd via NNTP. Newsbatches coming in over a UUCP link, for instance, are traditionally handled by the rnews program. INN's rnews decompresses the batch if necessary, and breaks it up into individual articles; it then offers them to innd one by one.

Newsreaders can deliver news when a user posts an article. Since the handling of newsreaders deserves special attention, we will come back to this a little later.

Figure 23.1: INN architecture (simplified for clarity)

When receiving an article, innd first looks up its message ID in the history file. Duplicate articles are dropped and the occurrences are optionally logged. The same goes for articles that are too old or lack some required header field, such as Subject:.[2] If innd finds that the article is acceptable, it looks at the Newsgroups: header line to find out what groups it has been posted to. If one or more of these groups are found in the active file, the article is filed to disk. Otherwise, it is filed to the special group junk.

[2] This is indicated by the Date: header field; the limit is usually two weeks.

Individual articles are kept below /var/spool/news, also called the news spool. Each newsgroup has a separate directory, in which each article is stored in a separate file. The file names are consecutive numbers, so that an article in comp.risks may be filed as comp/risks/217, for instance. When innd finds that the directory it wants to store the article in does not exist, it creates it automatically.

Apart from storing articles locally, you may also want to pass them on to outgoing feeds. This is governed by the newsfeeds file that lists all downstream sites along with the newsgroups that should be fed to them.

Just like innd's receiving end, the processing of outgoing news is handled by a single interface, too. Instead of doing all the transport-specific handling itself, innd relies on various backends to manage the transmission of articles to other news servers. Outgoing facilities are collectively dubbed channels. Depending on its purpose, a channel can have different attributes that determine exactly what information innd passes on to it.

For an outgoing NNTP feed, for instance, innd might fork the innxmit program at startup, and, for each article that should be sent across that feed, pass its message ID, size, and filename to innxmit's standard input. For an outgoing UUCP feed, on the other hand, it might write the article's size and file name to a special logfile, which is head by a different process at regular intervals in order to create batches and queue them to the UUCP subsystem.

Besides these two examples, there are other types of channels that are not strictly outgoing feeds. These are used, for instance, when archiving certain newsgroups, or when generating overview information. Overview information is intended to help newsreaders thread articles more efficiently. Old-style newsreaders had to scan all articles separately in order to obtain the header information required for threading. This would put an immense strain on the server machine, especially when using NNTP; furthermore, it was very slow.[3] The overview mechanism alleviates this problem by prerecording all relevant headers in a separate file (called .overview) for each newsgroup. This information can then be picked up by newsreaders either by reading it directly from the spool directory, or by using the XOVER command when connected via NNTP. INN has the innd daemon feed all articles to the overchan command, which is attached to the daemon through a channel. We'll see how this is done when we discuss configuring news feeds later.

[3] Threading 1,000 articles when talking to a loaded server could easily take around five minutes, which only the most dedicated Usenet addict would find acceptable.

Newsreaders and INN

Newsreaders running on the same machine as the server (or having mounted the server's news spool via NFS) can read articles from the spool directly. To post an article composed by the user, they invoke the inews program, which adds any header fields that are missing and forwards them to the daemon via NNTP.

Alternatively, newsreaders can access the server remotely via NNTP. This type of connection is handled differently from NNTP-based news feeds, to avoid tying up the daemon. Whenever a newsreader connects to the NNTP server, innd forks a separate program called nnrpd, which handles the session while innd returns to the more important things (receiving incoming news, for example).[4] You may be wondering how the innd process can distinguish between an incoming news feed and a connecting newsreader. The answer is quite simple: the NNTP protocol requires that an NNTP-based newsreader issue a mode reader command after connecting to the server; when this command is received, the server starts the nnrpd process, hands the connection to it, and returns to listening for connections from another news server. There used to be at least one DOS-based newsreader which was not configured to do this, and hence failed miserably when talking to INN, because innd itself does not recognize any of the commands used to read news if it doesn't know the connection is from a news reader.

[4] The name apparently stands for NetNews Read & Post Daemon.

We'll talk a little more about newsreader access to INN under "Controlling Newsreader Access," later in the chapter.

Installing INN

Before diving into INN's configuration, let's talk about its installation. Read this section, even if you've installed INN from one of the various Linux distributions; it contains some hints about security and compatibility.

Linux distributions included Verson INN-1.4sec for quite some time. Unfortunately, this version had two subtle security problems. Modern versions don't have these problems and most distributions include a precompiled Linux binary of INN Version 2 or later.

If you choose, you can build INN yourself. You can obtain the source from ftp.isc.org in the /isc/inn/ directory. Building INN requires that you edit a configuration file that tells INN some detail about your operating system, and some features may require minor modifications to the source itself.

Compiling the package itself is pretty simple; there's a script called BUILD that will guide you through the process. The source also contains extensive documentation on how to install and configure INN.

After installing all binaries, some manual fixups may be required to reconcile INN with any other applications that may want to access its rnews or inews programs. UUCP, for instance, expects to find the rnews program in /usr/bin or /bin, while INN installs it in /usr/lib/bin by default. Make sure /usr/lib/bin/ is in the default search path, or that there are symbolic links pointing to the actual location of the rnews and inews commands.

Configuring INN: the Basic Setup

One of the greatest obstacles beginners may face is that INN requires a working network setup to function properly, even when running on a standalone host. Therefore, it is essential that your kernel supports TCP/IP networking when running INN, and that you have set up the loopback interface as explained in Chapter 5, Configuring TCP/IP Networking.

Next, you have to make sure that innd is started at boot time. The default INN installation provides a script file called boot in the /etc/news/ directory. If your distribution uses the SystemV-style init package, all you have to do is create a symbolic link from your /etc/init.d/inn file pointing to /etc/news/boot. For other flavors of init, you have to make sure /etc/news/boot is executed from one of your rc scripts. Since INN requires networking support, the startup script should be run after the network interfaces are configured.

INN Configuration Files

Having completed these general tasks, you can now turn to the really interesting part of INN: its configuration files. All these files reside in /etc/news. Some changes to configurations files were introduced in Version 2, and it is Version 2 that we describe here. If you're running an older version, you should find this chapter useful to guide you in upgrading your configuration. During the next few sections, we will discuss them one by one, building the Virtual Brewery's configuration as an example.

If you want to find out more about the features of individual configuration files, you can also consult the manual pages; the INN distribution contains individual manual pages for each of them.

Global Parameters

There are a number of INN parameters that are global in nature; they are relevant to all newsgroups carried.

The inn.conf file

INN's main configuration file is inn.conf. Among other things, it determines the name by which your machine is known on Usenet. Version 2 of INN allows a baffling number of parameters to be configured in this file. Fortunately, most parameters have default values that are reasonable for most sites. The inn.conf(5) file details all of the parameters, and you should read it carefully if you experience any problems.

A simple example inn.conf might look like:

# Sample inn.conf for the Virtual Brewery server:          vlager.vbrew.com domain:          vbrew.com fromhost:        vbrew.com pathhost:        news.vbrew.com organization:    The Virtual Brewery mta:             /usr/sbin/sendmail -oi %s moderatormailer: %s@uunet.uu.net # # Paths to INN components and files. # pathnews:               /usr/lib/news pathbin:                /usr/lib/news/bin pathfilter:             /usr/lib/news/bin/filter pathcontrol:            /usr/lib/news/bin/control pathdb:                 /var/lib/news pathetc:                /etc/news pathrun:                /var/run/news pathlog:                /var/log/news pathhttp:               /var/log/news pathtmp:                /var/tmp pathspool:              /var/spool/news patharticles:           /var/spool/news/articles pathoverview:           /var/spool/news/overview pathoutgoing:           /var/spool/news/outgoing pathincoming:           /var/spool/news/incoming patharchive:            /var/spool/news/archive pathuniover:            /var/spool/news/uniover overviewname:           .overview

The first line tells the programs rnews and inews which host to contact when delivering articles. This entry is absolutely crucial; to pass articles to innd, they have to establish an NNTP connection with the server.

The domain keyword should specify the domain portion of the host's fully qualified domain name. A couple of programs must look up your host's fully qualified domain name; if your resolver library returns the unqualified hostname only, the name given in the domain attribute is tacked onto it. It's not a problem to configure it either way, so it's best to define domain.

The next line defines what hostname inews is going to use when adding a From: line to articles posted by local users. Most newsreaders use the From: field when composing a reply mail message to the author of an article. If you omit this field, it will default to your news host's fully qualifed domain name. This is ot always the best choice. You might, for example, have news and mail handled by two different hosts. In this case, you would supply the fully qualified domain name of your mail host after the fromhost statement.

The pathhost line defines the hostname INN is to add to the Path: header field whenever it receives an article. In most cases, you will want to use the fully qualified domain name of your news server; you can then omit this field since that is the default. Occasionally you may want to use a generic name, such as news.vbrew.com, when serving a large domain. Doing this allows you to move the news system easily to a different host, should you choose to at some time.

The next line contains the organization keyword. This statement allows you to configure what text inews will put into the Organization: line of articles posted by your local users. Formally, you would place a description of your organization or your organization's name in full here. Should you not wish to be so formal, it is fashionable for organizations with a sense of humor to exhibit it here.

The organization keyword is mandatory and specifies the pathname of the mail transport agent that will be used for posting moderator messages. %s is replaced by the moderator email address.

The moderatormailer entry defines a default address used when a user tries to post to a moderated newsgroup. The list of moderator addresses for each newsgroup is usually kept in a separate file, but you will have a hard time keeping track of all of them. The moderatormailer entry is therefore consulted as a last resort; if it is defined, inews will replace the %s string with the (slightly transformed) newsgroup name and send the entire article to this address. For instance, when posting to soc.feminism, the article is mailed to soc-feminism@uunet.uu.net, given the above configuration. At UUNET, there should be a mail alias installed for each of these submissions addresses that automatically forwards all messages to the appropriate moderator.

Finally, each of the remaining entries specifies the location of some component file or executable belonging to INN. If you've installed INN from a package, these paths should have been configured for you. If you're installing from source, you'll need to ensure that they reflect where you've installed INN.

Configuring Newsgroups

The news administrator on a system is able to control which newsgroups users have access to. INN provides two configuration files that allow the administrator to decide which newsgroups to support and provide descriptions for them.

The active and newsgroups files

The active and newsgroups files are used to store and describe the newsgroups hosted by this news server. They list which newsgroups we are interested in receiving and serving articles for, and administrative information about them. These files are found in the /var/lib/news/ directory.

The active file determines which newsgroups this server supports. Its syntax is straightforward. Each line in the active file has four fields delimited by whitespace:

name himark lomark flags

The name field is the name of the newsgroup. The himark field is the highest number that has been used for an article in that newsgroup. The lomark field is the lowest active number in use in the newsgroup. To illustrate how this works, consider the follow scenario. Imagine that we have a newly created newsgroup: himark and lowmark are both 0 because there are no articles. If we post 5 articles, they will be numbered 1 through 5. himark will now equal 5, the highest numbered article, and lowmark will equal 1, the lowest active article. If article 5 is cancelled there will be no change; himark will remain at 5 to ensure that that article number is not reallocated and lowmark will remain at 1, the lowest active article. If we now cancel article 1, himark will remain unchanged, but lowmark will now equal 2, because 1 is no longer active. If we now post a new article, it will be assigned article number 6, so himark will now equal 6. Article 5 has been in use, so we won't reassign it. lowmark remains at 2. This mechanism allows us to easily allocate unique article numbers for new articles and to calculate approximately how many active articles there are in the group: himark-lowmark.

The field may contain one of the following:

y

Posting directly to this news server is allowed.

n

Posting directly to this news server is not allowed. This prevents newsreaders from posting directly to this news server. New articles may only be received from other news servers.

m

The group is moderated. Any articles posted to this newsgroup are forwarded to the newsgroup moderator for approval before they enter the newsgroup. Most newsgroups are unmoderated.

j

Articles in this group are not kept, but only passed on. This causes the news server to accept the article, but all it will do with it is pass it to the "up-stream" news servers. It will not make the articles available to newsreaders reading from this server.

x

Articles cannot be posted to this newsgroup. The only way that news articles are delivered to this server is by feeding them from another news server. Newsreaders may not directly write articles to this server.

=foo.bar

Articles are locally filed into the ``foo.bar'' group.

In our simple server configuration we'll carry a small number of newsgroups, so our /var/lib/news/active file will look like:

control 0000000000 0000000001 y junk 0000000000 0000000001 y rec.crafts.brewing 0000000000 0000000001 y rec.crafts.brewing.ales 0000000000 0000000001 y rec.crafts.brewing.badtaste 0000000000 0000000001 y rec.crafts.brewing.brandy 0000000000 0000000001 y rec.crafts.brewing.champagne 0000000000 0000000001 y rec.crafts.brewing.private 0000000000 0000000001 y

The himark and lomark numbers in this example are those you would use when creating new newsgroups. The himark and lomark numbers will look quite different for a newsgroup that has been active for some time.

The newsgroups file is even simpler. It provides one-line descriptions of newsgroups. Some newsreaders are able to read and present this information to a user to help them decide whether they want to subscribe.

The format of the newsgroups file is simply:

name description

The name field is the name of a newsgroup, and the <description is a single line description of that newsgroup.

We want to describe the newsgroups that our server supports, so we'll build our newsgroups file as follows:

rec.crafts.brewing.ales         Home brewing Ales and Lagers rec.crafts.brewing.badtaste     Home brewing foul tasting brews rec.crafts.brewing.brandy       Home brewing your own Brandy rec.crafts.brewing.champagne    Home brew your own Champagne rec.crafts.brewing.private      The Virtual Brewery home brewers group

Configuring Newsfeeds

INN provides the news administrator the ability to control which newsgroups are forwarded on to other news servers and how they will be forwarded. The most common method uses the NNTP protocol described earlier, but INN also allows newsfeeds via other protocols, such as UUCP.

The newsfeeds file

The newsfeeds file determines where news articles will be sent. It normally resides in the /etc/news/ directory.

The format of the newsfeeds is a little complicated at first. We'll describe the general layout here, and the newsfeeds(5) manual page describes what we leave out. The format is as follows:

# newsfeeds file format site:pattern:flags:param site2:pattern2\         :flags2:param2

Each news feed to a site is described by a single line, or may be spread across multiple lines using the \ continuation character. The : characters delimit the fields in each line. The # character at the start of a line marks that line as a comment.

The site field names the site to which this feed description relates. The sitename can be coded any way you like and doesn't have to be the domain name of the site. The site name will be used later and will refer to an entry in a table that supplies the hostname to the innxmit program that transmits the news articles by NNTP to the remote server. You may have multiple entries for each site; each entry will be treated individually.

The pattern field specifies which news groups are to be sent to this site. The default is to send all groups, so if that is what you want, just make this field empty. This field is usually a comma-delimited list of pattern-matching expressions. The * character matches zero or more of any character, the . character has no special significance, the ! character (if used at the start of an expression) performs a logical NOT, and the @ character at the start of a newsgroup name means "Do not forward any articles that are posted or crossposted to this group." The list is read and parsed from left to right, so you should ensure that you place the more specific rules first. The pattern:

rec.crafts.brewing*,!rec.crafts.brewing.poison,@rec.crafts.brewing.private

would send all of the rec.crafts.brewing news heirarchy except the rec.crafts.brewing.poison. It would not feed any articles that were either posted or crossposted to the rec.crafts.brewing.private newsgroup; these articles will be trapped and available only to those people who use this server. If you reversed the first two patterns, the first pattern would be overridden by the second and you would end up feeding articles for the rec.crafts.brewing.poison newsgroup. The same is true of the first and last patterns; you must always place the more specific patterns before any less specific patterns for them to take effect.

flags controls and places constraints on the feed of news articles to this site. The flags field is a comma delimited list can contain any of the items from the following list, delimited by commands:

<size

Article must be less then size bytes.

A items

Article checks. items can be one or more of d (must have Distribution header) or p (don't check for site in Path header).

B high/low

Internal buffer size before writing to output.

H[count]

Article must have less then count hops; the default is 1.

I size

Internal buffer size (for a file feed).

M pattern

Only moderated groups that match the pattern.

N pattern

Only unmoderated groups that match the pattern.

S size

Start spooling if more than size bytes get queued.

T type

Feed types: f (file), m (funnel; the param field names the entry that articles will be funneled to), p (pipe to program), c (send to stdin channel of the param field's subprocess), and x (like c, but handles commands on stdin).

W items

What to write: b (article bytesize), f (full path), g (first newsgroup), m (Message ID), n (relative path), s (site that fed article), t (time received), * (names of funnel feed-ins or all sites that get the article), N (newsgroups header), D (distribution header), H (all headers), O (overview data), and R (replication data).

The param field has special coding that is dependent on the type of feed. In the most common configuration it is where you specify the name of the output file to which you will write the outgoing feed. In other configurations you can leave it out. In yet other configurations it takes on different meanings. If you want to do something unusual, the newsfeeds(5) manual page will explain the use of the param field in some detail.

There is a special site name that should be coded as ME and should be the first entry in the file. This entry is used to control the default settings for your news feeds. If the ME entry has a distribution list associated with it, this list will be prepended to each of the other site entries before they are sent. This allows you to, for example, declare some newsgroups to be automatically fed, or automatically blocked from feeding, without having to repeat the pattern in each site entry.

We mentioned earlier that it was possible to use some special feeds to generate thread data that makes the newsreader's job easier. We'll do this by exploiting the overchan command that is part of the INN distribution. To do this, we've created a special local feed called overview that will pass the news articles to the overchan command for processing into overview data.

Our news server will provide only one external news feed, which goes to the Groucho Marx University, and they receive articles for all newsgroups except the control and junk newsgroups, the rec.crafts.brewing.private newsgroup, which will be kept locally, and the rec.crafts.brewing.poison newsgroup, which we don't want people from our brewery seen posting to.

We'll use the nntpsend command to transport the news via NNTP to the news.groucho.edu server. nntpsend requires us to use the "file" delivery method and to write the article's pathname and article ID. Note that we've set the param field to the name of the output file. We'll talk a little more about the nntpsend command in a moment. Our resulting newsfeed's configuration is:

# /etc/news/newsfeeds file for the Virtual Brewery # # Send all newsgroups except the control and junk ones by default ME:!control,!junk:: # # Generate overview data for any newsreaders to use. overview::Tc,WO:/usr/lib/news/bin/overchan # # Feed the Groucho Marx University everything except our private newsgroup # and any articles posted to the rec.crafts.brewing.poison newsgroup. gmarxu:!rec.crafts.brewing.poison,@rec.crafts.brewing.private:\         Tf,Wnm:news.groucho.edu #

The nntpsend.ctl file

The nntpsend program manages the transmission of news articles using the NNTP protocol by calling the innxmit command. We saw a simple use of the nntpsend command earlier, but it too has a configuration file that provides us with some flexibility in how we configure our news feeds.

The nntpsend command expects to find batch files for the sites it will feed. It expects those batch files to be named /var/spool/news/out.going/sitename . innd creates these batch files when acting on an entry in the newsfeeds, which we saw in the previous sections. We specified the sitename as the filename in the param field, and that satisfies the nntpsend command's input requirements.

The nntpsend command has a configuration file called nntpsend.ctl that is usually stored in the /etc/news/ directory.

The nntpsend.ctl file allows us to associate a fully qualified domain name, some news feed size constraints, and a number of transmission parameters with a news feed site name. The sitename is a means of uniquely identifying a logical feed of articles. The general format of the file is:

sitename:fqdn:max_size:[args]

The following list describes the elements of this format:

sitename

The sitename as supplied in the newsfeeds file

fqdn

The fully qualified domain name of the news server to which we will be feeding the news articles

max_size

The maximum volume of news to feed in any single transfer

args

Additional arguments to pass to the innxmit command

Our sample configuration requires a very simple nntpsend.ctl file. We have only one news feed. We'll restrict the feed to a maximum of 2 MB of traffic and we'll pass an argument to the innxmit that sets a 3-minute (180 second) timeout. If we were a larger site and had many news feeds, we'd simply create new entries for each new feed site that looked much the same as this one:

# /etc/news/nntpsend.ctl # gmarxu:news.groucho.edu:2m:-t 180 #

Controlling Newsreader Access

Not so many years ago, it was common for organizations to provide public access to their news servers. Today it is difficult to locate public news servers; most organizations carefully control who has access to their servers, typically restricting access to users supported on their network. INN provides configuration files to control this access.

The incoming.conf file

We mentioned in our introduction to INN that it achieves some of its efficiency and size by separating the news feed mechanism from the newsreading mechanism. The /etc/news/incoming.conf file is where you specify which hosts will be feeding you news using the NNTP protocol, as well as where you define some parameters that control the way articles are fed to you from these hosts. Any host not listed in this file that connects to the news socket will not be handled by the innd daemon; instead, it will be handled by the nnrpd daemon.

The /etc/news/incoming.conf file syntax is very simple, but it takes a moment to come to terms with. Three types of valid entries are allowed: key/value pairs, which are how you specify attributes and their values; peers, which is how you specify the name of a host allowed to send articles to us using NNTP; and groups, a means of applying key/value pairs to groups of peers. Key/value pairs can have three different types of scope. Global pairs apply to every peer defined in the file. Group pairs apply to all peers defined within that group. Peer pairs apply only to that one peer. Specific definitions override less specific ones: therefore, peer definitions override group definitions, which in turn override global pairs.

Curly brace characters ({}) are used to delimit the start and end of the group and peer specifications. The # character marks the rest of the line it appears on as a comment. Key/value pairs are separated by the colon character and appear one to a line.

A number of different keys may be specified. The more common and useful are:

hostname

This key specifies a comma-separated list of fully qualifed names or IP addresses of the peers that we'll allow to send us articles. If this key is not supplied, the hostname defaults to the label of the peer.

streaming

This key determines whether streaming commands are allowed from this host. It is a Boolean value that defaults to true.

max-connections

This key specifies the maximum number of connections allowed from this group or peer. A value of zero means unlimited (which can also be specified using none).

password

This key allows you to specify the password that must be used by a peer if it is to be allowed to transfer news. The default is to not require a password.

patterns

This key specifies the newsgroups that we accept from the associated peer. This field is coded according to precisely the same rules as we used in our newsfeeds file.

In our example we have only one host that we are expecting to feed us news: our upstream news provider at Groucho Marx University. We'll have no password, but we will ensure that we don't accept any articles for our private newsgroup from outside. Our hosts.nntp looks like:

# Virtual Brewery incoming.conf file.  # Global settings streaming:       true max-connections: 5  # Allow NNTP posting from our local host. peer ME {     hostname: "localhost, 127.0.0.1" }  # Allow groucho to send us all newsgroup except our local ones. peer groucho {     hostname: news.groucho.edu     patterns: !rec.crafts.brewing.private }

The nnrp.access file

We mentioned earlier that newsreaders, and in fact any host not listed in the hosts.nntp, that connect to the INN news server are handled by the nnrpd program. nnrpd uses the /etc/news/nnrp.access file to determine who is allowed to make use of the news server, and what permissions they should have.

The nnrp.access file has a similar structure to the other configuration files we've looked at. It comprises a set of patterns used to match against the connecting host's domain name or IP address, and fields that determine what access and permission it should be given. Each entry should appear on a line by itself, and fields are separated by colons. The last entry in this file that matches the connecting host will be the one used, so again, you should put general patterns first and follow them with more specific ones later in the file. The five fields of each entry in the order they should appear are:

Hostname or IP address

This field conforms to wildmat(3) pattern-matching rules. It is a pattern that describes the connecting host's name or IP address.

Permissions

This field determines what permissions the matching host should be granted. There are two permissons you may configure: R gives read permissions, and P gives posting permissions.

Username

This field is optional and allows you to specify a username that an NNTP client must log into the server before being allowed to post news articles. This field may be left blank. No user authentication is required to read articles.

Password

This field is optional and is the password accompanying the username field. Leaving this field blank means that no password is required to post articles.

Newsgroups

This field is a pattern specifying which newsgroups the client is allowed to access. The pattern follows the same rules as those used in the newsfeeds file. The default for this field is no newsgroups, so you would normally have a pattern configured here.

In the virtual brewery example, we will allow any NNTP client in the Virtual Brewery domain to both read and post to all newsgroups. We will allow any NNTP client read-only access to all newsgroups except our private internal newsgroup. Our nnrp.access file will look like this:

# Virtual Brewery - nnrp.access # We will allow public reading of all newsgroups except our private one. *:R:::*,!rec.crafts.brewing.private  # Any host with the Virtual Brewery domain may Read and Post to all  # newsgroups *.vbrew.com:RP::*

Expiring News Articles

When news articles are received by a news server, they are stored to disk. News articles need to be available to users for some period of time to be useful, so a large operating news server can consume lots of disk space. To ensure that the disk space is used effectively, you can opt to delete news articles automatically after a period of time. This is called article expiration. Naturally, INN provides a means of automatically expiring news articles.

The expire.ctl file

The INN server uses a program called expire to delete expired news articles. The expire program in turn uses a file called /etc/news/expire.ctl to configure the rules that govern article expiration.

The syntax of /etc/news/expire.ctl is fairly simple. As with most configuration files, empty lines or lines beginning with the # character are ignored. The general idea is that you specify one rule per line. Each rule defines how article expiration will be performed on newsgroups matching a supplied pattern. The rule syntax looks like this:

pattern:modflag:keep:default:purge

The following list describes the fields:

pattern

This field is a comma-delimited list of patterns matching names of newsgroups. The wildmat(3) routine is used to match these patterns. The last rule matching a newsgroup name is the one that is applied, so if you want to specify wildcard (*) rules, they should be listed first in this file.

modflag

This flag describes how this rule applies to moderated newsgroups. It can be coded with an M to mean that this rule applies only to moderated newsgroups, a U to mean that this rule applies only to unmoderated newsgroups, or an A to mean that this rule ignores the moderated status and applies to all groups.

keep

This field allows you to specify the minimum time an article with an "Expires" header will be kept before it is expired. The units are days, and are a floating point, so you may specify values like 7.5 for seven-and-a-half days. You may also specify never if you wish articles to stay in a newsgroup forever.

default

This field is the most important. This field allows you to specify the time an article without an Expires header will be kept. Most articles won't have an Expires header. This field is coded in the same way as the keep field, with never meaning that articles without Expires headers will never be expired.

purge

This field allows you to specify the maximum time an article with an Expires header will be kept before it is expired. The coding of this field is the same as for the keep field.

Our requirements are simple. We will keep all articles in all newsgroups for 14 days by default, and between 7 and 21 days for articles that have an Expires header. The rec.crafts.brewing.private newsgroup is our internal newsgroup, so we'll make sure we don't expire any articles from it:

# expire.ctl file for the Virtual Brewery  # Expire all articles in 14 days by default, 7-21 days for those with # Expires: headers *:A:7:14:21  # This is a special internal newsgroup, which we will never expire. rec.crafts.brewing.private:A:never:never:never

We will mention one special type of entry you may have in your /etc/news/expires.ctl file. You may have exactly one line that looks like this:

/remember/:days

This entry allows you to specify the minimum number of days that an article will be remembered in the history file, irrespective of whether the article itself has been expired or not. This might be useful if one of the sites that is feeding you articles is infrequent and has a habit of sending you old articles every now and again. Setting the /remember/ field helps to prevent the upstream server from sending you the article again, even if it has already been expired from your server. If your server remembers it has already received the article, it will reject attempts to resend it. It is important to remember that this setting has no effect at all on article expiration; it affects only the time that details of an article are kept in the history database.

Handling Control Messages

Just as with C News, INN can automatically process control messages. INN provides a powerful configuration mechanism to control what action will occur for each of a variety of control messages, and an access control mechanism to control who can initiate actions against which newsgroups.

The control.ctl file

The control.ctl file is fairly simple in structure. The syntax rules for this file are much the same as for the other INN configuration files. Lines beginning with # are ignored, lines may be continued using /, and fields are delimited by :.

When a control message is received, it is tested against each rule in turn. The last rule in the file that matches the message is the rule that will be used, so you should put any generic rules at the start of the file and more specific rules at the end of the file. The general syntax of the file is:

message:from:newsgroups:action

The meanings of each of the fields are:

message

This is the name of the control message. Typical control messages are described later.

from

This is a shell-style pattern matching the email address of the person sending the message. The email address is converted to lowercase before comparison.

newsgroups

If the control message is newgroup or rmgroup, this field is a shell-style pattern matching the newsgroup created or removed.

action

This field specifies what action to take for any message matching the rule. There are quite a number of actions we can take; they are described in the next list.

The message field of each line can have one of the following values:

checkgroups

This message requests that news administrators resynchonrize their active newsgroups database against the list of newsgroups supplied in the control message.

newgroup

This message requests the creation of a new newsgroup. The body of the control message should contain a short description of the purpose of the newsgroup to be created.

rmgroup

requests that a newsgroup be removed.

sendsys

This message requests that the sys file of this news server be transmitted by mail to the originator of the control message. RFC-1036 states that it is a requirement of Usenet membership that this information be publicly available because it is used to keep the map of Usenet up to date.

version

This message requests that the hostname and version of news server software be returned to the originator of the control message.

all

This is a special coding that will match any control message.

The message field may include the following actions:

doit

The requested command is performed. In many cases, a mail message will be sent to the administrator to advise them that the action has taken place.

doit=file

This is the same as the doit action except that a log message will be written to the file log file. If the specified file is mail, the log entry is sent by email. If the specified file is the null string, the log message is written to /dev/null and is equivalent to using the unqualified doit action. If the file name begins with a / character, the name is taken to be an absolute filename for the logfile; otherwise, the specified name is translated to /var/log/news/file.log.

doifarg

The requested command is performed if the command has an argument. If the command has no argument, the control message is ignored.

drop

The requested command is ignored.

log

A log message is sent to the stderr output of the innd process. This is normally directed out to the /var/log/news/errlog file.

log=file

This is the same as a log action, except the logfile is specified as per the rules given for the doit=file action.

mail

An email message is sent to the news administrator containing the requested command details. No other action takes place.

verify-*

If an action begins with the string "verify-", then the control message is authenticated using PGP (or GPG).[5]

[5] PGP and GPG are tools designed to authenticate or encrypt messages using public key techniques. GPG is the GNU free version of PGP. GPG may be found at http://www.gnupg.org/, and PGP may be found at http://www.pgp.com/.

So that you can see what a control.ctl file would look like in practice, here is a very short illustrative sample:

## Sample /etc/news/control.ctl ## ## Warning: You should not use this file, it is illustrative only.  ##      Control Message Handling all:*:*:mail checkgroups:*:*:mail ihave:*:*:drop sendme:*:*:drop sendsys:*:*:log=sendsys senduuname:*:*:log=senduuname version:*:*:log=version newgroup:*:*:mail rmgroup:*:*:mail  ##  Handle control messages for the eight most important news heirarchies ##  COMP, HUMANITIES, MISC, NEWS, REC, SCI, SOC, TALK checkgroups:*:comp.*|humanities.*|misc.*|news.*|rec.*|sci.*|soc.*|talk.*:drop newgroup:*:comp.*|humanities.*|misc.*|news.*|rec.*|sci.*|soc.*|talk.*:drop rmgroup:*:comp.*|humanities.*|misc.*|news.*|rec.*|sci.*|soc.*|talk.*:drop checkgroups:group-admin@isc.org:*:verify-news.announce.newgroups newgroup:group-admin@isc.org:comp.*|misc.*|news.*:verify-news.announce.newgroups newgroup:group-admin@isc.org:rec.*|sci.*|soc.*:verify-news.announce.newgroups newgroup:group-admin@isc.org:talk.*|humanities.*:verify-news.announce.newgroups rmgroup:group-admin@isc.org:comp.*|misc.*|news.*:verify-news.announce.newgroups rmgroup:group-admin@isc.org:rec.*|sci.*|soc.*:verify-news.announce.newgroups rmgroup:group-admin@isc.org:talk.*|humanities.*:verify-news.announce.newgroups  ## GNU ( Free Software Foundation ) newgroup:gnu@prep.ai.mit.edu:gnu.*:doit newgroup:news@*ai.mit.edu:gnu.*:doit rmgroup:gnu@prep.ai.mit.edu:gnu.*:doit rmgroup:news@*ai.mit.edu:gnu.*:doit  ## LINUX (Newsfeed from news.lameter.com) checkgroups:christoph@lameter.com:linux.*:doit newgroup:christoph@lameter.com:linux.*:doit rmgroup:christoph@lameter.com:linux.*:doit

Running INN

The inn source package provides a script suitable for starting inn at boot time. The script is usually called /usr/lib/news/bin/rc.news. The script reads arguments from another script, usually called /usr/lib/news/innshellvars, which contains definitions of the filenames and filepaths that inn will use to locate components it needs. It is generally considered a good idea to execute inn with the permissions of a non-root user, such as news.

To ensure that inn is started at boot time, you should check that /usr/lib/news/innshellvars is configured correctly and then call the /usr/lib/news/bin/rc.news script from a script executed at boot time.

Additionally, there are administrative tasks that must be performed periodically. These tasks are usually configured to be executed by the cron command. The best way to do this is to add the appropriate commands to your /etc/crontab file, or even better, create a file suitable for the /etc/cron.d directory, if your distribution provides one. An example of such a file might look like:

# Example /etc/cron.d/inn file, as used in the Debian distribution. # SHELL=/bin/sh PATH=/usr/lib/news/bin:/sbin:/bin:/usr/sbin:/usr/bin  # Expire old news and overview entries nightly, generate reports.  15 0 * * *      news    news.daily expireover lowmark delayrm  # Every hour, run an rnews -U. This is not only for UUCP sites, but # also to process queued up articles put there by in.nnrpd in case # innd wasn't accepting any articles.  10 * * * *      news    rnews -U

These commands will ensure that old news is automatically expired each day, and that any queued articles are processed each hour. Note also that they are executed with the permissions of the news user.

Managing INN: The ctlinnd Command

The INN news server comes with a command to manage its day-to-day operation. The ctlinnd command can be used to manipulate newsgroups and newsgroup feeds, to obtain the status, of the server, and to reload, stop, and start the server.

You'd normally get a summary of the ctlinnd command syntax using:

# ctlinnd -h

We'll cover some of the more important uses of ctlinnd here; please consult the ctlinnd manual page for more detail.

Add a New Group

Use the following syntax to add a new group:

ctlinnd newgroup group rest creator

The arguments are defined as follows:

group

The name of the group to create.

rest

This argument should be coded in the same way as the flags field of the active file. It defaults to y if not supplied.

creator

The name of the person creating the group. Enclose it in quotes if there are any spaces in the name.

Change a Group

Use the following syntax to change a group:

ctlinnd changegroup group rest

The arguments are defined as follows:

group

The name of the group to change.

rest

This argument should be coded in the same way as the flags field of the active file.

This command is useful to change the moderation status of a group.

Remove a Group

Use the following syntax to remove a group:

ctlinnd rmgroup group

The argument is defined as follows:

group

The name of the group to remove.

This command removes the specified newsgroup from the active file. It has no effect on the news spool. All articles in the spool for the specified group will be expired in the usual fashion, but no new articles will be accepted.

Renumber a Group

Use the following syntax to renumber a group:

ctlinnd renumber group

The argument is defined as follows:

group

The name of the group to renumber. If a group is an empty string, all groups are renumbered.

This command updates the low-water mark for the specified group.

Allow/Disallow Newsreaders

Use the following syntax to allow or disallow newsreaders:

ctlinnd readers flag text

The arguments are defined as follows:

flag

Specifying n causes all newsreader connections to be disallowed. Specifying y allows newsreader connections.

text

The text supplied will be given to newsreaders who attempt to connect, and usually describes the reason for disabling newsreader access. When reenabling newsreader access, this field must be either an empty string or a copy of the text supplied when the newsreader was disabled.

This command does not affect incoming newsfeeds. It only controls connections from newsreaders.

Reject Newsfeed Connections

Use the following syntax to reject newsfeed connections:

ctlinnd reject reason

The argument is defined as follows:

reason

The text supplied should explain why incoming connections to innd are rejected.

This command does not affect connections that are handed off to nnrpd (i.e., newsreaders); it only affects connections that would be handled by innd directly, such as remote newsfeeds.

Allow Newsfeed Connections

Use the following syntax to allow newsfeed connections:

ctlinnd allow reason

The argument is defined as follows:

reason

The supplied text must be the same as that supplied to the preceding reject command or an empty string.

This command reverses the effect of a reject command.

Disable News Server

Use the following syntax to disable the news server:

ctlinnd throttle reason

The argument is defined as follows:

reason

The reason for throttling the server.

This command is simultaneously equivalent to a newsreaders no and a reject, and is useful when emergency work is performed on the news database. It ensures that nothing attempts to update it while you are working on it.

Restart News Server

Use the following syntax to restart the news server:

ctlinnd go reason

The argument is defined as follows:

reason

The reason given when stopping the server. If this field is an empty string, the server will be reenabled unconditionally. If a reason is given, only those functions disabled with a reason matching the supplied text will be restarted.

This command is used to restart a server function after a throttle, pause, or reject command.

Display Status of a Newsfeed

Use the following syntax to display the status of a newsfeed:

ctlinnd feedinfo site

The argument is defined as follows:

site

The site name (taken from the newsfeeds file) for which you wish to display the newsfeed's status.

Drop a Newsfeed

Use the following syntax to drop a newsfeed:

ctlinnd drop site

The argument is defined as follows:

site

The name of the site (taken from the newsfeeds file) to which feeds are dropped. If this field is an empty string, all active feeds will be dropped.

Dropping a newsfeed to a site halts any active feeds to the site. It is not a permanent change. This command would be useful if you've modified the feed details for a site and a feed to that site is active.

Begin a Newsfeed

Use the following syntax to begin a newsfeed:

ctlinnd begin site

The argument is defined as follows:

site

The name of the site from the newsfeeds file to which feeds are started. If a feed to the site is already active, a drop command is done first automatically.

This command causes the server to reread the newsfeeds file, locate the matching entry, and commence a newsfeed to the named site using the details found. You can use this command to test a new news feed to a site after you've added or modified its entry in the newsfeeds file.

Cancel an Article

Use the following syntax to cancel an article:

ctlinnd cancel Message-Id

The argument is defined as follows:

Message-ID

The ID of the article to be cancelled.

This command causes the specified article to be deleted from the server. It does not generate a cancel message.