Chapter 17. Usenet News
Usenet has been around since late 1979. The current implementation is based on the Network NewsTransfer Protocol (NNTP), defined in RFC 977 and released in March 1986. Information is propagated through Usenet by a system of newsfeeds in which one site requests a newsfeed from another site, and a third site requests a newsfeed from the second site, etc. There is no central Usenet authority—like Perl, it runs on the spirit of cooperation and sharing. When you run a news reader, such as tin or the news reader in a web browser, your client software talks to the NNTP server on the news host. When you post a message to a newsgroup, this posting is received by your NNTP server and passed on to other servers throughout the distribution area you specified for the posting. Each server periodically receives updated newsgroup information and newly posted news articles.
This chapter explores NNTP commands and responses. It introduces Net::NNTP, which implements NNTP commands and simplifies the process of writing a Perl-based NNTP news client. It also describes News::Newsrc, a module that provides methods for managing a .newsrc file.
There are two kinds of NNTP commands: the official set of commands as defined in RFC 977 and a number of extensions that have been added since the RFC was written. The extensions are described in an IETF Internet Draft document, "Common NNTP Extensions" by Stan Barber, which can be found at ftp://ftp.academ.com/pub/nntp/ietf/nntpext.txt.
17.1. The NNTP Protocol
Before you write your own news client, you should have some idea of how the NNTP protocol works. Like other servers, an NNTP server is bound to a port (usually port 119). It listens for incoming connections, takes the appropriate action, and returns a response to the client. When a news client connects with an NNTP server, or to the port on which the NNTP server is running, a message such as the following is produced:
Trying 0.0.0.0... Connected to hostname.mydomain.com. Escape character is '^]'. 200 newshost.mydomain.com InterNetNews NNRP server INN 1.5.1 17-Dec-1996 ready (posting ok).
Many NNTP servers understand the help (or HELP) command. When a client issues a help command, many NNTP servers respond with a list of all available commands. For example:
200 news.mydomain.com InterNetNews NNRP server INN 1.5.1 17-Dec-1996 ready (posting ok). HELP 100 Legal commands authinfo user Name|pass Password|generic <prog> <args> article [MessageID|Number] body [MessageID|Number] date group newsgroup head [MessageID|Number] help ihave last list [active|active.times|newsgroups|distributions|distrib.pats| overview.fmt|subscriptions] listgroup newsgroup mode reader newgroups yymmdd hhmmss ["GMT"] [<distributions>] newnews newsgroups yymmdd hhmmss ["GMT"] [<distributions>] next post slave stat [MessageID|Number] xgtitle [group_pattern] xhdr header [range|MessageID] xover [range] xpat header range|MessageID pat [morepat...] xpath MessageID Report problems to <email@example.com> .
After connecting to the NNTP server, you can get a list of available newsgroups with the list active command, and you can then select a newsgroup. If you were to connect directly to news.mydomain.com and use the group command to select the newsgroup local.test, your session might look like this:
200 news.mydomain.com InterNetNews NNRP server INN 1.5.1 17-Dec-1996 ready (posting ok). group local.test 211 4 1 4 local.test QUIT 205 .
The four numbers (in this example, 211 4 1 4) preceding the group name (local.test) represent the success code, total number of articles, and the first and last article numbers, respectively. So in this example, 211 is the success code. There are four articles in the local.test group, starting with article number 1 and ending with number 4.
Every article in a newsgroup has two identifiers associated with it. The first is the message number, and the second is a message ID. The message number is the number of that article on the server to which you are connecting. The message ID is an identifier that is always associated with the article on every server that receives the article. An article that has the message number 4 on your server might be number 83 on someone else's server. The message ID, however, will be the same on both. So, for example, if you want to post an article that refers to another article, use the message ID.
Once the group command has successfully identified a newsgroup, the client can request a particular article by sending an article request to the server. The article command takes either a message ID or a message number as an argument. For example:
article 4 Path: newshost.mydomain.com!news-w.ans.net!newsfeeds.ans.net!philabs ... From: firstname.lastname@example.org (User Name) Newsgroups: local.test Subject: Is anybody there? Date: 21 Apr 1997 ...
If the article command is successful, the server returns the selected message.
The xhdr command returns the selected headers ("X-headers") from articles in a certain newsgroup. The X-headers contain all the information about the news article, including the poster's email address, the subject of the message, the date and time the message was posted, the newsgroup(s) the message appears in, and the message ID. For example, if you want to see the subjects of the articles in the local.test newsgroup, you can run xhdr subject 1-4, as follows:
xhdr subject 1-4 221 subject fields follow 1 Is anybody there? 2 Re: Is anybody there? 3 Re: Is anybody there? 4 Get a life!
The 221 status code indicates that the xhdr completed successfully and returned the requested headers. A Usenet client might display these headers to the user, who can then select the articles to read based on the subject lines.
xpat subject 1-9 *anybody* 221 subject matches follow. 1 Is anybody there? 2 Re: Is anybody there? 3 Re: Is anybody there?
You can also post news articles with NNTP. When you issue the post command, the server returns a code of 340 if posting is permitted. If you get an error, such as 501, you cannot post articles with this command. Most servers require that you include Newsgroups, Subject, and From headers. You will receive a 240 reply if your message has been successfully posted. Here's an example of posting to local.test using NNTP:
200 news.mydomain.com InterNetNews NNRP server INN 1.5.1 17-Dec-1996 ready (posting ok). post 340 Ok Newsgroups: local.test Subject: talking NNTP From: email@example.com This is a test posting. . 240 Article posted QUIT 205 .
The server returns a code of 205 upon disconnecting. When the article arrives in local.test, it looks like this:
From news.mydomain.com!not-for-mail Sun Mar 9 19:18:37 1997 Path: news.mydomain.com!not-for-mail From: firstname.lastname@example.org Newsgroups: local.test Subject: talking NNTP Date: 10 Mar 1997 00:09:33 GMT Organization: Perl in a Nutshell Lines: 2 Distribution: local Message-ID: <email@example.com> NNTP-Posting-Host: newshost.mydomain.com Xref: newshost.mydomain.com local.test:6 This is a test posting.
Copyright © 2002 O'Reilly & Associates. All rights reserved.