The Idea of a Terminal Database (Unix Power Tools, 3rd Edition)

5.2. The Idea of a Terminal Database

In the past few years, terminals have been standardized to a few types. In fact, most terminals nowadays are terminal emulators (like xterm) that simulate a terminal on a graphical display. Years ago, though, terminals differed widely. Rather than simply being implemented in software, they were hardware -- keyboards and monitors or even teletypes, with which the user interacted to communicate with an often faraway mainframe or other big iron. All were specialized, and differences between them often came down to how much you paid and to what manufacturer. This lets you take advantage of other features of the manufacturer's primary hardware -- the big computers they considered their main product. Manufacturers produced a variety of terminals, each one including a particular set of features for a certain price. There were smart terminals and dumb ones, terminals with big screens and terminals with small screens, printing terminals and video displays, and terminals with all sorts of special features.

Differences between terminals do not matter much to programs like cat (Section 12.2) or who (Section 2.8) that use the terminal screen as a sort of typewriter with an endless scroll of paper. These programs produce sequential output and do not make use of the terminal's special features; they do not need to know much to do their job. Only programs such as screen editors, which make use of screen-handling features, need to know a lot about differences between terminals.

However, even today, we find a wide variety of terminal emulators across a multitude of platforms. My new Kyocera Smartphone, for example, is a Palm device integrated with a PCS telephone; one of the main reasons I bought it was for remote, emergency ssh access to my servers, using a tiny terminal emulator that runs on the PalmOS. Many Unix programs assume a basic environment that this terminal emulator does not provide -- an 80-column screen -- so even simple commands such as w, which prints a list of who is logged in, where they logged in from, and what they're currently running, become impossible to run. But let's go back to the early days and revisit some of the old problems that plagued early Unix developers, so that we might better understand how to deal with today's problems.

In the late 1970s, Bill Joy created the vi (Section 17.2) text editor at UC Berkeley. Like all screen-oriented editors, vi uses the terminal screen nonsequentially (in stark contrast to earlier editors such as ed, which were designed for a teletype, and so use even more terse commands and feature even more terse output). A program performing nonsequential output does not just print character after character, but must manipulate the text that was sent before, scroll the page, move the cursor, delete lines, insert characters, and more. While it would be possible to keep redrawing the screen in its entirety, many features are provided in hardware or firmware by the terminal itself, saving too much time and trouble to be ignored.

The first version of vi was written specifically for Lear Siegler ADM3a terminals. vi was such an improvement over line-oriented editors that there was great demand to port vi to other brands of terminals. The problem was that each terminal had different features and used different control codes to manipulate the features that they did have in common.

Rather than write separate terminal drivers for each terminal type, Bill Joy did something very clever, which all Unix users now take for granted. He wrote a version of vi with generic commands to manipulate the screen instead of hardcoding the control codes and dimensions for a particular terminal.[15]

[15]When we refer to terminals throughout this and other chapters, understand that we mean, more often than not, the set of standard terminal-emulation control codes implemented by terminal emulators, such as vt100 or ANSI color. So, though we may refer to a vt100 terminal, we're more likely referring to any terminal-emulator software that can understand and react to that set of control codes.

Joy came up with a generic terminal-handling mechanism that had two parts: a database describing the capabilities of each of the terminals to be supported and a subroutine library that allows programs to query that database and make use of the capability values it contains. Both the library and the database were given the name termcap, which is short for terminal capabilities.

At this point, users take for granted that you can use just about any terminal with a Unix system and use screen-oriented programs like vi without any problem. But this is really quite remarkable!

The termcap database is contained in a single text file, which grew quite large over the years to include descriptions of hundreds of different terminals. To improve performance, AT&T later introduced a database called terminfo, which stores terminal descriptions in compiled form in a separate file for each terminal.

If a program is designed to use termcap or terminfo, it queries an environment variable called TERM to determine the terminal type (or terminal type being emulated), then looks up the entry for that terminal in the terminal database, and reads the definition of any capabilities it plans to use as external variables. Programs that use termcap or terminfo range from screen editors like vi and emacs (Section 19.1), which use the complete terminal description, to a program like clear, which needs to know only one capability (the escape sequence to clear the screen). Other programs include more, pg, rogue, tset (Section 5.3), ul, and nroff.

--JS and SJC