3.3. Files managed under NISNow that we've walked through the setup procedure, we will discuss how the NIS maps relate to the files that they replace. In particular, we'll discuss how to modify the files that are appended by NIS so they can take advantage of NIS features. We will also pay special attention to the netgroups NIS map, a confusing but nevertheless important part of the overall picture. Table 3-2 lists the most common files managed by NIS. Not all vendors use NIS for all of these files, so it is best to check your documentation for a list of NIS-supported files.
Table 3-2. Summary of NIS maps
It's now time to face up to some distortions we've been making for the sake of simplicity. We've assumed that there's a one-to-one correspondence between files and maps. In fact, there are usually several maps for each file. A map really corresponds to a particular way of accessing a file: for example, the passwd.byname map looks up data in the password database by username. There's also a passwd.byuid that looks up users according to their user ID number. There could be (but there aren't) additional maps that looked up users on the basis of their group ID number, home directory, or even their choice of login shell. To make things a bit easier, the most commonly used maps have "nicknames," which correspond directly to the name of the original file: for example, the nickname for passwd.byname is simply passwd. Using nicknames as if they were map names rarely causes problems -- but it's important to realize that there is a distinction. It's also important to realize that nicknames are recognized by only two NIS utilities: ypmatch and ypcat.Another distortion: this is the first time we've seen the netid.byname map. On the master NIS server, this map is not based on any single source file, but instead is derived from information in the group, password, and hosts files, via /var/yp/Makefile. It contains one entry for each user in the password file. The data associated with the username is a list of every group to which the user belongs. The netid is used to determine group memberships quickly when a user logs in. Instead of reading the entire group map, searching for the user's name, the login process performs a single map lookup on the netid map. You usually don't have to worry about this map -- it will be built for you as needed -- but you should be aware that it exists. If NIS is not running, and if an NIS client has an /etc/netid file, then the information will be read from /etc/netid.
3.3.1. Working with the mapsEarlier, we introduced the concept of replaced files and appended files. Now, we'll discuss how to work with these files. First, let's review: these are important concepts, so repetition is helpful. If a map replaces the local file, the file is ignored once NIS is running. Aside from making sure that misplaced optimism doesn't lead you to delete the files that were distributed with your system, there's nothing interesting that you can do with these replaced files. We won't have anything further to say about them. Conversely, local files that are appended to by NIS maps are always consulted first, even if NIS is running. The password file is a good example of a file augmented by NIS. You may want to give some users access to one or two machines, and not include them in the NIS password map. The solution to this problem is to put these users into the local passwd file, but not into the master passwd file on the master server. The local password file is always read before getpwuid( ) goes to an NIS server. Password-file reading routines find locally defined users as well as those in the NIS map, and the search order of "local, then NIS" allows local password file entries to override values in the NIS map. Similarly, the local aliases file can be used to override entries in the NIS mail aliases map, setting up machine-specific expansion of one or more aliases. There is yet another group of files that can be augmented with data from NIS. These files are not managed by NIS directly, but you can add special entries referring to the NIS database (in particular, the netgroups map). Such files include hosts.equiv and .rhosts. We won't discuss these files in this chapter; we will treat them as the need arises. For example, we will discuss hosts.equiv in Chapter 12, "Network Security". Now we're going to discuss the special netgroups map. This new database is the basis for the most useful extensions to the standard administrative files; it is what prevents NIS from becoming a rigid, inflexible system. After our discussion of netgroups, we will pay special attention to the appended files.
3.3.2. NetgroupsIn addition to the standard password, group, and host file databases, NIS introduces a new database for creating sets of users and hosts called the netgroups map. The user and hostname fields are used to define groups (of hosts or users) for administrative purposes. For example, to define a subset of the users in the passwd map that should be given access to a specific machine, you can create a netgroup for those users. A netgroup is a set of triples of the form:
A single netgroup contains one or more of these triples. Host and usernames have their usual meanings, but a domain name in this instance refers to the NIS domain in which the netgroup is valid. If an entry in the triple is left blank, that field becomes a wildcard. If the entry is specified as a dash (-), the field can take no value. Netgroups are typically used to augment other maps and files; for example, adding a selected group of users to the password file. The definitions and behavior of netgroups are confusing because their syntax doesn't exactly match the way the netgroup information is used. Even though the netgroup syntax allows you to specify user and hostnames in the same triple, user and hostnames are rarely used together. For example, when a netgroup is used to add users to an NIS-managed password file, only the usernames are taken from the netgroup. The hostnames are ignored, because hostnames have no place in the password file. Similarly, when using a netgroup to grant filesystem access permissions to a set of NFS clients, only the hostname fields in the netgroup are used. Usernames are ignored in this case, which means a hostname will be included in the list even if - is used as the username in its triple. Some examples are helpful:(hostname, username, domain name)
In the first example, source is a group of three users; in this respect, the netgroup is similar to an entry in /etc/group. The source netgroup in this case grants no specific permissions, although it could be included in the password file for the source archive machine, granting selected users access to that host. The second example shows a definition for a set of hosts, and would be of no use in a password file. In the third example, stern and johnc are members of the trusted-users group when it is parsed for usernames. Hosts bitatron and corvette are members of trusted-users when it is parsed for hostnames. Note that there is no interpretation of the netgroup that associates user stern with host bitatron. In the fourth example, dave and jimc are members of dangerous-users, but no hosts are included in this group. The domain name field is used when multiple NIS domains exist on the same network and it is necessary to create a group that is valid in only one or the other domain. These groups are very different from those in /etc/group. The group file (or equivalent NIS map) explicitly grants permissions to users while the netgroup mechanism simply creates shorthand notations or nicknames. A netgroup can be used in many places where a user or hostname would appear, such as the password file or in the list of hosts that can access an NFS filesystem. You can also build netgroups from other netgroups. For example, you could create the netgroup hosts-n-users from the following entry:source (-,stern,nesales), (-,julie,nesales), (-,peter,nesales) trusted-hosts (bitatron,,), (corvette,,) trusted-users (bitatron,stern,), (corvette,johnc,) dangerous-users (,jimc,), (,dave,)
This netgroup contains all the members of both trusted-hosts and trusted-users. By using netgroups carefully, you can create special-purpose groups that can be managed separately. For example, you could create a group of "administrators" that can easily be added to the password list of every machine, or a group of "visitors" who are only added to the password files of certain machines. A final note about netgroups: they are accessible only through NIS. The library routines that have been modified to use NIS maps have also been educated about the uses of the netgroup map, and use the netgroup, password, and host maps together. If NIS is not running, netgroups are not defined. This implies that any netgroup file on an NIS client is ignored, because the NIS netgroup map replaces the local file. A local netgroup file does nothing at all. The uses of netgroups will be revisited as a security mechanism.hosts-n-users trusted-hosts, trusted-users
3.3.3. Hostname formats in netgroupsThe previous section used nonfully qualified hostnames, which are hostnames without a domain name suffix. This is the norm when using the hosts map in NIS to store hostnames. If you have hostnames that are available only in DNS, then you can and must use fully qualified hostnames in the netgroup map if you want those hosts to be members of a particular netgroup. See Chapter 5, "Living with Multiple Directory Servers " for more details on running NIS and DNS on the same network.
3.3.4. Integrating NIS maps with local filesFor files that are augmented by NIS maps, you typically strip the local file to the minimum number of entries needed for bootstrap or single-user operation. You then add in entries that are valid only on the local host -- for example, a user with an account on only one machine -- and then integrate NIS services by adding special entries that refer to the NIS map files. The /etc/nsswitch.conf file is used to control how NIS maps and local files are integrated. Normally if the two are integrated, the file is interpreted first, followed by the NIS map. For example, look at the passwd entry in the default nsswitch.conf for NIS clients:
The keyword files tells the system to read /etc/passwd first, and if the desired entry is not found, search passwd.byname or passwd.byuid, depending on whether the system is searching by account name or user identifier number. The reason why the passwd file is examined before the NIS map is that some accounts, such as root, are not placed in NIS, for security reasons (see Section 3.2.2, "Installing the NIS master server" in this chapter). If NIS were searched before the local passwd file, and if root were in NIS, then there would effectively be one global password for root. This is not desirable, because once an attacker figured out the root password for one system, he'd know the root password for all systems. Or, even if root were not in NIS, if clients were configured to read NIS before files for passwd information, the attacker that successfully compromised a NIS server, would be able to insert a root entry in the passwd map and gain access to every client.passwd: files nis
TIP: The default files and NIS integration will have your clients getting hostname and address information from NIS. Since you will likely have DNS running, you will find it better to get host informaton from DNS. See Chapter 5, "Living with Multiple Directory Servers ".At this point, we've run through most of what you need to know to get NIS running. With this background out of the way, we'll look at how NIS works. Along the way, we will give more precise definitions of terms that, until now, we have been using fairly loosely. Understanding how NIS works is essential to successful debugging. It is also crucial to planning your NIS network. NIS is built on the RPC protocol, and uses the UDP transport to move requests from the client host to the server. NIS services are integrated into the standard Unix library calls so that they remain transparent to processes that reference NIS-managed files. If you have a process that reads /etc/passwd, most of the queries about that file will be handled by NIS RPC calls to an NIS server. The library calling interface used by the application does not change at all, but the implementations of library routines such as getpwuid( ) that read the /etc/passwd file are modified to refer to NIS or to NIS and local files. The application using getpwuid( ) is oblivious to the change in its implementation. Therefore, when you enable NIS, you don't have to change any existing software. A vendor that supports NIS has already modified all of the relevant library calls to have them make NIS RPC calls in addition to looking at local files where relevant. Any process that used to do lookups in the host table still works; it just does something different in the depths of the library calls.
3.3.5. Map filesConfiguration files managed by NIS are converted into keyword and value pair tables called maps. We've been using the term "map" all along, as if a map were equivalent to the ASCII files that it replaces or augments. For example, we have said that the passwd NIS map is appended to the NIS client's /etc/passwd file. Now it's time to understand what a map file really is. NIS maps are constructed from DBM database files. DBM is the database system that is built into BSD Unix implementations; if it is not normally shipped as part of your Unix system, your vendor will supply it as part of the NIS implementation. Under DBM, a database consists of a set of keys and associated values organized in a table with fast lookup capabilities. Every key and value pair may be located using at most two filesystem accesses, making DBM an efficient storage mechanism for NIS maps. A common way to use the password file, for example, is to locate an entry by user ID number, or UID. Using the flat /etc/passwd file, a linear search is required, while the same value can be retrieved from a DBM file with a single lookup. This performance improvement in data location offsets the overhead of performing a remote procedure call over the network. Each DBM database, and therefore each NIS map, comprises two files: a hash-table accessed bitmap of indices and a data file. The index file has the .dir extension and the data file uses .pag. A database called addresses would be stored in:
A complete map contains both files. Consecutive records are not packed in the data file; they are arranged in hashed order and may have empty blocks between them. As a result, the DBM data file may appear to be up to four times as large as the data that it contains. The Unix operating system allows a file to have holes in it that are created when the file's write pointer is advanced beyond the end of the file using lseek( ). Filesystem data blocks are allocated only for those parts of the file containing data. The empty blocks are not allocated, and the file is only as large as the total number of used filesystem blocks and fragments. The holes in DBM files make them difficult to manipulate using standard Unix utilities. If you try to copy an NIS map using cp, or move it across a filesystem boundary with mv, the new file will have the holes expanded into zero-filled disk blocks. When cp reads the file, it doesn't expect to find holes, so it reads sequentially from the first byte until the end-of-file is found. Blocks that are not allocated are read back as zeros, and written to the new file as all zeros as well. This has the unfortunate side effect of making the copied DBM files consume much more disk space than the hole-filled files. Furthermore, NIS maps will not be usable on a machine of another architecture: if you build your maps on a SPARC machine, you can't copy them to an Intel-based machine. Map files are not ASCII files. For the administrator, the practical consequence is that you must always use NIS tools (like ypxfr and yppush, discussed in Section 4.2.1, "Map distribution") to move maps from one machine to another.addresses.dir index file addresses.pag data file
3.3.6. Map namingASCII files are converted into DBM files by selecting the key field and separating it from the value field by spaces or a tab. The makedbm utility builds the .dir and .pag files from ASCII input files. A limitation of the DBM system is that it supports only one key per value, so files that are accessed by more than one field value require an NIS map for each key field. With a flat ASCII file, you can read the records sequentially and perform comparisons on any field in the record. However, DBM files are indexed databases, so only one field -- the key -- is used for comparisons. If you need to search the database in two different ways, using two fields, then you must use two NIS maps or must implement one of the searches as a linear walk through all of the records in the NIS map. The password file is a good example of an ASCII file that is searched on multiple fields. The getpwnam( ) library call opens the password file and looks for the entry for a specific username. Equal in popularity is the getpwuid( ) library routine, which searches the database looking for the given user ID value. While getpwnam( ) is used by login and chown, getpwuid( ) is used by processes that need to match numeric user ID values to names, such as ls -l. To accommodate both access methods, the standard set of NIS maps includes two maps derived from the password file: one that uses the username as a key and one that uses the user ID field as a key. The map names used by NIS indicate the source of the data and the key field. The convention for map naming is:
The two NIS maps generated from the password file, for example, are passwd.byname (used by getpwnam( )) and passwd.byuid (used by getpwuid( )). These two maps are stored on disk as four files:filename.bykeyname
The order of the records in the maps will be different because they have different key fields driving the hash algorithm, but they contain exactly the same sets of entries.passwd.byname.dir passwd.byname.pag passwd.byuid.dir passwd.byuid.pag
3.3.7. Map structureTwo extra entries are added to each NIS map by makedbm. The master server name for the map is embedded in one entry and the map's order, or modification timestamp, is put in the other. These additional entries allow the map to describe itself fully, without requiring NIS to keep map management data. Again, NIS is ignorant of the content of the maps and merely provides an access mechanism. The maps themselves must contain timestamp and ownership information to coordinate updates with the master NIS server. Some maps are given nicknames based on the original file from which they are derived. Map nicknames exist only within the ypwhich and ypmatch utilities (see Section 13.4, "NIS tools") that retrieve information from NIS maps. Nicknames are neither part of the NIS service nor embedded in the maps themselves. They do provide convenient shorthands for referring to popular maps such as the password or hosts files. For example, the map nickname passwd refers to the passwd.byname map, and the hosts nickname refers to the hosts.byname map. To locate the password file entry for user stern in the passwd.byname map, use ypmatch with the map nickname:
In this example, ypmatch expands the nickname passwd to the map name passwd.byname, locates the key stern in that map, and prints the data value associated with the key. The library routines that use NIS don't retain any information from the maps. Once a routine looks up a hostname, for example, it passes the data back to the caller and "forgets" about the transaction. On Solaris, if the name service cache daemon (nscd) is running, then the results of queries from the passwd, group, and hosts maps are cached in the nscd daemon. Subsequent queries for the same entry will be satisfied out of the cache. The cache will keep the result of an NIS query until the entry reaches its time to live (ttl) threshold. Each cached NIS map has different time to live values. You can invoke nscd with the -g option to see what the time to live values are.% ypmatch stern passwd stern:passwd:1461:10:Hal Stern:/home/thud/stern:/bin/csh
3.3.8. NIS domains"Domain" is another term that we have used loosely; now we'll define domains more precisely. Groups of hosts that use the same set of maps form an NIS domain. All of the machines in an NIS domain will share the same password, hosts, and group file information. Technically, the maps themselves are grouped together to form a domain, and hosts join one or more of these NIS domains. For all practical purposes, though, an NIS domain includes both a set of maps and the machines using information in those map files. NIS domains define spheres of system management. A domain is a name applied to a group of NIS maps. The hosts that need to look up information in the maps bind themselves to the domain, which involves finding an NIS server that has the maps comprising the domain. It's easy to refer to the hosts that share a set of maps and the set of maps themselves interchangeably as a domain. The important point is that NIS domains are not just defined as a group of hosts; NIS domains are defined around a set of maps and the hosts that use these map files. Think of setting up NIS domains as building a set of database definitions. You need to define both the contents of the database and the users or hosts that can access the data in it. When defining NIS domains, you must decide if the data in the NIS maps applies to all hosts in the domain. If not, you may need to define multiple domains. This is equivalent to deciding that you really need two or more groups of databases to meet the requirements of different groups of users and hosts. As we've seen, the default domain name for a host is set using the domainname command:
This usually appears in the boot scripts as:nisclient# domainname nesales
Only the superuser can set or change the default domain. Without an argument, domainname prints the currently set domain name. Library calls that use NIS always request maps from the default domain, so setting the domain name must be the first step in NIS startup. It is possible for an application to request map information from more than one domain, but assume for now that all requests refer to maps in the current default domain. Despite the long introduction, a domain is implemented as nothing more than a subdirectory of the top-level NIS directory, /var/yp. Nothing special is required to create a new domain -- you simply assign it a name and then put maps into it using the server initialization procedures described later. The map files for a domain are placed in its subdirectory:/usr/bin/domainname `cat /etc/defaultdomain`
You can create multiple domains by repeating the initialization using different NIS domain names. Each new domain initialization creates a new subdirectory in the NIS map directory /var/yp. An NIS server provides service for every domain represented by a subdirectory in /var/yp. If multiple subdirectories exist, the NIS server answers binding requests for all of them. You do not have to tell NIS which domains to serve explicitly -- it figures this out by looking at the structure of its map directory. It's possible to treat NIS as another administrative tool. However, it's more flexible than a simple configuration file management system. NIS resembles a database management system with multiple tables. As long as the NIS server can locate map information with well-known file naming and key lookup conventions, the contents of the map files are immaterial to the server. A relational database system such as Oracle provides the framework of schemas and views, but it doesn't care what the schemas look like or what data is in the tables. Similarly, the NIS system provides a framework for locating information in map files, but the information in the files and the existence or lack of map files themselves is not of consequence to the NIS server. There is no minimal set of map files necessary to define a domain. While this places the responsibility for map synchronization on the system manager, it also affords the flexibility of adding locally defined maps to the system that are managed and accessed in a well-known manner./var/yp/domainname/mapname
126.96.36.199. Internet domains versus NIS domainsThe term "domain" is used in different ways by different services. In the Internet community, a domain refers to a group of hosts that are managed by an Internet Domain Name Service. These domains are defined strictly in terms of a group of hosts under common management, and are tied to organizations and their hierarchies. These domains include entire corporations or divisions, and may encompass several logical TCP/IP networks. The Internet domain east.sun.com, for example, spans six organizations spread over at least 15 states. Domains in the NIS world differ from Internet name service domains in several ways. NIS domains exist only in the scheme of local network management and are usually driven by physical limits or political "machine ownership" issues. There may be several NIS domains on one network, all managed by the same system administrator. Again, it is the set of maps and the hosts that use the maps that define an NIS domain, rather than a particular network partitioning. In general, you may find many NIS domains in an Internet name service domain; the name service's hostname database is built from the hostname maps in the individual NIS domains. Integration of NIS and name services is covered in Section 5.1, "Domain name servers". From here on, "domain" refers to an NIS domain unless explicitly noted.
3.3.9. The ypserv daemonNIS service is provided by a single daemon, ypserv, that handles all client requests. It's simple to tell whether a system is an NIS server: just look to see whether ypserv is running. In this section we'll look at the RPC procedures implemented as part of the NIS protocol and the facilities used to transfer maps from master to slave servers. Three sets of procedure calls make up the NIS protocol: client lookups, map maintenance calls, and NIS internal calls. Lookup requests are key-driven, and return one record from the DBM file per call. There are four kinds of lookups: match (single key), get-first, get-next, and get-all records. The get-first and get-next requests are used to scan the NIS map linearly, although keys are returned in a random order. "First" refers to the first key encountered in the data file based on hash table ordering, not the first key from the ASCII source file placed into the map. Map maintenance calls are used when negotiating a map transfer between master and slave servers, although they may be made by user applications as well. The get-master function returns the master server for a map and the get-order request returns the timestamp from the last generation of the map file. Both values are available as records in the NIS maps. Finally, the NIS internal calls are used to effect a map transfer and answer requests for service to a domain. An NIS server replies only positively to a service request; if it cannot serve the named domain it will not send a reply. The server daemon does not have any intrinsic knowledge of what domains it serves or which maps are available in those domains. It answers a request for service if the domain has a subdirectory in the NIS server directory. That is, a request for service to domain polygon will be answered if the /var/yp/polygon directory exists. This directory may be empty, or may not contain a full complement of maps, but the server still answers a service request if the map directory exists. There is no NIS RPC procedure to inquire about the existence of a map on a server; a "no such map" error is returned on a failed lookup request for the missing map. This underscores the need for every NIS server to have a full set of map files -- the NIS mechanism itself can't tell when a map is missing until an NIS client asks for information from it. If the log file /var/yp/ypserv.log exists when ypserv is started, error and warning messages will be written to this file. If an NIS server receives a service request for a domain it cannot serve, it logs messages such as:
indicating that it ignored a broadcast request for an unknown domain. If each server handles only its default domain, binding attempts overheard from other domains generate large numbers of these log messages. Running multiple NIS domains on a single IP network is best done if every server can handle every domain, or if you turn off logging. If not, you will be overwhelmed with these informational error messages that do nothing but grow the log file. ypserv keeps the file open while it is running, so a large log file must be cleaned up by truncating it:ypserv: Domain financials not supported (broadcast)
Removing the file with rm clears the directory entry, but does not free the disk space because the ypserv process still has the file open. If you have multiple domains with distinct servers on a single network, you probably shouldn't enable NIS logging.# cat /dev/null > /var/yp/ypserv.log
3.3.10. The ypbind daemonThe ypbind daemon is central to NIS client operation. Whenever any system is running ypbind, it is an NIS client -- no matter what else it is doing. Therefore, it will be worth our effort to spend some time thinking about ypbind. When ypbind first starts, it finds a server for the host's default domain. The process of locating a server is called binding the domain. If processes request service from other domains, ypbind attempts to locate servers for them as needed. ypbind reads a file like /var/yp/binding/bedrock/ypservers to get the name of an NIS server to bind to. If the NIS server chosen for a domain crashes or begins to respond slowly due to a high load, ypbind selects the next NIS server in the ypservers file to re-bind. The NIS timeout period varies by implementation, but is usually between two and three minutes. Each client can be bound to several domains at once; ypbind manages these bindings and locates servers on demand for each newly referenced NIS domain. A client in the NIS server-client relationship is not just a host, but a process on that host that needs NIS map information. Every client process must be bound to a server, and they do so by asking ypbind to locate a server on their behalf. ypbind keeps track of the server to which it is currently directing requests, so new client binding requests can be answered without having to contact an NIS server. ypbind continues to use its current server until it is explicitly told, as the result of an NIS RPC timeout, that the current server is not providing prompt service. After an RPC timeout, ypbind will try the next server in the ypservers file in an attempt to locate a faster NIS server. Because all client processes go through ypbind, we usually don't make a distinction between the client processes and the host on which they are running -- the host itself is called the NIS client. Once ypbind has created a binding between a client and a server, it never talks to the server again. When a client process requests a binding, ypbind simply hands back the name of the server to which the queries should be directed. Once a process has bound to a server, it can use that binding until an error occurs (such as a server crash or failure to respond). A process does not bind its domain before each NIS RPC call. Domain bindings are shown by ypwhich:
Here, ypwhich reports the currently bound server for the named domain. If the default or the named domain is not bound, ypwhich reports an error:% domainname nesales % ypwhich wahoo
An NIS client can be put back in standalone operation by modifying /etc/nsswitch.conf:gonzo% ypwhich -d financials Domain financials not bound on gonzo
client# cp /etc/nsswitch.files /etc/nsswitch.conf
3.3.11. NIS server as an NIS clientPreviously, we recommended that NIS servers also be NIS clients. This has a number of important effects on the network's behavior. When NIS servers are booted, they may bind to each other instead of to themselves. A server that is booting executes a sequence of commands that keep it fairly busy; so the local ypbind process may timeout trying to bind to the local NIS server, and bind successfully with another host. Thus multiple NIS servers usually end up cross-binding -- they bind to each other instead of themselves. If servers are also NIS clients, then having only one master and one slave server creates a window in which the entire network pauses if either server goes down. If the servers have bound to each other, and one crashes, the other server rebinds to itself after a short timeout. In the interim, however, the "live" server is probably not doing useful work because it's waiting for an NIS server to respond. Increasing the number of slave servers decreases the probability that a single server crash hangs other NIS servers and consequently hangs their bound clients. In addition, running more than two NIS servers prevents all NIS clients from rebinding to the same server when an NIS server becomes unavailable.
Copyright © 2002 O'Reilly & Associates. All rights reserved.