Managing map files (Managing NFS and NIS, 2nd Edition)

4.2. Managing map files

Keeping map files updated on all servers is essential to the proper operation of NIS. There are two mechanisms for updating map files: using make and the NIS Makefile, which pushes maps from the master server to the slave servers, and the ypxfr utility, which pulls maps from the master server. This section starts with a look at how map file updates are made and how they get distributed to slave servers.

Having a single point of administration makes it easier to propagate configuration changes through the network, but it also means that you may have more than one person changing the same file. If there are several system administrators maintaining the NIS maps, they need to coordinate their efforts, or you will find that one person removes NIS map entries added by another. Using a source code control system, such as SCCS or RCS, in conjunction with NIS often solves this problem. In the second part of this section, we'll see how to use alternate map source files and source code control systems with NIS.

4.2.1. Map distribution

Master and slave servers are distinguished by their ability to effect permanent changes to NIS maps. Changes may be made to an NIS map on a slave server, but the next map transfer from the master will overlay this change. Modify maps only on the master server, and push them from the master server to its slave servers. On the NIS master server, edit the source file for the map using your favorite text editor. Source files for NIS maps are listed in Table 3-1. Then go to the NIS map directory and build the new map using make, as shown here:

# vi /etc/hosts 
# cd /var/yp 
# make 
...New hosts map is built and distributed...

Without any arguments, make builds all maps that are out-of-date with respect to their ASCII source files. When more than one map is built from the same ASCII file, for example the passwd.byname and passwd.byuid maps built from /etc/passwd, they are all built when make is invoked.

When a map is rebuilt, the yppush utility is used to check the order number of the same map on each NIS server. If the maps are out-of-date, yppush transfers the map to the slave servers, using the server names in the ypservers map. Scripts to rebuild maps and push them to slave servers are part of the NIS Makefile, which is covered in Section 4.2.3, "Map file dependencies".

Map transfers done on demand after source file modifications may not always complete successfully. The NIS slave server may be down, or the transfer may timeout due to severe congestion or server host loading. To ensure that maps do not remain out-of-date for a long time (until the next NIS map update), NIS uses the ypxfr utility to transfer a map to a slave server. The slave transfers the map after checking the timestamp on its copy; if the master's copy has been modified more recently, the slave server will replace its copy of the map with the one it transfers from the server. It is possible to force a map transfer to a slave server, ignoring the slave's timestamp, which is useful if a map gets corrupted and must be replaced. Under Solaris, an additional master server daemon called ypxfrd is used to speed up map transfer operations, but the map distribution utilities resort to the old method if they cannot reach ypxfrd on the master server.

The map transfer -- both in yppush and in ypxfr -- is performed by requesting that the slave server walk through all keys in the modified map and build a map containing these keys. This seems quite counterintuitive, since you would hope that a map transfer amounts to nothing more than the master server sending the map to the slave server. However, NIS was designed to be used in a heterogeneous environment, so the master server's DBM file format may not correspond to that used by the slave server. DBM files are tightly tied to the byte ordering and file block allocation rules of the server system, and a DBM file must be created on the system that indexes it. Slave servers, therefore, have to enumerate the entries in an NIS map and rebuild the map from them, using their own local conventions for DBM file construction. Indeed, it is theoretically possible to have NIS server implementation that does not use DBM. When the slave server has rebuilt the map, it replaces its existing copy of the map with the new one. Schedules for transferring maps to slave servers and scripts to be run out of cron are provided in the next section.

4.2.2. Regular map transfers

Relying on demand-driven updates is overly optimistic, since a server may be down when the master is updated. NIS includes the ypxfr tool to perform periodic transfers of maps to slave servers, keeping them synchronized with the master server even if they miss an occasional yppush. The ypxfr utility will transfer a map only if the slave's copy is out-of-date with respect to the master's map.

Unlike yppush, ypxfr runs on the slave. ypxfr contacts the master server for a map, enumerates the entries in the map, and rebuilds a private copy of the map. If the map is built successfully, ypxfr replaces the slave server's copy of the map with the newly created one. Note that doing a yppush from the NIS master essentially involves asking each slave server to perform a ypxfr operation if the slave's copy of the map is out-of-date. The difference between yppush and ypxfr (besides the servers on which they are run) is that ypxfr retrieves a map even if the slave server does not have a copy of it, while yppush requires that the slave server have the map in order to check its modification time.

ypxfr map updates should be scheduled out of cron based on how often the maps change. The passwd and aliases maps change most frequently, and could be transferred once an hour. Other maps, like the services and rpc maps, tend to be static and can be updated once a day. The standard mechanism for invoking ypxfr out of cron is to create two or more scripts based on transfer frequency, and to call ypxfr from the scripts. The maps included in the ypxfr_1perhour script are those that are likely to be modified several times during the day, while those in ypxfr_2perday, and ypxfr_1perday may change once every few days:

ypxfr_1perhour script: 
/usr/lib/netsvc/yp/ypxfr passwd.byuid 
/usr/lib/netsvc/yp/ypxfr passwd.byname 
 
ypxfr_2perday script: 
/usr/lib/netsvc/yp/ypxfr hosts.byname
/usr/lib/netsvc/yp/ypxfr hosts.byaddr
/usr/lib/netsvc/yp/ypxfr ethers.byaddr
/usr/lib/netsvc/yp/ypxfr ethers.byname
/usr/lib/netsvc/yp/ypxfr netgroup
/usr/lib/netsvc/yp/ypxfr netgroup.byuser
/usr/lib/netsvc/yp/ypxfr netgroup.byhost
/usr/lib/netsvc/yp/ypxfr mail.aliases 

ypxfr_1perday script: 
/usr/lib/netsvc/yp/ypxfr group.byname
/usr/lib/netsvc/yp/ypxfr group.bygid 
/usr/lib/netsvc/yp/ypxfr protocols.byname
/usr/lib/netsvc/yp/ypxfr protocols.bynumber
/usr/lib/netsvc/yp/ypxfr networks.byname
/usr/lib/netsvc/yp/ypxfr networks.byaddr
/usr/lib/netsvc/yp/ypxfr services.byname
/usr/lib/netsvc/yp/ypxfr ypservers

crontab entry: 
0 *    * * * /usr/lib/netsvc/yp/ypxfr_1perhour 
0 0,12 * * * /usr/lib/netsvc/yp/ypxfr_2perday
0 0    * * * /usr/lib/netsvc/yp/ypxfr_1perday

ypxfr logs its activity on the slave servers if the log file /var/yp/ypxfr.log exists when ypxfr starts.

4.2.3. Map file dependencies

Dependencies of NIS maps on ASCII source files are maintained by the NIS Makefile, located in the NIS directory /var/yp on the master server. The Makefile dependencies are built around timestamp files named after their respective source files. For example, the timestamp file for the NIS maps built from the password file is passwd.time, and the timestamp for the hosts maps is kept in hosts.time.

The timestamp files are empty because only their modification dates are of interest. The make utility is used to build maps according to the rules in the Makefile, and make compares file modification times to determine which targets need to be rebuilt. For example, make compares the timestamp on the passwd.time file and that of the ASCII /etc/passwd file, and rebuilds the NIS passwd map if the ASCII source file was modified since the last time the NIS passwd map was built.

After editing a map source file, building the map (and any other maps that may depend on it) is done with make:

# cd /var/yp 
# make passwd    Rebuilds only password map. 
# make           Rebuilds all maps that are out-of-date.

If the source file has been modified more recently than the timestamp file, make notes that the dependency in the Makefile is not met and executes the commands to regenerate the NIS map. In most cases, map regeneration requires that the ASCII file be stripped of comments, fed to makedbm for conversion to DBM format, and then pushed to all slave servers using yppush.

Be careful when building a few selected maps; if other maps depend on the modified map, then you may distribute incomplete map information. For example, Solaris uses the netid map to combine password and group information. The netid map is used by login shells to determine user credentials: for every user, it lists all of the groups that user is a member of. The netid map depends on both the /etc/passwd and /etc/group files, so when either one is changed, the netid map should be rebuilt.

But let's say you make a change to the /etc/groups file, and decide to just rebuild and distribute the group map:

nismaster# cd /var/yp 
nismaster# make group

The commands in this example do not update the netid map, because the netid map doesn't depend on the group map at all. The netid map depends on the /etc/group file -- as does the group map -- but in the previous example, you would have instructed make to build only the group map. If you build the group map without updating the netid map, users will become very confused about their group memberships: their login shells will read netid and get old group information, even though the NIS map source files appear correct.

The best solution to this problem is to build all maps that are out-of-date by using make with no arguments:

nismaster# cd /var/yp 
nismaster# make

Once the map is built, the NIS Makefile distributes it, using yppush, to the slave servers named in the ypservers map. yppush walks through the list of NIS servers and performs an RPC call to each slave server to check the timestamp on the map to be transferred. If the map is out-of-date, yppush uses another RPC call to the slave server to initiate a transfer of the map.

A map that is corrupted or was not successfully transferred to all slave servers can be explicitly rebuilt and repushed by removing its timestamp file on the master server:

master# cd /var/yp 
master# rm hosts.time 
master# make hosts

This procedure should be used if a map was built when the NIS master server's time was set incorrectly, creating a map that becomes out-of-date when the time is reset. If you need to perform a complete reconstruction of all NIS maps, for any reason, remove all of the timestamp files and run make:

master# cd /var/yp 
master# rm *.time 
master# make

This extreme step is best reserved for testing the map distribution mechanism, or recovering from corruption of the NIS map directory.

4.2.4. Password file updates

One exception to the yppush push-on-demand strategy is the passwd map. Users need to be able to change their passwords without system manager intervention. The hosts file, for example, is changed by the superuser and then pushed to other servers when it is rebuilt. In contrast, when you change your password, you (as a nonprivileged user) modify the local password file. To change a password in an NIS map, the change must be made on the master server and distributed to all slave servers in order to be seen back on the client host where you made the change.

yppasswd is a user utility that is similar to the passwd program, but it changes the user's password in the original source file on the NIS master server. yppasswd usually forces the password map to be rebuilt, although at sites choosing not to rebuild the map on demand, the new password will not be distributed until the next map transfer. yppasswd is used like passwd, but it reports the server name on which the modifications are made. Here is an example:

[wahoo]% yppasswd 
Changing NIS password for stern on mahimahi. 
Old password: 
New password: 
Retype new password: 
NIS entry changed on mahimahi

Some versions of passwd (such as Solaris 2.6 and higher) check to see if the password file is managed by NIS, and invoke yppasswd if this is the case. Check your vendor's documentation for procedures particular to your system.

NIS provides read-only access to its maps. There is nothing in the NIS protocol that allows a client to rewrite the data for a key. To accept changes to maps, a server distinct from the NIS server is required that modifies the source file for the map and then rebuilds the NIS map from the modified ASCII file. To handle incoming yppasswd change requests, the master server must run the yppasswdd daemon (note the second "d" in the daemon's name). This RPC daemon gets started in the /usr/lib/netsvc/yp/ypstart boot script on the master NIS server only:

if [ "$master" = "$hostname" -a X$YP_SERVER = "XTRUE" ]; then
                ...
                if [ -x $YPDIR/rpc.yppasswdd ]; then
                        PWDIR=`grep "^PWDIR" /var/yp/Makefile 2> /dev/null` \
                        && PWDIR=`expr "$PWDIR" : `.*=[         ]*<[^  ]*>``
                        if [ "$PWDIR" ]; then
                                if [ "$PWDIR" = "/etc" ]; then
                                        unset PWDIR
                                else
                                        PWDIR="-D $PWDIR"
                                fi
                        fi
                        $YPDIR/rpc.yppasswdd $PWDIR -m \
                                && echo ` rpc.yppasswdd\c`
                fi
                ...
        fi

The host making a password map change locates the master server by asking for the master of the NIS passwd map, and the yppasswdd daemon acts as a gateway between the user's host and a passwd-like utility on the master server. The location of the master server's password file and options to build a new map after each update are given as command-line arguments to yppasswdd, as shown in the previous example.

The -D argument specifies the name of the master server's source for the password map; it may be the default /etc/passwd or it may point to an alternative password file.[7] The -m option specifies that make is to be performed in the NIS directory on the master server. You can optionally specify arguments after -m that are passed to make. With a default set up, the fragment in ypstart would cause yppasswdd to invoke make as:

[7]Recall from Section 3.2.2, "Installing the NIS master server" that we changed PWDR to /etc/nispw.

# ( cd /var/yp; make )

after each change to the master's password source file. Since it is likely only the password file will have changed, only the password maps get rebuilt and pushed. You can ensure that only the password maps get pushed changing the yppaswdd line in ypstart to:

$YPDIR/rpc.yppasswdd $PWDIR -m passwd \
                                && echo ` rpc.yppasswdd\c`

4.2.5. Source code control for map files

With multiple system administrators and a single point of administration, it is possible for conflicting or unexplained changes to NIS maps to wreak havoc with the network. The best way to control modifications to maps and to track the change history of map source files is to put them under a source code control system such as SCCS.

Source code files usually contain the SCCS headers in a comment or in a global string that gets compiled into an executable. Putting SCCS keywords into comments in the /etc/hosts and /etc/aliases files allows you to track the last version and date of edit:

header to be added to file: 
#       /etc/hosts header 
#       %M%      %I%         %H% %T% 
#       %W% 
keywords filled in after getting file from SCCS: 
#       /etc/hosts header 
#       hosts        1.32        12/29/90 16:37:52 
#       @(#)hosts        1.32

Once the headers have been added to the map source files, put them under SCCS administration:

nismaster# cd /etc 
nismaster# mkdir SCCS 
nismaster# /usr/ccs/bin/sccs admin -ialiases aliases 
nismaster# /usr/ccs/bin/sccs admin -ihosts hosts 
nismaster# /usr/ccs/bin/sccs get aliases hosts

The copies of the files that are checked out of SCCS control are read-only. Someone making a casual change to a map is forced to go and check it out of SCCS properly before doing so. Using SCCS, each change to a file is documented before the file gets put back under SCCS control. If you always return a file to SCCS before it is converted into an NIS map, the SCCS control file forms an audit trail for configuration changes:

nismaster# cd /etc 
nismaster# sccs prs hosts 
D 1.31 00/05/22 08:52:35 root 31 30     00001/00001/00117 
MRs: 
COMMENTS: 
added new host for info-center group 
D 1.30 00/06/04 07:19:04 root 30 29     00001/00001/00117 
MRs: 
COMMENTS: 
changed bosox-fddi to jetstar-fddi 
D 1.29 90/11/08 11:03:47 root 29 28     00011/00011/00107 
MRs: 
COMMENTS: 
commented out the porting lab systems.

If any change to the hosts or aliases file breaks, SCCS can be used to find the exact lines that were changed and the time the change was made (for confirmation that the modification caused the network problems).

The two disadvantages to using SCCS for NIS maps are that all changes must be made as root and that it won't work for the password file. The superuser must perform all file checkouts and modifications, unless the underlying file permissions are changed to make the files writable by nonprivileged users. If all changes are made by root, then the SCCS logs do not contain information about the user making the change. The password file falls outside of SCCS control because its contents will be modified by users changing their passwords, without being able to check the file out of SCCS first. Also, some files, such as /etc/group, have no comment lines, so you cannot use SCCS keywords in them.

4.2.6. Using alternate map source files

You may decide to use nonstandard source files for various NIS maps on the master server, especially if the master server is not going to be an NIS client. Alternatively, you may need to modify the standard NIS Makefile to build your own NIS maps. Approaches to both of these problems are discussed in this section.

Some system administrators prefer to build the NIS password map from a file other than /etc/passwd, giving them finer control over access to the server. Separating the host's and the NIS password files is also advantageous if there are password file entries on the server (such as those for dial-in UUCP) that shouldn't be made available on all NIS clients. To avoid distributing UUCP password file entries to all NIS clients, the NIS password file should be kept separately from /etc/passwd on the master server. The master can include private UUCP password file entries and can embed the entire NIS map file via nsswitch.conf.

If you de-couple the NIS password map from the master server's password file, then the NIS Makefile should be modified to reflect the new dependency. Refer back to the procedure described in Section 3.2.2, "Installing the NIS master server".