13.2. MAC and IP layer toolsThe tools covered in this section operate at the MAC and IP layers of the network protocol stack. Problems that manifest themselves as NFS or NIS failures may be due to an improper host or network configuration problem. The tools described in this section are used to ascertain that the basic network connectivity is sound. Issues that will be covered include setting network addresses, testing connectivity, and burst traffic handling.
13.2.1. ifconfig: interface configurationifconfig sets or examines the characteristics of a network interface, such as its IP address or availability. At boot time, ifconfig is used to initialize network interfaces, possibly doing this in stages since some information may be available on the network itself through NIS. You can also use ifconfig to examine the current state of an interface and compare its address assignments with NIS map information. Interfaces may be physical devices, logical devices associated with a physical network interface, IP tunnels, or pseudo-devices such as the loopback device. Examples of physical devices include Ethernet interfaces or packet drivers stacked on top of low-level synchronous line drivers. IP tunnels are point-to-point interfaces that enable an IP packet to be encapsulated within another IP packet, appearing as a physical interface. For example, an IPv6-in-IPv4 tunnel allows IPv6 packets to be encapsulated within IPv4 packets, allowing IPv6 traffic to cross routers that understand only IPv4.
22.214.171.124. Examining interfacesTo list all available network interfaces, invoke ifconfig with the -a option:
The protocols listed will depend on the contents of inet_type(4). Both IPv6 and IPv4 will be listed if /etc/default/inet_type does not exist, or if it defines DEFAULT_IP=BOTH. Only IPv4 will be listed if DEFAULT_IP=IP_VERSION4. The network interface Ethernet address will also be reported when ifconfig is invoked as root.
In this example, ifconfig lists four different interfaces, lo0, hme0, hme0:1, and hme0:2. lo0 is the loopback pseudo-device used by IP to communicate between network applications that specify the local host on both end-points. hme0 is the actual physical Ethernet device configured on the host. Note that lo0 is listed in two different lines: the first line reports the loopback configuration in use by IPv4, and the third line reports the loopback configuration in use by IPv6. IPv4 specifies 127.0.0.1 as the loopback address; IPv6 specifies ::1/128. Similarly, the second line reports the IPv4 address used by the hme0 device (126.96.36.199), and the fourth line reports the device's IPv6 link-local address (fe80::a00:20ff:fe81:23f1/10). Solaris supports multiple logical interfaces associated with a single physical network interface. This allows a host to be assigned multiple IP addresses (even if the host only has a single network interface). This is particularly useful when a host communicates over various IPv6 addresses. In this example, hme0:1 and hme0:2 are logical interfaces associated with the physical network interface hme0. hme0:1 uses the site-local IPv6 address fec0::56:a00:20ff:fe81:23f1/64, and hme0:2 uses the global IPv6 address 2100::56:a00:20ff:fe81:23f1/64. To examine a particular network interface, invoke ifconfig with its name as an argument. By default, the IPv4 interface configuration is reported, unless you specify the address family you are interested in, as in the third example:% ifconfig -a lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 hme0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 188.8.131.52 netmask ffffff00 broadcast 184.108.40.206 lo0: flags=2000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6> mtu 8252 index 1 inet6 ::1/128 hme0: flags=2000841<UP,RUNNING,MULTICAST,IPv6> mtu 1500 index 2 inet6 fe80::a00:20ff:fe81:23f1/10 hme0:1: flags=2080841<UP,RUNNING,MULTICAST,ADDRCONF,IPv6> mtu 1500 index 2 inet6 fec0::56:a00:20ff:fe81:23f1/64 hme0:2: flags=2080841<UP,RUNNING,MULTICAST,ADDRCONF,IPv6> mtu 1500 index 2 inet6 2100::56:a00:20ff:fe81:23f1/64
If the specified interface does not exist on the system or is not configured into the kernel, ifconfig reports the error "No such device." The flags field is a bitmap that describes the state of the interface. Values for the flags may be found in /usr/include/net/if.h. The most common settings are:% ifconfig hme0 hme0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 220.127.116.11 netmask ffffff00 broadcast 18.104.22.168 % ifconfig lo0 lo0: flags=1000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 % ifconfig hme0 inet6 hme0: flags=2000841<UP,RUNNING,MULTICAST,IPv6> mtu 1500 index 2 inet6 fe80::a00:20ff:fe81:23f1/10
This interface is a serial line that connects networks 22.214.171.124 and 126.96.36.199; the machine on the other end of the line has a similar point-to-point interface configuration with the local and destination IP addresses reversed:this-side% ifconfig ipdptp0 ipdptp0: flags=10088d1<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST,PRIVATE,IPv4> mtu 8232 index 3 inet 188.8.131.52 --> 184.108.40.206 netmask ffffff00
Marking the line PRIVATE means that the host-to-host connection will not be advertised to routers on the network. Note also that the Address Resolution Protocol (ARP) is not used over point-to-point links.that-side% ifconfig ipdptp0 ipdptp0: flags=10088d1<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST,PRIVATE,IPv4> mtu 8232 index 5 inet 220.127.116.11 --> 18.104.22.168 netmask ffffff00
22.214.171.124. Initializing an interfaceIn addition to displaying the status of a network interface, ifconfig is used to configure the interface. During the boot process, Solaris identifies the network interfaces to be configured by searching for /etc/hostname.*[0-9] and /etc/hostname6.*[0-9] files. For example the presence of /etc/hostname.hme0 and /etc/hostname.hme1 indicate that the two network interfaces hme0 and hme1 need to be assigned an IPv4 address at boot time. Similarly, the presence of /etc/hostname6.hme0 indicates that hme0 needs to be configured to use IPv6. You can statically assign an IP address to the interface by specifying the corresponding hostname in the /etc/hostname.*[0-9] or /etc/hostname6.*[0-9] file. Hostnames and their corresponding IP addresses may be managed through NIS, which requires a functioning network to retrieve map values. This chicken-and-egg problem is solved by invoking ifconfig twice during the four steps required to bring a host up on the network:
126.96.36.199. Multiple interfacesYou can place a system on more than one network by either installing multiple physical network interfaces, or by configuring multiple logical interfaces associated with a physical network interface. In the first case, each network uses separate physical media, in the second case the networks are on the same physical media. A host that acts as a gateway between two networks is a good example of a system connected to physically separate networks. A host configured to run over both IPv4 and IPv6 is an example of a system with multiple logical interfaces and a single physical network. ifconfig can configure the interfaces one at a time, or in groups. For example, if a host has several interfaces, they can be enabled individually by using ifconfig:
As in the previous example, the plus signs (+) make ifconfig read the netmasks database for its data. In both examples, the interfaces are marked up and configured with a single command. ifconfig can also configure multiple interfaces at once using the -a option:... ifconfig hme0 acadia up netmask + broadcast + ... ifconfig hme1 acadia-gw up broadcast 188.8.131.52 netmask +
The -auD4 set of options instructs ifconfig to update the netmask and broadcast configuration for all IPv4 up devices that are not under DHCP control. Each network interface has a distinct hostname and IP address. One convention for two-network systems is to append -gw to the "primary" hostname. In this configuration, each network interface is on a separate IP network. Host acadia from the previous example appears in the NIS ipnodes map on network 184.108.40.206 and 220.127.116.11:ifconfig -auD4 netmask + broadcast +
To hosts on the 131.40.52 network, the machine is acadia-gw, but on the 192.254.1 network, the same host is called acadia. Systems with more than two network interfaces can use any convenient host naming scheme. For example, in a campus with four backbone Ethernet segments, machine names can reflect both the "given" name and the network name. A host sitting on all four IP networks is given four hostnames and four IP addresses:18.104.22.168 acadia 22.214.171.124 acadia-gw
If the additional interfaces are configured after NIS is started, then the NIS ipnodes map is relied upon to provide the IP address for each interface. To configure an interface early in the boot process -- before NIS is started -- the appropriate hostname and IP address must be in /etc/inet/ipnodes on the local machine. Note that you can configure the multiple physical network interfaces to be on separate IP networks. You can turn on IP interface groups on the host, such that it can have more than one IP address on the same subnet, and use the outbound networks for multiplexing traffic. You can also enable interface trunking on the host to use the multiple physical network interfaces as a single IP address. Trunking offers a measure of fault tolerance, since the trunked interface keeps working even if one of the network interfaces fails. It also scales as you add more network interfaces to the host, providing additional network bandwidth. We revisit IP interface groups and trunking in Section 17.3, "Network infrastructure".ipnodes file: 126.96.36.199 boris-bb1 188.8.131.52 boris-bb2 184.108.40.206 boris-bb3 220.127.116.11 boris-bb4
18.104.22.168. Mismatched host informationIf you have inconsistent hostname and IP address information in the NIS hosts map and the local hosts file, or the NIS ipnodes map and the local ipnodes file, major confusion will result. The host may not be able to start all of its services if its host IP address changes during the boot process, and other machines will not know how to map the host's name to an IP address that is represented on the network. You will find that some network activity works fine, where others fail. For example, you will be able to telnet into other systems from your misconfigured host, but the other systems will not be able to telnet into your misconfigured host. This is because the other hosts are using a different IP address than the one ifconfig used to configure your network interface. You will be able to mount NFS filesystems exported without restrictions, but will not be able to mount filesystems that are exported to your specific host (either explicitly or via netgroups) since the NFS server sees your request as coming from a different host. This kind of failure indicates that the local host's IP address has changed between the early boot phase and the last ifconfig. You may find that the local /etc/inet/hosts file disagrees with the NIS hosts map or the local /etc/inet/ipnodes file disagrees with the NIS ipnodes map. Mismatched IPv4 addresses between the hosts and ipnodes maps will lead to inconsistent behavior between IPv6-aware or -enabled applications and IPv6-unaware applications, because they obtain their address information from different sources. If the hosts database contains the correct information but the ipnodes database is corrupted, then IPv6-unaware applications will work correctly, while the IPv6-aware and -enabled applications will experience problems. The reverse is true when the corrupted information is in the hosts database.
13.2.2. Subnetwork masksThe second ifconfig in the boot process installs proper masks and broadcast addresses if subnetting is used to divide a larger IP address space. Default subnetwork masks and broadcast addresses are assigned based on IP address class, as shown in Table 13-3.
Table 13-3. Default broadcast addresses
The NIS netmasks map contains an association of network numbers and subnetwork masks and is used to override the default network masks corresponding to each class of IP address. A simple example is the division of a Class B network into Class C-like subnetworks, so that each subnetwork number can be assigned to a distinct physical network. To effect such a scheme, the netmasks NIS map contains a single entry for the Class B address:
Broadcast addresses are derived from the network mask and host IP address by performing a logical and of the two. Any bits that are not masked out by the netmask become part of the broadcast address, while those that are masked out are set to all ones in Solaris (other systems may set them to all zeros). Network numbers are matched based on the number of octets normally used for an address of that class. IP address 22.214.171.124 has a Class B network number, so the first two octets in the IP address are used as an index into the netmasks map. Similarly, IP address 126.96.36.199 is a Class A address; therefore, only the first octet is used as a key into netmasks. This scheme simplifies the management of netmasks. By listing the network number to be partitioned, you do not have to itemize all subnetworks in the netmasks file. Continuing the previous example, consider this ifconfig:188.8.131.52 255.255.255.0
Using a plus sign (+) as the netmask instead of an explicit network mask forces the second ifconfig to read the NIS netmasks map for the correct mask. The four-octet mask is logically and-ed with the IP address, producing the broadcast network number. In the preceding example, the broadcast address is in the ones form. Note that the network mask is actually displayed as a hexadecimal mask value, and not as an IP address. A more complex example involves dividing the Class C network 192.6.4 into four subnetworks. To get four subnetworks, we need an additional two bits of network number, which are taken from the two most significant bits of the host number. The netmask is therefore extended into the next two bits, making it 26 bits instead of the default 24-bit Class C netmask:ipnodes excerpt: 184.108.40.206 mahimahi netmasks map: 220.127.116.11 255.255.255.0 ifconfig line: ifconfig hme0 mahimahi netmask + Resulting interface configuration: % ifconfig hme0 hme0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 18.104.22.168 netmask ffffff00 broadcast 22.214.171.124
Again, only one entry in netmasks is needed, and the key for the entry matches the Class C network number that is being divided. You use variable length subnetting when using Classless IP addressing. You specify how many bits of the IP address to use for the network, and how many to use for the host by setting the appropriate netmask entry. The format of the netmask entry is the same as before, however, there should be an entry for each subnet defined. ifconfig uses the longest possible matching mask. Say your engineering organization has been given control of the 126.96.36.199 network (addresses 188.8.131.52 -> 184.108.40.206). You decide to partition it into four separate subnetworks that map the four groups in your organization: Systems Engineering, Applications Engineering, Graphics Engineering, and Customer Support. You plan to use a single system to serve as your gateway between the four separate subnets and the enterprise network. Your enterprise network address is 220.127.116.11, and is therefore connected to the 18.104.22.168 enterprise network. In order to partition the 131.40.86 address space into four separate subnets, you need to use the two upper bits of the last octet to identify the network. Table 13-4 shows the distribution of the IP addresses to the different networks.Partitioning requires: 24 bits of Class C network number 2 additional bits of subnetwork number 6 bits left for host number Last octet has 2 bits of netmask, 6 of host number: 11000000 binary = 192 decimal Resulting netmasks file entry: 22.214.171.124 255.255.255.192
Table 13-4. Network assignment
The last octet of the address will have two bits of netmask and six of host number:
The resulting netmasks file is:11000000 binary = 192 decimal The resulting netmask: 255.255.255.192
The first entry indicates that the Class B network 126.96.36.199 is subnetted. The next four entries represent the four variable-length subnets for the classless addresses for the different groups. Addresses 188.8.131.52 through 184.108.40.206 have a subnet mask with 26 bits in the subnet fields and 6 bits in the host field. All other addresses in the range 220.127.116.11 through 18.104.22.168 have a 24 bit subnet field. The IP address assignments for the five network interfaces are shown in Table 13-22.214.171.124.0 255.255.255.0 126.96.36.199 255.255.255.192 188.8.131.52 255.255.255.192 184.108.40.206 255.255.255.192 220.127.116.11 255.255.255.192
Table 13-5. Assigning addresses to interfaces
For example, the server would direct network traffic to the hme0 interface when communicating with IP address 18.104.22.168, since it is part of the 22.214.171.124 subnet; hme1 when communicating with 126.96.36.199, since it is part of the 188.8.131.52 subnet; hme2 when communicating with 184.108.40.206, and so on.ifconfig only governs the local machine's interface to the network. If a host cannot exchange packets with a peer host on the same network, then it is necessary to verify that a datagram circuit to the remote host exists and that the remote node is properly advertising itself on the network. Tools that perform these tests are arp and ping.
13.2.3. IP to MAC address mappingsApplications use IP addresses and hostnames to identify remote nodes, but packets sent on the Ethernet identify their destinations via a 48-bit MAC-layer address. The Ethernet interface on each host only receives packets that have its MAC address of a broadcast address in the destination field. IP addresses are completely independent of the 48-bit MAC-level address; several disjoint networks may use the same sets of IP addresses although the 48-bit addresses to which they map are unique worldwide. You can tell who makes an Ethernet interface by looking at the first three octets of its address. Some of the most popular prefixes are shown in Table 13-6. Fortunately, newer diagnostic tools such as ethereal know how to map the prefix number to the vendor of the interface. ethereal is introduced later in this chapter in Section 13.5.2, "ethereal / tethereal".
Table 13-6. Ethernet address prefixes
ARP, the Address Resolution Protocol, is used to maintain tables of 32- to 48-bit address translations. The ARP table is a dynamic collection of MAC-to-IPv4 address mappings. To fill in the MAC-level Ethernet packet headers, the sending host must resolve the destination IPv4 address into a 48-bit address. The host first checks its ARP table for an entry keyed by the IPv4 address, and if none is found, the host broadcasts an ARP request containing the recipient's IPv4 address. Any machine supporting ARP address resolution responds to an ARP request with a packet containing its MAC address. The requester updates its ARP table, fills in the MAC address in the Ethernet packet header, and transmits the packet.If no reply is received for the ARP request, the transmitting host sends the request again. Typically, a delay of a second or more is inserted between consecutive ARP requests to prevent a series of ARP packets from saturating the network. Flurries of ARP requests sometimes occur when a malformed packet is sent on the network; some hosts interpret it as a broadcast packet and attempt to get the Ethernet address of the sender via an ARP request. If many machines are affected, the ensuing flood of network activity can consume a considerable amount of the available bandwidth. This behavior is referred to as an ARP storm, and is most frequently caused by an electrical problem in a transceiver that damages packets after the host has cleanly written them over its network interface. To examine the current ARP table entries, use arp -a:
The arp -a output listing reports the interface over which the ARP notification arrived, the IP address (or hostname) and its Ethernet address mapping. The unresolved entry (denoted by the U flag) is for a host that did not respond to an ARP request; after several minutes the entry is removed from the table. Complete entries in the ARP table may be static or dynamic, indicating how the address mappings were added and the length of their expected lifetimes. Solaris identifies static entries with the S flag. The host's own Ethernet address as well as all multicast address entries (identified by the M flag) will always be static.The previous example was run on the host roger, therefore the static nature of the entry for its own Ethernet address and multicast entries. The absence of the S flag identifies a dynamic or learned entry. Dynamic entries are added on demand during the course of normal IP traffic handling. Infrequently used mappings added in this fashion have a short lifetime; after five minutes without a reference to the entry, the ARP table management routines remove it. This ongoing table pruning is necessary to minimize the overhead of ARP table lookups. The ARP table is accessed using a hash table; a smaller, sparser table has fewer hash key collisions. A host that communicates regularly with many other hosts may have an ARP table that is fairly large, while a host that is quiescent or exchanging packets with only a few peers has a small ARP table. The difference between dynamic and permanent entries is how they are added to the ARP table. Dynamic entries are added on the fly, as a result of replies to ARP requests. Permanent entries are loaded into the ARP table once at boot time, and are useful if a host must communicate with a node that cannot respond to an ARP request during some part of its startup procedure. For example, a diskless client may not have ARP support embedded in the boot PROM, requiring its boot server to have a permanent ARP table entry for it. Once the diskless node is running the Unix kernel, it should be able to respond to ARP requests to complete dynamic ARP table entries on other hosts. The arp -a output reports a mask for every entry. This mask is used during lookup of an entry in the ARP table. The lookup function in the kernel applies the mask to the address being queried and compares it with the one in the table. If the resulting addresses match, the lookup is successful. A mask of 255.255.255.255 (all ones) means that the two addresses need to be exactly the same in order to be considered equivalent. A mask of 240.0.0.0 means that only the upper four bits of the address are used to find a matching address. In the previous example, all multicast addresses use the Ethernet address corresponding to the 240.0.0.0 entry. The ARP mask does not provide much useful information to the regular user. Be sure not to confuse this ARP mask with the netmask specified by the ifconfig command. The ARP mask is generated and used only by the internal kernel routines to reduce the number of entries that need to be stored in the table. The netmask specified by the ifconfig command is used for IP routing. A variation of the permanent ARP table entry is a published mapping. Published mappings are denoted by the P flag. Published entries include the IP address for the current host, and the addresses that have been explicitly added by the -s or -f options (explained later in this chapter). Publishing ARP table entries turns a host into an ARP server. Normally, a host replies only to requests for its own IP address, but if it has published entries then it replies for multiple IP addresses. If an ARP request is broadcast requesting the IP address of a published entry, the host publishing that entry returns an ARP reply to the sender, even though the IP address in the ARP request does not match its own. This mechanism is used to cope with machines that cannot respond to ARP requests due to lack of ARP support or because they are isolated from broadcast packets by a piece of network partitioning hardware that filters out broadcast packets. This mechanism is also useful in SLIP or PPP configurations. When any of these situations exist, a machine is designated as an ARP server and is loaded with ARP entries from a file containing hostnames, Ethernet addresses, and the pub qualifier. For example, to publish the ARP entries for hosts relax and stress on server irie, we put the ARP information into a configuration file /etc/arptable and then load it using arp -f:% arp -a Net to Media Table: IPv4 Device IP Address Mask Flags Phys Addr ------ -------------------- --------------- ----- --------------- hme0 caramba 255.255.255.255 08:00:20:b9:2b:f6 hme1 socks 255.255.255.255 08:00:20:e7:91:5d hme0 copper 255.255.255.255 00:20:af:9d:7c:92 hme0 roger 255.255.255.255 SP 08:00:20:a0:33:90 hme0 universo 255.255.255.255 U hme0 peggy 255.255.255.255 SP 08:00:20:81:23:f1 hme1 duke 255.255.255.255 00:04:00:20:56:d7 hme0 220.127.116.11 240.0.0.0 SM 01:00:5e:00:00:00 hme1 18.104.22.168 240.0.0.0 SM 01:00:5e:00:00:00 hme1 daisy 255.255.255.255 08:00:20:b5:3d:d7
The -f option forces arp to read the named file for entries, alternatively the -s option can be used to add a single mapping from the command line:irie# cat /etc/arptable relax 08:00:20:73:3e:ec pub stress 08:00:20:b9:18:3d pub irie# arp -f /etc/arptable
As a diagnostic tool, arp is useful for resolving esoteric point-to-point connectivity problems. If a host's ARP table contains an incorrect entry, the machine using it will not be reachable, since outgoing packets will contain the wrong Ethernet address. ARP table entries may contain incorrect Ethernet addresses for several reasons:irie# arp -s relax 08:00:20:73:3e:ec pub
An example involving intermittent ARP failures is presented in Chapter 15, "Debugging Network Problems". IPv6 nodes use the neighbor discovery mechanism to learn the link layer address (MAC in the case of Ethernet) of the other nodes connected to the link. The IPv6 neighbor discovery mechanism delivers the functionality previously provided by the combination of ARP, ICMP router discovery, and ICMP redirect mechanisms. This is done by defining special ICMP6 message types: neighbor solicitation and neighbor advertisement. A node issues neighbor solicitations when it needs to request the link-layer (MAC) address of a neighbor. Nodes will also issue neighbor advertisement messages in response to neighbor solicitation messages, as well as when their link-layer address changes.# arp -d fenwick fenwick (22.214.171.124) deleted # telnet fenwick ...Telnet times out... # arp -a | grep fenwick hme0 fenwick 255.255.255.255 08:00:20:79:61:eb
13.2.4. Using ping to check network connectivityping is similar to arp in that it provides information about hosts on a network rather than information about data that is sent on the network. arp provides a low-level look at the MAC addressing used by a host, but it is not that powerful for diagnosing connectivity problems. ping is a more general purpose tool for investigating point-to-point connectivity problems and areas of questionable physical network topology. ping uses the Internetwork Control Message Protocol (ICMP) echo facility to ask a remote machine for a reply. ICMP is another component of the network protocol stack that is a peer of IP and ARP. The returned packet contains a timestamp added by the remote host which is used to compute the round trip packet transit time. In its simplest form, ping is given a hostname or IP address and returns a verdict on connectivity to that host:
The -s option puts ping into continuous-send mode, and displays the sequence numbers and transit times for packets as they are returned. Optionally, the packet size and packet count may be specified on the command line:% ping shamrock shamrock is alive % ping 126.96.36.199 188.8.131.52 is alive
For example:ping [-s] host [packet-size] [packet-count]
and:% ping -s mahimahi PING mahimahi: 56 data bytes 64 bytes from mahimahi (184.108.40.206): icmp_seq=0. time=3. ms 64 bytes from mahimahi (220.127.116.11): icmp_seq=1. time=2. ms 64 bytes from mahimahi (18.104.22.168): icmp_seq=2. time=2. ms 64 bytes from mahimahi (22.214.171.124): icmp_seq=3. time=3. ms 64 bytes from mahimahi (126.96.36.199): icmp_seq=4. time=2. ms ^C ----mahimahi PING Statistics---- 5 packets transmitted, 5 packets received, 0% packet loss round-trip (ms) min/avg/max = 2/2/3
The eight bytes added to each ICMP echo request in the corresponding reply are the timestamp information added by the remote host. If no explicit count on the number of packets is specified, then ping continues transmitting until interrupted. By default, ping uses a 56-byte packet, which is the smallest IP packet, complete with headers and checksums, that will be transmitted on the Ethernet. The ping utility is good for answering questions about whether the remote host is attached to the network and whether the network between the hosts is reliable. Additionally, ping can indicate that a hostname and IP address are not consistent across several machines. The replies received when the host is specified by name may contain an incorrect IP address. Conversely, if pinging the remote host by name does not produce a reply, try the IP address of the host. If a reply is received when the host is specified by address, but not by name, then the local machine has an incorrect view of the remote host's IP address. These kinds of problems are generally machine specific, so intermittent ping failures can be a hint of IP address confusion: machines that do not agree on the IP addresses they have been assigned. If NIS is used, this could indicate that the NIS ipnodes map was corrupted or changed (incorrectly) since the remote host last booted. The NIS ipnodes map supersedes the local /etc/inet/ipnodes file, so a disparity between the two values for a remote machine is ignored; the NIS ipnodes map takes precedence. However, in the absence of NIS, the failure of a remote node to answer a ping to its hostname indicates the /etc/inet/ipnodes files are out of synchronization.% ping -s mahimahi 100 3 PING mahimahi: 100 data bytes 108 bytes from mahimahi (188.8.131.52): icmp_seq=0. time=3. ms 108 bytes from mahimahi (184.108.40.206): icmp_seq=1. time=3. ms 108 bytes from mahimahi (220.127.116.11): icmp_seq=2. time=3. ms ----mahimahi PING Statistics---- 3 packets transmitted, 3 packets received, 0% packet loss round-trip (ms) min/avg/max = 3/3/3
You can change the search order for hosts and ipnodes in /etc/nsswitch.conf in order to reverse the precedence order.Larger packet sizes may be used to test connectivity through network components that are suspected of damaging large packets or trains of packets. ping only sends one packet at a time, so it won't test the capacity of a network interface. However, it tells you whether packets close to the network's MTU can make it from point to point intact, through all of the network hardware between the two hosts. Using the packet count indicators and transit times, ping can be used to examine connectivity, network segment length, and potential termination problems. Electrical problems, including poor or missing cable termination, are among the most difficult problems to diagnose and pinpoint without repeatedly splitting the network in half and testing the smaller segments. If ping shows that packets are dropped out of sequence, or that return packets are received in bursts, it is likely that either a network cable segment has an electrical fault or that the network is not terminated properly. These problems are more common in older 10Base-5 and 10Base-2 networks than in newer CAT5 twisted pair networks. For example, the following output from ping indicates that the network is intermittently dropping packets; this behavior is usually caused by improper termination and is quite random in nature:
The gap between packets 1 and 16, along with the exceptionally long packet delay, indicates that a low-level network problem is consuming packets.% ping -s mahimahi PING mahimahi: 56 data bytes 64 bytes from mahimahi (18.104.22.168): icmp_seq=0. time=3. ms 64 bytes from mahimahi (22.214.171.124): icmp_seq=1. time=2. ms 64 bytes from mahimahi (126.96.36.199): icmp_seq=16. time=1295. ms 64 bytes from mahimahi (188.8.131.52): icmp_seq=17. time=3. ms 64 bytes from mahimahi (184.108.40.206): icmp_seq=18. time=2. ms
13.2.5. Gauging Ethernet interface capacityEven with a well-conditioned network and proper host configuration information, a server may have trouble communicating with its clients because its network interface is overloaded. If an NFS server is hit with more packets than it can receive through its network interface, some client requests will be lost and eventually retransmitted. To the NFS clients, the server appears painfully slow, when it's really the server's network interface that is the problem. The spray utility provides a very coarse estimate of network interface capacity, both on individual hosts and through network hardware between hosts. spray showers a target host with consecutive packets of a fixed length by making remote procedure calls to the rpc.sprayd daemon on the remote host. After the last packet is sent, the rpc.sprayd daemon is queried for a count of the packets received; this value is compared to the number of packets sent to determine the percentage dropped between client and server. On its own, spray is of limited usefulness as a measure of the packet handling capability of a machine. The packet containing the RPC call may be lost by the client, due to other activity on its network interface; it may be consumed by a collision on the network; or it may be incident to the server but not copied from the network by the server's network interface due to a lack of buffer space or excessive server CPU loading. Many packets are lost on the sending host, and spray has no knowledge of where the packets vanish once they get pass the application layer. Due to these factors, spray is best used to gauge the relative packet-handling speeds of two or more machines. Here are some examples of using spray to test various network constraints. spray requires a hostname and takes a packet count, delay value, and packet length as optional arguments:
For example:spray [-c count] [-d delay] [-l length] host
spray reports the number of packets received, as well as the transfer rate. The packet drop rates are only meaningful when used to compare the relative network input and output rates of the two machines under test. It's important to note that network interface speed depends upon much more than CPU speed. A faster CPU helps a host process network protocols faster, but the network interface and bus hardware usually determine how quickly the host can pull packets from the network. A fast network interface may be separated from the CPU by a bus that has a high latency. Even a high-throughput I/O system may exhibit poor network performance if there is a large time overhead required to set up each packet transfer from the network interface to the CPU. Similar hosts stress each other fairly, since their network interfaces have the same input capacity. Even on a well-conditioned, little-used network, a client machine that has a significantly faster CPU than its server may perform worse under the stress of spray than the same two machines with the client and server roles reversed. With increased CPU speed comes increased packet handling speed, so a faster machine can transmit packets quickly enough to outpace a slower server. If the disparity between client and server is great, then the client is forced to retransmit requests and the server is additionally burdened with the duplicate requests. Use spray to exercise combinations of client and server with varying packet sizes to identify cases in which a client may race ahead of its server. When a fast NFS client is teamed with a slower server, the NFS mount parameters require tuning as described in Section 18.1, "Slow server compensation". Send various sized packets to an NFS server to see how it handles "large" and "small" NFS requests. Disk write operations are "large," usually filling several full-size IP packets. Other operations, such as getting the attributes of a file, fit into a packet of 150 bytes or less. Small packets are more easily handled by all hosts, since there is less data to move around, but NFS servers may be subject to bursts of large packets during intense periods of client write operations. If no explicit arguments are given, spray sends 1162 packets of 86 bytes. In most implementations of spray, if either a packet count or packet length are given, the other argument is chosen so that 100 kbytes of data are transferred between client and server. Try using spray with packet sizes of 1500 bytes to judge how well an NFS server or the network handle write requests. Normally, no delay is inserted between packets sent by spray, although the -d option may be used to specify a delay in microseconds. Insert delays between the packets to simulate realistic packet arrival rates, under "normal" conditions. Client requests may be separated by several tens of microseconds, so including a delay between packets may give you a more accurate picture of packet handling rates. In Figure 13-1, baxter and arches are identical machines and acadia is a faster machine with a faster network interface. spray produces the following output:% spray wahoo sending 1162 packets of length 86 to wahoo ... 675 packets (58.090%) dropped by wahoo 1197 packets/sec, 103007 bytes/sec
Fast machine to slow machine: [acadia]% spray baxter -c 100 -l 1160 sending 100 packets of length 1162 to baxter ... 39 packets (39.000%) dropped by baxter 520 packets/sec, 605037 bytes/sec Fast machine to slow machine, with delay: [acadia]% spray baxter -c 100 -l 1160 -d 1 sending 100 packets of length 1162 to baxter ... no packets dropped by baxter 99 packets/sec, 115680 bytes/sec Slow machine to fast machine: [baxter]% spray acadia -c 100 -l 1160 sending 100 packets of length 1162 to acadia ... no packets dropped by acadia 769 packets/sec, 893846 bytes/sec Slow machine to identical machine: [baxter]% spray arches -c 100 -l 1160 sending 100 packets of length 1162 to arches ... no packets dropped by arches 769 packets/sec, 893846 bytes/sec
Figure 13-1. Testing relative packet handling ratesWhen the fast machine sprays the slower one, a significant number of packets are dropped; but adding a one-microsecond delay between the packets allows the slow machine to keep pace and receive all incident packets. The slow machine to fast machine test produces the same packet handling rate as the slow machine showering an identical peer; if the slow machine sprays the fast one, the network bandwidth used is more than 30% greater than when the fast machine hammers the slow one. Note that you couldn't get NFS to insert delays like this, but performing the test with delays may indicate the location of a bottleneck. Knowing your constraints, you can change other configuration parameters, such as NFS client behavior, to avoid the bottleneck. We'll look at these tuning procedures more in Chapter 18, "Client-Side Performance Tuning". The four tools discussed to this point -- ifconfig, arp, ping, and spray -- focus on the issues of packet addressing and routing. If they indicate a problem, all network services, such as telnet and rlogin, will be affected. We now move up through the network and transport layers in the network protocol stack, leaving the MAC and IP layers for the session and application layers.
Copyright © 2002 O'Reilly & Associates. All rights reserved.