To eliminate single points of failure for networking, each
subnet accessed by a cluster node is required to have redundant
network interfaces. Redundant cables are also needed to protect
against cable failures. Each interface card is connected to a different
cable, and the cables themselves are connected by a component such
as a hub or a bridge. This arrangement of physical cables connected
to each other via a bridge or concentrator or switch is known as
a bridged net.
IP addresses can be associated with interfaces on a bridged
net. An interface that has an IP address associated with it is known
as a primary interface, and an
interface that does not have an IP address associated with it is
known as a standby interface.
Standby interfaces are those which are available for switching by
Serviceguard if a failure occurs on the primary. When Serviceguard
detects a primary interface failure, it will switch the IP addresses
and any associated connections from the failed interface card to
a healthy standby interface card.
Serviceguard supports a maximum of 30 network interfaces per
node. For this purpose an interface is defined as anything represented
as a LAN interface in the output of lanscan (1m), so the total of 30 can comprise any combination of
physical LAN ports, VLAN ports, IPoIB interfaces and APA aggregates.
(A node can have more than 30 such interfaces, but only 30 can be
part of the cluster configuration.)
A selection of network configurations is described further
in the following sections. See also “How
the Network Manager Works ”. For detailed information about supported
network configurations, consult Hewlett-Packard support.
A single subnet
cannot be configured on different network interfaces (NICs) on the
For IPv4 subnets, Serviceguard
does not support different subnets on the same LAN interface.
IPv6, Serviceguard supports up to two subnets per LAN interface
(site-local and global).
Serviceguard does support
different subnets on the same bridged network (this applies at both
the node and the cluster level).
As of Serviceguard A.11.18 it is possible to configure multiple
subnets, joined by a router, both for the cluster heartbeat and
for data, with some nodes using one subnet and some another.
A cross-subnet configuration allows:
Automatic package failover from a
node on one subnet to a node on another
A cluster heartbeat that spans subnets.
Cluster and package configuration tasks are affected as follows:
You must use the -w full option
to cmquerycl to discover actual or potential nodes
and subnets across routers.
You must configure two new parameters in the package configuration
file to allow packages to fail over across subnets:
ip_subnet_node - to indicate which nodes the subnet is configured
monitored_subnet_access - to indicate whether the subnet is configured on all
nodes (FULL) or only some (PARTIAL)
(For legacy packages, see “Configuring
You should not use the wildcard (*)
for node_name in the package configuration file, as this could allow
the package to fail over across subnets when a node on the same
subnet is eligible. Instead, list the nodes in order of preference.
The following restrictions apply:
All nodes in the cluster must belong
to the same network domain (that is, the domain portion
of the fully-qualified domain name must be
The nodes must be fully connected at the IP level.
A minimum of two heartbeat paths must be configured
for each cluster node.
There must be less than 200 milliseconds of latency
in the heartbeat network.
Each heartbeat subnet on each node must be physically
routed separately to the heartbeat subnet on another node; that
is, each heartbeat path must be physically separate:
The heartbeats must be statically
routed; static route entries must be configured on each node to
route the heartbeats through different paths.
Failure of a single router must not affect both
heartbeats at the same time.
Because Veritas Cluster File System from Symantec
(CFS) requires link-level traffic communication
(LLT) among the nodes, Serviceguard cannot be configured in cross-subnet
configurations with CFS alone.
But CFS is supported in specific cross-subnet
configurations with Serviceguard and HP add-on products such as
Serviceguard Extension for Oracle RAC (SGeRAC);
see the documentation listed below.
Each package subnet must be configured with a standby
interface on the local bridged net. The standby interface can be
shared between subnets.
Deploying applications in
this environment requires careful consideration; see “Implications
for Application Deployment”.
cmrunnode will fail if the “hostname LAN” is
down on the node in question. (“Hostname LAN” refers
to the public LAN on which the IP address that the node’s
hostname resolves to is configured).
If a monitored_subnet is configured for PARTIAL monitored_subnet_access in a package’s configuration file, it must be
configured on at least one of the nodes on the node_name list for that package. Conversely, if all of the subnets
that are being monitored for this package are configured for PARTIAL access,
each node on the node_name list must have at least one of these subnets configured.
As in other configurations, a package
will not start on a node unless the subnets configured on that node,
and specified in the package configuration file as monitored subnets,
Failed Network Cards
Depending on the system configuration, it is possible to replace
failed network cards while the cluster is running. The process is
described under “Replacement of LAN Cards” in
the chapter “Troubleshooting Your Cluster.” With
some restrictions, you can also add and delete LAN interfaces to
and from the cluster configuration while the cluster is running;
the Cluster Networking Configuration while the Cluster Is Running”.