 |
» |
|
|
|
You can check status using Serviceguard Manager or from a
cluster node’s command line. Reviewing
Cluster and Package Status with the cmviewcl Command | |
Information about cluster
status is stored in the status database, which is maintained on
each individual node in the cluster. You can display information
contained in this database by issuing the cmviewcl command:cmviewcl -v You can use the cmviewcl command without root access; in clusters running Serviceguard
version A.11.16 or later, grant access by assigning the Monitor
role to the users in question. In earlier versions, allow access by
adding <nodename> <nonrootuser> to the cmclnodelist file. cmviewcl -v displays information about all the nodes and packages
in a running cluster, together with the settings of parameters that
determine failover behavior. | | | |  | TIP: Some commands take longer to complete in large configurations.
In particular, you can expect Serviceguard’s CPU usage
to increase during cmviewcl -v as the number of packages and services increases. | | | | |
You can also specify that the output should be formatted as
it was in a specific earlier release by using the -r option to specify the release format you want, for example: cmviewcl -r A.11.16 (See the cmviewcl (1m) manpage for the supported release formats.) The formatting
options let you choose a style: the tabulated format is designed
for viewing; the line format is designed for scripting, and is easily
parsed. See the manpage for a detailed description of other cmviewcl options. The cmviewcl -v command output lists dependencies throughout the cluster.
For a specific package’s dependencies, use the -p <pkgname> option. Viewing CFS
Multi-Node Information On systems that support Veritas Cluster File System (CFS), you can use cfs commands to see multi-node package configuration information, status,
and dependencies in a CFS cluster; for example: cfsdgadm show_package <diskgroup> cfsmntadm show_package <mountpoint> getconf -p <mnpkg> | grep dependency Types
of Cluster and Package States A cluster or its component nodes may be in several different
states at different points in time. The following sections describe
many of the common conditions the cluster or package may be in. The status of a cluster, as shown by cmviewcl, can be one of the following: up - At least one node has a running cluster daemon,
and reconfiguration is not taking place. down - No cluster daemons are running on any cluster
node. starting - The cluster is in the process of determining
its active membership. At least one cluster daemon is running. unknown - The node on which the cmviewcl command is issued cannot communicate with other nodes
in the cluster.
The status of a node is either up (active as
a member of the cluster) or down (inactive in
the cluster), depending on whether its cluster daemon is running
or not. Note that a node might be down from the cluster perspective,
but still up and running HP-UX. A node may also be in one of the following states: Failed. A node never
sees itself in this state. Other active members of the cluster will
see a node in this state if that node was in an active cluster,
but is no longer, and is not halted. Reforming. A node is in this state
when the cluster is re-forming. The node is currently running the
protocols which ensure that all nodes agree to the new membership
of an active cluster. If agreement is reached, the status database
is updated to reflect the new cluster membership. Running. A node in this state has
completed all required activity for the last re-formation and is
operating normally. Halted. A node never sees itself
in this state. Other nodes will see it in this state after the node
has gracefully left the active cluster, for instance with a cmhaltnode command. Unknown. A node never sees itself
in this state. Other nodes assign a node this state if it has never
been an active cluster member.
The status of a
package can be one of the following: up - The package master control script is active. down - The package master control script is not active. start_wait - A cmrunpkg command is in progress for this package. The package
is waiting for packages it depends on (predecessors)
to start before it can start. starting - The package is starting.
The package master control script is running. halting - A cmhaltpkg command is in progress for this package and the halt
script is running. halt_wait - A cmhaltpkg command is in progress for this package. The package
is waiting to be halted, but the halt script cannot start because
the package is waiting for packages that depend on it (successors)
to halt. The parameter description for “successor_halt_timeout” provides more information. failing - The package is halting
because it, or a package it depends on, has failed. fail_wait - The package is waiting
to be halted because the package or a package it depends on has
failed, but must wait for a package it depends on to halt before
it can halt. relocate_wait - The package’s
halt script has completed or Serviceguard is still trying to place
the package. unknown - Serviceguard could
not determine the status at the time cmviewcl was run.
A system multi-node package is up when it is running on all the
active cluster nodes. A multi-node package is up if it is running
on any of its configured nodes. The state of the
package can be one of the following: starting - The package is starting. The package master
control script is running. start_wait - A cmrunpkg command is in progress for this package. The package
is waiting for packages it depends on (predecessors)
to start before it can start. running - Services are active and being monitored. halting - A cmhaltpkg command is in progress for this package and the halt
script is running. halt_wait - A cmhaltpkg command is in progress for this package. The package
is waiting to be halted, but the halt script cannot start because
the package is waiting for packages that depend on it (successors)
to halt. The parameter description for “successor_halt_timeout” provides more information. halted - The package is down and halted. failing - The package is halting because it, or a package
it depends on, has failed. fail_wait - The package is waiting to be halted because
the package or a package it depends on has failed, but must wait
for a package it depends on to halt before it can halt. failed - The package is down and failed. relocate_wait - The package’s halt script has completed
or Serviceguard is still trying to place the package. unknown - Serviceguard could not determine the state at
the time cmviewcl was run.
Package
Switching Attributes cmviewcl shows the following package switching information: AUTO_RUN: Can be enabled or disabled. For failover packages, enabled means that the package starts when the cluster
starts, and Serviceguard can switch the package to another node
in the event of failure. For system multi-node packages, enabled means an instance of the package can start on
a new node joining the cluster (disabled means it will not). (Switching for a node): For failover packages, enabled means that the package can switch to the specified
node. disabled means that the package cannot switch to the specified
node until the node is enabled to run the package via the cmmodpkg command. Every failover package is marked enabled or disabled for each node that is either a primary or adoptive
node for the package. For multi-node packages, node switching disabled means the package cannot start on that node.
Services have only status, as follows: Up. The service is being monitored. Down. The service is not running. It may not have started,
or have halted or failed.
The network interfaces have only status, as follows: Unknown. Serviceguard cannot determine whether the interface
is up or down. A standby interface has this status.
Failover
and Failback PoliciesFailover packages can be configured with one of two values
for the failover_policy parameter (see “failover_policy”),
as displayed in the output of cmviewcl -v: configured_node.
The package fails over to the next node in the node_name list in the package configuration file (see “node_name”). min_package_node. The package
fails over to the node in the cluster that has the fewest running
packages.
Failover packages can also be configured with one of two values
for the failback_policy parameter (see “failback_policy”),
and these are also displayed in the output of cmviewcl -v: automatic: Following
a failover, a package returns to its primary node when the primary
node becomes available again. manual: Following a failover,
a package will run on the adoptive node until moved back to its
original node by a system administrator.
Examples
of Cluster and Package States The following sample output from the cmviewcl -v command shows status for the cluster in the sample
configuration. If the cluster is using a quorum server for tie-breaking services,
the display shows the server name, state and status following the
entry for each node, as in the following excerpt from the output
of cmviewcl -v: CLUSTER STATUS
example up
NODE STATUS STATE
ftsys9 up running
Quorum Server Status:
NAME STATUS STATE
lp-qs up running
...
NODE STATUS STATE
ftsys10 up running
Quorum Server Status:
NAME STATUS STATE
lp-qs up running |
If the cluster is using the Veritas Cluster Volume Manager
(CVM), version 3.5, for disk storage, the system multi-node package VxVM-CVM-pkg must be running on all active nodes for applications
to be able to access CVM disk groups. The system multi-node package
is named SG-CFS-pkg if the cluster is using version 4.1 or later of the
Veritas Cluster Volume Manager. | | | |  | NOTE: Check the Serviceguard,
SGeRAC, and SMS Compatibility and Feature Matrix and
the latest Release Notes for your version of Serviceguard for up-to-date
information about support for CVM and CFS (http://www.docs.hp.com -> High Availability -> Serviceguard). | | | | |
The VxVM-CVM-pkg package is shown in the following output of the cmviewcl command: CLUSTER STATUS
example up
NODE STATUS STATE
ftsys7 down halted
ftsys8 down halted
ftsys9 up running
ftsys10 up running
MULTI_NODE_PACKAGES:
PACKAGE STATUS STATE AUTO_RUN SYSTEM
VxVM-CVM-pkg up running enabled yes |
When you use the -v option, the display shows
the system multi-node package associated with each active node in
the cluster, as in the following: MULTI_NODE_PACKAGES:
PACKAGE STATUS STATE AUTO_RUN SYSTEM
VxVM-CVM-pkg up running enabled yes
NODE STATUS SWITCHING
ftsys7 down disabled
NODE STATUS SWITHCHING
ftsys8 down disabled
NODE STATUS SWITCHING
ftsys9 up enabled
Script_Parameters:
ITEM STATUS MAX_RESTARTS RESTARTS NAME
Service up 0 0 VxVM-CVM-pkg.srv
NODE STATUS SWITCHING
ftsys10 up enabled
Script_Parameters:
ITEM STATUS MAX_RESTARTS RESTARTS NAME
Service up 0 0 VxVM-CVM-pkg.srv |
Status
After Halting a PackageAfter we halt the failover package pkg2 with the cmhaltpkg command, the output of cmviewcl-v is as follows: |
CLUSTER STATUS
example up
NODE STATUS STATE
ftsys9 up running
Network_Parameters:
INTERFACE STATUS PATH NAME
PRIMARY up 56/36.1 lan0
STANDBY up 60/6 lan1
PACKAGE STATUS STATE AUTO_RUN NODE
pkg1 up running enabled ftsys9
Policy_Parameters:
POLICY_NAME CONFIGURED_VALUE
Failover configured_node
Failback manual
Script_Parameters:
ITEM STATUS MAX_RESTARTS RESTARTS NAME
Service up 0 0 service1
Subnet up 15.13.168.0
Resource up /example/float
Node_Switching_Parameters:
NODE_TYPE STATUS SWITCHING NAME
Primary up enabled ftsys9 (current)
Alternate up enabled ftsys10
NODE STATUS STATE
ftsys10 up running
Network_Parameters:
INTERFACE STATUS PATH NAME
PRIMARY up 28.1 lan0
STANDBY up 32.1 lan1
UNOWNED_PACKAGES
PACKAGE STATUS STATE AUTO_RUN NODE
pkg2 down halted disabled unowned
Policy_Parameters:
POLICY_NAME CONFIGURED_VALUE
Failover configured_node
Failback manual
Script_Parameters:
ITEM STATUS NODE_NAME NAME
Resource up ftsys9 /example/float
Subnet up ftsys9 15.13.168.0
Resource down ftsys10 /example/float
Subnet up ftsys10 15.13.168.0
Node_Switching_Parameters:
NODE_TYPE STATUS SWITCHING NAME
Primary up enabled ftsys10
Alternate up enabled ftsys9
|
|
pkg2 now has the status down, and it is shown as unowned, with package switching disabled. Resource /example/float, which is configured as a dependency of pkg2, is down on one node. Note that switching is enabled for
both nodes, however. This means that once global switching is re-enabled
for the package, it will attempt to start up on the primary node. Status
After Moving the Package to Another NodeIf we use the following command and then run cmviewcl -v, we’ll see: |
CLUSTER STATUS
example up
NODE STATUS STATE
ftsys9 up running
Network_Parameters:
INTERFACE STATUS PATH NAME
PRIMARY up 56/36.1 lan0
STANDBY up 60/6 lan1
PACKAGE STATUS STATE AUTO_RUN NODE
pkg1 up running enabled ftsys9
Policy_Parameters:
POLICY_NAME CONFIGURED_VALUE
Failover configured_node
Failback manual
Script_Parameters:
ITEM STATUS MAX_RESTARTS RESTARTS NAME
Service up 0 0 service1
Subnet up 15.13.168.0
Resource up /example/float
Node_Switching_Parameters:
NODE_TYPE STATUS SWITCHING NAME
Primary up enabled ftsys9 (current)
Alternate up enabled ftsys10
PACKAGE STATUS STATE AUTO_RUN NODE
pkg2 up running disabled ftsys9
Policy_Parameters:
POLICY_NAME CONFIGURED_VALUE
Failover configured_node
Failback manual
Script_Parameters:
ITEM STATUS MAX_RESTARTS RESTARTS NAME
Service up 0 0 service2.1
Subnet up 15.13.168.0
Node_Switching_Parameters:
NODE_TYPE STATUS SWITCHING NAME
Primary up enabled ftsys10
Alternate up enabled ftsys9 (current)
NODE STATUS STATE
ftsys10 up running
Network_Parameters:
INTERFACE STATUS PATH NAME
PRIMARY up 28.1 lan0
STANDBY up 32.1 lan1
|
|
Now pkg2 is running on node ftsys9. Note that switching is still disabled. Status
After Auto Run is EnabledThe following command changes the AUTO_RUN package switching flag back to enabled: The output of the cmviewcl command is now as follows: CLUSTER STATUS
example up
NODE STATUS STATE
ftsys9 up running
PACKAGE STATUS STATE AUTO_RUN NODE
pkg1 up running enabled ftsys9
pkg2 up running enabled ftsys9
NODE STATUS STATE
ftsys10 up running |
Both packages are now running on ftsys9 and pkg2 is enabled for switching. ftsys10 is running the cmcld daemon but no packages. Status
After Halting a NodeAfter we halt ftsys10 with the following command we’ll see the following output from cmviewcl: CLUSTER STATUS
example up
NODE STATUS STATE
ftsys9 up running
PACKAGE STATUS STATE AUTO_RUN NODE
pkg1 up running enabled ftsys9
pkg2 up running enabled ftsys9
NODE STATUS STATE
ftsys10 down halted |
This output can be seen on both ftsys9 and ftsys10. Viewing Information
about Unowned PackagesThe following example shows packages that are currently unowned,
that is, not running on any configured node. cmviewcl provides information on monitored resources for each
node on which the package can run; this allows you to identify the
cause of a failure and decide where to start the package up again. UNOWNED_PACKAGES
PACKAGE STATUS STATE AUTO_RUN NODE
PKG3 down halted disabled unowned
Policy_Parameters:
POLICY_NAME CONFIGURED_VALUE
Failover min_package_node
Failback automatic
Script_Parameters:
ITEM STATUS NODE_NAME NAME
Resource up manx /resource/random
Subnet up manx 192.8.15.0
Resource up burmese /resource/random
Subnet up burmese 192.8.15.0
Resource up tabby /resource/random
Subnet up tabby 192.8.15.0
Resource up persian /resource/random
Subnet up persian 192.8.15.0
Node_Switching_Parameters:
NODE_TYPE STATUS SWITCHING NAME
Primary up enabled manx
Alternate up enabled burmese
Alternate up enabled tabby
Alternate up enabled persian
|
Viewing
Information about System Multi-Node PackagesThe following example shows a cluster that includes system
multi-node packages as well as failover packages. The system multi-node
packages are running on all nodes in the cluster, whereas the standard
packages run on only one node at a time. CLUSTER STATUS
cats up
NODE STATUS STATE
manx up running
PACKAGE STATUS STATE AUTO_RUN NODE
pkg1 up running enabled manx
NODE STATUS STATE
tabby up running
PACKAGE STATUS STATE AUTO_RUN NODE
pkg2 up running enabled tabby
SYSTEM_MULTI_NODE_PACKAGES:
PACKAGE STATUS STATE
VxVM-CVM-pkg up running
|
Status
of the Packages with a Cluster File System Installed You can use cmviewcl to see the status of the package and the cluster file system
on all nodes, as shown in the example below: cmviewcl -v -p SG-CFS-pkg MULTI_NODE_PACKAGES
PACKAGE STATUS STATE AUTO_RUN SYSTEM
SG-CFS-pkg up running enabled yes
NODE_NAME STATUS SWITCHING
soy up enabled
Script_Parameters:
ITEM STATUS MAX_RESTARTS RESTARTS NAME
Service up 0 0 SG-CFS-vxconfigd
Service up 5 0 SG-CFS-sgcvmd
Service up 5 0 SG-CFS-vxfsckd
Service up 0 0 SG-CFS-cmvxd
Service up 0 0 SG-CFS-cmvxpingd
NODE_NAME STATUS SWITCHING
tofu up enabled
Script_Parameters:
ITEM STATUS MAX_RESTARTS RESTARTS NAME
Service up 0 0 SG-CFS-vxconfigd
Service up 5 0 SG-CFS-sgcvmd
Service up 5 0 SG-CFS-vxfsckd
Service up 0 0 SG-CFS-cmvxd
Service up 0 0 SG-CFS-cmvxpingd
|
Status
of CFS Disk Group PackagesTo see the status of the disk group, use the cfsdgadm display command. For example, for the diskgroup logdata, enter: cfsdgadm display -v logdata
NODE NAME ACTIVATION MODE
ftsys9 sw (sw)
MOUNT POINT SHARED VOLUME TYPE
ftsys10 sw (sw)
MOUNT POINT SHARED VOLUME TYPE
...
To see which package is monitoring a disk group, use the cfsdgadm show_package command. For example, for the diskgroup logdata, enter: cfsdgadm show_package logdata SG-CFS-DG-1 Status
of CFS Mount Point PackagesTo see the status of the mount point package, use the cfsmntadm display command. For example, for the mount point /tmp/logdata/log_files, enter: cfsmntadm display -v /tmp/logdata/log_files Mount Point : /tmp/logdata/log_files
Shared Volume : lvol1
Disk Group : logdata To see which package is monitoring a mount point, use the cfsmntadm show_package command. For example, for the diskgroup logdata: cfsmntadm show_package /tmp/logdata/log_files SG-CFS_MP-1
|
Printable version
