Managing Faults with Cisco UGM

With the Event Browser in Cisco UGM, you can identify all faults, also known as alarm events and take appropriate action to resolve them quickly and efficiently; in addition, you can forward user-specified SNMP traps to any configured remote host, and continuously export all alarm events, as they are raised, to a user-specified text file.

Trap handling in Cisco UGM is handled by the ASFaultStandAlone process, and constitutes the main function of the fault management component. A standalone process is started by sysmgr which restarts it in case of a crash.

Monitored Events

You can use the Event Browser to view alarm events raised against a device object; various filtering criteria are provided by the Query Editor.

Table 7-1 Traps and Alarm Events from Cisco UGM-Managed Devices

Alarm Event	Severity Level	Explanation
ciscoColdStart	Warning	The device object was started from a power-off state. Note Clear this event manually.
ciscoWarmStart	Warning	The SNMP server was shut down and restarted. Note Clear this event manually.
ciscoLinkDown	Major	A DS1, DS3, or Ethernet interface is down.
ciscoLinkUp	Normal	A DS1, DS3, or Ethernet interface is up.
ciscoAuthenticationFailure	Major	The device received a message that was improperly authenticated.
cachePopFailed	Major	Indicates that chassis initialization failed.
cachePopInterrupted	Major	Indicates that chassis initialization was interrupted.
cardInsertedTrap	Warning	An OIR trap indicated that a card was inserted in the device; Cisco UGM initiates discovery on the device.
cardInserted	Warning	Results from the cardInsRemDetected or cardInsertedTrap; contains the card slot number.
cardRemovedTrap	Warning	An OIR trap indicated that a card was removed from the device; Cisco UGM initiates discovery on the device.
cardRemoved	Warning	Results from the cardInsRemDetected or cardRemovedTrap; contains the card slot number.
Card inserted in slot	Informational	A new card was inserted in the device; Cisco UGM completes discovery on the device.
cardInsRemDetected	Informational	Card presence polling detected that cards were moved in the device.
Card removed in slot	Informational	A card was removed from the device; Cisco UGM completes discovery on the device.
Chassis initialization interrupted	Major	The device was removed from the initializing state before the initialization was completed.
chassisTypeMismatch	Major	A chassis was deployed by using the wrong deployment template.
chassisRebootDetected	Informational	The polling mechanism using sysUpTime detected that the device was rebooted.
communicationLost	Major	Cisco UGM lost SNMP connectivity with the device.
communicationEstablished	Normal	Cisco UGM established SNMP connectivity with the device.
discoveryFailed1	Major	Indicates that device component discovery failed due to loss of communication with the device.
discoveryInterrupted	Major	Indicates that device component discovery was interrupted.
discoveryFailed2	Major	Indicates that device component discovery failed due to Cisco UGM or Cisco EMF internal errors.
discoveryFinished	Normal	Indicates that device component discovery was completed successfully.
discoveryStarted	Normal	Indicates that device component discovery has started.
deploymentFailed	Major	Indicates that device component deployment failed due to an internal error.
deploymentInterrupted	Major	Indicates that device component deployment was interrupted.
deploymentFinished	Normal	Indicates that device component deployment was completed successfully.
deploymentStarted	Normal	Indicates that device component deployment has started.
downloadImageCompleted	Normal	Received a trap indicating that an image was downloaded.
envMonShutdown	Critical	A critical environmental condition is detected, and a device shutdown is imminent.
envMonVoltage	Major	A voltage threshold was exceeded on the device.
envMonNormalVoltage	Major	The environment monitor detected normal voltage on the device.
envMonWarningVoltage	Major	The environment monitor detected voltage that exceeded the warning level.
envMonCriticalVoltage	Major	The environment monitor detected voltage that exceeded the critical level.
envMonShutdownVoltage	Major	The environment monitor detected voltage that exceeded the shutdown level.
envMonVoltageNotPresent	Major	Voltage monitoring is not present on this device.
envMonVoltageDisabled	Major	Voltage monitoring is disabled on this device.
envMonTemperature	Major	A temperature threshold was exceeded on the device.
envMonNormalTemperature	Major	The environment monitor detected normal temperature on the device.
envMonWarningTemperature	Major	The environment monitor detected that the temperature exceeded the warning level.
envMonCriticalTemperature	Major	The environment monitor detected that the temperature exceeded the critical level.
envMonShutdownTemperature	Major	The environment monitor detected that the temperature exceeded the shutdown level.
envMonTemperatureNotPresent	Major	Temperature monitoring is not present on this device.
envMonTemperatureDisabled	Major	Temperature monitoring is disabled on this device.
envMonFan	Major	The fan on the device has failed.
envMonNormalFan	Major	The environment monitor detected that the fan is in a normal state.
envMonWarningFan	Major	The environment monitor detected that the fan is at the warning level.
envMonCriticalFan	Major	The environment monitor detected that the fan is at the critical level.
envMonShutdownFan	Major	The environment monitor detected that the fan is at the shutdown level.
envMonFanNotPresent	Major	Fan monitoring is not present on this device.
envMonFanDisabled	Major	Fan monitoring is disabled on this device.
envMonRedundantSupply	Major	The redundant power supply on the device has failed.
envMonNormalRedundantSupply	Major	The environment monitor detected that the redundant power supply is in a normal state.
envMonWarningRedundantSupply	Major	The environment monitor detected that the redundant power supply is at the warning level.
envMonCriticalRedundantSupply	Major	The environment monitor detected that the redundant power supply is at the critical level.
envMonShutdownRedundantSupply	Major	The environment monitor detected that the redundant power supply is at the shutdown level.
envMonRedundantSupplyNotPresent	Major	Redundant power supply monitoring is not present on this device.
envMonRedundantSupplyDisabled	Major	Redundant power supply monitoring is disabled on this device.
entityDecommissioned	Informational	Device or card object has been decommissioned.
entityCommissioned	Informational	Device or card object has been commissioned.
initialClearSysAlarms	Normal	Clears previous file system usage alarms during initialization.
fileSysAboveMajor	Major	Server disk usage is over the user-defined major threshold.¹
fileSysAboveCritical	Critical	Server disk usage is over the user-defined critical threshold.²
fileSysBelowMajor	Normal	Server disk usage is below the user-defined major threshold.
fileSysBelowCritical	Normal	Server disk usage is below the user-defined critical threshold.
modemGoesOffline	Warning	Indicates that the modem or Universal Port is offline due to a failure.
modemGoesOffline	Minor	Indicates that the modem or Universal Port is administratively offline.
modemGoesOnline	Normal	Indicates that the modem or Universal Port is online.
modemStatusClear	Normal	Indicates that the device is being deployed using the wrong deployment template.
gracefulShutdownInterrupted	Major	During a Graceful Shutdown operation, loss of communication with the device occurred or it was decommissioned. Note Clear this event manually.
acceptTrafficInterrupted	Major	During an Accept Traffic operation, loss of communication with the device occurred or it was decommissioned. Clear this event manually.
redStatusChange	Warning	Cisco AS5800 device operation (in the redundant mode) switched to the standby router shelf.
upgradeSPEImageInterrupted	Major	Indicates that the SPE image upgrade operation was interrupted.
upgradeModemImageInterrupted	Major	Indicates that the modem image upgrade operation was interrupted.
upgradeIOSImageInterrupted	Major	Indicates that the IOS image upgrade operation was interrupted.
upgradeVFCImageInterrupted	Major	Indicates that the VFC image upgrade operation was interrupted.
upgradeSPEImageFailed	Major	Indicates that the SPE image upgrade operation failed.
upgradeModemImageFailed	Major	Indicates that the modem image upgrade operation failed.
upgradeIOSImageFailed	Major	Indicates that the Cisco IOS image was not upgraded.
upgradeVFCImageFailed	Major	Indicates that the VFC image was not upgraded.

1For details on changing this threshold, see the "Example: Sample Configuration File for Fault Management" section.

2For details on changing this threshold, see the "Example: Sample Configuration File for Fault Management" section.

Incoming Alarm	Alarms Cleared
ciscoLinkDown	ciscoLinkUp
ciscoLinkUp	ciscoLinkDown
communicationEstablished	communicationLost discoveryFailed1
discoveryFinished	discoveryStarted
deploymentFinished	deploymentStarted
deploymentFinished	deploymentFailed
discoveryFinished	discoveryFailed2
cardInserted	cardInsertedTrap
cardRemoved	cardRemovedTrap
initialClearSysAlarms	fileSysAboveMajor fileSysAboveCritical initialClearSysAlarms
envMonNormalVoltage	envMonWarningVoltage envMonCriticalVoltage envMonShutdownVoltage envMonVoltageNotPresent envMonVoltageDisabled
envMonNormalTemperature	envMonWarningTemperature envMonCriticalTemperature envMonShutdownTemperature envMonTemperatureNotPresent envMonTemperatureDisabled
envMonNormalfan	envMonWarningFan envMonCriticalFan envMonShutdownFan envMonFanNotPresent envMonFanDisabled
envMonNormalRedundantSupply	envMonWarningRedundantSupply envMonCriticalRedundantSupply envMonShutdownRedundantSupply envMonRedundantSupplyNotPresent envMonRedundantSupplyDisabled
modemStatusClear	modemGoesOffline modemGoesOnline modemStatusClear
fileSysAboveMajor	fileSysBelowMajor
fileSysAboveCritical	fileSysBelowCritical
upgradeSPEImageFailed	downloadImageCompleted
upgradeModemImageFailed	downloadImageCompleted
upgradeIOSImageFailed	downloadImageCompleted
upgradeVFCImageFailed	downloadImageCompleted

Overview of Alarm Events

The Map Viewer shows all managed device objects with current alarms. These alarm events are indicated by colored dots next to the objects in the Map Viewer tree, and also by the color and appearance of object device icons in the Map Viewer (right) pane.

Color Identification of Alarms

In the Map Viewer tree, you can see raised alarm events by the presence of colored dots next to tree objects and by object icons in the Map Viewer pane.

The dots are color coded to reflect the following severity levels (highest to lowest): critical, major, minor, warning, informational, and normal.

Objects and Icons Representing Device States

Icon	Device State and Appearance
	Decommissioned. Diagonal lines through the object.
	Deploying Green outline with diagonal lines through the object; arrow icon in upper left corner.
	Errored Orange outline; explosive icon in upper left corner.
	Initializing Green outline with diagonal lines through the object; lock icon and arrow icon on upper side.
	Normal Green outline.
	Reload Green outline; clipboard icon in upper left corner.

Alarms Generated by Commissioning or Decommissioning Objects

A device or card object can be in either commissioned or decommissioned state within Cisco UGM.

If an object is in a commissioned state, most alarm events against that object are propagated to the physical tree in the Map Viewer and appear in the parent objects at the region level.

For decommissioned objects, alarm events are not propagated up to the physical tree in the Map Viewer.

Table 7-1 describes Cisco UGM alarm events, their severity, explanation, and recovery procedures.

Cl earing Alarm Events

If you manually clear an alarm event for an object in the Event Browser, that object appears in the Map Viewer with an alarm notification reflecting the next highest alarm present for that object (assuming that there is more than one alarm for the object).

Cisco UGM does not generate all alarm events again, even if the alarm conditions are still present; therefore, be cautious in clearing alarm events.

Step 2 Right-click the object whose list of alarm events you want to view and choose Tools > Open Event Browser.

You can acknowledge and clear individual alarm events by clicking the appropriate box next to each event.

Overview of the Event Browser

You can start the Event Browser from the Launchpad or from the pop-up menu for the individual object within Map Viewer.

You can see all events—regardless of your access privilege. The events are color coded to the corresponding alarm. (See "Color Identification of Alarms" section, and the Cisco Element Management Framework User Guide.

In the Event Browser window, you can check the Ack (acknowledge) box next to an event to communicate to other users that you are planning to deal with that particular event. When you resolve the event, click the Clear box.

You can view additional alarm details by using the Event browser. For more information, refer to the Cisco Element Management Framework User's Guide.

Using the Event Browser

Step 2 Right-click the object whose list of alarm events you want to view and choose Tools > Open Event Browser.

Using the Query Editor

If you do not want to view all events in the system, set up a query by using the Query Editor to view only specific events.

The criteria that you use to specify a query are on individual tabs. The Event Browser is updated with only those events that match the query criteria. A progress bar when the event browser is opened indicates that Cisco UGM is querying events and the window is being updated.

If you have specified different queries, you can open more than one Event Browser session at a time.

For details about the Query Editor refer to the Cisco Element Manager Framework User's Guide.

Overview of Trap Forwarding

Specifying New Trap Forwarding Hosts

Step 3 In the Map viewer window, choose ASEMSConfig > Trap Forwarding > Trap Forwarding Properties.

Specifying New Trap Specifiers for a Trap Forwarding Host

Step 4 Right-click the host destination for which you wish to add a new trap specifier and select Deploy Trap Specifiers.

Step 6 In the Map Viewer, choose ASEMSConfig > Trap Forwarding > Trap Forwarding Properties.

Trap forwarding reflects any changes made (and saved) in this dialog box. Any previously specified trap forwarding settings are replaced.

Changing Previously Specified Trap Forwarding Data

Trap forwarding reflects any changes made (and saved) in this dialog box. Any previously specified trap forwarding settings are replaced.

Removing Previously Specified Trap Forwarding Data

Step 4 Right-click the object to be deleted (a host destination, or a specific trap specifier for a given host destination) and choose Deployment > Delete Objects.

Step 5 In the Map Viewer, choose ASEMSConfig > Trap Forwarding > Trap Forwarding Properties.

Trap forwarding reflects any changes made (and saved) in this dialog box. Any previously specified trap forwarding settings are replaced.

Example: Cisco UGM Trap Mapping Tables

Tables 7-4 through 7-8 provide detailed information about SNMP v1 traps handled by Cisco UGM. Each trap is uniquely identified by Enterprise ID, Generic ID, and Specific ID.

Class Mapping	Enterprise	Generic ID	Alarm Severity
ciscoColdStart	1.3.6.1.4.1.9.1.162	0	warning
ciscoWarmStart	1.3.6.1.4.1.9.1.162	1	warning
ciscoLinkDown	1.3.6.1.4.1.9.1.162	2	major
ciscoLinkUp	1.3.6.1.4.1.9.1.162	3	normal
ciscoAuthenticationFailure	1.3.6.1.4.1.9.1.162	4	major

Class Mapping	Enterprise	Generic ID	Alarm Severity
ciscoColdStart	1.3.6.1.4.1.9.1.313	0	warning
ciscoWarmStart	1.3.6.1.4.1.9.1.313	1	warning
ciscoLinkDown	1.3.6.1.4.1.9.1.313	2	major
ciscoLinkUp	1.3.6.1.4.1.9.1.313	3	normal
ciscoAuthenticationFailure	1.3.6.1.4.1.9.1.313	4	major

Class Mapping	Enterprise	Generic ID	Alarm Severity
ciscoColdStart	1.3.6.1.4.1.9.1.274	0	warning
ciscoWarmStart	1.3.6.1.4.1.9.1.274	1	warning
ciscoLinkDown	1.3.6.1.4.1.9.1.274	2	major
ciscoLinkUp	1.3.6.1.4.1.9.1.274	3	normal
ciscoAuthenticationFailure	1.3.6.1.4.1.9.1.274	4	major

Class Mapping	Enterprise	Generic ID	Alarm Severity
ciscoColdStart	1.3.6.1.4.1.9.1.188	0	warning
ciscoWarmStart	1.3.6.1.4.1.9.1.188	1	warning
ciscoLinkDown	1.3.6.1.4.1.9.1.188	2	major
ciscoLinkUp	1.3.6.1.4.1.9.1.188	3	normal
ciscoAuthenticationFailure	1.3.6.1.4.1.9.1.188	4	major

Class Mapping	Enterprise	Generic ID	Alarm Severity
ciscoColdStart	1.3.6.1.4.1.9.1.308	0	warning
ciscoWarmStart	1.3.6.1.4.1.9.1.308	1	warning
ciscoLinkDown	1.3.6.1.4.1.9.1.308	2	major
ciscoLinkUp	1.3.6.1.4.1.9.1.308	3	normal
ciscoAuthenticationFailure	1.3.6.1.4.1.9.1.308	4	major

Table 7-9 provides detailed information about SNMP v2 traps handled by
Cisco UGM.

Table 7-9 SNMP V2 Trap Mapping

Class Mapping	Enterprise	Generic ID	Specific ID	Alarm Severity
ciscoColdStart	1.3.6.1.6.3.1.1.5.1	-1	-1	warning
ciscoWarmStart	1.3.6.1.6.3.1.1.5.2	-1	-1	warning
ciscoLinkDown	1.3.6.1.6.3.1.1.5.3	-1	-1	major
ciscoLinkUp	1.3.6.1.6.3.1.1.5.4	-1	-1	normal
ciscoAuthenticationFailure	1.3.6.1.6.3.1.1.5.5	-1	-1	major
cardInserted	1.3.6.1.4.1.9.9.117.2	6	3	warning
cardRemoved	1.3.6.1.4.1.9.9.117.2	6	4	warning
redC5800StatusChange	1.3.6.1.4.1.9.9.105.2.0.1	6	1	warning
redRFStatusChange	1.3.6.1.4.1.9.9.176.2.0.2	6	2	warning
flashCopyCompletion	1.3.6.1.4.1.9.9.10.1.3.0.1	-1	-1	normal
envMonShutdown	1.3.6.1.4.1.9.9.13.3	6	1	critical
envMonVoltage	1.3.6.1.4.1.9.9.13.3	6	2	major
envMonTemperature	1.3.6.1.4.1.9.9.13.3	6	3	major
envMonFan	1.3.6.1.4.1.9.9.13.3	6	4	major
envMonRedundantSupply	1.3.6.1.4.1.9.9.13.3	6	5	major

Overview of Exporting Alarm Events

With Cisco UGM, you can capture and export all alarm data to an ASCII text file; this file can then be examined locally by an external system or retrieved by an external system by using File Transfer Protocol (FTP). The external system is responsible for parsing the contents of this file.

Exporting SNMP traps consists of capturing traps from managed devices and writing them to a text file.

Exporting Alarm Events to a File

Step 6 Specify the maximum size (in KBytes) of a file before the selected aging action begins. When the maximum file size is reached, export then continues to the newly created file.

Step 7 Specify where the file is moved to (or moveTarCompressed to) when aging occurs.

Example: Alarm Data Export Format and Sample

Example: Sample Configuration File for Fault Management

Table of Contents