Troubleshooting APS Problems

This chapter describes how to troubleshoot APS (Automatic Protection Switching) problems. This chapter contains the following sections:

• Overview

• Initial Troubleshooting Checklist

• Troubleshooting Specific APS Problems

9.1 Overview

APS provides protection against signal transmission failure. The Cisco ONS 15540 ESPx supports the following APS features:

•1+1 path protection

•Splitter protection

•Line card protection

–Client based

–Y-cable based

–Switch fabric based

•Trunk fiber protection

•Redundant switch fabric protection

•Bidirectional and unidirectional path switching

For more information on APS support on the Cisco ONS 15540 ESPx, refer to the Cisco ONS 15540 ESPx Configuration Guide.

9.2 Initial Troubleshooting Checklist

Follow this initial checklist before proceeding with the troubleshooting procedures:

•Issue show interfaces commands to ensure that the interfaces along the signal paths are administratively up and that there are no errors on the interfaces.

•Issue the show connect command to verify the status of the cross connections.

•Issue the show aps detail command on both nodes to verify the following:

–The working and protection interfaces are correct.

–The aps state field shows "enabled (associated)."

–The msg-channel field shows "Up" on the desired message channel.

–The direction field shows the same expected values (either "uni" or "bi") on both nodes.

–AFOV (auto-failover) is enabled.

•Check that the LEDs on the cards show the proper state.

•Issue the show facility-alarm status command to display the alarms on the interfaces.

•If ITU cards are present, check that the ITU cards are patched to the correct mux/demux module ports. Issue a show patch detail command to verify that there are no frequency mismatches.

•Verify that all required patches are configured.

•Ensure that all optical connectors are clean. Refer to the Cisco ONS 15540 ESPx Cleaning Procedures for Fiber Optic Connections document.

9.3 Troubleshooting Specific APS Problems

This section contains troubleshooting information for specific APS problems.

9.3.1 APS Group State Enabled But Not Associated

Symptom The show aps group command or show aps detail command outputs show an APS group state is enabled but the group is not associated.

Table 9-1 describes the potential causes of the symptoms and the solutions.

Table 9-1 APS Group State Enabled But Not Associated
Possible Problem	Solution
Either the working or protection channel is not present.	Verify that the channel is not administratively down. Then make sure that all of the cards are properly seated and that the LEDs are showing the proper state. Verify that all of the interfaces of the APS group are in the up/up state.
For switch fabric based line card protection, the cross connections through the switch fabric are not configured correctly.	1. Issue the show connect command to verify that the working and protection cross connections are correctly configured. 2. Use the connect command to correct any problems.

9.3.2 Bidirectional APS Configured But Remote Node Direction, Architecture, and Receive k1/k2 Are Unknown

Symptom The show aps group command or the show aps detail command output shows an APS group state is configured for bidirectional switching but the remote node direction, remote node architecture, and receive k1/k2 are unknown.

Table 9-2 describes the potential causes of the symptoms and the solutions.

Table 9-2 Bidirectional APS Configured But Remote Node Direction, Architecture, and Receive k1/k2 Are Unknown
Possible Problem	Solution
The configured message channel is not up.	1. Verify that the ethernetdcc, OSC, and fastethernet interfaces are up. 2. Verify that all required patches are configured. 3. Verify that bidirectional APS, message-channel, APS group name, and the far end IP address are configured correctly.
The client signal has errors.	Use the show interfaces command to check the error counters on the active interface. If they are increasing, the line could be bad.

9.3.3 Message Channel Interface Up But APS Msg-Channel Status Down

Symptom The configured message channel interface is up but the msg-channel status in the show aps group or show aps detail command output is down.

Table 9-3 describes the potential causes of the symptoms and the solutions.

Note Check both the local and remote systems for message channel problems.

Table 9-3 Message Channel Interface Up But APS msg-channel Status Down
Possible Problem	Solution
The line cards are not correctly patched to the mux/demux modules.	Check the patch connections on the shelf. Ensure that ITU trunk cards are connected to the correct filter ports on the mux/demux module.
The OSC interfaces are not correctly patched to the mux/demux modules.	Check that the OSC interfaces are correctly patched to the mux/demux module.
The patches between the line cards or the OSC interfaces and the mux/demux modules are not configured in the CLI.	Issue the show patch command to verify the patch connections are correctly configured. If not, issue the patch command to correct the configuration.
The unused wavepatch on a splitter line card in a line card protected configuration is not disabled.	Use the shutdown command to disable the unused wavepatch interfaces.
If far-end group names are used in the APS message channel configuration, the names are not configured correctly.	1. Use the show aps group command or the show aps detail command to verify the far-end group name configuration. 2. Use the aps message-channel command to correct the far-end group name configuration. 3. Use the show aps detail command determine the current message channel.
The message channel is IP and the NME¹ connection is down.	Use the show interfaces fastethernet 0 command to verify the status of the NME. If the line or the protocol is down, see Chapter 2, "Troubleshooting Processor Card Problems."
The optical connectors are dirty.	Refer to the Cisco ONS 15540 ESPx Cleaning Procedures for Fiber Optic Connections document.

¹NME = network management Ethernet

9.3.4 APS Does Not Switch to Protection Signal When the Working Signal Fails

Symptom When the working signal fails, APS does not switch over to the protection signal.

Table 9-4 describes the potential causes of the symptoms and the solutions.

Table 9-4 APS Does Not Switch to Protection Signal When the Working Signal Fails
Possible Problem	Solution
An APS switchover request is pending.	1. Use the show aps detail command to verify that auto-failover is enabled. 2. Use the show aps group command or the show aps detail command to determine the pending APS switchover request. 3. Use the aps clear command to remove the APS request.
A trunk failure occurred on the protection signal.	Correct the failure on the protection signal.

9.3.5 Lockout from Protection Request Fails

Symptom A request to lock out an APS switchover to the protection path made with an aps lockout command failed.

Table 9-5 describes the potential cause of the symptom and the solution.

Table 9-5 Lockout from Protection Request Fails
Possible Problem	Solution
The active signal is already switched to the protection path.	1. Use the aps switch group-name force protection-to-working command to ensure that the active signal is on the working path and then use the aps lockout command. 2. If the aps switch group-name force protection-to-working command fails, check the status of the working path using the show interfaces command and resolve the signal failure.

9.3.6 Remote Switchover Does Not Occur After Local Switchover

Symptom The remote system does not switch over after the local system switches over.

Table 9-6 describes the potential causes of the symptoms and the solutions.

Table 9-6 Remote Switchover Does Not Occur After Local Switchover
Possible Problem	Solution
Message channel is down.	1. Issue show aps detail commands on both systems to verify the APS direction configuration. 2. Issue aps direction commands to correct the APS direction configuration, if necessary.
The protection path on the remote system has failed.	1. Issue the show interfaces command for the protection interface on the remote system. 2. Resolve any problems on the interface.

9.3.7 Manual or Forced Switchover Fails

Symptom A request for a manual or forced APS switchover fails.

Table 9-7 describes the potential cause of the symptom and the solution.

Table 9-7 Manual or Forced Switchover Fails
Possible Problem	Solution
A higher priority request is in effect. For bidirectional APS, the higher priority request might originate from the remote node.	1. Use the show aps group command or the show aps detail command to determine if the request is user generated or system generated. 2. For user generated requests, use the aps clear command to remove the higher priority request. 3. For system generated requests, correct the failure that is preventing the switchover.

9.3.8 APS Group Transmitting k1k2 sf-lp to Peer APS Group

Symptom The transmit k1k2 field in the show aps group or show aps detail command output indicates sf-lp is sent to the peer APS group in a y-cable configuration.

Table 9-8 describes the potential cause of the symptoms and the solution.

Table 9-8 APS Group Transmitting k1k2 sf-lp to Peer APS Group
Possible Problem	Solution
A failure occurred on the client receive signal.	1. Check the show facility status command output for Loss of Signal and Loss of Sync alarms on the active interface. 2. Verify that there are no breaks on the client fiber and that the connector are clean. Refer to the Cisco ONS 15540 ESPx Cleaning Procedures for Fiber Optic Connections document. 3. Ensure that the SFP optics are properly seated and that the LEDs are on. 4. Issue the show interfaces command to verify the protocol encapsulation. Use the encapsulation command to correct any misconfiguration.

Table Of Contents