Troubleshooting 8-Port FC/GE Aggregation Card Problems

Table Of Contents

Troubleshooting 8-Port FC/GE Aggregation Card Problems

6.1  Overview

6.2  Initial Troubleshooting Checklist

6.3  Troubleshooting 8-Port FC/GE Aggregation Card Interface Problems

6.3.1  FC/FICON Encapsulated Gigabitphy Interface Is Down

6.3.2  GE Encapsulated Gigabitphy Interface Is Down

6.3.3  Gigabitphy Interface Is Administratively Down

6.3.4  Client Equipment Interface Connected to the 8-Port FC/GE Aggregation Card Is Not Up

6.3.5  Client Equipment Detects CVRD Errors

6.3.6  Transmit Frame Count Is Not Incrementing

6.3.7  Local Interface with GE Encapsulation Receives Frames But Not the Remote Interface

6.3.8  FC/FICON Encapsulated Gigabitphy Interface Receives CRC Errors from Trunk Card

6.3.9  Both the Local and Remote Gigabit Interfaces Are Down

6.3.10  Gigabitphy Interface Not Created

6.4  Debugging Problems Using show controller Command Output

6.5  Troubleshooting 8-Port FC/GE Aggregation Card Problems Using Loopbacks

6.5.1  Facility Loopbacks

6.5.2  Terminal Loopbacks

Troubleshooting 8-Port FC/GE Aggregation Card Problems

This chapter describes how to troubleshoot 8-port FC/GE aggregation card interface problems. This chapter includes the following sections:

• Overview

• Initial Troubleshooting Checklist

• Troubleshooting 8-Port FC/GE Aggregation Card Interface Problems

• Troubleshooting 8-Port FC/GE Aggregation Card Problems Using Loopbacks

6.1  Overview

The 8-port FC/GE aggregation card uses SFP (small form-factor pluggable) optical transceivers to provide up to eight configurable client interfaces. Each interface can be configured in the CLI (command-line interface) for FC (Fibre Channel), FICON (fiber connection), or GE (Gigabit Ethernet) traffic.

Figure 6-1shows an example path of a client signal through the Cisco ONS 15530 and the associated interfaces.

Figure 6-1 Interfaces for an 8-Port FC/GE Aggregation Card

6.2  Initial Troubleshooting Checklist

Follow this initial checklist before proceeding with the troubleshooting procedures:

•Check that the client receive signal power level is between -18 dBm and -13.5 dBm for multimode SFP optics and between -20 dBm and -3 dBm for single-mode SFP optics. If the receive signal power is not within the range, adjust the attenuation.

•Issue show interfaces commands to ensure that the interfaces on the signal path are administratively up and that there are no errors on the interfaces.

•Issue a show connect command to verify the status of the cross connections between the 8-port FC/GE aggregation card and the ITU trunk card or uplink card.

•Check that the LEDs on the card and SFP optics show the proper state.

•Issue a show facility-alarm status command to display the alarms on the interfaces.

•Check that the ITU cards are patched to the correct OADM ports. Issue a show patch command to verify that there are no frequency mismatches.

•Ensure that all optical connectors are clean. Refer to the Cisco ONS 15530 Cleaning Procedures for Fiber Optic Connections document.

6.3  Troubleshooting 8-Port FC/GE Aggregation Card Interface Problems

This section contains troubleshooting procedures for 8-port FC/GE aggregation card interface problems.

6.3.1  FC/FICON Encapsulated Gigabitphy Interface Is Down

Symptom    A gigabitphy interface encapsulated for FC or FICON traffic is down because of Loss of Light.
Table 6-1 describes the potential causes of the symptom and the solutions.

Table 6-1 FC/FICON Encapsulated Gigabitphy Interface Is Down

Possible Problem

Solution

The SFP optics and the client side optical fiber are different types.

1. Verify that the fiber type, either single-mode or multimode, matches the type of SFP optics installed on the card.

2. Switch to the correct type of fiber if there is a mismatch.

The cabling between the client equipment and the SFP optics are reversed.

Check that the Tx and Rx ports are correctly cabled to the client equipment. If not, then correct the cabling.

The optical fiber between the client equipment and the SFP optics is faulty.

Check the optical fibers connecting the client equipment to the SFP optics. If it they are faulty, replace them.

The connectors on the optical fiber between the client equipment and the SFP optics are dirty.

Refer to the Cisco ONS 15530 Cleaning Procedures for Fiber Optic Connections document.

6.3.2  GE Encapsulated Gigabitphy Interface Is Down

Symptom    A gigabitphy interface encapsulated for GE traffic is down because of Loss of Light or Loss of Sync.
Table 6-2 describes the potential causes of the symptom and the solutions.

Table 6-2 GE Encapsulated Gigabitphy Interface Is Down

Possible Problem

Solution

The SFP optics and the client side optical fiber are different types.

1. Verify that the fiber type, either single-mode or multimode, matches the type of SFP optics installed on the card.

2. Switch to the correct type of fiber if there is a mismatch.

The cabling between the client equipment and the SFP optics are reversed.

Check that the Tx and Rx ports are correctly cabled to the client equipment. If not, then correct the cabling.

The optical fiber between the client equipment and the SFP optics is faulty.

Check the optical fibers connecting the client equipment to the SFP optics. If it they are faulty, replace them.

The connectors on the optical fiber between the client equipment and the SFP optics are dirty.

Refer to the Cisco ONS 15530 Cleaning Procedures for Fiber Optic Connections document.

Autonegotiation is enabled on the client equipment.

1. Disable autonegotiation on the client equipment.

2. Upgrade the functional image on the 8-port FC/GE aggregation card to version 2.30 and the Cisco IOS release to 12.2(18)SV or later.

Note Functional image versions earlier than 2.27 do not support autonegotiation. Functional image versions between 2.27 and 2.3 support autonegotiation, but not end-to-end autonegotiation. Functional image version 2.3 and above support end-to-end autonegotiation. Link defects such as a broken fiber from the FC/GE card to the client device at one end are not propagated to the client at the other end.

6.3.3  Gigabitphy Interface Is Administratively Down

Symptom    A gigabitphy interface is administratively down and cannot carry any type of traffic.
Table 6-3 describes the potential causes of the symptom and the solutions.

Table 6-3 Gigabitphy Interface Is Administratively Down

Possible Problem

Solution

The SFP optics is not Cisco certified.

1. Issue a show interfaces command for the gigabitphy interface to verify that the Optical Transceiver field shows "Unknown Vendor."

2. Replace it with a Cisco certified SFP optics.

3. Issue a no shutdown command on the interface.

The flow identifier is not configured.

1. Issue a show interfaces command for the gigabitphy interface to verify that the Flow-identifier field contains a valid value.

2. Correct an invalid value with the cdl flow identifier command.

3. Issue a no shutdown command on the interface.

6.3.4  Client Equipment Interface Connected to the 8-Port FC/GE Aggregation Card Is Not Up

Symptom    The client signal to one gigabitphy interface fell below an alarm threshold falls below an alarm threshold value.
Table 6-4 describes the potential causes of the symptom and the solutions.

Table 6-4 Client Equipment Interface Connected to the 8-Port FC/GE Aggregation Card Is Not Up

Possible Problem

Solution

The cross connection is not configured.

1. Issue a show interfaces command for the gigabitphy interface to verify that the Client Laser Status field shows "Down due to Local Condition."

2. Issue a show connect command to verify the status of the cross connection.

3. Issue the connect command to establish the cross connection.

The client side laser on the 8-port FC/GE aggregation card is turned off due to forward laser control activity.

1. Issue a show interfaces command for the remote gigabitphy interface to check for keepalive timeouts.

2. Issue a show interfaces command for the local gigabitphy interface to verify that the Client Laser Status field shows "Down due to keep-alive timeout" and that forward laser control is configured.

3. Resolve the problem on the remote gigabitphy interface.

6.3.5  Client Equipment Detects CVRD Errors

Symptom    The client equipment detects CVRD (code violation and running disparity) errors and the gigabitphy interface shows Loss of Sync.
Table 6-5 describes the potential causes of the symptom and the solutions.

Table 6-5 Client Equipment Detects CVRD Errors

Possible Problem

Solution

The protocol encapsulation configuration is incorrect.

Issue a show interfaces command for the gigabitphy interface. Issue an encapsulation command if the protocol is incorrect.

The connectors on the optical fiber between the client equipment and the SFP optics are dirty.

Refer to the Cisco ONS 15530 Cleaning Procedures for Fiber Optic Connections document.

Autonegotiation is enabled on the client equipment.

Disable autonegotiation on the client equipment.

Note The 8-port FC/GE aggregation card does not support autonegotiation for GE traffic.

6.3.6  Transmit Frame Count Is Not Incrementing

Symptom    The Transmit Frame Count field in the show interfaces command output for the gigabitphy interface is not incrementing.
Table 6-6 describes the potential causes of the symptom and the solutions.

Table 6-6 Transmit Frame Count Is Not Incrementing

Possible Problem

Solution

An interface in the path is administratively shut down.

1. Issue show interface commands for all interfaces in the signal path, especially on the ITU trunk card or uplink card.

2. Make sure that the interfaces are up and the lasers are on.

The client receive signal power is not strong enough.

1. Verify that the cross connection exists.

a. Issue a show connect command to verify the status of the cross connection.

b. Issue the connect command to establish the cross connection if it is not present.

2. Verify the status of the waveethernetphy interface and the laser on the ITU trunk card.

a. Issue a show interfaces command for the waveethernetphy interface.

b. Issue a no shutdown command if the interface is administratively down. If the interfaces remain down, see Chapter 8, "Troubleshooting 2.5-Gbps ITU Trunk Card Problems," Chapter 9, "Troubleshooting 10-Gbps ITU Trunk Card Problems," or Chapter 10, "Troubleshooting 10-Gbps ITU Tunable Trunk Card Problems."

c. Issue a no laser shutdown command if the laser is off.

The cross connection is not properly configured.

1. Issue a show interfaces command to verify that the Client Laser Status field shows "Down due to Local Condition."

2. Issue a show connect command to verify the status of the cross connection.

3. Issue the connect command to establish the cross connection.

The flow identifier is not configured.

1. Issue a show interfaces command to verify that the Flow-identifier field contains a valid value.

2. Correct an invalid value with the cdl flow identifier command.

3. Issue a no shutdown command on the interface.

6.3.7  Local Interface with GE Encapsulation Receives Frames But Not the Remote Interface

Symptom    A gigabitphy interface on the local system receives frames but the remote interface does not receive any frames.
Table 6-7 describes the potential causes of the symptom and the solutions.

Table 6-7 Local Interface With GE Encapsulation Receives Frames But Not the Remote Interface

Possible Problem

Solution

The frame size of the GE traffic is too large.

Issue a show interfaces command for the gigabitphy interface to verify whether the Giant Packets field and Runt Packet field are incrementing. If they are, the signal contains traffic that the Cisco ONS 15530 does not support. The maximum frame size supported by is 10232 bytes.

The flow identifier is not configured correctly.

1. Issue a show interfaces command for both the local and remote gigabitphy interfaces to verify that the Flow-identifier field contains a correct value. Both the local and remote interfaces must have the same value.

2. Correct an incorrect value with the cdl flow identifier command.

3. Issue a no shutdown command on the interface.

6.3.8  FC/FICON Encapsulated Gigabitphy Interface Receives CRC Errors from Trunk Card

Symptom    A gigabitphy interface encapsulated with either FC or FICON shows CRC (cyclic redundancy check) errors received from the trunk cards. The show interfaces command for the interface shows NO TX CRC.
Table 6-8 describes the potential cause of the symptom and the solution.

Table 6-8 FC/FICON Encapsulated Gigabitphy Interface Receives CRC Errors from Line Card

Possible Problem

Solution

The transmit buffer is not correctly configured.

1. Issue show controller command for the interface to determine if the FC Egress FIFO Underflow field is incrementing. If it is, the transmit buffer is not large enough.

2. Issue the tx-buffer size command to increase the buffer size.

For information on setting the transmit buffer size on 8-port FC/GE aggregation card, refer to the Cisco ONS 15530 Configuration Guide .

The power of the signal received from the trunk card is close to or below the low warning threshold.

1. Verify that the receive signal power level is between -28 dBm and -8 dBm for a 2.5-Gbps ITU trunk card and between -22 dBm and -6 dBm for a 10-Gbps ITU trunk card.

2. Adjust the attenuation if the signal power is outside the power range for the trunk card.

6.3.9  Both the Local and Remote Gigabit Interfaces Are Down

Symptom    The local gigabitphy interface is down due to a local condition while the remote gigabitphy interface is down and the remote gigabitphy interface is down due to a keepalive timeout.
Table 6-9 describes the potential cause of the symptom and the solution.

Table 6-9 Both the Local and Remote Gigabit Interfaces Are Down

Possible Problem

Solution

Forward laser control is configured on one interface but not on the other interface.

1. Issue a show interfaces command for both gigabitphy interfaces to verify the status of the forward laser control configuration.

2. Issue the laser control forward enable command to enable forward laser control on the interface or the no laser control forward command to disable it.

6.3.10  Gigabitphy Interface Not Created

Symptom    A gigabitphy interface does not appear in the configuration and the system does not recognize it.
Table 6-10 describes the potential cause of the symptom and the solution.

Table 6-10 Gigabitphy interface not created

Possible Problem

Solution

Wrong SFP optics is installed.

Issue a show interfaces command for the gigabitphy interface and verify that the Optical Transceiver field shows a valid value. If not, replace the SFP optics with the correct part.

6.4  Debugging Problems Using show controller Command Output

You can use the show controllers command output to determine and resolve problems on your 8-port FC/GE aggregation card. The following is an example of the command output:

Switch# show controllers gigabitphy 4/0/2
Line card base addr: 0x400000
Optical Transceiver: Multi-Mode
FC Egress FIFO Underflow: 0
FC Egress FIFO Overflow: 0
Uplink Request FIFO full: 0
Uplink Data FIFO full: 0
Downlink Descriptor FIFO full: 0
Downlink Data FIFO full: 0
Tx CRC errors: 0
Registers specific to fabric port 2:
Encapsulation FPGA 0 base addr: 0x80000
Version.................: 0x50410621
ID......................: 0x40000
Port Control:
Line interface type................: GigabitEthernet
Line port transmit.................: enabled
Line port receive..................: enabled
Port Control 1..........: 0x960
Loopback Control:
Gigabit interface loopback.........: disabled
Line interface loopback............: disabled
Port State..............: 0x0
FC interface.......................: active
Port Fail Cause.........: 0x11
Loss Of sync.......................: yes
Loss Of signal.....................: no
Receive degrade....................: no
Transmit fault.....................: no
Inactive inflow control mode.......: yes
Egress fifo full...................: no
Transceiver present................: no
Port Fail Mask..........: 0x3F
Loss Of sync.......................: yes
Loss Of signal.....................: yes
Receive degrade....................: yes
Transmit fault.....................: yes
Inactive in flow control mode......: yes
Egress fifo full...................: yes
Transceiver present................: no
Port Fail Status........: 0xE2
Loss Of signal from xcvr...........: no
Loss Of signal from phy............: yes
Egress fifo full...................: no
Transmit fault from phy............: no
Transceiver present................: no
Loss of sync.......................: yes
Inactive...........................: yes
Egress fifo empty..................: yes
Port Fail Status Mask...: 0xF6
Loss Of signal from xcvr...........: no
Loss Of signal from phy............: yes
Receive degrade....................: yes
Transmit fault from phy............: no
Transceiver present................: yes
Loss of sync.......................: yes
Reset Register..........: 0xFF00
Ordered-Set Control.....: 0x0
Credit Mgmt Control:
Credit memory size.................: 0x800
Port login mode enable.............: yes
Flow control enable................: no
Credit Mgmt Status:
Hudson Login Complete..............: no
Client Login Complete..............: no
Excess Credit Detect...............: no
Flow Control Active................: no
Indirect Address........: 0x8130007F
Indirect Data...........: 0x11
FC Port 2 Source ID.....: 0x0
FIFO Hi Control.........: 0x4
FIFO Low Control........: 0x2
OFC Control.............: 0x8400C3
PHY Control:
External phy loopback..............: disabled
External phy.......................: locked
External phy byte sync.............: no
External phy internal byte sync....: enabled
Transceiver Control:
Auto laser shut....................: disabled
Transceiver bandwidth..............: full
Laser shut control for simi interface: enabled
Transceiver........................: present
Laser Control:
External transceiver transmit......: disabled
Write for ext transceiver tx enable: disabled
External transceiver transmit......: OFF
Hardware FLC.......................: disabled
Clock Control...........: 0x0
Mux/Demux FPGA 0 base addr: 0x40000
Registers specific to client port 2:
Uplink0 SII Register....: 0xFF (255)
GE0 MTU.................: 0x27F80040
GE0 Tx CRC Threshold....: 0x64

Table 6-11 describes some of the fields in the show controllers command output that are very useful.

Table 6-11 show controllers Command Output Field Descriptions

Field

Problem Description

FC FIFO Egress Underflow

Indicates transmit buffer underflow and invalid transmit buffer size.

Tx CRC Errors

Indicates GE CRC errors.

Port Control:

Indicates protocol encapsulation and the status of the Tx and Rx for the port.

Loopback Control:

Indicates the status of the interface loopback.

FIFO Hi Control

Indicates the value for the transmit buffer size in hexadecimal.

External Phy

Indicates if the interface is in sync.

Uplink0 SII Register

Indicates the flow identifier value.

6.5  Troubleshooting 8-Port FC/GE Aggregation Card Problems Using Loopbacks

This section describes how to use software loopbacks to perform fault isolation for signals on 8-port FC/GE aggregation cards. The 8-port FC/GE aggregation card supports two types of software loopbacks:

•Facility loopbacks

•Terminal loopbacks

To perform further loopback operations, see the "8.4  Troubleshooting 2.5-Gbps ITU Trunk Card Problems Using Loopbacks" section on page 8-5, the "9.4  Troubleshooting 10-Gbps ITU Trunk Card Problems Using Loopbacks" section on page 9-5, and the "10.4  Troubleshooting 10-Gbps ITU Tunable Trunk Card Problems Using Loopbacks" section on page 10-6.

6.5.1  Facility Loopbacks

Facility loopbacks on 8-port FC/GE aggregation cards verify the functioning of the SFP optics from the client side (see Figure 6-2).

Figure 6-2 Facility Loopback Example

To create a facility loopback:

Step 1 Issue a loopback facility command on the gigabitphy interface.

Step 2 Check that the traffic is reaching the client equipment.

Step 3 If the signal does not reach the client equipment, replace the SFP optics.

6.5.2  Terminal Loopbacks

Terminal loopbacks verify the functioning of the 8-port FC/GE aggregation cards from the trunk side (see Figure 6-3).

Figure 6-3 Terminal Loopback Example

To create a terminal loopback:

Step 1 Issue a loopback terminal command on the gigabitphy interface.

Step 2 Check that the traffic is reaching the client equipment.

Step 3 If the signal does not reach the far end, check the trunk fiber and the interfaces along the signal path. If the fiber is intact, replace the card.

Possible Problem	Solution
The SFP optics and the client side optical fiber are different types.	1. Verify that the fiber type, either single-mode or multimode, matches the type of SFP optics installed on the card. 2. Switch to the correct type of fiber if there is a mismatch.
The cabling between the client equipment and the SFP optics are reversed.	Check that the Tx and Rx ports are correctly cabled to the client equipment. If not, then correct the cabling.
The optical fiber between the client equipment and the SFP optics is faulty.	Check the optical fibers connecting the client equipment to the SFP optics. If it they are faulty, replace them.
The connectors on the optical fiber between the client equipment and the SFP optics are dirty.	Refer to the Cisco ONS 15530 Cleaning Procedures for Fiber Optic Connections document.

Possible Problem	Solution
The SFP optics is not Cisco certified.	1. Issue a show interfaces command for the gigabitphy interface to verify that the Optical Transceiver field shows "Unknown Vendor." 2. Replace it with a Cisco certified SFP optics. 3. Issue a no shutdown command on the interface.
The flow identifier is not configured.	1. Issue a show interfaces command for the gigabitphy interface to verify that the Flow-identifier field contains a valid value. 2. Correct an invalid value with the cdl flow identifier command. 3. Issue a no shutdown command on the interface.

Possible Problem	Solution
The cross connection is not configured.	1. Issue a show interfaces command for the gigabitphy interface to verify that the Client Laser Status field shows "Down due to Local Condition." 2. Issue a show connect command to verify the status of the cross connection. 3. Issue the connect command to establish the cross connection.
The client side laser on the 8-port FC/GE aggregation card is turned off due to forward laser control activity.	1. Issue a show interfaces command for the remote gigabitphy interface to check for keepalive timeouts. 2. Issue a show interfaces command for the local gigabitphy interface to verify that the Client Laser Status field shows "Down due to keep-alive timeout" and that forward laser control is configured. 3. Resolve the problem on the remote gigabitphy interface.

Possible Problem	Solution
The protocol encapsulation configuration is incorrect.	Issue a show interfaces command for the gigabitphy interface. Issue an encapsulation command if the protocol is incorrect.
The connectors on the optical fiber between the client equipment and the SFP optics are dirty.	Refer to the Cisco ONS 15530 Cleaning Procedures for Fiber Optic Connections document.
Autonegotiation is enabled on the client equipment.	Disable autonegotiation on the client equipment. Note The 8-port FC/GE aggregation card does not support autonegotiation for GE traffic.

Possible Problem	Solution
An interface in the path is administratively shut down.	1. Issue show interface commands for all interfaces in the signal path, especially on the ITU trunk card or uplink card. 2. Make sure that the interfaces are up and the lasers are on.
The client receive signal power is not strong enough.	1. Verify that the cross connection exists. a. Issue a show connect command to verify the status of the cross connection. b. Issue the connect command to establish the cross connection if it is not present. 2. Verify the status of the waveethernetphy interface and the laser on the ITU trunk card. a. Issue a show interfaces command for the waveethernetphy interface. b. Issue a no shutdown command if the interface is administratively down. If the interfaces remain down, see Chapter 8, "Troubleshooting 2.5-Gbps ITU Trunk Card Problems," Chapter 9, "Troubleshooting 10-Gbps ITU Trunk Card Problems," or Chapter 10, "Troubleshooting 10-Gbps ITU Tunable Trunk Card Problems." c. Issue a no laser shutdown command if the laser is off.
The cross connection is not properly configured.	1. Issue a show interfaces command to verify that the Client Laser Status field shows "Down due to Local Condition." 2. Issue a show connect command to verify the status of the cross connection. 3. Issue the connect command to establish the cross connection.
The flow identifier is not configured.	1. Issue a show interfaces command to verify that the Flow-identifier field contains a valid value. 2. Correct an invalid value with the cdl flow identifier command. 3. Issue a no shutdown command on the interface.

Possible Problem	Solution
The frame size of the GE traffic is too large.	Issue a show interfaces command for the gigabitphy interface to verify whether the Giant Packets field and Runt Packet field are incrementing. If they are, the signal contains traffic that the Cisco ONS 15530 does not support. The maximum frame size supported by is 10232 bytes.
The flow identifier is not configured correctly.	1. Issue a show interfaces command for both the local and remote gigabitphy interfaces to verify that the Flow-identifier field contains a correct value. Both the local and remote interfaces must have the same value. 2. Correct an incorrect value with the cdl flow identifier command. 3. Issue a no shutdown command on the interface.

Possible Problem	Solution
The transmit buffer is not correctly configured.	1. Issue show controller command for the interface to determine if the FC Egress FIFO Underflow field is incrementing. If it is, the transmit buffer is not large enough. 2. Issue the tx-buffer size command to increase the buffer size. For information on setting the transmit buffer size on 8-port FC/GE aggregation card, refer to the Cisco ONS 15530 Configuration Guide .
The power of the signal received from the trunk card is close to or below the low warning threshold.	1. Verify that the receive signal power level is between -28 dBm and -8 dBm for a 2.5-Gbps ITU trunk card and between -22 dBm and -6 dBm for a 10-Gbps ITU trunk card. 2. Adjust the attenuation if the signal power is outside the power range for the trunk card.

Field	Problem Description
FC FIFO Egress Underflow	Indicates transmit buffer underflow and invalid transmit buffer size.
Tx CRC Errors	Indicates GE CRC errors.
Port Control:	Indicates protocol encapsulation and the status of the Tx and Rx for the port.
Loopback Control:	Indicates the status of the interface loopback.
FIFO Hi Control	Indicates the value for the transmit buffer size in hexadecimal.
External Phy	Indicates if the interface is in sync.
Uplink0 SII Register	Indicates the flow identifier value.