Chapter 10, Troubleshooting

This chapter presents basic fault information and diagnosis (troubleshooting) procedures for the 2.1.0, 2.2.1, and 2.3.0 product releases of the Cisco ONS 15216 EDFA2. Only TL1 information about the alarm notifications and commands is provided.

This chapter describes the behavior of the ONS 15216 EDFA2 in the most expected troubleshooting situations. It does not address all possible causes of a particular situation or all possible solutions. For additional help, please review the Release Notes for the software version(s) being used (see "Obtaining Documentation" section on page xix).

10.1 Alarm Indicators

10.1.1 LEDs and Office Alarms

On the left side of the front panel of the ONS 15216 EDFA2 are the POWER, FAIL, and LOS alarm LEDs. The Ethernet LEDs are located at the top left and right sides of the Ethernet LAN port. When the module is powered on, a LED test is performed.

The POWER, FAIL, and LOS LEDs correspond to Office Alarm conditions sent via the RJ-45 ALARM port (if used). For additional alarm contact pinout information, see the "Alarm Out Relay Interface (RJ-45)" section on page 3-5.

10.1.1.1 POWER LED (Green)

•

On: -48 VDC power supply is within tolerance. (Power Bus A and B are powered normally.)

•

Off: If in duplex mode, both Power Bus A and B have failed (below -40 VDC) and remain below the minimum turn-on supply voltage (-43 VDC). If in simplex mode, Power Bus A has failed (below -40 VDC) and remains below the minimum turn-on supply voltage (-43 VDC). Note that the power bus turn-off threshold has a 1.0V tolerance.

•

Flashing: PWRBUSA or PWRBUSB alarm. If in duplex mode, Power Bus A, or B, or both are out of the user-settable tolerance range. If in simplex mode, Power Bus A is out of the user-settable tolerance range. The tolerance range is defined by PWRBUSMIN and PWRBUSMAX. The power bus alarm threshold has a 1.0V tolerance and a 1.0V hysteresis.

If flashing, the problem may be an inappropriate setting of the Power Bus alarm or a power failure. First, check the alarm thresholds using RTRV-TH-ENV. The default values are POWERBUSMIN=400 (-40 VDC) and POWERBUSMAX=570 (-57 VDC). If the POWER LED is flashing, it means that the current voltage applied to one of the power buses is outside that range. The command RTRV-ALARM-ENV responds with which POWERBUS is outside of the range. If Power Bus B is absent on purpose, use the command ED-ENV to set the POWERBUSMODE to SIMPLEX. If all the settings are correct, verify the DC voltage of the power source.

Office Alarm: In the off condition, the first pair (0) of alarm relay contacts in the RJ-45 ALARM connector changes from a normally open condition to a closed condition. The LED and alarm relay automatically reset when the alarm condition clears.

In the flashing condition, the fourth pair (3) of alarm relay contacts in the RJ-45 ALARM connector changes from a normally open condition to a closed condition. The LED and alarm relay automatically reset when the alarm condition clears.

10.1.1.2 FAIL LED (Red)

The FAIL LED is red when on. This LED indicates all alarm conditions except POWER and LOS, which have their own LEDs. The FAIL LED functions as follows:

•

Off: The gain, output power, case temperature, laser pump current, and laser pump temperature are all within their specified ranges (or -48 VDC power has failed).

•

On: GAIN, LPOUT, CTMP, LCRNT1, LCRNT2, LTMP1, and/or LTMP2 alarm. The gain, output power, case temperature, laser pump current, and/or laser pump temperature are out of tolerance. (A major internal failure has occurred.)

If on, access the ONS 15216 EDFA2 to determine which alarms have occurred. See the sections Optical Alarms, Equipment Alarms, and Environmental Alarms that follow for information about how to troubleshoot a particular alarm.

Office Alarm: In the on condition, the second pair (1) of alarm relay contacts in the RJ-45 ALARM connector changes from a normally open to a closed condition. The LED and alarm relay automatically reset when the alarm condition clears.

10.1.1.3 LOS LED (Yellow)

•

Off: The optical input power is above the user-settable minimum threshold (or -48 VDC power has failed).

•

On: LOS alarm. The optical input power to the ONS 15216 EDFA2 is below the threshold defined by LOSTH . The value of LOSHYST provides the hysteresis required for the alarm to clear.

If on, see the Optical Alarms section for information about how to troubleshoot the LOS alarm.

Office Alarm: In the on condition, the third pair (2) of alarm relay contacts in the RJ-45 ALARM connector changes from a normally open condition to a closed condition. The LED and alarm relay automatically reset when the alarm condition clears.

10.1.1.4 Ethernet Socket LEDs

Two LEDs are located at the top left and right sides of the Ethernet socket (LAN port). These LEDs are both green. These LEDs function as follows:

•

If the right Ethernet socket LED is on or flashing, there is Ethernet activity (traffic).

Note If the system is powered up without an Ethernet cable in the Ethernet socket (LAN port), the activity LED (right side) turns on and remains on even if there is not Ethernet traffic. See "LAN Port Activity LED Stays On" section.

10.1.2 Optical Alarms

The ONS 15216 EDFA2 amplifier has three main categories of alarms: optical (DWDM), equipment (EQPT), and environmental (ENV). The optical alarms are directly related to the signal amplification and are usually service affecting. The ONS 15216 EDFA2 has three Optical alarms:

Note Most optical failures are due to dirty or damaged connectors, so it is very important to clean all connectors. Note that there is a connector inside the transmission equipment and that a problem may come from there as well as from the patchcord.

Warning Follow all directions and warning labels when working with optical fibers. To prevent eye damage, never look directly into a fiber or connector.

10.1.2.1 LOS (Loss of Signal) Alarm

This alarm is generated when the input signal is below the value of LOSTH. The value of LOSHYST provides the hysteresis required for the alarm to clear. Possible causes for this alarm are:

•

Input power is below threshold value: Verify that the threshold value is set to the correct value for the particular ONS 15216 EDFA2 application (RTRV-TH-DWDM). Read the input power using the RTRV-DWDM command.

•

No optical signal is present at the input (RTRV-DWDM): Verify that the transmitter is working correctly and that the upstream fiber is not broken (RTRV-DWDM).

•

Power meter value is higher than the reading of the ONS 15216 EDFA2 software: Clean connector of fiber patchcord as well as the one from inside the ONS 15216 EDFA2 by removing the fiber access panel.

Warning Follow all directions and warning labels when working with optical fibers. To prevent eye damage, never look directly into a fiber or connector.

10.1.2.2 GAIN (Gain Out of Range) Alarm

This alarm is only valid when the ONS 15216 EDFA2 is in GAINTEMP mode. It occurs when the value of OVERALLGAINMEASURED has deviated more than 1.25 dB from the manufacturer-defined set point's allowable range. This condition has a 0.25 dB hysteresis.

The GAIN alarm is also triggered if the value of INPWRDBM goes outside the manufacturer-defined range by more than 0.7 dB. This condition has a 0.2 dB hysteresis.

Use the RTRV-DWDM command to retrieve the values of OVERALLGAINMEASURED and INPWRDBM.

If both GAIN and LOS alarm conditions occur at the same time, only the LOS alarm is generated via TL1. The GAIN alarm has a soak period of 1-2 seconds before being raised or cleared.

•

Input power is out of range: Verify that for the current gain setting, the input power is within the range. For example, if the Gain = 22, then the input range is -27 to -5 dBm. Every dB taken from the Gain is added to the input range, so if the Gain =21, then the input range is -26 to -4 dBm.

•

VOA attenuation is drifting out of range: Verify that the VOA set value approximately equals the VOA current value.

To determine this, you must use the ASH shell. You can begin an ASH Telnet session using the IP address of the ONS 15216 EDFA2 followed by port 8023.

telnet ONS_15216_EDFA2_IP_Address 8023

"VOA set value (in units of 10 dB)" =
"snmp attribute get local cerent15216EdfaPumpStatusGain 1" -
"snmp attribute get local cerent15216EdfaConstGainOverallGain"

The value of cerent15216EdfaPumpStatusGain 1 is the fixed gain of the amplifier module. The value of cerent15216EdfaConstGainOverallGain is the set target gain. Note that the minimum VOA value is approximately 1 dB, which corresponds to the maximum Gain of approximately 22 dB.

"VOA current value (in units of 100 dB)" =
"snmp attribute get local cerent15216EdfaInPowerdBm" -
"snmp attribute get local cerent15216EdfaPumpStatusInPowerdBm 1"

The value of cerent15216EdfaInPowerdBm (INPWRDBM) is the current signal power at the input before the VOA. The value of cerent15216EdfaPumpStatusInPowerdBm 1 is the signal power after the VOA.

If the "VOA set value" is different than "VOA current value" by more than a typical variation of up to 0.5 dB, the unit must be returned to Cisco for repair.

10.1.2.3 LPOUT (Loss of Output Power) Alarm

This alarm is valid only when the ONS 15216 EDFA2 is in OPWR mode, which is rare because this mode applies to few applications and is generally not recommended. This alarm occurs if the unit is in OPWR mode and the output power is deviating more than the value of LPOUTDEV from the value of LPOUTSETPT. The value of LPOUTHYST is used to clear the alarm. Use RTRV-TH-DWDM to find the threshold values.

If both LPOUT and GAIN or LOS alarm conditions occur at the same time, only the GAIN or LOS alarm is generated via TL1.

•

The input power may be too low to achieve the output power setting: Reduce the value of PUMP2CTRLVALUE using the ED-DWDM command until the alarm clears.

•

The LPOUTDEV value may be set too low for practical use: Increase the value of LPOUTDEV using the RTRV-TH-DWDM command.

10.1.3 Equipment Alarms

The ONS 15216 EDFA2 amplifier has three main categories of alarms: optical (DWDM), equipment (EQPT), and environmental (ENV). The equipment alarms are directly related to general equipment conditions in the unit. The ONS 15216 EDFA2 has five equipment alarms:

10.1.3.1 CTMP (Case Temperature Out of Range) Alarm

This alarm occurs when the case temperature is out of the threshold range as defined by MINCTMP and MAXCTMP. The values of MINCTMPHYST and MAXCTMPHYST provide the hysteresis required for the alarm to clear. Possible causes for this alarm are:

•

The Min and Max thresholds may not reflect the ambient temperature where the ONS 15216 EDFA2 is located: Use RTRV-TH-EQPT to get the Min and Max temperature threshold and change the threshold using SET-TH-EQPT.

•

The ambient temperature is too high for a legitimate case temperature alarm to clear: Shut down the ONS 15216 EDFA2 and allow time for it to cool, and do not repower it until the ambient temperature is reduced.

10.1.3.2 LCRNT1 and LCRNT2 (Excessive Pump Current) Alarms

These alarms occur when the input power exceeds input range and the ONS 15216 EDFA2 is overdriving the pump laser to try to get the target Gain value. The manufacturer defines this condition as the drive current at greater than 95% of end of life value. The current must drop to 90% of the end-of-life value for the alarm to clear. Possible causes for this alarm are:

•

The input power is too high for the gain setting: Reduce the input power by using an external attenuator or by changing the OVERALLGAIN value using the ED-DWDM command.

•

One pump laser is degrading: The unit needs to be replaced. Contact the Cisco TAC.

10.1.3.3 LTMP1 and LTMP2 (Excessive Pump Temperature) Alarms

These alarms occur when the pump laser has reached a critical temperature threshold. That is, the chip temperature is deviating more than 10 degrees C from the manufacturer-defined set point.

•

Ambient temperature is too high so the laser can't cool down: This condition is usually associated with a CTMP alarm. See the CTMP (Case Temperature Out of Range) Alarm section above.

•

The ONS 15216 EDFA2 has been operating with excessive pump current for a long period of time, causing the pump(s) to overheat: This condition is usually associated with a LCRNT1 or LCRNT2 alarm. See the LCRNT1 and LCRNT2 (Excessive Pump Current) Alarms section above.

10.1.4 Environmental Alarms

The ONS 15216 EDFA2 amplifier has three main categories of alarms: optical (DWDM), equipment (EQPT), and environmental (ENV). The environmental alarms are directly related to the power bus voltage. The ONS 15216 EDFA2 has two equipment alarms:

10.1.4.1 PWRBUSA and PWRBUSB (Power Bus) Alarms

These alarms occur when the Power Bus voltage is out of the threshold range as defined by PWRBUSMIN and PWRBUSMAX. The power bus threshold has a 1.0V tolerance and a 1.0V hysteresis. The 1.0V tolerance prevents most early alarm conditions due to a potential �1.5V inaccuracy in the ONS 15216 EDFA2 voltage measurement.

The problem may be an inappropriate setting of the Power Bus alarm or a power failure. First, check the alarm thresholds using RTRV-TH-ENV. The default values are PWRBUSMIN=400 (-40 VDC) and PWRBUSMAX=570 (-57 VDC). If the POWER LED is flashing, it means that the current voltage applied to one of the power buses is outside that range. The command RTRV-ALARM-ENV responds with which power buses outside of the range. If Power Bus B is not being used, check that PWRBUSMODE=SIMPLE using RTRV-TH-ENV. If all the settings are correct, then verify the DC voltage of the power source.

10.2 Troubleshooting Typical Scenarios

10.2.1 No Output Power after Adjusting Gain Settings

Release Notes 2.1.0 Caveat CSCdy50894

Description: The VOA pre-attenuator is stuck at maximum attenuation if the gain setting is modified when the input power is below the minimum signal level.

Workaround: Use the following series of sample commands to recover from the VOA being stuck at high attenuation:

name:ONS15216 EDFA2> ED-DWDM:name:all:100:::PUMP1CTRLMODE=PWR,PUMP1CTRLVALUE=50;

name:ONS15216 EDFA2> ED-DWDM:name:all:101:::PREATT=10;

name:ONS15216 EDFA2> ED-DWDM:name:all:102:::PUMP1CTRLMODE=GAINTEMP;

name:ONS15216 EDFA2> ED-DWDM:name:all:103:::OVERALLGAIN=160;

ash:hostname:ONS15216 EDFA2> snmp attribute set local cerent15216EdfaPumpCfgConstPumpPower 1 50

ash:hostname:ONS15216 EDFA2> snmp attribute set local cerent15216EdfaPumpCfgControlMode 1 constPower

ash:hostname:ONS15216 EDFA2> snmp attribute set local cerent15216EdfaVariableGainPreAttenuation 10

ash:hostname:ONS15216 EDFA2> snmp attribute set local cerent15216EdfaConstGainOverallGain 160

10.2.2 2.0.1 to 2.2.1 Upgrade Attempt

To achieve a smooth software upgrade, the ONS 15216 EDFA2 must first get upgraded from software version v2.0.1 to v2.1.0 before being upgraded from v2.1.0 to v2.2.1.

If a unit has been upgraded directly from v2.0.1 to v2.2.1, some features of v2.2.1 may not operate properly. In addition, the CISCO15 user has only read write access and cannot perform administrative commands.

To fix this, the software must be reverted back to v2.0.1 using the uninstall procedure described in the release notes, and then all firmware and software upgrade procedures must be performed in series, from v2.0.1 to v2.1.0 and then from v2.1.0 to v2.2.1. See the 2.1.0 and 2.2.1 Release Notes for the procedures.

10.2.3 Image File Download Incomplete

If there is insufficient space in the flash file system (FFS) during an image file download, the ONS 15216 EDFA2 may report that the download has completed successfully even though the file is incomplete. Follow the upgrade instructions precisely to ensure that the file downloads completely and the file length is checked.

If a new image does not boot correctly, check its file size using the ASH ffs file list command. The 2.0.1 image should be 1,683,299 bytes, the 2.1.0 image should be 2,021,467 bytes, and the 2.2.1 image should be 1,963,195 bytes. See the 2.3.0 Release Notes for the 2.3.0 image file size.

If the file length is incorrect, delete the file, clear FFS space, and download the image file again. See the 2.1.0, 2.2.1, and 2.3.0 Release Notes for the procedures.

10.2.4 Boot Up Failure

If the primary image fails to boot, the ONS 15216 EDFA2 boots the second image. If both fail, the ONS 15216 EDFA2 continues to try to alternately boot the images unsuccessfully.

If this occurs, the cause could be either corrupt or incomplete images, or incorrectly addressed or named boot entries. Follow the instructions below to correct these problems.

Note If the ONS 15216 EDFA2 firmware is not version 2.3.14 or greater and the software has not been upgraded to 2.1.0 or greater, the CISCO15 user resets to read_write access level, not read_write_admin, which has full administrative control. Contact the Cisco TAC for assistance.

Step 2

Open HyperTerminal. (HyperTerminal can be found in the Microsoft Windows Accessories menu.)

Step 3

While the dots are present in the HyperTerminal window during the reboot process, enter Ctrl C on the keyboard.

Step 4

At the hostname prompt, enter the following command within 60 seconds of rebooting:

hostname:edfaboot% login CISCO15

hostname:edfaboot% password

hostname:edfaboot% ffs file list

Determine which image files are present. Ensure that there is enough space on the FFS for two images and that the images are complete. The 2.0.1 image should be 1,683,299 bytes, the 2.1.0 image should be 2,021,467 bytes, and the 2.2.1 image should be 1,963,195 bytes. See the 2.3.0 Release Notes for the 2.3.0 image file size. Use the ffs file delete command to remove any unwanted files as needed. Use the network host ftp command to download new images as needed.

hostname:edfaboot% srom cfg boot display

Check that the entries are complete and accurately reflect the image file names. Use the srom cfg boot modify command to correct the boot entries as needed.

Step 7

When the problems with the images and boot entries have been completed, enter the following command:

hostname:edfaboot% processor reset

The ONS 15216 EDFA2 should boot up properly. If not, use this procedure again to correct any remaining problems.

10.2.5 No Response from RS-232 Port

If the ONS 15216 EDFA2 does not respond when connected to the RS-232 (EIA/TIA-232) port, check that the physical connection is good, that power is supplied to the unit (POWER LED is on or flashing), that the DB-9 pinout is correct (see "PC Connection via Modem" section on page 3-14), that the port speed and communications configuration are correct (see "Serial Connection Procedure" section on page 3-7), and that any terminal server settings (if used) are correct.

10.2.6 No Response from LAN Port

If the ONS 15216 EDFA2 does not respond on the RJ-45 LAN port, check that the physical connection is good, that power is supplied to the unit (POWER LED is on or flashing), that the IP address and subnet mask are correct (RTRV-NE-GEN command from RS-232 port), and that the application (Telnet) is trying to access the correct IP address.

Check the two LAN port LEDs to be sure that there is a connection to the unit (the left Ethernet socket LED is on) and that traffic is flowing (the right Ethernet socket LED is on or flashing). If the unit is not booting correctly and is continuously resetting itself, you must us the RS-232 port to address the issue as detailed in the "Boot Up Failure" section.

10.2.7 LAN Port Activity LED Stays On

Release Notes 2.1.0, 2.2.1, and 2.3.0 Caveat CSCdx41604

Description: When powered up without an Ethernet cable, the activity light remains on.

Work Around: The Ethernet activity light works correctly when connected to the network.

10.2.8 Lost Password

If the root password for CISCO15 is lost, there is a procedure to reset it to regain full administrative control of the ONS 15216 EDFA2. The user must be connected to the ONS 15216 EDFA2 by serial communication, reset the power to the unit, and perform a series of commands within 60 seconds of rebooting, according to the following procedure. This procedure resets the default user password only; no other settings are affected.

Warning The ONS 15216 EDFA2 ceases its optical amplification function when power is off during this process. Traffic should be rerouted before this procedure begins.