Troubleshooting

MAJOR/MINOR alarm flashing on affected console screen.

• Failed connection
  
  
• Failed circuit lines
  
  
• Failed trunks
  
  
• Failed cards
  
  
• Unreachable node
  
  
• High error rate on circuit lines or trunks.
  
  

1.

Use dspnw command to identify the node(s).

2.

Use vt command to place yourself at the affected node, and use dspalms to identify the alarm type.

a. If the alarm display indicates a failed connection, go to probable cause 1.

b. If the alarm display indicates a failed circuit line, go to probable cause 2.

c. If the alarm display indicates a failed trunk, go to probable cause 3.

d. If the alarm display indicates a failed card, go to probable cause 4.

e. If the alarm display indicates an unreachable node, go to probable cause 5.

1.

Failed connection.

1.

Use the dspcons command to identify which connections have failed and to determine the remote end connection assignments.

2.

Use the dsplog command to determine the cause of failure of the connections. These failures could consist of failed circuit lines, trunks cards, or clock over speeds.

a. If the connections have failed due to a circuit line failure, go to probable cause 2.

b. If the connections have failed due to a packet line failure, go to probable cause 3.

c. If the connections have failed due to a card failure, go to probable cause 4.

d. If connections have failed due to a clock over speed condition, go to probable cause 5.

2.

Failed circuit line.

1.

Use the dspclns command to identify the circuit line number and failure type.

a. If the failure is a circuit line local CGA (no pulses received at the local end of circuit line) go to probable cause 2a.

b. If the failure is a circuit line remote CGA (no pulses received at the remote end of circuit line), go to probable cause 2B.

c. If the failure is circuit line frame slips (indicating excessive frame slips on the T1 between the IGX node and the PBX) go to probable cause 2C.

d. If the failure is circuit line bipolar errors (indicating excessive bipolar errors on this circuit line) go to probable cause 2D.

2A

Circuit line local CGA.

1.

Use the dsplog command to determine date, time of day, and the duration of the CGA alarm.

2.

Determine if the PBX T1 subrate or the PBX E1 interface went down at the time the CGA alarm was logged by the IGX node.

3.

Check cabling between IGX node and the PBX and make necessary repairs if defective.

4.

Make a note of the steps taken and call the Cisco TAC.

2B

Circuit line remote CGA.

1.

Refer to remedies for probable cause 2A.

2.

Circuit line frame slips.

1.

Use the dsplog command to determine date, time of day, and duration of the frame slip alarm. Also determine if the clock source for this line has changed due to line failure in the network.

2.

Use the dspclnerrs command to quantify frame slips and rate information.

3.

Use the dspclnhist command to obtain historical information on frame slips.

4.

Use the dspcurclk command to identify the current clock source and path to the current clock source.

5.

Use the clrclnalm command to clear the circuit line alarms

6.

Make a note of the steps taken, and call the Cisco TAC.

2

Circuit bipolar errors.

1.

Use the dsplog command to determine when the bipolar error threshold was exceeded, and the duration of the alarm.

2.

Use the dspclnerrs command to quantify the bipolar errors.

3.

Use the dspclnhist command to obtain historical information on bipolar errors.

4.

Check cabling between IGX node and the PBX for loose connections, and tighten it if it is loose.

5.

Use the clrclnalm command to clear line alarms.

6.

Make a note of the steps taken, and call the Cisco TAC.

3.

Failed trunk.

1.

Use the dsptrks command to identify the remote end node name, trunk numbers at each end, and the type of failure.

a. If the display shows a communication failure, go to probable cause 3A.

b. If the display shows a local CGA, go to probable cause 3B.

c. If the display shows a remote CGA, go to probable cause 3C.

d. If the display shows a bipolar error, go to probable cause 3D.

e. If the display shows a frame slip error, go to probable cause 3E.

f. If the display shows an out-of-frame error, go to probable cause 3F.

g. If the display shows a time-stamped packet drop error, go to probable cause 3G.

h. If the display shows a non time-stamped packet drop error, go to probable cause 3H.

i. If the display shows a loop-back, go to probable cause 3I.

3A

Communication Failure.

1.

Use the dsplog command to determine when the communication failure or CGA occurred, and identify connections which may have failed due to lack of bandwidth on an alternate route.

2.

Use the dsptrkerrs command at each end of the packet line to quantify errors, and determine if they are unidirectional or bidirectional.

3.

Call telephone carrier and request span testing. Ask the carrier to perform BER tests using multiple test patterns, including standard quasi, all 1, and 3 and 24 patterns.

4.

Make a note of the steps taken, and call the Cisco TAC.

3B

Local CGA—indicates no pulses at the local end of the trunk.

1.

Refer to probable cause 3A remedies.

3C

Remote CGA—indicates no pulses at the remote end of the trunk.

1.

Refer to probable cause 3A remedies

3D

Bipolar errors—indicates excessive bipolar errors on this trunk.

1.

Use the dsplog command to determine the date, time of day, and the duration of the alarm.

2.

Use the dsptrkerrs command at each end of the trunk to quantify errors, and determine whether they are unidirectional, or bi-directional.

3.

Use the dsptrkhist command at each end of the trunk to collect historical information on line errors.

4.

Use the clrtrkalm command to clear trunk alarms.

5.

Call the Cisco TAC for assistance. Cisco personnel can monitor line errors, and may advise disruptive testing to be scheduled with telephone carrier.

6.

Call telephone carrier and request span testing. Ask the carrier to perform BER tests using multiple test patterns, including standard quasi, all 1, and 3 and 24 patterns.

7.

If telephone carrier is unable to isolate the problem on the span, contact Customer Support for assistance.

3E

Frame slip errors indicates excessive frame slips on this trunk

1.

Refer to probable cause 3D remedies.

3F.

Out-of-frame errors—indicates excessive out-of-frame errors on the trunk.

1.

Refer to probable cause 3D remedies

3G

Time-stamped packet drops—
indicates time-stamped packet drops have exceeded the threshold for generating an alarm.

1.

Use the dsplog command to determine when the dropped packet alarm threshold was exceeded, and determine the duration of the alarm.

2.

Use the dspload command alarm. to determine the current loading of this trunk.

3.

Make a note of steps taken and call Customer Support. Refer to probable cause 3G remedies.

3H

Non time-stamped packet drops—indicates that non time -stamped packet drops have exceeded the threshold for generating an alarm.

Refer to probable cause 3G remedies.

3I.

Loop-back.

1.

Determine if company personnel are performing span tests with CSU loop-backs, demarc, or DSX panel.

2.

If company personnel are performing loop-back tests, ask them to indicate when they have completed testing, and monitor the system to ensure that the loop-back indication disappears when testing is complete.

3.

If company personnel are not performing loop-back tests, telephone carrier most likely has the E1 span in loop-back mode.

4.

Call telephone carrier to verify that they are testing the E1 span, and ask them to indicate when they have completed their tests. Monitor the system to ensure that the loop-back indication disappears when testing is completed.

5.

Make a note of the alarm steps taken, and call Customer Support.

4.

Failed cards—indicates the number of cards that failed.

1.

Use the dspcds command to determine which card has failed, along with its status (active or standby).

2.

Use the dsplog command to determine time of day the card failed and whether or not any connections using this card are also in a failed condition.

3.

If the failed card is an HDM or LDM card, use the dspbob command at each end of the connection using this card to verify that data is passing. For a CDP, CVM, or UVM, use the dspchstats command.

4.

If a card has failed, make a note of the steps taken, and call Customer Support.

5.

Unreachable node—shows the number of unreachable nodes in the network.

1.

At any node, use the dsplog command to determine the date and time of day that the node became unreachable. A node is usually unreachable due to a trunk failure or a power outage.

2.

Contact personnel at that node to determine whether or not there was a power failure at the time logged by the IGX node.

3.

If there was a power failure, check that NPM comes up and run diagnostics.

4.

If there was not a power failure, call Customer Support.

6.

Clock Overspeed.

1.

Use the dspbob command to determine the incoming baud rate for this connection.

2.

Use the dspcon command to verify that the console incoming baud rate is the same as the configured baud rate.

3.

Reconfigure the incoming baud rate to match the configured baud rate.

4.

Make a note of the steps taken, and call Customer Support.