|
|
Table Of Contents
Troubleshooting CPU Switch Module Problems
2.2 Initial Troubleshooting Checklist
2.3 Verifying CPU Switch Module Configuration
2.4 Recovering a Lost Password
2.5 Verifying NME Interface Configurations
2.6 Troubleshooting CPU Switch Module Memory
2.7 Verifying Hardware and Software Versions
2.8 Verifying Hardware and Software Compatibility
2.9 Troubleshooting Redundant CPU Switch Modules
2.9.1 Verifying Hardware and Software Versions of Redundant CPU Switch Modules
2.9.2 Verifying Redundant CPU Switch Module Functions
2.10 Troubleshooting CPU Switch Module Problems
2.10.1 Active CPU Switch Module Boot Failure
2.10.2 Standby CPU Switch Module Boot Failure
2.10.3 Unable to Access CPU Switch Module Console
2.10.4 Unable to Access Enable Mode on Active CPU Switch Module
2.10.5 Unable to Access Enable Mode on Standby CPU Switch Module
Troubleshooting CPU Switch Module Problems
This chapter describes how to troubleshoot CPU switch module problems. This chapter includes the following sections:
•
Overview
•
Initial Troubleshooting Checklist
•
Verifying CPU Switch Module Configuration
•
Verifying NME Interface Configurations
•
Troubleshooting CPU Switch Module Memory
•
Verifying Hardware and Software Versions
•
Verifying Hardware and Software Compatibility
•
Troubleshooting Redundant CPU Switch Modules
•
Troubleshooting CPU Switch Module Problems
2.1 Overview
The Cisco ONS 15530 supports two CPU switch modules for redundancy, one in active mode and the other in hot-standby mode. CPU switch modules are installed in slot 5 and slot 6. Each CPU switch module has a processor, a switch fabric, a clock, an Ethernet switch for communication between CPU switch modules and with the LRC (line card redundancy controller) on the OADM modules and line cards, and an SRC (switch card redundancy controller). The active CPU switch module controls the system. All LRCs in the system use the system clock and synchronization signals from the active CPU switch module. Interfaces on the CPU switch modules permit access by 10/100 Ethernet, console terminal, or modem connections.
Note
For information on slot assignments, CPU switch module LEDs, alarm condition clear and reset button, interrupt clear and reset button, NME LEDs, and cabling, refer to the
Cisco ONS 15530 Hardware Installation Guide. For default configuration of the various modules, refer to the Cisco ONS 15530 Configuration Guide and the Cisco ONS 15530 Command Reference.2.2 Initial Troubleshooting Checklist
Follow this initial checklist before proceeding with the troubleshooting procedures:
•
Issue the show running-config command to check the running configuration.
•
Ensure the LEDs on the CPU switch modules show the proper state.
•
Ensure the Ethernet and Console cables are connected properly.
•
Issue the show facility-alarm status command to check for CPU switch module, fan, or power supply alarms.
•
Issue the show hardware detail command to verify the CPU switch module functional image.
•
Ensure online and power-on diagnostics do not report any alarms or failures for the CPU switch module.
•
Ensure the active and standby CPU switch modules are compatible.
•
Ensure the active and standby CPU switch module have same version of software installed.
2.3 Verifying CPU Switch Module Configuration
To display the CPU switch module configuration and status, issue the show running-config command.
Command Purposeshow running-config
Shows all components of the CPU switch module running a configuration.
The following example shows the show running-config command, which displays all the components of the CPU switch module configuration. For a detailed description of this command, refer to the Cisco IOS Configuration Fundamentals Command Reference.
Switch# show running-configBuilding configuration...Current configuration : 2971 bytes!version 12.2no service padservice timestamps debug uptimeservice timestamps log uptimeno service password-encryptionservice internal!hostname top!boot system bootflash:ons15530-i-mz.sar-f-repboot bootldr bootflash:ons15530-i-mz.sar-f-replogging snmp-authfaillogging queue-limit 100logging buffered 10000 debuggingenable password!diag onlineno diag power-onip subnet-zeroip ftp source-interface FastEthernet0ip ftp usernameip ftp passwordno ip domain-lookup!!!<information deleted>control-plane!!redundancyassociate group spaps working Wavepatch4/0/0aps protection Wavepatch4/0/1aps enablestandby privilege-mode enable!!interface Loopback0no ip address!interface FastEthernet0ip address 172.25.22.125 255.255.255.0duplex autospeed autono cdp enable!interface Fastethernet-sby0no ip addressshutdownduplex autospeed auto!<information deleted>!router ospf 100log-adjacency-changesredistribute connected subnetsredistribute static subnets!ip classlessip route 0.0.0.0 0.0.0.0 FastEthernet0ip route 172.25.18.0 255.255.255.0 FastEthernet0no ip http server!!!snmp-server engineID local 80000009030000016447A1D1snmp-server community public RWsnmp-server location san-jose-dev-testsnmp-server contact Edward.Ding : eding@cisco.comsnmp-server enable traps snmp authentication warmstartsnmp-server enable traps ttysnmp-server enable traps bgpsnmp-server enable traps oscpsnmp-server enable traps configsnmp-server enable traps syslogsnmp-server enable traps entitysnmp-server enable traps fru-ctrlsnmp-server enable traps topology throttle-interval 60snmp-server enable traps rfsnmp-server enable traps apssnmp-server enable traps patchsnmp-server enable traps alarmsbanner motd ^C*****^Calias associate-group g400 ag400!line con 0exec-timeout 0 0line aux 0line vty 0 4exec-timeout 0 0password labloginlength 0width 0!exception core-file /tftpboot/eding/CORE/h3exception protocol ftpexception dump 172.20.46.50end2.4 Recovering a Lost Password
This section describes the procedure to recover a lost login or to enable a password. The procedure differs depending on the platform and the software used, but in all cases, password recovery requires that the system be taken out of operation and powered down.
If you need to perform the following procedure, make certain that there are secondary systems that can temporarily serve the functions of the system undergoing the procedure. If this is not possible, advise all potential users and, if possible, perform the procedure during low-use hours.
Note
Make a note of your password, and store it in a secure place.
All of the procedures for recovering lost passwords depend on changing the configuration register of the system. This is done by reconfiguring the system software.
More recent Cisco platforms run from Flash memory or are netbooted from a network server and can ignore the contents of NVRAM (nonvolatile random-access memory) when booting. By ignoring the contents of NVRAM, you can bypass the configuration file (which contains the passwords) and gain complete access to the system. You can then recover the lost password or configure a new one.
Note
If your password is encrypted, you cannot recover it. You must configure a new password.
Follow these steps to recover a password:
Step 1
Enter the show version command and the configuration register value in the privileged EXEC mode. The default value is 0x2102.
Step 2
Power up the Cisco ONS 15530.
Step 3
Press the Break key sequence or send a break signal, which is usually ^] within 60 seconds of turning the system on. If you do not see the
>prompt with a system name, the terminal is not sending the correct break signal. In that case, check the terminal or terminal emulation setup.Step 4
Enter the confreg command at the
>prompt.Step 5
Answer yes to the
Do you wish to change configuration [y/n]?prompt.Step 6
Answer no to all the questions that appear until you reach the
Ignore system config info [y/n]prompt. Answer yes.Step 7
Answer no to the remaining questions until you reach the
Change boot characteristics [y/n]?prompt. Answer yes.Step 8
Enter 2 at the
enter to boot:prompt.Step 9
Answer no to the
Do you wish to change configuration [y/n]?prompt.Step 10
Enter the reset command at the
rommon>prompt.Step 11
Enter the enable command at the
Switch>prompt. You are in enable mode and see theSwitch#prompt.Step 12
Enter the show startup-config command to view your password.
Step 13
Proceed to Step 16 if your password is clear text. Or, continue with Step 14 if your password is encrypted.
Step 14
Enter the configure memory command to copy the NVRAM into memory if your password is encrypted.
Step 15
Enter the copy running-config startup-config command.
Step 16
Enter the configure terminal command.
Step 17
Enter the enable secret password command.
Step 18
Enter the config-register value command, where value is whatever value you entered in Step 1.
Step 19
Enter the exit command to exit configuration mode.
Step 20
Enter the copy running-config startup-config command.
Step 21
Enter the reload command at the prompt.
2.5 Verifying NME Interface Configurations
The administration interfaces provide simple command-line interfaces to all internal management and debugging facilities of the CPU switch module. To manage and debug the CPU switch module, you can use the NME (network management Ethernet) interface, the console port, and the auxiliary port.
For cable connection information for each of these interface ports, refer to the
Cisco ONS 15530 Hardware Installation Guide. For initial configuration information, refer to the Cisco ONS 15530 Configuration Guide and the Cisco ONS 15530 Command Reference.The NME interface has a full duplex, auto sensing connection with troubleshooting LEDs on the CPU switch module faceplate.
You can configure and monitor the NME connection using the CLI. The NME connection appears in the configuration as FastEthernet 0 or FastEthernet-sby 0 depending on the slot where the CPU switch module is installed.
To display the NME FastEthernet module configuration and status, use the following commands:
Follow these steps to verify the NME interface:
Step 1
Issue the show interfaces FastEthernet 0 slot/subcard/port command to check the NME interface configuration.
Switch# show interfaces FastEthernet 0FastEthernet0 is up, line protocol is up
Hardware is Gt96k FE, address is 0009.7c1a.cb50 (bia 0009.7c1a.cb50)Internet address is 172.25.22.125/24MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,reliability 255/255, txload 1/255, rxload 1/255Encapsulation ARPA, loopback not setKeepalive set (10 sec)Half-duplex, 100Mb/s, 100BaseTX/FX
ARP type: ARPA, ARP Timeout 04:00:00Last input 00:00:00, output 00:00:06, output hang never
Last clearing of "show interface" counters neverInput queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0Queueing strategy: fifoOutput queue: 0/40 (size/max)5 minute input rate 3000 bits/sec, 5 packets/sec5 minute output rate 0 bits/sec, 0 packets/sec131803 packets input, 8271274 bytesReceived 131333 broadcasts (0 IP multicast)0 runts, 0 giants, 0 throttles0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog0 input packets with dribble condition detected3254 packets output, 200502 bytes, 0 underruns0 output errors, 0 collisions, 2 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier
0 output buffer failures, 0 output buffers swapped out
Step 2
Check the FastEthernet field to see whether the interface is up. If it is down, check for the following:
•
Disconnected or faulty cabling. Check cables.
•
Hardware failure. Swap hardware.
If administratively down, the interface has been administratively taken down. Issue the no shutdown interface configuration command to reenable the interface.
Step 3
Check the line protocol field to see whether the status is up.
If the interface is down, the line protocol software processes might have determined that the line is unusable or the local or remote interface might be misconfigured. See if the interface can be brought up by following the recommendations in Step 2.
Step 4
Check the duplex mode field. It should match the speed of the interface and be configured as auto-negotiation.
Step 5
Check the last input and last output fields. They show the number of hours, minutes, and seconds since the last packet was successfully received or transmitted by the interface.
Step 6
Check the output hang field. It shows the number of hours, minutes, and seconds since the last reset caused by a lengthy transmission.
Step 7
Check the CRC field. The presence of many CRC errors, but not many collisions, indicates excessive noise. If the number of errors is too high, check the cables for damage. If you are using UTP cable, make sure you are using Category 5 cables and not another type, such as Category 3.
Note
Errors and the input and output difference should not exceed 0.5 to 2.0 percent of traffic on the interface.
Step 8
Check the collisions fields. These numbers indicate packet collisions and these numbers should be very low. The total number of collisions, with respect to the total number of output packets, should be 0.1 percent or less.
Step 9
Check the late collisions fields. Late collisions should never occur in a properly designed Ethernet network. They usually occur when Ethernet cables are too long or when there are too many repeaters in the network.
Step 10
Check carrier fields. These numbers indicate a lost carrier detect signal and can be caused by a malfunctioning interface that is not supplying the transmit clock signal or by a cable problem. If the system notices that the carrier detect line of an interface is up, but the line protocol is down, it periodically resets the interface in an effort to restart it. Interface resets can also occur when an interface is looped back or shut down.
Step 11
Check the buffer fields. These numbers indicate the number of received packets discarded because there was no buffer space. Broadcast storms on Ethernet networks, and bursts of noise on serial lines, are often responsible for no-input buffer events.
Step 12
Check the FastEthernet field to see whether the interface is up. If it is down, see if the interface can be brought up by following the recommendations in Step 2. If administratively down, the interface has been administratively taken down. Issue the no shutdown interface configuration command to reenable the interface.
If you determine that the connection is configured incorrectly, refer to the
Cisco ONS 15530 Configuration Guide.In addition, you can use the show controllers command to troubleshoot the status of the NME interface configuration:
Switch# show controllers fastethernet 0Interface FastEthernet0Hardware is GT96K FE ADDR: 62118CA0, FASTSEND: 0, MCI_INDEX: 0DIST ROUTE ENABLED: 0Route Cache Flag: 1GPIO 2 CONF= 7FFF7FFFGPIO 2 IO= 3D003DCIU arbit = 2A8PHY add register = 0x3E0PHY data register = 0xF1F003APort Configuration Register= 0x80ENABLE HT8K HMOD0Port Configuration Extend Register= 0xCD00TX1:1 RXPRI=DE(00) ~FLCNTL ~FLNKP MFL64KB EPort Command Register= 0x0Port Status Register= 0x9100MB HDPX FCTL EN LNK UP ~PAUSED TX oFFSerial Parameter Register= 0x218823Hash table pointer= 0x35A83C0Source ADDR L= 0x0Source ADDR H= 0x0SDMA configuration register= 0x2200RETX 0 RX BE TX BE FRINT BSIZE 4SDMA command register= 0x1000080SRT TXL EN RXInterrupt MASK= 0x80003DCDInterrupt Cause= 0x0Serial 0 mask 3Serial 0 cause 0IP DIFFSERV P0L= 0x0 IP DIFFSERV P0H= 0x0IP DIFFSERV P1L= 0x0 IP DIFFSERV P1H= 0x0IP VLAN TAG PRI= 0xF0CCIP VLAN TAG PRI= 0xF0CCFirst rxd Q0= 0x35E85A0 Curr rxd Q0= 0x35E85A0First rxd Q1= 0x35E88A0 Curr rxd Q1= 0x35E88A0First rxd Q2= 0x35E8D00 Curr rxd Q2= 0x35E8D00First rxd Q3= 0x35E9160 Curr rxd Q3= 0x35E9160First txd Q0= 0x35E99D0 First txd Q1= 0x35E9E00gt96kfe_instance=0x6211AA58, registers=0xB4088800rx ring entries=64, tx ring entries=128rxring0=0x35E8440, rxring1=0x35E88A0, rxring2=0x35E8D00, rxring3=0x35E9160malloc rxring0=0x35E8440, rxring1=0x35E88A0, rxring2=0x35E8D00, rxring3=0x35E9160Head rxring0=0xD, rxring1=0x0, rxring2=0x0, rxring3=0x0Tail rxring0=0x0, rxring1=0x0, rxring2=0x0, rxring3=0x0Shadow rxring0=0x6211ACE0, rxring1=0x6211AE20, rxring2=0x6211AF60, rxring3=0x62125CA0tx_limited=0(128)txring0=0x35E95C0, txring1=0x35E9E00Head txring0=0x41, txring1=0x0Tail txring0=0x41, txring1=0x0Tail COUNT txring0=0x0, txring1=0x0PHY registers:Register 0x00: 1000 782D 0040 6212 01E1 40A1 0003 0000Register 0x08:Register 0x10: D000 0301 0000 0000 0000 017F 0100 0000Register 0x18: 003A F33E 8F00 FF00 002A C000 20A0MIB counters:bytes_recvd =11564162bytes_sent =214232frames_recvd =156732frames_sent =3265total_bytes_recvd =11564162total_frames_recvd =156735bcast_frames_recvd =131833mcast_frames_recvd =22545crc_errors =0ovr_sized_frames =0fragments =3jabber =0collision =0late_collision =064bytes_frames =14631165_127bytes_frames =8619128_255bytes_frames =1015256_511bytes_frames =4056512_1023bytes_frames =01023_maxbytes_frames =0rx_error =0dropped_frames =0mcast_frames_tx =0bcast_frames_tx =2803sml_frame_recvd =0Software MAC address filter(hash:length/addr/mask/hits):0x00: 0 ffff.ffff.ffff 0000.0000.0000 1318032.6 Troubleshooting CPU Switch Module Memory
To troubleshoot the CPU switch module memory, use the following commands:
Command Purposeshow memory
Shows statistics about the Cisco ONS 15530 memory, including free pool statistics.
show buffers
Displays statistics for the buffer pools on the Cisco ONS 15530.
Troubleshooting Cisco ONS 15530 CPU switch module memory is the same as troubleshooting any Cisco route processor. Refer to the "Troubleshooting Hardware and Booting Problems" chapter of the Cisco IOS Internetwork Troubleshooting Handbook for more information.
If the Cisco ONS 15530 fails, it is sometimes useful to get a full copy of the memory image, called a core dump, to identify the cause of the failure. Core dumps are generally only useful to your technical support representative. For troubleshooting information relating to system management and information about creating core dumps, refer to the Cisco IOS Configuration Fundamentals Command Reference.
2.7 Verifying Hardware and Software Versions
A common problem is an incompatibility between a hardware module and the Cisco IOS software version needed to perform a particular function. This section describes troubleshooting that problem.
Display the hardware and software versions to ensure that they are the most recent. Very old hardware and software versions (two or three versions back) can have caveats that have been fixed in more recent versions. Use the following EXEC commands to display version information:
To verify hardware and software versions, use the following steps:
Step 1
Issue the show version command to display the system software version on the active CPU switch module.
Switch# show versionCisco Internetwork Operating System SoftwareIOS (tm) ONS-15530 Software (ONS15530-I-M), Version 12.2(20030711:0
04939) [sar-f-rep 108]Copyright (c) 1986-2003 by cisco Systems, Inc.Compiled Mon 14-Jul-03 14:44 by sarImage text-base: 0x60010BDC, data-base: 0x60A30000ROM: System Bootstrap, Version 12.1(10r)EV, RELEASE SOFTWARE (fc1)
top uptime is 8 hours, 2 minutesSystem returned to ROM by RPR Switchover at 20:01:26 UTC Fri Jun 23 2000System image file is "bootflash:ons15530-i-mz.sar-f-rep"cisco ONS15530 (RM7000) processor with 49152K/16384K bytes of memory.R7000 CPU at 234Mhz, Implementation 39, Rev 2.1, 256KB L2, 2048KB L3 CacheLast reset from s/w nmi2 FastEthernet/IEEE 802.3 interface(s)509K bytes of non-volatile configuration memory.16384K bytes of Flash internal SIMM (Sector size 256K).Standby CPU is upStandby CPU has 49152K/16384K bytes of memory.error - a Software forced crash, PC 0x602C1830ONS-15530 Software (ONS15530-I-M), Experimental Version 12.2(20030711:004939) [sar-f-rep 108]Compiled Mon 14-Jul-03 14:44 by sarImage text-base: 0x60010BDC, data-base: 0x60A30000Stack trace from system failure:FP: 0x625BF990, RA: 0x602C1830FP: 0x625BF9C0, RA: 0x6008DB90FP: 0x625BF9F8, RA: 0x625BFA88FP: 0x625BF9F8, RA: 0x602BF5D0FP: 0x625BFA18, RA: 0x60623578FP: 0x625BFA60, RA: 0x6062376CFP: 0x625BFCE8, RA: 0x60620998FP: 0x625BFD58, RA: 0x6060B7D4Configuration register is 0x2Step 2
Verify the ROM field. It indicates the release of Cisco IOS software loaded and running on the active CPU switch module.
Step 3
Issue the show hardware command to display the hardware revision levels for the CPU switch modules.
Switch# show hardware--------------------------------------------------------------------------------ONS 15530 Chassis, ETSI Version named Switch, Date: 04:04:48 UTC Sat Jun 24 2000----------------------------------------------------------------------------------------------------------------------------------------------------------------Back-Plane Information--------------------------------------------------------------------------------Orderable Product No. MAC-Address MAC-Size Serial No. Mfg. Date H/W Ver--------------------- ----------------- -------- ------------ ---------- -------15530-CHAS-E= 00-09-7c-1a-cb-50 16 TBC06101005 2002/06/24 3.1--------------------------------------------------------------------------------Slot Orderable Product No. Part No. Rev Serial No. Mfg. Date H/W Ver.---- ------------------------- ---------- --- ------------ ---------- ----------0/0 PROTO-HAMPTONS-MUX/DEMUX 73-7399-01 2 CAB0603MBAX 01/30/2002 1.00/1 PROTO-HAMPTONS-MUX/DEMUX 73-7399-01 2 CAB0603MB91 01/30/2002 1.05/* 15530-CPU= 73-6572-06 C0 CNH0651006X 01/21/2003 6.1
6/* 15530-CPU= 73-6572-06 C0 CNH0651006L 01/14/2003 6.1
Power Supply:Slot Part No. Rev Serial No. RMA No. Hw Vrs Power Consumption---- ---------------- ---- ----------- ----------- ------- -----------------Power Supply 0 Not presentUnable to read idprom for 1Power Supply 1 :type : 600W ACstatus : OKStep 4
Verify that the hardware versions listed in the H/W Ver column for the CPU switch modules in slots 5 and 6 are the same. If the hardware versions are not the same, continue with the "Verifying Hardware and Software Compatibility" section.
Step 5
Issue the show hardware detail command to display detailed information about the CPU switch module hardware, including the functional image versions.
Switch# show hardware detail--------------------------------------------------------------------------------ONS 15530 Chassis, ETSI Version named Switch, Date: 04:05:37 UTC Sat Jun 24 2000----------------------------------------------------------------------------------------------------------------------------------------------------------------Back-Plane Information--------------------------------------------------------------------------------Slot Number : N/AController Type : 0x1106On-Board Description : ONS 15530 Chassis, ETSI VersionOrderable Product Number: 15530-CHAS-E=Board Part Number : 73-6573-03Board Revision : 02Serial Number : TBC06101005Manufacturing Date : 2002/06/24Hardware Version : 3.1RMA Number : 0RMA Failure Code : 0MAC Address : 00-09-7c-1a-cb-50MAC Address Block Size : 16--------------------------------------------------------------------------------Slot Number : 0/0Controller Type : 0x1108On-Board Description : Prototype-Hamptons-MUX/DEMUXOrderable Product Number: PROTO-HAMPTONS-MUX/DEMUXBoard Part Number : 73-7399-01Board Revision : 2Serial Number : CAB0603MBAXManufacturing Date : 01/30/2002Hardware Version : 1.0RMA Number : 0x00RMA Failure Code : 0x00--------------------------------------------------------------------------------Slot Number : 0/1Controller Type : 0x1108On-Board Description : Prototype-Hamptons-MUX/DEMUXOrderable Product Number: PROTO-HAMPTONS-MUX/DEMUXBoard Part Number : 73-7399-01Board Revision : 2Serial Number : CAB0603MB91Manufacturing Date : 01/30/2002Hardware Version : 1.0RMA Number : 0x00RMA Failure Code : 0x00--------------------------------------------------------------------------------Slot Number : 5/*Controller Type : 0x1100On-Board Description : ONS 15530 CPU and Switch BoardOrderable Product Number: 15530-CPU=Board Part Number : 73-6572-06Board Revision : C0Serial Number : CNH0651006XManufacturing Date : 01/21/2003Hardware Version : 6.1
RMA Number :RMA Failure Code :Functional Image Version: 1.43
Function-ID : 0--------------------------------------------------------------------------------Slot Number : 6/*Controller Type : 0x1100On-Board Description : ONS 15530 CPU and Switch BoardOrderable Product Number: 15530-CPU=Board Part Number : 73-6572-06Board Revision : C0Serial Number : CNH0651006LManufacturing Date : 01/14/2003Hardware Version : 6.1
RMA Number :RMA Failure Code :Functional Image Version: 1.43
Function-ID : 0Power Supply:Slot Part No. Rev Serial No. RMA No. Hw Vrs Power Consumption---- ---------------- ---- ----------- ----------- ------- -----------------Power Supply 0 Not presentUnable to read idprom for 1Power Supply 1 :type : 600W ACstatus : OKStep 6
Verify that the Hardware Version and Functional Image Version fields for the CPU switch modules in slots 5 and 6 are the same. If they are not the same, continue with the following process to confirm that they are compatible.
Step 7
Use the show functional-image command to display detailed information about the functional images for the route processors, switch processors, and Fast Ethernet interface for the Cisco ONS 15530. The following example shows how to display the functional image for the route processor in slot 4:
Switch# show functional-image slot XStep 8
Verify the FunctionalVersion and #HardwareRequired fields to determine the FPGA version and the hardware version required for the FPGA. Compare this with the hardware version using the show hardware command output. If the FPGA version does not support the hardware version, download a new FPGA image, upgrade the hardware, or both.
2.8 Verifying Hardware and Software Compatibility
You can verify your hardware and software version compatibility by using the following EXEC command to display CPU switch module compatibility information:
Command Purposeshow redundancy capability
Displays the software version compatibility information.
show functional-image slot slot
Displays functional image information.
To verify hardware and software compatibility of the CPU switch modules and modules, use the following steps:
Step 1
Issue the show redundancy capability command to display the system software version compatibility with the various modules installed.
Switch# show redundancy capabilityCPU capability supportActive CPU Sby CPU Sby Compat CPU capability description---------- ---------- ----------- ----------------------------------------48 MB 48 MB OK CPU DRAM size
16 MB 16 MB OK CPU PMEM size
512 KB 512 KB OK CPU NVRAM size
16 MB 16 MB OK CPU Bootflash size6.1 6.1 OK CPU hardware major.minor version
1.43 1.43 OK CPU functional major.minor version
Linecard driver major.minor versions, (counts: Active=13, Standby=13)
Active CPU Sby CPU Sby Compat Drv/Ch/F ID Driver description---------- ---------- ----------- ----------- ------------------------------1.3 1.3 OK 0x1100/0/0 CPU with Switch Fabric2.3 2.3 OK 0x1101/0/0 10 Port ESCON line card2.1 2.1 OK 0x110A/0/0 8 Port GE-FC line card3.1 3.1 OK 0x1105/0/0 2.5G Transparent line card1.9 1.9 OK 0x1105/1/0 2.5G Transparent line card3.1 3.1 OK 0x1109/0/0 2.5G Transparent line card1.9 1.9 OK 0x1109/1/0 2.5G Transparent line cardActive CPU Sby CPU Sby Compat Drv/Ch/F ID Driver description---------- ---------- ----------- ----------- ------------------------------1.3 1.3 OK 0x1103/0/0 OSC line card0.1 0.1 OK 0x1107/1/0 OSC daughter card2.1 2.1 OK 0x1102/0/0 10G trunk card1.0 1.0 OK 0x110B/0/0 2.5G trunk card2.1 2.1 OK 0x1110/0/0 PSM wdm splitter1.1 1.1 OK 0x1100/0/1 ONS15530 RommonSoftware sync client versions, listed as version range X-Y.
X indicates the oldest peer version it can communicate with.Y indicates the current sync client version.Sync client counts: Active=6, Standby=6Active CPU Sby CPU Sby Compat Cl ID Redundancy Client description---------- ---------- ----------- ----- ------------------------------------ver 1-2 ver 1-2 OK 17 CPU Redundancyver 1-1 ver 1-1 OK 19 Interface Syncver 1-1 ver 1-1 OK 36 MetOpt Password Syncver 1-2 ver 1-2 OK 18 Online Diagnosticsver 1-2 ver 1-2 OK 6 OIR Clientver 1-1 ver 1-1 OK 27 metopt cm db syncBackplane IDPROM comparison
Backplane IDPROM field Match Local CPU Peer CPU--------------------------- ----- -------------------- --------------------idversion YES 1 1magic YES 153 153card_type YES 4358 4358order_part_num_str YES 15530-CHAS-E= 15530-CHAS-E=description_str YES ONS 15530 Chassis, ETSI VersionONS 15530 Chassis, ETSIVersionboard_part_num_str YES 73-6573-03 73-6573-03board_revision_str YES 02 02serial_number_str YES TBC06101005 TBC06101005date_of_manufacture_str YES 2002/06/24 2002/06/24deviation_numbers_str YES 0 0manufacturing_use YES 0 0rma_number_str YES 0 0rma_failure_code_str YES 0 0oem_str YES Cisco_Systems Cisco_Systemsclei_str YESsnmp_oid_substr YES 3.326 3.326schematic_num_str YES 92-4568-03 92-4568-03hardware_major_version YES 3 3Backplane IDPROM field Match Local CPU Peer CPU--------------------------- ----- -------------------- --------------------hardware_minor_version YES 1 1engineering_use_str YEScrc16 OK 26352 9285user_track_string YESdiagst YES ^A ^Aboard_specific_revision YES 1 1board_specific_magic_number YES 153 153board_specific_length YES 56 56mac_address_block_size YES 16 16mac_address_base_str YES 00097c1acb50 00097c1acb50cpu_number OK 0 1optical_backplane_type YES 255 255Step 2
Check the CPU memory sizes and versions in the CPU Capability Description column. The numbers in the Active CPU and Sby CPU (Standby CPU) columns should match. If not, check the Sby Compat (Standby Compatibility) column. If this column indicates the values are OK, then these values will function as compatible redundant CPU switch modules. If not, swap the CPU switch modules with versions that are compatible.
Step 3
Check the CPU hardware major.minor versions and CPU functional major.minor versions in the CPU Capability Description column. The numbers in the Active CPU and Sby CPU (Standby CPU) columns should match. If not, check the Sby Compat (Standby Compatibility) columns. If this column indicates the values are OK, then these values will function as compatible redundant CPU switch modules. If not, swap the CPU switch modules with versions that are compatible.
Step 4
Check the information in the Linecard driver section of the display. This section shows the compatibility of the software versions installed on the active and standby CPU switch modules with the various modules installed in the system.
Step 5
Check the Sby Compat (Standby Compatibility) and the Driver description columns. An OK in the Sby Compat column indicates the software version installed on the CPU switch modules supports the drivers on the modules listed.
Step 6
Check the Software sync client version section of the display. The Active CPU, Sby CPU and Redundancy Client description columns indicate the software versions the two CPU switch modules can use to synchronize their configurations. The version range in the display, shown as X-Y, indicates oldest-current peer client versions. For example, if the version lists 1-2, that indicates version 1 is the oldest version that the current version 2 could synchronize with its configuration.
Step 7
Check the Backplane IDPROM comparison section of the display. Check the Match column. This indicates which elements match, are acceptable, or fail. Some elements do not match but the range is acceptable. For example, the crc16 elements fields never match because the information in the IDPROMs of the two CPU switch modules are different so the checksums never match. But they do appear as OK or compatible.
If any of the drivers are not supported or appear as OK, try updating the images installed on the CPU switch modules. Use the information in the "1.10 Checking Release Notes for Workarounds" section on page 1-16 to upgrade to a more recent version. That should solve a CPU switch module image compatibility problem.
2.9 Troubleshooting Redundant CPU Switch Modules
The Cisco ONS 15530 supports fault tolerance by allowing a standby CPU switch module to take over if the active CPU switch module fails. This standby, or redundant, CPU switch module runs in hot-standby mode. In hot-standby mode, the standby CPU switch module is partially booted with the Cisco IOS software; however, no configuration is loaded.
At the time of a switchover, the standby CPU switch module takes over as the active CPU switch module and loads the configuration as follows:
•
If the running configurations on the active and standby CPU switch module match, the new active CPU switch module uses the running configuration file.
•
If the running configurations on the active and standby CPU switch modules do not match, the new active CPU switch module uses the last saved configuration file in its NVRAM (not the NVRAM of the former active CPU switch module).
The former active CPU switch module then becomes the standby CPU switch module.
Note
If the standby CPU switch module is unavailable, a major alarm is reported. Issue the show facility-alarm status command to display the redundancy alarm status.
For redundant CPU switch modules to function correctly, your Cisco ONS 15530 CPU switch modules must meet the following requirements:
•
Both CPU switch modules must have compatible hardware configurations.
•
ROMMON version 12.1(10r)EV.
•
Both CPU switch modules must have compatible releases of Cisco IOS software.
A common error you may encounter is the incompatibility of hardware modules and the Cisco IOS software version needed to perform a particular function.
2.9.1 Verifying Hardware and Software Versions of Redundant CPU Switch Modules
To troubleshoot the CPU switch module hardware and software versions for redundancy, use the following commands:
To confirm that your system CPU switch modules meet the redundancy requirements, complete the following steps:
Step 1
Use the show version command to confirm the system hardware and software status of the active CPU switch module.
Switch# show versionCisco Internetwork Operating System SoftwareIOS (tm) ONS-15540 Software (manopt-M0-M), 12.1(X:X)Copyright (c) 1986-2001 by cisco Systems, Inc.Compiled Fri 23-Feb-01 15:23 by ffrazerImage text-base:0x60010950, data-base:0x604E8000ROM:System Bootstrap, Version 12.1(X:X)
BOOTFLASH:ONS-15540 Software (manopt-M0-M), 12.1(X:X)Switch uptime is 30 minutesSystem returned to ROM by power-onSystem image file is "tftp://test/eng/manopt-m0-mz.010223.6"cisco (QUEENS-CPU) processor with 98304K/32768K bytes of memory.R7000 CPU at 234Mhz, Implementation 39, Rev 2.1, 256KB L2, 2048KB L3 CacheLast reset from power-on2 Ethernet/IEEE 802.3 interface(s)509K bytes of non-volatile configuration memory.20480K bytes of Flash PCMCIA card at slot 0 (Sector size 128K).16384K bytes of Flash internal SIMM (Sector size 64K).Configuration register is 0x102Step 2
Verify the ROM field. It indicates the release of Cisco IOS software loaded and running on the active CPU switch module.
Step 3
Use the show hardware detail command to compare the hardware versions of the active and standby CPU switch modules.
Switch# show hardware detail--------------------------------------------------------------------------------named Switch, Date: 04:36:29 UTC Fri Apr 20 2001--------------------------------------------------------------------------------.{Information Deleted].--------------------------------------------------------------------------------Slot Number : 6Controller Type : Queens CPUOn-Board Description : Queens_CPU_PHASE_0Orderable Product Number: N/ABoard Part Number : 73-5621-02Board Revision : 03Serial Number : CAB0505GZHDManufacturing Date : 02/16/2001Hardware Version : 2.1
RMA Number : 0x00RMA Failure Code : 0x00Functional Image Version: 1.8--------------------------------------------------------------------------------Slot Number : 7Controller Type : Queens CPUOn-Board Description : Queens_CPU_PHASE_0Orderable Product Number: N/ABoard Part Number : 73-5621-02Board Revision : 03Serial Number : CAB0505GZHVManufacturing Date : 02/16/2001Hardware Version : 2.1
RMA Number : 0x00RMA Failure Code : 0x00Functional Image Version: 1.11--------------------------------------------------------------------------------Back-Plane EEPROM--------------------------------------------------------------------------------Slot Number : N/AController Type : N/AOn-Board Description :Orderable Product Number:Board Part Number :Board Revision :Serial Number :Manufacturing Date : 01/01/2000Hardware Version : 0.0RMA Number : 0x00RMA Failure Code : 0x00Optical Back-Plane Type : Unknown Optical BackplaneMAC Address : 00-ab-00-00-00-MAC Address Block Size : 1--------------------------------------------------------------------------------Power-Supply Module--------------------------------------------------------------------------------Primary Power-Supply is : Not workingBackup Power-Supply is : Not workingStep 4
In the slots labeled 6 and 7, compare the Image version fields. These numbers must all match or be compatible, otherwise redundancy will not function correctly on your Cisco ONS 15530. For additional information, see the "Verifying Hardware and Software Versions" section.
To troubleshoot the hardware and software versions on redundant CPU switch module, use the following steps:
Step 1
Issue the show version command to display the system software version on the active CPU switch module as described in the "Verifying Hardware and Software Versions" section.
Step 2
Issue the show redundancy summary command to check the configuration and status of the active and standby CPU switch module.
Switch# show redundancy summaryRedundant system information----------------------------Available Uptime: 12 hours, 50 minutessysUpTime (switchover clears): 7 hours, 52 minutesSwitchover Count: 5Inter-CPU Communication State: UPLast Restart Reason: Switch overReported Switchover Reason: Active unit failed (error - a Software forced crash, PC 0x602C1830)Software state at switchover: STANDBY HOTLast Running Config sync: 7 hours, 52 minutes
Running Config sync status: In SyncLast Startup Config sync: 7 hours, 52 minutes
Startup Config sync status: In SyncThis CPU is the Active CPU.-------------------------------Slot: 5
Time since CPU Initialized: 8 hours, 7 minutesImage Version: ONS-15530 Software (ONS15530-I-M), Experimental Version 12.2(20030711:004939) [sar-f-rep 108]Image File: bootflash:ons15530-i-mz.sar-f-repSoftware Redundancy State: ACTIVEHardware State: ACTIVEHardware Severity: 0Peer CPU is the Standby CPU.-------------------------------Slot: 6
Time since CPU Initialized: 7 hours, 52 minutesImage Version: ONS-15530 Software (ONS15530-I-M), Version 12.2(20030711:004939) [sar-f-rep 108]Image File (on sby-CPU): bootflash:ons15530-i-mz.sar-f-repSoftware Redundancy State: STANDBY HOTHardware State: STANDBYHardware Severity: 0Privilege Mode: EnabledStep 3
Verify the Last Running Config sync and Last Startup Config sync fields. They indicate the last time the running configuration and startup configuration were synchronized between the CPU switch modules.
Step 4
Verify the active, standby, and Slot fields. They indicate in which slot the active CPU switch module is configured.
2.9.2 Verifying Redundant CPU Switch Module Functions
To troubleshoot the CPU switch module function capabilities and redundancy, use the following commands:
Follow these steps to troubleshoot CPU switch module and redundancy capabilities on the system:
Step 1
Issue the show redundancy capability command to display capabilities of the active or standby CPU switch modules described in the "Verifying Hardware and Software Versions" section.
Step 2
Check the CPU memory sizes and versions in the column, CPU capability description. The numbers in the columns Active CPU and Sby CPU (Standby CPU) should match. If not, check the column, Sby Compat (Standby Compatibility). If this column indicates the values are OK then these values will function as compatible redundant CPU switch modules. If not, swap the CPU switch modules with versions that are compatible.
Step 3
Check the CPU hardware and functional major.minor versions in the column, CPU capability description. The numbers in the columns Active CPU and Sby CPU (Standby CPU) should match. If not, check the column, Sby Compat (Standby Compatibility). If this column indicates the values are OK then these values will function as compatible redundant CPU switch modules. If not, swap the CPU switch modules with versions that are compatible.
Step 4
Check the information in the column Driver description. This column lists the hardware drivers on the system components that are supported by the CPU switch module version for both the Active and Sby (Standby) CPU switch modules. OK indicates both versions of CPU switch modules support these drivers.
Step 5
Check the Software sync client version section of the display. The Active and Sby CPU columns Redundancy Client description columns indicate the software versions the two CPU switch modules can use to synchronize their configurations. The version range in the display, shown as X-Y, indicates oldest-current peer client versions. For example, if the version lists 1-2, that indicates version 1 is the oldest version that the current version 2 could synchronize with its configuration.
Step 1
Check the IDPROM comparison section of the display. Check the Match column. This indicates which elements match, are acceptable, or fail. Some elements do not match but the range is acceptable. For example, the crc16 elements fields never match because the information in the IDPROMs of the two CPU switch modules are different so the checksums never match. But they do appear as OK or compatible.
Step 2
Issue the show redundancy clients command to display a list of internal redundancy clients.
Switch# show redundancy clientsclientID = 0 clientSeq = 0 RF_INTERNAL_MSGclientID = 6 clientSeq = 180 OIR ClientclientID = 7 clientSeq = 190 APSclientID = 17 clientSeq = 230 CPU RedundancyclientID = 18 clientSeq = 280 Online DiagnosticsclientID = 19 clientSeq = 300 Interface SyncclientID = 27 clientSeq = 330 metopt cm db syncclientID = 35 clientSeq = 360 History RF ClientclientID = 36 clientSeq = 370 MetOpt Password SyncclientID = 65000 clientSeq = 65000 RF_LAST_CLIENTStep 3
Issue the show redundancy counters command to display internal redundancy software counters.
Switch# show redundancy countersRedundancy Facility OMscomm link up = 2comm link down down = 1invalid client tx = 1null tx by client = 0tx failures = 1tx msg length invalid = 0client not rxing msgs = 0rx peer msg routing errors = 0null peer msg rx = 0errored peer msg rx = 0buffers tx = 2668tx buffers unavailable = 0buffers rx = 10858buffer release errors = 0duplicate client registers = 0failed to register client = 0Invalid client syncs = 0Step 4
Issue the show redundancy history command to display internal redundancy software history.
Switch# show redundancy history4w5d client added: RF_INTERNAL_MSG(0) seq=04w5d client added: RF_LAST_CLIENT(65000) seq=6500000:00:00 client added: History RF Client(35) seq=36000:00:01 client added: CPU Redundancy(17) seq=23000:00:02 client added: Interface Sync(19) seq=30000:00:02 client added: MetOpt Password Sync(36) seq=37000:00:02 *my state = INITIALIZATION(2) *peer state = DISABLED(1)00:00:02 RF_PROG_INITIALIZATION(100) RF_INTERNAL_MSG(0) op=0 rc=1100:00:02 RF_PROG_INITIALIZATION(100) CPU Redundancy(17) op=0 rc=1100:00:02 RF_PROG_INITIALIZATION(100) Interface Sync(19) op=0 rc=1100:00:02 RF_PROG_INITIALIZATION(100) History RF Client(35) op=0 rc=1100:00:02 RF_PROG_INITIALIZATION(100) MetOpt Password Sync(36) op=0 rc=1100:00:02 RF_PROG_INITIALIZATION(100) RF_LAST_CLIENT(65000) op=0 rc=1100:00:02 *my state = NEGOTIATION(3) peer state = DISABLED(1)00:00:02 RF_STATUS_PEER_PRESENCE(400) op=100:00:02 RF_STATUS_PEER_PRESENCE(400) CPU Redundancy(17) op=100:00:02 RF_STATUS_PEER_PRESENCE(400) Interface Sync(19) op=100:00:02 RF_STATUS_PEER_PRESENCE(400) MetOpt Password Sync(36) op=100:00:03 RF_STATUS_PEER_COMM(401) op=100:00:03 RF_STATUS_PEER_COMM(401) CPU Redundancy(17) op=100:00:03 RF_STATUS_PEER_COMM(401) Interface Sync(19) op=100:00:03 RF_STATUS_PEER_COMM(401) MetOpt Password Sync(36) op=100:15:12 RF_EVENT_PEER_PROG_DONE(506) RF_LAST_CLIENT(65000) op=10500:15:16 *my state = ACTIVE(13) *peer state = STANDBY HOT(8)Information deleted-------------------------------------------------------Step 5
Issue the show redundancy running-config-file command to display running configuration on the standby CPU switch module.
sby-Switch# show redundancy running-config-file!version 12.2no service padservice timestamps debug uptimeservice timestamps log uptimeno service password-encryptionservice internal!hostname top!boot system bootflash:ons15530-i-mz.sar-f-repboot bootldr bootflash:ons15530-i-mz.sar-f-replogging snmp-authfaillogging queue-limit 100logging buffered 10000 debuggingenable password lab!diag onlineno diag power-onip subnet-zeroip ftp source-interface FastEthernet0ip ftp username rhinoip ftp password godzillano ip domain-lookup!!!!!Information deleted-----------------------------------------------------end^@^@Step 6
Issue the show redundancy states command to display internal redundancy software state information.
Switch# show redundancy statesmy state = 13 -ACTIVE
peer state = 8 -STANDBY HOT
Mode = DuplexUnit ID = 5Split Mode = DisabledManual Swact = EnabledCommunications = Upclient count = 10client_notification_TMR = 30000 millisecondskeep_alive TMR = 12000 millisecondskeep_alive count = 0keep_alive threshold = 17RF debug mask = 0x0Refer to the Cisco ONS 15530 Configuration Guide and the Cisco ONS 15530 Command Reference for the following:
•
Configuring CPU switch module redundancy
•
Upgrading the software image on the redundant CPU switch module
•
Downloading the system image on the CPU switch modules
2.10 Troubleshooting CPU Switch Module Problems
This section includes CPU switch module troubleshooting procedures.
2.10.1 Active CPU Switch Module Boot Failure
Symptom The active CPU switch module fails to boot.
Table 2-1 describes the potential causes of the symptom and the solutions.
2.10.2 Standby CPU Switch Module Boot Failure
Symptom The standby CPU switch module fails to boot.
Table 2-2 describes the potential causes of the symptom and the solutions.
2.10.3 Unable to Access CPU Switch Module Console
Symptom The CPU switch module console cannot be accessed.
Table 2-3 describes the potential causes of the symptom and the solutions.
2.10.4 Unable to Access Enable Mode on Active CPU Switch Module
Symptom The system does not allow access to the enable mode.
Table 2-4 describes the potential causes of the symptom and the solutions.
Table 2-4 Unable to Access Enable Mode
Possible Problem SolutionPassword incorrect.
Perform the password recovery procedure. See the "Recovering a Lost Password" section.
2.10.5 Unable to Access Enable Mode on Standby CPU Switch Module
Symptom The system does not allow access to the enable mode on the standby CPU switch module.
Table 2-4 describes the potential causes of the symptom and the solutions.
Table 2-5 Unable to Access Enable Mode
Possible Problem SolutionPassword incorrect.
Perform the password recovery procedure. See the "Recovering a Lost Password" section.
Password synchronization.
Check the image on the active and standby CPU switch modules. Update to the latest image if necessary. If the images are the same, issue the show tech and the show log cammands and provide the outputs to Cisco technical support.
Posted: Mon Apr 30 12:21:39 PDT 2007
All contents are Copyright © 1992--2007 Cisco Systems, Inc. All rights reserved.
Important Notices and Privacy Statement.