Cross-Platform Release Notes for Cisco IOS Release 12.0 S, Part 3: Caveats for 12.0(30)S through 12.0(33)S

Caveats describe unexpected behavior in Cisco IOS software releases. Severity1 caveats are the most serious caveats; severity 2 caveats are less serious. Severity 3 caveats are moderate caveats, and only select severity 3 caveats are included in the caveats document.

Because Cisco IOS Release 12.0S is based on Cisco IOS Release 12.0, many caveats that apply to Cisco IOS Release 12.0 will also apply to Cisco IOS Release 12.0S. For information on severity 1 and severity 2 caveats in Cisco IOS Release 12.0, see the Caveats for Cisco IOS Release 12.0 document located on Cisco.com.

Note If you have an account on Cisco.com, you can also use the Bug Toolkit to find select caveats of any severity. To reach the Bug Toolkit, log in to Cisco.com and click Technical Support: Tools & Resources: Bug Toolkit. (The Bug Toolkit is listed under Troubleshooting.) Another option is to go to http://www.cisco.com/cgi-bin/Support/Bugtool/launch_bugtool.pl. (If the defect that you have requested cannot be displayed, this may be due to one or more of the following reasons: the defect number does not exist, the defect does not have a customer-visible description yet, or the defect has been marked Cisco Confidential.)

The Dictionary of Internetworking Terms and Acronyms contains definitions of acronyms that are not defined in this document:

Cross-Platform Release Notes for Cisco IOS Release 12.0S, Part 3:
Caveats for 12.0(33) through 12.0(30)S

Cross-Platform Release Notes for Cisco IOS Release 12.0S, Part 4:
Caveats for 12.0(29)S1 through 12.0(27)S

Cross-Platform Release Notes for Cisco IOS Release 12.0S, Part 5:
Caveats for 12.0(26)S6 through 12.0(24)S

Cross-Platform Release Notes for Cisco IOS Release 12.0S, Part 6:
Caveats for 12.0(23)S6 through 12.0(6)S

Resolved Caveats—Cisco IOS Release 12.0(33)S

All the caveats listed in this section are resolved in Cisco IOS Release 12.0(33)S. This section describes only severity 1, severity 2, and select severity 3 caveats.

ISO CLNS

Symptoms: A MPLS tunnel may not come up after a stateful switchover (SSO) has occurred.

Conditions: This symptom is observed on a Cisco router when Cisco IS-IS NSF is enabled and when IS-IS is used as the IGP for MPLS TE tunnels.

Second Alternate Workaround: Enter the shutdown interface configuration command followed by the no shutdown interface configuration command on the interface that is used for the MPLS TE tunnels after the SSO has occurred.

Miscellaneous

Conditions: This symptom is observed on a Cisco router when you first disable and then re-enable IPv6 on an interface.

Workaround: Enter the shutdown interface configuration command followed by the no shutdown interface configuration command on the affected interface.

Symptoms: A Cisco 12000 series may generate the following error message and reload unexpectedly because of a bus error:

%MEM_ECC-2-MBE: Multiple bit error detected at XXXXXXXX: %MEM_ECC-3-SYNDROME_MBE: 8-bit Syndrome for the detected Multi-bit error: 0x99

Conditions: This symptom is observed on a Cisco 12000 series that is configured for CEF and MPLS.

Conditions: The symptom is observed when you modify an existing access control list (ACL) that is attached to an interface.

Workaround: Do not modify an ACL that is attached to an interface. If you cannot remove the ACL from the interface, create a new ACL and apply it to the interface.

Conditions: This symptom is observed on a Cisco router that has an L2TPv3 tunnel.

Symptoms: A Cisco 7200 series that is configured for QoS may crash when traffic is sent.

Conditions: This symptom is observed on a Cisco 7200 series that has an NPE-G1 or NPE-G2 and that has a Port Adapter Jacket Card in which a 2-port OC-3/STM-1 POS port adapter (PA-POS-2OC3) in installed that has an interface with a service policy.

Conditions: Occurs while executing the command no control-plane slot <slot no> command.

Symptoms: Traffic forwarding stops upon mic-reloading the egress card when E4+ is ingress.

Workaround: Mic-reload the E4+ line card to recover. You can also enter the clear cef linecard <ingress card slot#> command to recover traffic.

Conditions: Occurs when the high-availability feature is configured on the RPR of a Cisco 7500 router.

Conditions: Occurs when the router receives a "pim join" message. The router allocates blocks of memory that are never released. Occurs when there are more than three IPv6 PIM hosts on the same network segment.

Open Caveats—Cisco IOS Release 12.0(33)S

This section describes possibly unexpected behavior by Cisco IOS Release 12.0(33)S. All the caveats listed in this section are open in Cisco IOS Release 12.0(33)S. This section describes only severity 1, severity 2, and select severity 3 caveats.

IP Routing Protocols

Symptoms: The remove-private-as command does not work as expected when applied to a neighbor for which a route-map with a "set as-path prepend" and "continue" statement are configured.

route-map test permit 10

set as-path prepend 2

continue

route-map test permit 20

set metric 200

Workaround: Do not use the remove-private-as command with the above configuration.

Symptoms: BGP convergence is very slow and CPU utilization at BGP Router process is always near 100% during the convergence at aggregation router.

Conditions: Occurs if the number of component prefixes belonging to the aggregate-address entry increases. Also occurs if the number of duplicate aggregation component prefixes for aggregate-address entry increases

Symptoms: When you enter the neighbor ip address send-label explicit-null command, the RP may crash.

Conditions: This symptom is observed on a Cisco router that is configured for BGP.

Symptoms: After the command route-map test-comm-in permit 20 is issued, one of the peers should be denied and should not be advertised to any other peers.But this behavior is not seen with this image.

Symptoms: On a Cisco router running Cisco IOS release 12.0(32)S4 and configured with BGP and peer-groups, if the Fast Peering Session Deactivation feature is configured in the peer-group, the router automatically configures on the command a route-map with the same name as the peer-group.

RR#conf t

Enter configuration commands, one per line. End with CNTL/Z.

RR(config)#router bgp 65001

RR(config-router)#neighbor rrs-client fall-over ?

bfd Use BFD to detect failure

route-map Route map for peer route

<cr>

RR(config-router)#neighbor rrs-client fall-over

RR#sh ru

<snip>

router bgp 65001

neighbor rrs-client peer-group

neighbor rrs-client remote-as 20959

neighbor rrs-client update-source Loopback0

neighbor rrs-client fall-over route-map rrs-client <<<<<<<

the route-map does not exist.

Symptoms: A carrier supporting carrier (CsC) Multiprotocol Border Gateway Protocol (MPBGP) connection between two PE routers may remain in the active state but never becomes established.

Symptoms: A PE that is part of a confederation and receives a VPNv4 prefix from an internal and an external confederation peer assigns a local label to the prefix despite the fact that the prefix is not local to this PE and that the PE is not changing the BGP next-hop.

Conditions: Occurs when receiving this prefix via two paths from confederation peers.

Further Problem Description: Whether or not the PE will chose to allocate a local label depends on the order at which the multiple paths for this vpnv4 prefix are learned. The immediate impact is that the local label that is allocated takes up memory in the router as the router will populate the LFIB with the labels.

Symptoms: When converting BGP from NLRI to AF form, route-maps are not applied to the multicast neighbor.

Conditions: Occurs when a neighbor is supporting both unicast and multicast. If a route-map is applied to the neighbor, it will only be applied to the IPv4 unicast address-family after converting with the bgp upgrade-cli command

Workaround: If the route-map is needed for the unicast and multicast address-family, then the command match nlri unicast multicast should be added to the route-map, even though this is the default behavior.

Symptoms: Prefixes are allowed by the outbound route-map even though the match condition is met and the action is set to deny

Workaround: If there is single match clause based on NLRI, the condition is avoided. The issue is not observed.

Symptoms: The output of the show stacks command may show a very large number of blank lines (for example, 280,000) instead of a process name before the next line of command output is shown.

Conditions: This symptom is observed on a Cisco 12000 series. The symptom may also occur when you enter a command that executes the show stacks command such as the show tech command or the show tech cef command.

ISO CLNS

Symptoms: A Cisco 7600 router may encounter a condition such that when IS-IS and Traffic Engineering are configured, IS-IS should remove the native path from its local RIB and call RIB code to remove the path from global RIB but fails by either not passing the "delete" msg to RIB properly or RIB doesn't react when it received the "delete" call.

Conditions: Output from the show mpls traffic-engineering tunnel command may indicate the "Removal Trigger: setup timed out" status.

Workaround: Shut/no shut the interface or change the metric temporarily to force an update by using the tunnel mpls traffic-eng autoroute metric 1 command.

Miscellaneous

Symptoms: Incorrect police percent conversions occur in the second and third levels of a policy.

Conditions: This symptom is observed on a Cisco 7200 series that runs Cisco IOS Release 12.2SB. However, the symptom is platform-independent.

Symptoms: When configuring frame relay queueing, bandwidth is taken as 28kbps and more than 28 kbps cannot be configured.

Conditions: This happens only when service policy is applied under map-class frame-relay and then binding it under the DLCI with frame-relay traffic shaping enabled under the interface.

Conditions: Occur when dLFIoLL+QoS+SSO is configured on a Cisco 7500 router. Input service policy is configured on MCT1E1 interface.

Conditions: Occurs when 200 network commands are configured using RIP version 2.

Symptoms: During setup of vc-class provision for l2tpv3 ATM tunnels, the initial end-to-end is just fine, but after removing vc-class on both PE at the same time by script, one of ATM sub-interface on CE router went down, and ping was not passing through anymore.

Conditions: Occurs when the vc-class on PE1 and PE2 are removed at almost at the same time.

Workaround: Remove vc-class on one PE router first, and then remove vc-class on the other PE router.

Symptoms: IPv6 EBGP sessions fail with the following message in "debug bgp events":

%BGP-4-INCORRECT_TTL: Discarded message with TTL 32 from <ip>

Symptoms: Cisco 7206VXR with PA-MC-8TE1+ experiences interface flaps when there is a service policy configured on the interface.

75Q2-R3(config)#policy-map multiple_action_1

75Q2-R3(config-pmap)# class 35

75Q2-R3(config-pmap-c)# police cir 8000 bc 1000 pir 10000 be 12000

75Q2-R3(config-pmap-c-police)#conform-action set-mpls-exp-imposition-transmit 5

75Q2-R3#sh policy-map

Policy Map multiple_action_1

Class 35

police cir 8000 bc 1000 pir 10000 be 12000

conform-action set-mpls-exp-transmit 5 <<<should be set-mpls-exp-imposition-transmit 5

exceed-action drop

violate-action drop

Symptoms: Using the show ip mds stats linecard command shows MDFS reloads on all LCs when multicast distributed routing is added on a VRF through the configuration of ip multicast-routing vrf vpn distributed.

Note that while the MDFS reload is a real reload, it is without a preceding clear so it will not generally cause traffic interruption as it merely causes the same information to be downloaded to the linecards again. However in a highly scaled system running close to the limit, the additional load introduced by a full MDFS reload of every linecard may cause additional failures because of CPU utilization.

Conditions: This issue is seen on a Cisco 7500 router configured for dLFIoLL. The policy is attached to ATM and multilink interfaces.

Conditions: Occurs in traffic between a Cisco 12000 router and a Cisco 7500 router. The Cisco 7500 does not receive MDT update.

Symptoms: Some FRR database entries become active after reoptimization. Traffic on the LSP which become FRR active is forwarded wrong path and continues to drop.

- This problem may happen when manual or timer reoptimization is performed during convergence

- This problem may happen when "Tunnel head end item" and "LSP midpoint item" in FRR database have more than one entry in each item.

- This problem may happen when midpoint entry in "LSP midpoint item" is the LSP using "loose" path-option on a headend router.

Symptoms: The Unicast and Multicast VPN traffic packets are dropped on a MLFR bundle link while increasing the traffic rate and bringing back to normal.

Conditions: The drops are seen only after an increase in the traffic rate and bringing back to normal value.

Conditions: dLFIoLL is configured on a Cisco 7500 router and MDR reload is done on the VIP

Wide-Area Networking

Symptom 1: Multicast packets that traverse a Frame Relay virtual circuit (VC) bundle are dropped.

Condition 1: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.0S.

Symptom 2: Multicast packets that traverse a Frame Relay virtual circuit (VC) bundle are process-switched.

Resolved Caveats—Cisco IOS Release 12.0(32)S9

Cisco IOS Release 12.0(32)S9 is a rebuild of Cisco IOS Release 12.0(32)S. The caveats listed in this section are resolved in Cisco IOS Release 12.0(32)S9 but may be open in previous Cisco IOS releases. This section describes only severity 1, severity 2, and select severity 3 caveats.

Basic System Services

Symptoms: Cisco 7500 router with a RSP8 running Cisco IOS Release 12.0(27)S5b may experience traceback errors when the command bgp regexp deterministic is issued.

%SYS-4-REGEXP: new engine: regexp compilation had failed.

-Process= "BGP Router", ipl= 0, pid= 172

-Traceback= 40E601F4 40E5E8C0 40E5F950 40E5E128 4060BD4C 407CCF14 407CD440 407CE4A0 
407A944C 407AB594 407ABE90 4079BA34

Conditions: Traceback appears after issuing the command bgp regexp deterministic.

%SYS-5-CONFIG_I: Configured from scp://userid:password@10.1.1.1/config.txt by console

Conditions: When using SNMP to modify a configuration by means of the CISCO-CONFIG-COPY-MIB, selection of ConfigCopyProtocol of SCP or FTP may result in the password being exposed in a syslog message.

Workaround: When using SNMP to modify a configuration by means of the CISCO-CONFIG-COPY-MIB, use the ConfigCopyProtocol of RCP to avoid exposure of the password.

Symptoms: After you have performed a microcode reload on a router, a ping may not go through for 100 percent.

Conditions: This symptom is observed on a Cisco router that has an RSP after you have entered the microcode reload command.

May 9 08:12:02.449: %CONTROLLER-5-UPDOWN: Controller E1 3/1/7, changed state to down

May 9 08:12:03.449: %CONTROLLER-5-UPDOWN: Controller E1 3/1/7, changed state to up

May 9 08:48:16.354: %CONTROLLER-5-UPDOWN: Controller E1 3/1/7, changed state to down

May 9 08:48:16.354: %RSP-3-ACCERROR: Serial3/1/7:1 acc E80002BA had bad value 39

-Traceback= 40348B24 404E4DEC 4050B378 405086B4 405087AC 4050CAAC 4050E754

Symptoms: snmpEngineBoots variable, used in SNMPv3 packets, not incremented correctly during RP switchover when using RPR+ on a Cisco 12000 Series router. This may cause issues for third party mediation devices that attempt to utilize this variable to decide if the SNMPv3 configuration needs to be reinstated after a reboot or RP switchover of the device.

Conditions: Add SNMPv3 configuration, then do an RP switchover two times. At least one of the times, the snmpEngineBoots variable will not have incremented. Variable can be decoded via packet capture of SNMPv3 packets from router, or with snmpwalk:

snmpwalk -v 3 -Os -u USERNAME -l authNoPriv -A PASSWORD Z.Z.Z.Z snmpEngineBoots

Interface counters on E5 line cards may show incorrect packet input stats in the output of show interface.

Conditions: Problem is seen when the line card CPU spikes to over 90% due to the process "CEF LC IPC Backg"

Workaround: Issuing the clear counters command will temporarily correct the counters.

Symptoms: Customer experienced excessive environmental messages on the logs when environmental temperature is still normal.

Jun 22 19:44:18.229: %CI-6-ENVNORMAL: RSP(6) Inlet measured at 19C/66F

Jun 22 19:44:21.765: %CI-6-ENVNORMAL: RSP(6) Hotpoint measured at

25C/77F

Jun 22 19:44:25.301: %CI-6-ENVNORMAL: RSP(6) Exhaust measured at

25C/77F

Jun 22 19:44:28.836: %CI-6-ENVNORMAL: RSP(7) Inlet measured at 19C/66F

Jun 22 19:44:32.372: %CI-6-ENVNORMAL: RSP(7) Hotpoint measured at

24C/75F

Jun 22 19:44:35.908: %CI-6-ENVNORMAL: RSP(7) Exhaust measured at

24C/75F

Jun 22 19:44:39.556: %CI-6-ENVNORMAL: +12 Voltage measured at 12.26

Jun 22 19:44:43.092: %CI-6-ENVNORMAL: +5 Voltage measured at 5.11

Jun 22 19:44:46.627: %CI-6-ENVNORMAL: -12 Voltage measured at -12.03

Jun 22 19:44:50.163: %CI-6-ENVNORMAL: +24 Voltage measured at 21.60

Jun 22 19:44:53.699: %CI-6-ENVNORMAL: 2.5 Reference measured at 2.47

Conditions: These are called ENVMIB clearance messages and indicate that a problem condition has returned to normal condition. Unless there is a problem condition these should not be seen and if once seen they should not repeat.

EXEC and Configuration Parser

00:45:48: %IPCGRP-3-ERROR: set cfg-exited: timeout 00:46:11: %IPCGRP-3-ERROR: standby 
set time: timeout

Conditions: If we give the any "do commands" under config mode/(interface) and If we wait until (exec-timeout 1) timer times out, we can observe these errors in active and also output of "do command" is displayed in standby.

Interfaces and Bridging

Symptoms: High number of input errors and ignores when locally switching traffic on a VIP6-80. Traffic coming in on a PA-2FE and out a PA-A6-OC3-SMI on the same VIP we will see input errors on the PA-2FE.

Conditions: Occurs on a Cisco 7500 series router with an RSP16 with VIP6-80s running 12.4 PA-2FE and PA-A6-OC3-SMI.

IP Routing Protocols

Symptoms: The set ip next-hop in-vrf vrf-name command does not work in conjunction with import maps.

Conditions: This symptom is observed on a Cisco router that is configured for BGP.

Symptoms: A OSPF router in a NSSA that summarizes an address will keep the summary learned via a neighbor router after using the clear ip route * command.

Conditions: Both neighbors are summarizing the same network in a NSSA area and are advertising it.

Symptoms: A router that is configured for OSPF Sham-Link and BGP redistribution may crash.

Conditions: This symptom is observed only in network topologies with OSPF routes that traverse two or more sham links. For example, the symptom may occur in a hub-and-spoke topology with sham links between the hub and two or more individual spokes. This symptom was observed on a Cisco 10000 series but may also occur on other platforms.

Symptoms: A router may reload when Border Gateway Protocol (BGP) neighbor statements are removed from the configuration.

Conditions: This symptom is observed in rare circumstances on a Cisco router when BGP neighbors are removed very quickly by a script at a much faster rate than manually possible and when a large BGP table is already present on the router before the script adds and removes the BGP neighbors.

Further Problem Description: If you manually remove the BGP neighbors, it is less likely that the symptom occurs.

Symptoms: In an MPLS VPN where OSPF is being used between PE and CE, if the BGP VPNv4 routes received over the VPN backbone are advertised by a Juniper device, OSPF Route-Type and OSPF Router-ID extended community may not be interpreted correctly. This may result in the PE redistributing the incorrect route-type via OSPF.

Conditions: - OSPF being used between PE-CE - BGP VPNv4 prefixes received from a Juniper device

Workaround: If it is possible to configure OSPF Route Type Extended Community/OSPF Router ID Extended Community type to be advertised by the Juniper device, then using the 0x8000/0x8001 would avoid this problem.

RFC4577 defines two encodings for Route Type and Router ID BGP communities that are used in VPN context:

OSPF Route Type Extended Community - 0x0306 or 0x8000 OSPF Router ID Extended Community - 0x0107 or 0x8001

Cisco uses the original encodings (0x8000 and 0x8001) when advertising routes over the VPN backbone and does not currently recognize the new types (0x0306 and 0x0107).

Conditions: This symptom is observed on a Cisco router that is configured for BGP and that has VPNs configured.

Symptoms: A Route Reflector (RR) does not withdraw a prefix that redistributes itself even if this prefix is removed from the BGP table.

Conditions: This symptom is observed on a Cisco router that functions as an RR that advertises two of the same prefixes with different Route Distinguishers (RDs) when one of these prefixes redistributes itself and when the other prefix is a route that is learned from an RR client via iBGP.

Symptoms: After addition/deletion/modification of a VRF and the re-addition of associated configuration, it becomes apparent that the RIB is not being updated by BGP after reconvergence, and LDP neighborship is reestablished. As the RIB is not updated, neither is CEF. While BGP VPNv4 has the correct information, the RIB is empty of remote PE VRF subnets, and CEF has a default entry.

Conditions: This symptom is observed on Cisco 12000 series router that is running Cisco IOS Release 12.0(32)S6.

Symptoms: After a reload, the following error message may be displayed if an OSPFv3 router redistributes large numbers of the external routes:

%OSPFv3-3-DBEXIST: DB already exist

Symptoms: Repetitive CPU spikes are seen on Cisco 12000 series router, running 12.0(32)S and configured for BGP multipath with several iBGP and eBGP peers.

Conditions: TblVer is incrementing every 5 minutes, thus causing the BGP Router process to inflict CPU spikes at every 5 minutes.

ISO CLNS

Symptoms: A Cisco router running ISIS routing protocol may experience spurious memory access issue. The following messages can be seen:

*Sep 5 10:40:11.781 JST: %ALIGN-3-SPURIOUS: Spurious memory access made at 0x61081F84 
reading 0x4

*Sep 5 10:40:11.781 JST: %ALIGN-3-TRACE: -Traceback= 61081F84 610833C0 61083D8C 
6108438C 610748D0 610751B4 00000000 00000000

Symptoms: The interface command isis metric xxx on the MPLS tunnel interface is not shown in the configuration, nor does it take effect when a non-default value is configured.

Conditions: When MPLS TE tunnel is configured and ISIS is used as routing protocol.

Symptoms: CEF drops seen when new link comes up, and IS-IS picks the new link as the best path.

Conditions: Only seen when IS-IS the IGP routing protocol used.The problem is that IS-IS is not an IP based routing protocol, so IS-IS does not depend on IP connectivity between the routers, and can install routes with a next-hop for which no ARP entry exists yet. This leads to a short window of loss until the ARP entry is resolved.

Symptoms: IS-IS adjacencies may flap after a stateful switchover (SSO) has occurred.

Conditions: This symptom is observed when there are large number of adjacencies (for example, 16) and when the IS-IS database is large (for example, one LSP containing 5000 routes).

Workaround: Increase the hold time that is advertised in the IS-IS Hello (IIH) packet by entering the router isis nsf advertise holdtime 90 command on the router on which the SSO occurs.

Symptoms: The prefix of a serial interface that is configured for PPP or HDLC and that functions as a passive interface for IS-IS may not be installed in the local IS-IS database.

Conditions: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.2(18)SXF6 but is not release-specific.

Second Alternate Workaround: Enter any command that triggers the generation of the local IS-IS database.

Symptoms: For routers with parallel links connected through switches, if neighbor shuts down ethernet interface, ISIS routes in the routing table use that interface as nexthop will not be deleted even after it receives new LSP from the neighbor reflecting the topology change until adjacency times out, thus create temporary blackhole and delay the convergence.

Symptoms: IPV6 Traffic is hit in one direction after resuming when OIR is performed on the connecting SIP on the router at the other end.

Occurs when an OIR is performed on GigE Line Card of CRS1 connecting to the GigE of GSR the IPV6 traffic going in one direction (from GSR to CRS) is hit again for about 4 seconds after the traffic has resumed after the OIR.

Symptoms: A quick shut/no shut (less than 3 sec) of a traffic eng tunnel interface results in the tunnel being removed from the routing table.

Conditions: Occurs on a Cisco 7600 series router running Cisco IOS Release 12.2(33)SRB1. Routing protocol is ISIS.

Workaround: Wait at least 7 seconds to do the no shut after the initial shut on the tunnel interface. If you can not wait for than 7 sec to shut/no shut the tunnel interface then you can configure: .ip routing protocol purge interface

Conditions: If there is a DR change and ISIS SPF finishes before neighbor sends the router with correct LAN ID reflecting the new DR.

Workaround: Anything that can trigger another SPF run will correct this problem.

Miscellaneous

Symptoms: Whenever an ACE is deleted on the RP from a numbered extended or standard ACL, the corresponding ACL is removed from the LC.

If it is really necessary to use numbered ACLs, modify them offline, delete the whole ACL and re-apply it on the RP.

Symptoms: The Hierarchal Queuing Framework (HQF) is not removed after you have removed a service policy.

Conditions: This symptom is observed on a Cisco 7200 series that has an NPE-G1 and that runs Cisco IOS Release 12.2SB.

Symptoms: While running a Cisco IOS Release 12.0(32)S image, an Engine 3 line card on a Cisco 12000 series router may report some CPUHOG error messages similar to the following:

%SYS-3-CPUHOG: Task ran for 2264 msec (282/155), process = Per-Second Jobs, PC = 400FC51C. -Traceback= 400FC524 400ACC0C 40102FCC 400ACC38 400D7580 400D756C

Symptoms: The standby RP resets continuously because of synchronization failures.

Conditions: This symptom is observed on a Cisco router when you first perform and OIR of a VIP in which a port adapter is installed that supports both T1 and E1 (for example, a PA-MC-8TE1+ port adapter) and then an SSO switchover occurs.

Workaround: There is no workaround. You must power-cycle the standby RP to enable it to come up.

Symptoms: When an InARP request is received on an AAL5SNAP PVC, the router does not respond with an InARP reply.

Conditions: This symptom has been observed when the source address contained in InARP request is not in the subnet of the sub-interface on which PVC is configured.

Conditions: This problem is seen when there is P2P on one side and multipoint on the other. The map is deleted on the multipoint side, and the inARP configuration is changed on P2P side.

Symptoms: Configuring an SDCC interfaces on a 2xOC48 POS SPA on the Cisco 12000 Series platform causes the router to display the following error message.

"%EELC_QOS_RES_MGR-3-HW_IDB_INDEX_TO_TX_PORT_MAPPING_FAILED:

Mapping of hwidb_index to tx_port failed. hwidb_index = 5"

The message is displayed every time and SDCC interface is configured for this SPA. There is no other adverse effect other than the message being displayed.

Conditions: This symptom is observed on a Cisco 12000 series router that is running Cisco IOS Release 12.0(32)S.

Conditions: This symptom occurs after edge interface on a PE is micro reloaded, for example 4t3/e3 serial interface with PPPoMPLS AToM circuit.

Symptoms: Cisco Catalyst 4500 Supervisors and Cisco Catalyst 4948 that are running Cisco IOS Release 12.2(31)SG crash when one of the following commands are issued:

Workaround: Do not use any of the above commands. For troubleshooting high CPU issues use the steps indicated in the following tech tip instead:

SLOT 2: Aug 31 21:41:09.908: %SW_MGR-3-CM_ERROR: Connection Manager Error - provision 
segment failed [ADJ:FR:1774000] - no interface available.

-Traceback= 6010C364 6010C8C0 60819570 60819610 60781D4C 608171E0 60829C4C 60829D0C 
60817A9C 60817C2C 6080A4B0 60829C4C 60829D0C 6080B058 60806A30 608098E4

Symptoms: When both ACL and SNF are configured on one 3GE-GBIC-SC, SNF cannot work, and the CPU usage is high.

Conditions: This problem appears in Cisco IOS Release 12.0(28.4)S1 and is not seen in previous releases.

Symptoms: When reloading a Cisco 7500 router (lsnt-ap-pe) with Cisco IOS Release 12.0(32)S5, several IDBINDEX_SYNC-3-IDBINDEX_ENTRY_LOOKUP and tracebacks occur in the standby log.

Conditions: This symptom has been observed on a Cisco 7500 router platform with MVPN configured.

Symptoms: IPv6 traffic drops (in 6PE scenario) on PE (core facing interface). IPv6 version errors are also seen.

Conditions: This symptom is seen in Engine3 line card. This issue is seen in Cisco IOS Release 12.0(33)S and Release 12.0(32)SY2 images.

Symptoms: The following message may be seen on manual RP switchover in SSO mode:

%FM-2-BAD_TLV: Error in internal messaging - bad tlv 0

Conditions: This is Observed on Cisco 12000 series router with SSO S/w and IPv6 ACL(with Traffic).

Symptoms: On the E3 line card "non channelized 4 port OC12 -- 4OC12X/POS" when an IPv6 ACL is applied on an outgoing interface to match on "Routing" extension header, it will instead match on "Fragments" extension header.

Conditions: The same behavior is seen on E3 line card "Channelized OC12 CHOC12" on both incoming and outgoing interface.

Workaround: Match on "Routing" extension header, or match on "Fragments" extension header.

Symptoms: The following may be seen on physical OIR of a SPA-4XT3/E3 from a SIP-501 linecard:

%SPA_PLUGIN-4-LED_WRITE: SPA-4XT3/E3[8/2]: LED write failed for port 1, status 3.

Conditions: This symptom is observed on a Cisco 12000 series router that runs the c12kprp-p-mz image of Cisco IOS Release 12.0(32)SY2

Symptoms: BGP Neighbor drop after receiving a high rate of MCAST traffic without previously defined state.

Conditions: 100K PPS of Mcast arriving at input interface on a GSR E5 card causing neighbor loss.

Conditions: This symptom has been observed when doing RPR+ switchover. The traffic is lost.

Conditions: Occurs when using shut/no shut on a multilink interface through which traffic is flowing. Also occurs when clear interface multilink # command is issued.

Further Problem Description: IPv6 Traffic is dropping in IPv6-IP tunnel with E2,E3,E4, E5 cards

Symptoms: Upon an RPR+ switchover on a PE router followed by a switching of core traffic to an auxiliary interface, some mroutes may not resume traffic.

Conditions: This affects E3 line cards. Line card is core for one vrf and edge for the other

Symptoms: A channelized STM-1 port adapter is not transmitting equal traffic compared to a Multichannel T1/E1 port adapter when links from both port adapters are part of the same multilink interface.

Conditions: The problem is seen under heavy load on the multilink PPP bundle with or without multilink fragmentation enabled.

Workaround: Place all member links from the bundle on the same type of port adapter.

Symptoms: After a microcode reload of SPA-1XCHSTM1/OC3 has been executed, the sequence of interfaces as it appears in show run and show ip interface brief is changed.

Conditions: SPA-1XCHSTM1/OC3 is configured on an E5 Line Card. When the router is reloaded and the SPA-1XCHSTM1/OC3 is configured for the first time, the sequence in which the interfaces appears fine. However, when the microcode reload is executed on the LC, the sequence in which the interfaces are displayed is not as expected. It has no other known side effects. Specifically this does not affect the traffic, nor the SNMP interface indexes.

Symptoms: Using snmpget of an OID using the ifindex value of an interface for its index will result in an error:

snmpget -c <community> -v1 <device>

IF-MIB::ifDescr.92 Error in packet

Reason: (noSuchName) There is no such variable name in this MIB.

Failed object: IF-MIB::ifDescr.92

Symptoms: Connect the TGN to one of the interface of OC48 SPA. Inject the PAIS alarm and check whether it is reported or not using command the show controller pos x/y/z. Then clear the alarm, immediately after this we can see B3-TCA alarm declared and cleared.

Symptoms: When the no bert command is issued to remove a configured bert test before the timer has expired, the bert test may be reset and the slave RSP may also reset

Workaround: Do not issue no bert, instead change the interval in the bert pattern to 1 and let the test time out naturally. Example: bert pattern 2^20-O153 interval 1

Symptoms: All E1 interfaces on a PA-MC-E3 port adapter may flap continuously even after the traffic has been stopped.

Conditions: This symptom is observed on a Cisco Catalyst 6500 series switch and Cisco 7600 series router that have a PA-MC-E3 port adapter when you configure 16 or 128 channel groups on each time slot (that is, time slots 1-31) and then generate traffic just above line rate traffic through all the channel groups. Note that the symptom is not platform-specific.

Symptoms: Configuration of "hw-module slot x tcam compile acl no-merge" is missing after OIR of E3 POS card.

Symptoms: The following error message appears while removing an MFR bundle interface:

%FIB-2-IF_NUMBER_ILLEGAL: Attempt to create CEF interface for MFR1 with illegal 
if_number: 0

Conditions: The error occurs if the MFR bundle interface is removed without removing the member links from the bundle.

Workaround: Remove the MFR bundle, wait approximately 1 minute, then re-add the configuration for the affected MFR bundle.

Symptoms: Using CLI to delete a child policy on a Cisco 7500 series router causes the VIP to crash.

Conditions: The router has a hierarchical QoS policy attached to an interface. Traffic is flowing through the QoS policy. There are BGP updates happening on the router. The no policy- map command is executed to delete the child policy. The router is running Cisco IOS Release 12.0(32)S6.

Symptoms: On enabling/disabling mcast hw-acceleration, the following error occurs:

1d06h: %REDUNDANCY-3-CONFIG_SYNC: Active and Standby lbl configuration out of sync" 
message may be thrown.

Symptoms: After changing redundancy mode via SNMP on GSR redundancy mode description is missing on SNMP O/P.

Conditions: Only on changing redundancy mode, this does not affect the functionality

Conditions: The problem occurs when egress is any SIP-X01 card and there is congestion on the outgoing interface and no default queue-limit is configured explicitly for the traffic class.

Workaround: The problem does not occur when a low limit is configured on the class and WRED is also configured.

Conditions: This command is used to capture corrupt packet data when assembler RP detects it.

Symptoms: Unexpected flows are seen on a GSR configured with aggregate netflow and acting as a provider edge router (PE). The unexpected flows is seen in the output direction of the CE-facing interface (vrf interface).

Conditions: When traffic is generated (FTP, UDP, ICMP), we can observe strange flows of IP precedence 5 (TOS A0). If there is no traffic flowing through, then the flows are not seen. The router runs Cisco IOS Software 12.0(32)S6. Both ingress and egress linecards are ISE Engine 3 linecards. The P-facing interfaces are bundled in a port-channel.

Further Problem Description: The flows can be seen through the following command:

clk15ra#execute-on slot 3 sh ip cac ver flow agg as-tos | i Gi3/1.999 ========= Line 
Card (Slot 3) ========= Gi3/1.999 0 PortCh1 0 00 64 121K 46 147.2 Gi2/0 0 Gi3/1.999* 0 
00 96 238K 1300 144.6 Gi2/0 0 Gi3/1.999* 0 A0 76 3996 442 115.8 >>> Unexpected

Symptoms: Following unknown trigger events, a GSR running Cisco IOS version 120(32)S6 may not properly classify PIM MDT-encapsulated multicast traffic via an MQC queueing policy attached to a POS interface on an E3 POS linecard [4OC3X/POS-IR-LC-B=]. This symptom can be observed via executing the show policy-map interface POS X/Y command while multicast traffic with non-zero IP Precedence or DSCP Value is exiting the interface.

Conditions: The issue is observed on MDT encapsulated GRE multicast tunnel packets on a P router in an MPLS core. The P router's failing Line-Card is 4OC3X/POS-IR-LC-B= and the P router is running Cisco IOS version c12kprp-p-mz.120-32.S3.

Symptoms: Tracebacks and error messages seen on E2 POS LC when changing from the encap from FR-HDLC encap

Conditions: Occurs when unconfiguring the Frame-relay interfaces to the default encap HDLC.

Symptoms: The standby crashes when the 3-port Gigabit ethernet card is in admin down state after a switchover has happened

Conditions: The 3 port Gigabit Ethernet Card has to be in admin down mode and then 2 switchovers need to be done to get into this situation.

Workaround: Use the no hardware-module slot <x> shut command on the LC and admin shut it again.

Symptoms: When the router is in transit mode (SRP shut down), the show controllers command will behave as follows:

–

Active defects are detected properly (SLOS SLOF PLOP) as if the SRP interface were up

–

Active defects will not get cleared even if the issue goes away until the SRP is brought up

Workaround: Use no shut command on the SRP interface without assigning it an IP address.

Symptoms: GRP may crash continuously on reload if a MFR interface in the router has bfd neighbor command configured.

Workaround: Issue send brk from "telnet>" prompt and boot the router with a boot image. Then remove the "bfd neighbor" config from MFR and reload the router with the regular image.

Conditions: BFD sessions remain in down state and do not transition to up state on Cisco ISR routers.

Symptoms: Field diagnostics for a card such as the SIP-401, SIP-501, SIP-601 fails with messages:

"Timeout waiting for Field Diag individual test completion

Field Diagnostic: ****TIMEOUT FAILURE**** slot 4: first test failed: 163,

SPA LPBK damsl, error 0

Field Diag eeprom values: run 44 fail mode 3 (TIMEOUT FAILURE) slot 4

last test failed was 163, error code 0

Shutting down diags in slot 4"

Moreover, depending upon which verbosity and debugging flags are enabled you may see a series of messages such as:

"Field Diag Misc Services requested for a slot (4) not in Field Diag mode, dropping 
request."

Conditions: This problem occurs when field diagnostics is run on the SPA, "SPA-10x1GE" residing in any Jacket card.

Symptoms: In case that MPLS Traffic Engineering Tunnel (TE Tunnel) and Fast Reroute (FRR) are configured on 12000 Series router, FRR backup Tunnel does not restore MPLS LDP/TDP encapsulating packets inputting from Engine 4 Line Card.

Conditions: When Next hop is Head-end LDP/TDP over TE Tunnel and FRR become active, the labeled packets are not protected by the backup TE Tunnel so it causes continuous packet drop until new path is installed.

Symptoms: The tx-ring-limit value configured in PA-2CT3 may reset to its default value.

Symptoms: Holddown timer value is different in "show bfd nbr detail" and "show bfd neighbor". The detailed output of "show bfd neighbor" command displayed holddown timer value for async mode where as the summarized output showed the holddown timer value for echo mode. Change had to be made to have holddown field in both summarized and detailed output of "show bfd neighbor" to display same value.

Conditions: When holddown value in "show bfd summary" and "show bfd neighbor detail" are compared.

Symptoms: This issue is a traceback seen on boot when upgrading or downgrading an image. The traceback seen is:

'%MBUS_C2W-3-SPA_UNKNOWN_C2W_DEVICE: cannot find SPA's C2W device for slot:', and 
running config for all SPA's may be lost from an engine 5 card.

%RP-3-FABPINGTIMEOUT: Fabric ping timeout

Router#attach <slot number>

LC-SlotX>show controllers tofab queues outputq 0 2303 drr

Workaround: Enter valid queue numbers for "start_q" and "end_q" values in the command. Valid Queues are as follows.

Symptoms: A router may crash because of a bus error when you perform an OIR of a PA-MC-8TE1+ port adapter or when you enter the hw-module slot slot-number stop command for the slot in which the PA-MC-8TE1+ port adapter is installed.

Symptoms: When E4P is egress and policing is configured in the policy, matching ip prec did not match labelled packets and matching exp did not match pure IP packets.

Conditions: When E4P is egress and policing is configured in the policy, matching ip prec did not match labelled packets and matching exp did not match pure IP packets.

Symptoms: The following traceback occurs when some policy/class addition/deletion or interface flaps is done:

SLOT 2:Jan 25 19:13:34 UTC: ee192_tx_q_cleanup_guts(): EE_QM_QOS_INTERNAL_ERROR

Conditions: It happens when some policy or class addition/deletion or interface flaps is done.

Symptoms: When changing the APS mode using aps unidirectional or no aps unidirectional the following error message may be seen:

%SONET-4-APSMM: SONET9/0 (grp 1 chn 0: ACTIVE): APS mode mismatch - switching to 
unidirectional mode

Conditions: This problem is seen on a Cisco 12416 Internet Series router containing dual channelize OC-48 line cards configured for automatic protection switching. The router is running Cisco IOS Release 12.0(32)S6q.

Conditions: Occurs when cef linecard is cleared concurrently with the unprovisioning of the multilink bundle.

Workaround: Do not clear cef linecard and unprovision the bundle at the same time.

Symptoms: A spurious access error occurs in tfib_post_table_change_sanity_check () function.

Conditions: This symptom occurs if route is deleted. ROUTE_DOWN event is triggered in tfib_post_table_change() function which in turn calls tfib_post_table_sanity_check(). In that function, spurious access is reported, as the only path of route is down.

Symptoms: "%REDUNDANCY-3-CONFIG_SYNC: Active and Standby lbl configuration out of sync" message may be seen on hw-module shut of spas

Symptoms: Hardware CHEF entry on line card is incorrect. Incorrect slot is observed by show hip hardware-clef command.

Conditions: The problem occurs on Engine4+ Line Card of Cisco12000 router. The prefix has two next hop with load balancing.

SLOT X:Jul 29 09:40:52.844 UTC: %TX192-3-CPUIF_ERR: Underrun Error:

Read Pointer Crosses

Write Pointer.

-Traceback= 400310C4 411E4614 408E1934 4092B4C8 40A2FE1C 40A304F4

40110920

X=slot number

Conditions: Packets which have their IP header length corrupted hit the Tx E4+ card, and it causes this fatal error, causing the line card to crash.

When changing the access lists used in LDP outbound filtering configuration tag-switching advertise-tag for or mpls ldp advertise-label for, it may result in the LSR not advertising label bindings for some prefixes.

If tag-switching advertise-tags oldstyle or mpls ldp advertise-labels oldstyle is configured, and the access-list used as prefix-acl in tag-switching advertise-tags [for <prefix-acl>] [to <tsr-acl>] or mpls ldp advertise-labels [for <prefix-acl>] [to <lsr-acl>] has been changed, it may result the LSR not advertise label bindings to or withdraw label bindings from peers as expected.

The access-list explicitly permits/denies a prefix (including explicit "permit any" or "deny any") before the change, and after the change it explicitly denies/permits a prefix (including explicit "deny any" or "permit any").

a. After performing the changes of access lists which cause LDP withdraw label bindings, wait for a short time to ensure label bindings have been withdrawn before performing the changes of access lists which cause LDP re-advertise label bindings.

b. If problem already happened, recover it by performing the same changes in a pace as described in a., or toggle "mpls ip" on the interface.

a. Remove the access-list completely, with a short time to ensure label bindings have been withdrawn/advertised to all peers as expected, then add back the desired access-list.

Symptoms: In case that multiple MPLS Traffic Engineering(MPLS-TE) Tunnels and Fast Reroute(FRR) are configured on Cisco 12000 series router, in event of FRR invoking, unaffected MPLS-TE tunnel is also protected by FRR unexpectedly.

Conditions: The MPLS-TE Tunnels are Equal Cost Multipath for BGP Nexthop. Once one of tunnels is restored by FRR, the other tunnel which is not affected may be protected also and the backup Tunnel continue "active" status in FRR database. After FRR invoking and reoptimized to a secondary path, BGP routes are balanced into both one primary tunnel and one backup tunnel which should not be activated.

Symptoms: In a Cisco 7500 HA router in RPR+ Mode when configuring and unconfiguring channel groups under an E1 controller, the router reports the following:

*Aug 22 17:58:34.970: %HA-2-IPC_ERROR: Failed to open peer port. timeout *Aug 22 
17:58:34.974: %HA-3-SYNC_ERROR: CCB sync failed for slot: 1 *Aug 22 17:58:34.974: 
%HA-5-SYNC_RETRY: Reloading standby and retrying sync operation (retry 1).

Conditions: This symptom is observed when configuring and unconfiguring channel groups under an E1 controller.

Symptoms: A GSR with Eng5 line cards may not pass traffic when acting as an ASBR in an inter-as vpn option B setup. The problem happens after the removal of a VRF from the ASBR. The mpls labels advertised on the eBGP peering for the vpnv4 prefixes are not programmed in the line cards so traffic is dropped. The label for a prefix can be seen on the RP, but not on the LCs.

Conditions: When there are lots of prefixes in BGP. Seen with PRP2 with Eng5 line cards.

Workaround: Flapping the affected prefix triggers an update which gets the labels on the LCs.

%EE48-3-ALPHA_MCAST: Can't assign new hw_mdb

Conditions: Observed on E3-choc12 Line card with multicast-egress-qos enabled. At least 8 OIF for a VRF mroute on the same LC.

Workaround: Spread the VRF interface distribution so that none of the LC contain more then 7 interfaces.

Further Problem Description: Best known way to recover is through reload of the LC.

Symptoms: In an FRR setup, VPNv4 packets are black-holed around 4-5 seconds while the traffic switched over to the backup TE-Tunnel.

Conditions: Problem is seen with Cisco IOS Release 12.0(32)SY4 for VPNv4 traffic ingress to GSR Engine-2 line card.

Engine-2 linecard incorrectly sets the tunnel label for tunnel-end when the traffic is forwarded to the back-up tunnel. MPLS packets arrives at the tunnel-end with 0 TTL hence they are discarded.

Symptoms: RSP is not reused the IDB. When the maximum 2047 IDB is reached after delete and add a channel-group several times, the RSP did not assign the old IDB because is on non-share reuse mode and log the follow messages.

Cisco7507_test#sh idb ifnum

Interface Deleted IfIndex Feature

Se0/0/0:1 36 non-shared reuse,

Se0/0/0:1 21

Se0/0/1:3 37 non-shared reuse,

Se0/0/1:3 22

Se0/0/1:5 38 non-shared reuse,

Se0/0/1:5 23

Se0/0/1:6 39 non-shared reuse,

Se0/0/1:6 24

Se0/0/6:21 227 non-shared reuse,

Se0/0/6:21 69

Se0/0/6:21.16 70

Se0/0/6:28 600 non-shared reuse,

Se0/0/6:28 67

Se0/0/6:28.16 68

RSP logged the follow messages:

===============================

Cisco7507_test(config-controller)#controller E1 6/0/6

Cisco7507_test(config-controller)#no ch 3 timeslots 4

Cisco7507_test(config-controller)# ch 3 timeslots 4

*Sep 13 14:03:15: %FIB-2-HW_IF_INDEX_ILLEGAL: Attempt to create CEF

interface for Serial6/0/6:3 with illegal index: 2049

-Traceback= 403564FC 40278C60 4027B438 402A1E3C 404ECA1C 404DEAC4

*Sep 13 14:03:16: %IPC-5-SLAVELOG: VIP-SLOT6:

Sep 13 14:03:15: %MDS-2-LC_INVALID_INPUT_IF_INDEX: Invalid input

if_index (0x801) for HWIDB allocation exceeds platform_maxinterfaces

Conditions: This symptom is observed on a Cisco 7500 series that runs Cisco IOS Release 120(32)S. This Issue happens only when reconfigure channel-groups.

Symptoms: When an add/drop multiplexer is in bidirectional mode, changing the APS mode using the aps unidirectional or no aps unidirectional commands, there is no APS mode mismatch message seen.

Conditions: This problem is seen on a Cisco 12416 Internet Series router containing dual Channelize OC48 line cards configured for Automatic Protection Switching. The router is booted with Cisco IO S Release 12.0(32)S6q.

Symptoms: On router using HA RPR+, after an encapsulation change is done on serial interfaces of channelized port adapters, a reload of slave RSP is called.

Conditions: Encapsulation is changed on channelized interfaces and issue happens when you exit configuration mode.

Symptoms: If spa_dmlp_add_bundle(&bndl); in bflc_spa_dmlp_add_bundle_guts() routine failed for first link. The flag bundle->plim_provisioned was set to true, making bundle unusable.

Symptoms: When an engine 3 OC48 card is reloaded, PIM neighbors over the default MDT tunnel are lost for VRFs that have interfaces configured on a linecard other than the one which was reloaded.

Conditions: VRFs configured that have interfaces defined on more than one LC might see the default MDT tunnel flap when an LC is reloaded.

Conditions: The issue happens with Frame-relay interfaces configured on a SPA using any of the following SIP on a Cisco 12000 series router:

Symptoms: Tracebacks and spurious memory access occur when MR APS is unconfigured.

Conditions: The direction has to be changed from bidirection to unidirection before unconfiguration.

Symptoms: RP does not boot up if it is power cycled immediately on upgrading the ROMmon.

Workaround: Avoid power cycle of the PRP immediately after the upgrade. Boot IOS after the ROMmon upgrade.

Symptoms: There are no netflow data for the traffic received on the port-channel 1. There is netflow data for the traffic received on port-channel 2, but the data match with the traffic from first interface of the bundle. Netflow data are still correct for the traffic sent to a port-channel interface (netflow destination interface).

Conditions: Sampled netflow configured on GSR 12416 router, 2 port-channel configured Upgrade of the IOS from version 12.0(32).S5 to version 12.0(32).S8.

Symptoms: PIM neighbor will not form over tunnel interface when a VRF is re-added. If the neighbor does form, even then MCAST traffic will continue to drop on the E3 LC.

Conditions: Occurs after a VRF is removed and later on re-added. This was observed in Cisco IOS Release 12.0(32)S.

Workaround: Remove and re-add the hw-module slot X ip multicast hw-accelerate source-table size command. This is a traffic-impacting operation.

Symptoms: After a long haul fiber cut is restored on DWDM equipment, the interface on E4+ is stuck.

Test GSR (E4+) ---(local fiber)--- third-party DWDM ---- long hual fiber -------- third-party DWDM ---(local fiber)---- Customer GSR

LOF is seen on E4+ which is not getting cleared upon restoration of long haul fiber cut. This is causing the interface not to come up.

Workaround: Framer is not sending an interrupt to clear the LOF in software. When LOS interrupt comes to clear LOS, check the status of LOF also and clear it if it is not present.

Conditions: The issue happens with multilink frame-relay interfaces configured on a SPA using any of the following SIP on a Cisco 12000 series router:

%GSRSPA-3-UNEXPECTED_NULL_BAY error comes up when SPA is removed from linecard.

Router# SLOT 1:00:07:09: %GSRSPA-3-UNEXPECTED_NULL_BAY: hostType 0x75, slot 1, subSlot 
0, bayCount 2 -Traceback= 4003110C 406D1668 406DBBC0 40A3AA80 40110700 SLOT 
1:00:07:09: TFIA is halted. Waiting for RP to initiate recovery

Wide-Area Networking

Symptoms: When many MLP sessions come up at once, the router may leak packet memory. In some cases, this may cause the router to reload.

Conditions: This symptom has been observed on Cisco 7600 and 12000 series routers. It may also occur on other models.

Symptoms: Changing the encapsulation on a member of a multilink bundle while the bundle is up may cause the router to reload.

Conditions: This symptom has been observed when changing an interface that is an active member of a multilink bundle from PPP to frame relay encapsulation.

Symptoms: VIP2-50 is crashing on a Cisco 7500 series router running Cisco IOS Release 12.0(31)S02y due to a DBUS error

Conditions: This happens on a Cisco 7200 series router when removing CBWFQ with encapsulation frame-relay.

Resolved Caveats—Cisco IOS Release 12.0(32)S8

Cisco IOS Release 12.0(32)S8 is a rebuild of Cisco IOS Release 12.0(32)S. The caveats listed in this section are resolved in Cisco IOS Release 12.0(32)S8 but may be open in previous Cisco IOS releases. This section describes only severity 1, severity 2, and select severity 3 caveats.

Basic System Services

Symptoms: A Cisco router may reload if SNMP GetNextObjectInstance request are processed at clogHistoryEntry_get.

Conditions: This symptom is observed on a Cisco 7206VXR series router with NPE- 300 processor board running IOS 12.2(13)T5.

Workaround: The work around is not to query the CISCO-SYSLOG-MIB. You may create a SNMP view to exclude this MIB and attach this view to all communities configured on the device. This will prevent any managers from accessing the CISCO-SYSLOG-MIB.

Conditions: The crash is seen when the length of any token in the URL of the request is excessively long.

Symptoms: In the vrf-lite configuration , the superviser crashes when the vrfs are removed from the configuration. The crash persists even on interchanging the Wanda card and SIP-2.

OSPF was the common protocol and the crash has not been reported when bgp is configured on the router.

Workaround: This is a repeatable bug and no workarounds are known at this point.

Symptoms: The memd recarved on RSP never occur because of the VIP stuck trying to change the MTU under Serial interface.

=================

interface Serial5/0/1:1 mtu 2000 < ===== to be changed

ip address 10.10.10.1 255.255.255.0

no ip directed-broadcast no keepalive

===============

interface Serial5/0/1:1

Router(config-if)#mtu 900

RSP logged the follow messages:

=========================== 12:56:45:

%CBUS-3-CMD: Cmd failed: tx limit, response 0x8010, Serial5/0/1:1

-Traceback= 40349B68 404CFE38 404D5180 403C30B4 4037CF64 40390CF4 403F22C8 403F22B4

Router(config-if)#

Router(config-if)# 12:57:21:

%HA-2-IPC_ERROR: Failed to open peer port. retry queue flush 12:57:21:

%HA-3-SYNC_ERROR: CCB sync failed for slot: 1 12:57:21:

%HA-5-SYNC_RETRY: Reloading standby and retrying sync operation (retry 1). 12:57:36:

%RSP-3-SLAVECHANGE: Slave changed state from Slave to Non-participant

Conditions: This symptom is observed on a Cisco 7500 series that runs Cisco IOS Release 120(32)S. This Issue happens only on serial interfaces when you change the MTU.

Symptoms: A Cisco port adaptor CT3IP-50 running IOS version 12.0(32)S6 may reload unexpectedly. This has been experienced many times. The information gathered points to a software issue. This enclosure will be updated as more information is gathered.

%SYS-3-CPUHOG: Task ran for 123588 msec (2838/0), process = VIP Txacc loss compensation, PC = 60308350. -Traceback= 60308358 : %SYS-2-WATCHDOG: Process aborted on watchdog timeout, process = VIP Txacc loss compensation. -Traceback= 60030DC4 6011774C 6011C244 6010EDF4 603081D0 6030851C

20:32:43 UTC Mon Apr 16 2007: Breakpoint exception, CPU signal 23, PC = 0x6010CF38

IBM Connectivity

A vulnerability exists in the Data-link Switching (DLSw) feature in Cisco IOS where an invalid value in a DLSw message could result in a reload of the DLSw device. Successful exploitation of this vulnerability requires that an attacker be able to establish a DLSw connection to the device.

IP Routing Protocols

Symptoms: The issue is observed at big service provider whereas atm-sub interfaces are deleted and new one created on regular basis as they remove and add new end customers.

Because it is not a manual process as scripting is used to perform that task, old configuration from deleted sub-interface are showing up on new sub-interfaces and in some cases are creating outages.

Workaround: verify sub-interface configuration and if configuration cannot be deleted on that sub-interface, delete this sub-interface then create a dummy sub-interface which will pull that configuration. Then recreate prior sub-interface.

The Cisco Next Hop Resolution Protocol (NHRP) feature in Cisco IOS contains a vulnerability that can result in a restart of the device or possible remote code execution.

NHRP is a primary component of the Dynamic Multipoint Virtual Private Network (DMVPN) feature.

NHRP can operate in three ways: at the link layer (Layer 2), over Generic Routing Encapsulation (GRE) and multipoint GRE (mGRE) tunnels and directly on IP (IP protocol number 54). This vulnerability affects all three methods of operation.

This vulnerability is addressed by Cisco bug IDs CSCin95836 for non-12.2 mainline releases and CSCsi23231 for Cisco IOS Release 12.2 mainline releases.

Symptoms: A router may reload unexpectedly when you enter the transmit-interface interface configuration command on an interface that has a point-to-point OSPF adjacency.

Conditions: This symptom is observed on a Cisco router when the OSPF network type is configured as point-to-point, either because the interface is, for example, a serial interface, or because the ip ospf network point-to-point interface configuration command is enabled on the interface.

Workaround: When there is an OSPF adjacency on the interface that is being configured, first enter the shutdown interface configuration command before you enter the transmit-interface interface configuration command.

Symptoms: It has been observed that Cisco IOS Command Line Interface (CLI) allows user to enter multiple entries of no ip split-horizon eigrp AS# when the interface has not yet been assigned to a VRF interface.

Then when the interface is assigned to a VRF, these "no ip split-horizon eigrp AS#" entries stay under the interface and are not checked against the EIGRP AS number to which this interface belong.

This behavior has been observed with multiple Cisco IOS version included Cisco IOS Release 12.0(32)S5:

interface ATM6/0.64 point-to-point ip vrf forwarding NETWORK

ip address 10.150.10.113 255.255.255.252

no ip directed-broadcast

ip mtu 1500

no ip split-horizon eigrp 100

no ip split-horizon eigrp 1

no ip split-horizon eigrp 2

no ip split-horizon eigrp 4

no atm enable-ilmi-trap pvc 1/64

Workaround: before assigning an interface to a VRF, make sure no protocol configuration exist under the interface by using either the "no" statement command or the command itself to get the configuration into a default state.

Symptoms: CBB route reflector advertising the route with different next hop, and IPFR route reflector lsanca92c14-0391 still keeps the old route in the routing table. Soft clear needed to perform on lsanca92c14-0391 to refresh the route.

Conditions: The function bgp_suppress_ebgp_update() does not check if the neighbor has the flag BN_AF_NHOP_UNCHANGED. As a result, we supress the EBGP update incorrectly.

Symptoms: When an interface comes up, Cisco IOS triggers a PIM Hello, this hello does not always make it to the other side. The reason seems to be here that hello is sent before the port can actually forward IP packets. IGP manages to get its neighborship up but PIM not, then RPF will change to the new neighbor and in result blackholing happens up to 30 seconds.

issue observed at big service provider whereas atm-sub interfaces are deleted and new one created on regular basis as they remove and add new end customers.

Because it is not a manual process as scripting is used to perform that task, old configuration from deleted sub-interface are showing up on new sub-interfaces and in some cases are creating outages.

workaround: verify sub-interface configuration and if configuration cannot be deleted on that sub-interface, delete this sub-interface then create a dummy sub-interface which will pull that configuration. Then recreate prior sub-interface.

Symptoms: OSPF packets with IP Precedence 0 are classified by SPD as priority packets. This is an error because only IP Precedence 6 packets should be classified as priority packets by SPD.

Conditions: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.2(18) or a later release but may also affect other releases.

Workaround: Use ACLs to block invalid IP control packets from reaching the control plane.

ISO CLNS

Symptoms: BFD may not come up when an IP address on an interface is changed and when IS-IS is configured as the routing protocol.

Conditions: This symptom is observed only when you first enter the router isis command and then enter the bfd all-interfaces command.

Symptoms: IS-IS may not advertise the prefix of a passive interface to the IS-IS database on a local router.

Conditions: This symptom is observed on a Cisco router when you shut down an interface (for example, G9/1/1) of a 5-port GE SPA (SPA-5X1GE) that is installed in a SIP-600, replace the SPA-5X1GE with another card, and then enter the no shutdown interface configuration command on the interface at the same location (G9/1/1) on the new card. In this situation, the prefix for the interface (G9/1/1) is not advertised.

Possible Workaround: Enter the shutdown interface configuration command followed by the no shutdown interface configuration command on the affected interface.

Miscellaneous

Cisco Internetwork Operating System (IOS�) Software is vulnerable to a Denial of Service (DoS) and potentially an arbitrary code execution attack from a specifically crafted IPv6 packet. The packet must be sent from a local network segment. Only devices that have been explicitly configured to process IPv6 traffic are affected. Upon successful exploitation, the device may reload or be open to further exploitation.

Cisco has made free software available to address this vulnerability for all affected customers.

Symptoms: The output of "show ip mds forwarding vrf <X> <group>" command (executed on E3 Line cards) would normally show both the slowpath entries as well as the hardware switching entries. However, after upgrade to 32S2 nightly of 04/28, some of the mVRFs show only the slowpath entries for this command.

Conditions: First reported on 32S2 nightly image of 4/28. However, it could have existed before that.

Workaround: None is required. The hardware entries are still properly programmed and can be still seen using "show ip hardware-m rx vrf <X> <group>" command.

Further Problem Description: The main concersns are: - Output of the command is not consistent for different vrfs even on the same Cisco IOS image. This can lead to confusion or incorrect interpretation while troubleshooting problems. - Its not clear if the lack of hardware entries in the output of the command should be treated as normal or not. - The "expected" output of this command needs to be documented. It is suggested that this bug be used to document the 'normal'/'expected' output of the command (i.e whether hardware entries should be expected or not).

Conditions: This symptom is observed on a Cisco router when you first disable and then re-enable IPv6 on an interface.

Workaround: Enter the shutdown interface configuration command followed by the no shutdown interface configuration command on the affected interface.

Symptoms: Ethernet frames with layer 2 header comprised of BEEFF00D are sent out of an MPLS TE tunnel during MPLS TE tunnel reconvergence.

Conditions: This symptom has been observed on an Engine 3 Gigabit Ethernet line card of a Cisco 12000 series router. The symptom was not present on POS line cards of the same engine.

Symptom has been observed between the time when reconvergence of all tunnels is triggered and the time when reconvergence of the particular tunnel is completed.

Symptoms: If a multilink interface has one end connected to Cisco 12000 router and the other end connected to a non-Cisco-12000 router, then the multilink interface receiver, at the non-Cisco-12000 router side, may drop all received packets due to packet fragment loss or out-of-order.

Condition: This may happen immediately when the first member link comes back up again after all member links of the multilink interface have gone down.

2> Move the member links from the current multilink interface to the new multilink interface.

Symptoms: A Network Processing Engine G1 (NPE-G1) may restart unexpectedly and report the following message:

Conditions: This symptom is observed only on Cisco 7200 and Cisco 7301 series routers that are configured with an NPE-G1 Network Processing Engine.

Symptoms: When you make changes to an active QoS service policy, an already freed block may be accessed. If this situation occurs, a Malloc failure may occur, and the router may crash.

Conditions: This symptom is observed on a Cisco router when you make changes to an active QoS service policy while traffic is being processed.

Symptoms: A Cisco 7200 series router may unexpectedly reload if QOS is configured when a PA-POS-1OC3 or PA-POS-2OC3 is installed.

Conditions: This can occur on all 12.0S images prior to the fix if the device is configured with QOS and has a PA-POS-1OC3 or PA-POS-2OC3 installed.

Symptoms: A Cisco 7304 that has an NPE-G100 processor may access a bad virtual address and reload unexpectedly.

Conditions: This symptom is observed when traffic flows to an ATM VC that is configured for MLP with a QoS policy and when the Qos policy has a priority class.

Symptoms: On an Engine 3 GE line card, after several switchovers, the show policy-map interface command output shows MDRR queues unallocated.

Conditions: This symptom is observed on Cisco 12000 series Internet routers with Engine3 GE line card and with 2 RPs in SSO Mode. An output service policy is applied to the line card. It happens in a systematic behavior (after the second switchover) if 8Qs/LC have been configured on the line card. It happens in a random behavior (after a random number of switchover) if 4Qs/LC have been configured on the line card (default configuration).

Symptoms: CE to CE pings fail in an EoMPLS Network when the PE edge facing interface is newly provisioned in Port Mode (also known as the xconnect command is applied on the interface).

Conditions: This symptom is observed on Cisco IOS Release 12.0(32)SY and EoMPLS is configured in port mode, on an Engine 3 GiGE line card for a link between PE-CE, and the port is newly provisioned. This is seen only on one test bed and not seen on other test beds with the same configurations.

Workaround: The problem goes away by doing a shut command followed by a no shut command on the newly provisioned interface.

Further Problem Description: Once the workaround is applied, this problem does not happen for that port even after the router or line card reloads. It also does not affect any traffic or any customers that are added later on.

Symptoms: When passing full 10 Gig Traffic into an E5 LC wherein the Dual priority feature is enabled and all the 10 Gig traffic is matching the higher priority criteria, the LC crashes. Conditions: 1. Dual priority feature enabled on the E5 LC

This Unexpected Reload on Cisco 12000 series routers running either GRP or PRP is seen when they are connected back-to-back and one of them is reloaded or restarted. This problem happens when BGP IPV4 prefix A recurses over a nexthop PrefixB. Also @a label is configured for the Prefix A. All works fine in normal condition but when the Nexthop router crashes or when interface connecting Prefix B flaps, the router crashes.

1. we have a recursive BGP route ie BGP aggregate resolved via /32 and vice- versa.

Multiple vulnerabilities exist in the Cisco IOS File Transfer Protocol (FTP) Server feature. These vulnerabilities include Denial of Service, improper verification of user credentials and the ability to read or write any file in the device's filesystem, including the device's saved configuration, which may include passwords or other sensitive information.

The Cisco IOS FTP Server is an optional service that is disabled by default. Devices that are not specifically configured to enable the Cisco IOS FTP Server service are unaffected by these vulnerabilities.

Symptoms: While booting the Router the rf_db is not getting initilaized to correct values

Symptoms: The show policy-map interface command output counters are not incrementing if traffic stream is muticast and Eng3 is setting qos-group on ingress.

Conditions: - Ingress policy-map classifying, and setting qos-group for received muticast traffic streams on Eng3 of c12000 node

- Egress policy-map attached to one of Eng5 interfaces, matching the qos-group set previously by Eng3 policy-map.

Symptoms: fast_tagrew will be missing in cef leading to imposition failure for those recursive prefixes going over a tunnel.

Symptoms: Each multicast packet has 24 bytes overhead added on both eng3 with egress qos enabled and on eng5 LC. This overhead can be seen via "sh policy-map int" counter. This is not just a counter issue, it also impacts the throughput.

Symptoms: After router reload, the local switching traffic on the 4xgigE Eng3 LC is not restored.

Symptoms: OIR SFC on 12816 chassis may result in LC gets reset. The problem observed when there is L2 local switching traffic between the POS ports.

Conditions: OIR redundant SFC and there are L2 FR local switching traffic between the ports.

Symptoms: "show ip mds stats linecard" shows MDFS reloads on all LCs when multicast distributed routing is added on a VRF through the configuration of "ip multicast-routing vrf vpn distributed"

Further Problem Description: Note that whilst the MDFS reload is a real reload, it is without a preceding clear so it will not generally cause traffic interruption as it merely causes the same information to be downloaded to the linecards again. However in a highly scaled system running close to the limit, the additional load introduced by a full MDFS reload of every linecard may cause additional failures owing to maxing out of the CPUs.

Symptoms: VRF feature clean-up error messages occur When trying to delete T1 interfaces from a channel-group

Conditions: The condition occurs When trying to delete T1 interfaces from a channel-group

Workaround: Remove all the VRF configurations from the serial interface before trying to delete it.

Symptoms: During Cisco IOS upgrade procedure on GSR, configuration from interface may be removed from running configuration.

Conditions: This issue is detected when upgrading from Cisco IOS Release 12.0(31)S2 to Cisco IOS Release 12.0(32)S2 and the missing configuration is from Gigabit Ethernet interface of SPA-1XTENGE-XFP and SIP-600 card in slot 0. Problem only appear on rare situations. Adding back congiguration will restore service.

Symptoms: When the linecard runs into some abnormal situation, the communications between SPA and linecard may not be reliable. If it is occurs, it normally triggers the SPA IPC keepalive failure. The failure of SPA keepalive/heartbeat results in SPA IPC restart. However, because of SPA IPC communications, the restarting of SPA may never be completed, leaving SPA in out of service state.

Conditions: feature mode is enabled and traffic is passed through it in ingress direction in PE router in a 6PE setup

GSR router has been upgraded from Cisco IOS Release 12.0(31)S1C to Cisco IOS Release 12.0(32)S6. On router's reload the following error messages were printed in router's log:

*Jan 30 04:23:28.585 EST: %QM-4-SW_SWITCH: Interface Serial4/0/0/5:0 routed traffic 
will be software switched in ingress direction(s)

*Jan 30 04:23:29.589 EST: %QM-2-TCAM_ERROR: TCAM pgm error(46): LC based QOS Mgr 
failed RTR2-PE#

*Jan 30 04:23:29.589 EST: %QM-4-SW_SWITCH: Interface Serial4/0/0/10:0 routed traffic 
will be software switched in ingress direction(s)

GSR has SIP-400 with 4CT3/DS0 SPA, with approx 30 channalized interfaces. After reload ALL channalized interfaces had both Ingress and Egress TCAM disabled.

RTR2-PE#sh qm int Serial4/0/0/16:0 Interface: Serial4/0/0/16:0 IP is enabled

hw[EGRESS] = 0, hw[INGRESS] = 0 hw_force_default[EGRESS] = 1, 
hw_force_default[INGRESS] = 1 TCAM disabled for egress. All packets punted to LC CPU 
TCAM disabled for ingress. All packets punted to LC CPU flags: message replied

*Jan 30 04:23:51.159 EST: %ALIGN-3-SPURIOUS: Spurious memory access made at 0x7A935C 
reading 0x44

*Jan 30 04:23:51.159 EST: %ALIGN-3-TRACE:

-Traceback= 007A935C 007A8BE8 0079E1EC 007ACBE0 007AC270 002EC5CC 002F65C4 00000000

Symptoms: After LC reset of Engine 3 CHOC48 or 4xGE or RP toggle, 'sh policy-map interface' shows subinterface attached policy to have ouput queues unallocated. Output also fails to show policing or WRED information.

Conditions: Engine 3 channelized OC48 on c12000 using frame-relay encapsulation on serial and POS interfaces, as well as subinterfaces configured on 4xGE card. Triggered by LC reset or RP toggle.

Symptoms: A router may reload when you perform an snmpwalk on the ciscoMvpnMrouteMdtTable.

Conditions: This symptom is observed when all of the following conditions are present:

- The Multicast Distribution Tree (MDT) default group address is not configured for the VRF.

Workaround: Configure the MDT default group address for the VRF by entering the mdt default mdt group command in VRF configuration mode.

Symptoms: remove interfaces from one vrfA to another vrfB and cause mdt tunnel to flap. then remove the "mdt default" statement from vrfA, this causes LC to reset

Conditions: Some of the PIM CLIs are causing active RP to crash. This is happening ONLY when these CLIs are configured while in the sub-config mode for "control-plane policing". Normally, any global relevant config should automatically exit the sub-config prompt, and accept the CLI as well. In this case, teh PIM command is rejected and RP crash follows. The same PIM commands work fine when executed under the global config mode (where they belong) or under other sub-config modes.

Workaround: Use the "exit" command to exit the the main config prompt before configuring PIM related CLIs.

Symptoms: OSPFv3 neighbor down may occur after upgrading IOS to 12.0(32)S release.

Symptoms: A Cisco 12000 router running 12.0(32)SY2b image exhibits large number of memory usage on the RP and reduce in its Largest Free Block (fragmentation) compare to earlier release of 12.0(32)SY1.

This is seen on E0 ATM, E3 ATM, and E3 GE and is suspected to be impacting other types of engines and line cards as well.

Conditions: This appears to be impacting line cards with larger number of IDBs (subinterfaces.)

Symptoms: When ALPHA has an error due to some bug, the linecard crashes without error recovery.

Further Problem Description: During any ALPHA errors, there is no trace of the problem occuring. This happens only during SW bugs, but should be fixed to improve debuggablity.

Conditions: problem can be seen on c12kprp-p-mz.120-nightly.S for engine4+ card.

Symptoms: 4-port E3 GE LC stuck in UP state for about 45 minutes before coming to RUN state.

Conditions: This issue was seen in scaled IP+L3VPN+mVPN environment when the router was reloaded.

%LC_MOD48-3-SPAG_MULTIPLE_BAY_EFC_BP_MAPPED_TO_CONGA_PORT: Found EFC channels from 2 different bays mapped to the same Conga port. existing map {bay=1, efc_chn=238}, requested map {bay=2, efc_chn=238}

Conditions: This error message while reloading SPAs that has multilink bundle (Multilink FR or Multilink PPP) configuration. This issue is specific to the SPA interfaces on SIP-400.

Symptoms: The use of time-based WRED on an output service policy results in different min-max threshold values on E3 and on E5 for identical policy.

Symptoms: After inserting a 4OC48E/POS-SR-SC LC it is recognized but when command "upgrade mbus-agent-rom" is used LC is not recognized by the router anymore and displays:

%MBUS-6-OIR: 4 Port ES Packet Over SONET OC-48c/STM-16 Removed from Slot x MBus agent ROM upgrade failed on slot x (rc=5)

Conditions: New LC is inserted in the chassis and have never been upgraded its ROM code before.

Conditions: Service policy attached on MFR subinterfaces and microcode reload done

Symptoms: C12000 dual RP with Engine 5 line card traceback SEC 0:00:01:48: %EERP-3-INVALID_UIDB_HWIDB_MAP: slot 7, index= 00000004 orig_if= GigabitEthernet7/0/0 given_if= GigabitEthernet7/0/0 free= 0

-Traceback= 212BF8 212D00 82027C 81BDFC 81B0B4 12EEE4 181908 181B04 1826A0 125F6C 1990B4 199434 192BC8 192E2C 193128 193DD0

Conditions: C12000 Engine 5 interface configuration change when system has dual RP

Symptoms: After RP switchover, all configured WRED thresholds are set to 0 in "sh policy-map int" command

Workaround: Remove service-policy from the interface and re-attach it or shut/no shut the interface

Conditions: Execute CSC switchover by shutting primary CSC. EoMPLS traffic forwarding stops.

Conditions: When error recovery is triggered and traffic passes into the Mod48 , during error recovery , it fails to drain the half packets and fails to initialise the congs.

Condition: - configuring "service-policy output" on the interface, - flowing traffic heavy enough to be policed by the configuration.

Further Problem Description: Flowing about 110 pps traffic, the router crashes with %SYS-6-STACKLOW on the FR LMI process.

Symptoms: QoS on egress port of E4+ POS is not taking effect if the tunnel headpoint on the same port as well.

Conditions: The issue happens when we remove the "mpls ip" configuration under the tunnel and re-add it after a delay.

1.This corrupts and resets the feature processing flag in the ingress adjacency which normally enables "feature processing in the egress". Since this is reset to 0, the egress is not able to do QoS.

2. The adjacency rewrite info corresponding to the tunnel in the egress gets removed.

Workaround: Remove "mpls ip" from the tunnel interface once and immediately add it back. This corrects the issue.

Symptoms: 'show interface' command indicate that egress data rate on 4GE-SFP-LC shows doubled rate of actual traffic rate.

Conditions: EoMPLS setting on the interface may cause this issue. It can be seen with PRP-2 and GRP-B.

Symptoms: RPR+ mode, when I do the following sequence the slave RSP configs add a "shutdown" command under interface serial.

Symptoms: On a GSR running Cisco IOS Release 12.0(32)S6o all non ipc tofab queues are depleted on a choc12/ds1-ir-sc line card causing all traffic to be dropped including Layer 2 control traffic for the interfaces.

Symptoms: Local Switching -- like to like and Interworking is not working on E3 gig LC.

Conditions: Local Switching -- like to like and Interworking is not working on E3 gig LC. Traffic starts and stops forwarding after some time.

Symptoms: SPA-2XOC48POS/RPR goes to Out Of Service after encountering a SPA BUS ERROR. TRANSCEIVER-6-REMOVED messages were followed by SCC failure resulting the SPA to go to Out Of Service.

Conditions: Lots of L1 errors are found (B2-BER) on the link and the interfaces flapped lot of times before the BUS ERROR.

The channelized OC12 controller on a GSR might go down and remain down in case of a PLIM CPU reset. If this occurs, APS may not switch to the other channel.

"show aps" will display that the active channel's interface is down: router#sho aps CHOCx 3/0 APS Group 3: protect channel 0 (Active -interface down)

Recovery: In case of a PLIM cpu reset, if the controller stays in the down state, the line card must be reloaded with the "test mbus <slot> power off/on" commands.

Symptoms: BFD session goes down on removing and adding ip address from bfd enabled interface.

Conditions: This symptom is observed on a GSR router loaded with Cisco IOS Release 12.0(32)sy image. After BFD sessions are estalished on adding and removing ip address couple of times the BFD sessions go down

Symptoms: The following error message occurs on CHOC12/DS1-IR-SC linecard indicating that the PLIM is reset:

Conditions: When shutdown / no shutdown was executed on many multilink ppp bundles continuously.

Some CEF related commands on a previously removed sub interface are (wrongly) retained when configuring a new sub-interface. This is also true for sub-interfaces that are being reused.

Symptoms: The following CEF related commands are retained from previously removed sub-intrefaces:

Router(config-if)#int Serial3/3/1:0.100 point <<< ADD .100

Router(config-subif)#ip cef accounting non-recursive external

Router(config-subif)#ip load-sharing per-packet

Router(config-subif)#no int Serial3/3/1:0.100 <<< REMOVE

Router(config)#int Serial3/3/1:0.200 point <<< ADD .200

Router(config-subif)#do sh run int Serial3/3/1:0.200 Building configuration...

Current configuration : 166 bytes ! interface Serial3/3/1:0.200 point-to-point

no ip directed-broadcast ip load-sharing per-packet <<<<

ip cef accounting non-recursive external <<<<

no ip mroute-cache end

Conditions: This issue is observed in 12.0S on reused and new ATM or serial sub-interface on a GSR.

Workaround: Verify sub-interface configuration and if configuration cannot be deleted on that sub-interface, delete this sub-interface then create a dummy sub-interface which will pull that configuration. Then recreate the prior sub-interface.

Symptoms: When changing mtu on 4xOC3-V2 SPA in SIP-501, the following crash may be seen: %RP-3-CARVE_FAIL: FrFab BMA, slot 9 %RP-3-COREDUMP: Core dump incident on slot 9, error: Safe FrFab buffer carve failure %RP-4-RSTSLOT: Resetting the card in the slot: 9,Event: linecard error report %LCINFO-3-CRASH: Line card in slot 9 crashed

Conditions: This symptom is observed on a Cisco 12000 series router that runs the c12kprp-p-mz image of Cisco IOS pre-Release 12.0(32)SY4.

Symptoms: SPA may reset due to heartbeat failures on doing CSC switchover. Following messages may be seen. SLOT 4:1d19h: %SPA_PLIM-3-HEARTBEAT: Subslot 3 has experienced an heartbeat failure Current Sequence 31064 received Sequence 31057 Time since last keep 1000ms.

Symptoms: On rommon upgrade of secondary RP after an IOS upgrade, the active RP may crash

Conditions: Dual RP set up. IOS upgrade done by reloading the secondary RP followed by redundacy force switch over. When the router is up, perform rommon upgrade on secondary RP.

Symptoms: On doing OIR of CSC cards or during Cisco IOS upgrades, the CSC card may be recognised as SFC or vice versa. In case of non-redundant fabric configuration, the linecards may go into low bandwidth mode. This may result in configs being lost.

Symptoms: When upgrading from 120-28.S4c to 120-32.S6p some tcam carve configuration commands are lost for an engine 3 card.

hw-module slot 4 tcam carve RX_TOP_NF 3

hw-module slot 4 tcam carve RX_144b 35

hw-module slot 4 tcam carve RX_288b 45

hw-module slot 14 tcam carve RX_TOP_NF 3

hw-module slot 14 tcam carve RX_144b 35

hw-module slot 14 tcam carve RX_288b 45

Symptoms: SLOT 3:Jun 19 17:17:17.632 EDT: %DATACORRUPTION-*1*-DATAINCONSISTENCY: copy error -Traceback= 4003111C 40045CCC 4005F3A0 409CA48C 409CC660

Conditions: Just after upgrading one c12000 from 28S4c to 32S6p, while doing the upgrade all all, the following tracebacks were observed in the log.

Workaround: This is a non-intrusive trace. There is no need for a workaround or operator intervention.

Symptoms: After RPR+ switchover, native multicast traffic may get dropped on Engine 5 linecards

On the egress linecard do: sh ip mds foraffected s g and retreive the hw_mdb sh controller multicast mgid-table hw_mdb should return something like: table_value[0]:200043AFFFFFFFF table_value[1]:0 The table_value[0] should END In FFFFFFFF when this problem occurs

Conditions: Only affects E5 It has only been seen on the 32s release (32.S6o, 32.s6p]

Workaround: Any of the following will work: flap the affected interface clear the mroute [ clear ip mroute * reload mdfs [ clear ip mds line linecard num ]

Symptoms: When engine 3 4xGE linecard cpu is busy, packets may be forwarded to the wrong interface on the linecard when multiple configuration events are happening.

Conditions: Multiple configuration events, for example removing multiple output service policies on an interface or subinterface level.

Further Problem Description: Backing out this fix will cause MLPPP bundle unprovisioning with a traceback "%RP-3-ENCAP:Failure to get output encapsulation:unprovisioning MLPPP mem"

Symptoms: Output NF not working on 32S8 16/06 dated coded image for Engine5 card

Symptoms: MVPN Decap node classifying multicast traffic stream according to the GRE tunnel IP Precedence. show policy interface packets counter incrementing for the wrong class/queue.

Conditions: - P node (GSR), with ingress E3 line card. Ingress MQC policy re-marking (set ip precedence) multicast traffic. - C12000 act as MVPN Decap node, ingress E3 line card with egress multicast policy enabled. Traffic matching P re-marked IP Precedence.

Symptoms: In rare cases, C12000 router with SIP400 and one or more SPA-CT3/DS0 and SPA-T3E3 installed may display the following message:

SLOT 14:Jul 22 06:18:31.790 EDT: %SPA_PLIM-3-HEARTBEAT: Subslot 2 has experienced an 
heart beat failure Current Sequence 1980 received Sequence 1970 Time since last keep 
2952ms.

SPA-CT3/DS0 and SPA-T3E3 may stay in the state and the SPA may not recover in some cases.

Workaround: The following command may be used to disable SPA heartbeat to avoid the SPA failure.

execute-on <slot#> test hw-module subslot <subslot#> ipc keepalive disable

It is not recommended to use this command and it may lead SPA stuck in bad state. The test command shall be used under Cisco Support supervision.

Symptoms: E5 SIP-601 LC CPU stays almost at 100% for a very long time, the WAHOO MCAST DELE process utilizing most of the CPU.

Conditions: This is seen after the MSE router reload on the scale testbed with scaled IP, L3VPN and mVPN configs.

Wide-Area Networking

Symptoms: The interface level commands 'no ppp link reset' and 'no pp lcp fast-start' disappear from the configuration after performing a 'wr mem' or 'show run | inc ppp'. The commands do not reappear after reload.

Conditions: Interface level configuration of 'no ppp lcp fast-start' and 'no ppp link reset'

Resolved Caveats—Cisco IOS Release 12.0(32)S7

Cisco IOS Release 12.0(32)S7 is a rebuild of Cisco IOS Release 12.0(32)S. The caveats listed in this section are resolved in Cisco IOS Release 12.0(32)S7 but may be open in previous Cisco IOS releases. This section describes only severity 1, severity 2, and select severity 3 caveats.

Basic System Services

SEC  8:000049: Jan 31 22:25:00.760: %SNMP-3-DVR_DUP_REGN_ERR: Attempt for dupe

regn with SNMP by driver having ifIndex 709 and ifDescr Tunnel0

-Traceback= 204128 204230 92DB90 92DF6C B2CF8C BBF368 BC00C8 1C4EFC 1C5524

1C60B8 1C655C 2EC5CC

Conditions: This symptom has been observed when new interfaces are added (or existing interfaces like tunnel come up) after bootup, or when new or existing interfaces come up after RPR+ switchover when running Cisco IOS Release 12.0(32)S6. Also, this symptom occurs if the snmp ifindex persist command is configured on the router.

Further Problem Description: Though customer traffic is not affected, this symptom does impact the SNMP stats and other SNMP data for both the original and the new interface. Usually the message is from the standby RP, so once that standby RP becomes active, the data from SNMP polls of these interfaces would not be accurate.

Interfaces and Bridging

Symptoms: The data terminal ready (DTR) port adapter on a Cisco router does not continuously drop and pulse according to the time intervals configured with the pulse-time seconds interface configuration command. On the 4-port serial port adapter (PA-4T+) and Fast Serial Interface Processor (FSIP), the �RSP-3-RESTART� message may appear right before a Cisco router has to be restarted to recover a serial interface from a circuit outage that was configured by entering the pulse-time seconds interface configuration command

Conditions: This symptom is observed on a Cisco 7500 series router that a PA-4T+ port adapter, an 8-port serial port adapter (PA-8T), and an FSIP.

IP Routing Protocols

Symptoms: A Cisco router or switch may reload unexpectedly when you enter the show ip eigrp neighbors EXEC command.

Conditions: This symptom is platform independent and is observed when you enter the show ip eigrp neighbors EXEC command immediately after you have entered the shutdown interface configuration command followed by the no shutdown interface configuration command for the interface that connects the router or switch to the neighbor.

Workaround: Wait for the neighbor list to be completely rebuilt before you enter the show ip eigrp neighbors EXEC command.

Symptoms: Enhanced Interior Gateway Routing Protocol (EIGRP) routes cannot be seen even when message digest algorithm 5 (MD5) is authenticated on all routers. This symptom is intermittent and may occur when authentication is turned off and subsequently turned back on again. Sometimes, this symptom occurs just after authentication is enabled.

Conditions: This symptom has been observed when authentication is turned off and subsequently turned back on again and sometimes just after authentication is enabled.

Workaround: This symptom is intermittent and may be resolved by disabling and reenabling authentication a second time. This symptom may automatically be resolved after a few minutes.

Symptoms: A summarized entry may remain in the Enhanced Interior Gateway Routing Protocol (EIGRP) topology table after manual summarization is disabled.

Conditions: This symptom is observed when manual summarization is enabled and subsequently disabled on a network that is also being redistributed into EIGRP.

Symptoms: The interface-type and interface-number arguments in the distribute-list address family configuration command do not function.

Conditions: This symptom is observed on a Cisco platform that integrates the fix for caveat CSCea59206. A list of the affected releases can be found at http://www.cisco.com/cgi-bin/Support/Bugtool/onebug.pl?bugid=CSCea59206. Cisco IOS software releases that are not listed in the "First Fixed-in Version" field at this location are not affected.

Further Problem Description: The fix for CSCed84633 re-enables the interface-type and interface-number arguments in the distribute-list address family configuration command for both VRF interfaces and non-VRF interfaces.

Symptoms: When BGP updates are received, stale paths are not removed from the BGP table, causing the number of paths for a prefix to increase. When the number of BGP paths reaches the upper limit of 255 paths, the router resets.

Conditions: This symptom is observed on a Cisco router when the neighbor soft-reconfiguration inbound command is enabled for each BGP peer.

Workaround: Remove the neighbor soft-reconfiguration inbound command. A router that runs a Cisco IOS software image that has a route refresh capability, storing BGP updates is usually not necessary.

ISO CLNS

Symptoms: Tracebacks may be generated when you configure IS-IS and LDP features, for example, when you enter the no ip router isis area-tag command.

Conditions: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.0(32)SY but may also occur in other releases.

Miscellaneous

Symptoms: A Cisco 7200 series or another mid-range router may crash or may stop responding.

Conditions: This symptom is observed on a Cisco 7200 series or other mid-range router that runs Cisco IOS Release 12.3(6a). The crash occurs when an interface that is configured with a rate-limit command is deleted by entering the no interface command and then reenabled by entering the interface command.

Workaround: Remove the rate-limit configuration from the interface before deleting the interface.

Conditions: This symptom is observed on Cisco IOS Release 12.0(26)S with GSR Eng4+ LC sampled netflow.

Condition: This issue has been seen on a Cisco 12416 Cisco Internet Series router containing dual PRP-2 Route Processors running Cisco IOS Release 12.0(32)S6. The route contains 11 Engine 3 and 3 Engine 4+ line cards. The CEF inconsistency has been seen on both Engine 3 and Engine 4+ line cards.

Symptoms: A master timer may be accessed prior to being initialized. This can be seen by the following error message:

SEC 7:000048: Nov 30 00:00:29 GMT: %SYS-3-MGDTIMER: Uninitialized timer, init with 
uninitialized master, timer = 53E62C0. -Process= "Init", ipl= 0, pid= 3

Condition: This symptom is seen on a Cisco 12416 Cisco Internet Series Router containing dual PRP-2 processors running in RPR+ mode. This router is booted with Cisco IOS version 12.0(32)S6.

Symptoms: In a rare condition, the line card CPU may be hogged and fails to process SPA IPC messages. The back-pressure of SPA interface may cause SPA interface card overflow with its IPC internal FIFO and make SPA interfaces down.

Conditions: If the LC gets reset before the crashinfo file is closed, the open bit is set

Symptoms: The SIP-400 LC crashes and recovers when the CE2 is reloaded. The CE is a Cisco 7200 with PA-8TE1's connecting via a Mux to the GSR. The CT3 SPA on the GSR has quite a few MLP & MLFR interfaces for VPN services using biscuit. MVPN is also turned on for these VPNs.

Symptoms: On a GSR running Cisco IOS Release 12.0(31)S, the BFD session is dropped after lost of One BFD control packet.

The actual BFD interval is constantly higher than the interval configured, in the traces below the tx interval is configured to 2s and the actual Tx interval is around 3,5s

Conditions: This is a BFD-enabled BGP session between two GSR, x.x.x.1 with a tx timer of 5s, and x.x.x.x.2 with a tx timer of 2s. Both multiplier are 3. Both routers are running Cisco IOS Release 12.0(31)S, and both sides are Engine 3 linecards.

#30: BFD control packet A from .2 to .1

#31: BFD control packet A from .2 to .1 (3,575848s after #30)

#33: BFD control packet A from .2 to .1 (3,511843s after #31)

#34: BFD control packet A from .2 to .1 (3,623851s after #33)

#36: BFD control packet A from .2 to .1 (3,167859s after #34)

#37: BFD control packet A from .2 to .1 (3,655841s after #36)

#38: BFD control packet A from .2 to .1 (3,359866s after #37)

#42: BFD control packet A from .2 to .1 (3,255843s after #38)

#44: BFD control packet A from .2 to .1 (3,471862s after #42)

#46: BFD control packet A from .2 to .1 (3,639842s after #44)

#47: BFD control packet A from .2 to .1 (3,287859s after #46)

#48: .1 signals the session down 6.003069 s after having received #47

6.003069 s is the detection time 3 x 2 s (our mult x .1's tx timer)

Symptoms: When setup ATM to ATM local switch over a l2tpv3 tunnel, the PE router might reload at atm_l2trans_vc_get_segment_handle_fr_ac_ie_hdl

Conditions: The ATM setup is VPtrunk with cellrelay, and this symptom has only observed on Cisco 7500 platform as PE router in the topology.

Symptoms: On removal of a member from portchannel VRF interface, router crashes.

Conditions: This symptom has been observed on removal of a member from portchannel VRF interface.

Symptoms: Sending mpls traffic to an E5 LC with oc-192 SPA which doesn't have mpls configured, it will cause this LC crash.

Conditions: This symptom has been observed with an E5 LC with oc-192 SPA. This symptom does not happen if using 10G SPA or E4+ oc-192.

%TX192-3-PAM_MODULE:

Conditions: No conditions have been identified so far. This section is being completed as more information is available.

Further Problem Description: This is the complete log that is seen on the router:

SLOT 12:Aug 11 07:51:39: %TX192-3-PAM_MODULE: status = 0x2, mask= 0x3F -

MODULE: Error signal from PIM module.

-Traceback= 40030CBC 40862008 408625EC 4096046C 40960B08 4010F8C8

SLOT 12:Aug 11 07:51:39: %TX192-3-PAM_PIM: status = 0x3D6, mask= 0x181 - PIM:

header start offset >= 16kB.

-Traceback= 40030CBC 408621A4 40862634 4096046C 40960B08 4010F8C8

SLOT 12:Aug 11 07:51:39: %GSR-3-INTPROC: Process Traceback= 4011717C 40110290

40011180

-Traceback= 40030CBC 4075AB8C 40960D60 4010F8C8

SLOT 12:Aug 11 07:51:44: %GSR-3-INTPROC: Process Traceback= 400F9B34 40117140

-Traceback= 40030CBC 4075AB8C 40960D60 4010F8C8

Aug 11 07:51:49: %LDP-5-NBRCHG: TDP Neighbor 10.10.10:0 is DOWN (TCP

connection closed by peer)

Aug 11 07:51:57: %LDP-5-NBRCHG: TDP Neighbor 10.10.10.72:0 is UP

SLOT 12:Aug 11 07:52:28: %TX192-3-PAM_MODULE: status = 0x2, mask= 0x3F -

MODULE: Error signal from PIM module.

-Traceback= 40030CBC 40862008 408625EC 4096046C 40960B08 4010F8C8

SLOT 12:Aug 11 07:52:28: %TX192-3-PAM_PIM: status = 0x356, mask= 0x181 - PIM:

header pkt length >= 16kB.

-Traceback= 40030CBC 408621A4 40862634 4096046C 40960B08 4010F8C8

SLOT 12:Aug 11 07:52:28: %GSR-3-INTPROC: Process Traceback= 409AFC9C 40117178

-Traceback= 40030CBC 4075AB8C 40960D60 4010F8C8

SLOT 12:Aug 11 07:52:33: %GSR-3-INTPROC: Process Traceback= 409AFC9C 40117178

-Traceback= 40030CBC 4075AB8C 40960D60 4010F8C8

Symptoms: After perform a RP switchover on a long idle(20 hrs above) GSR router loaded with 092806 nightly build image, all LCs got reset.

Conditions: This symptom has been observed when all LCs are reset, after RP switchover on a long idle router.

Symptoms: IPv4/Ipv6 ping failed over L2TPv3 Tunnel. This is happening with PD also and here IPv6 ping fails.

Conditions: 1) Crate a L2TPv3 tunnel .. establesh a Ipv6/ipv4 adj. b/w the CE's 2) Send a Ipv4/ Ipv6 traffic. 3) Do a reload/SSO of the router ... you should hit this issue.

This issue is not related to TE/FRR specific. Issue in driver code handling read/write to hardware. Linecard is crashing because we are writting to hardware without stopping the pipeline. TE/FRR config has exposed this issue because TE/FRR updates happen in interrupt mode.

Symptoms: LC may crash at multicast code area after deprovision and reprovision mlp interface that has pim enabled

Symptoms: After deprovision and re-provision MLP interface and its associated serial link interfaces several times, error msg may appear. After this error msg, you can no longer define the same mlp interface.

%EERP-2-UIDB_ERR: Unable to allocate resources. Invalid slot for free -1

Symptoms: After adding and deleting MLPPP interface, RP switchover and then change the encapsulation to FR interface, error msg %IDBINDEX_SYNC-3-IDBINDEX_ENTRY_SET can occur and standby RP can crash.

Conditions: Recursive prefix going over attached parent and parent is load shared. That is, there is multiple path to reach parent prefix. And, there is no tag information for attached or connected parent prefix. When recursive child prefix is resolved, it will access NULL tag_info and tag_rewrite of connected/attached parent prefix and result in spurious access.

Workaround: In vrf address family, give the command "redistribute connected" and/or "redistribute static". Also, configure static route instead of static recursive route by specifying interface option in the static route configuration, if any exist. LFIB does no t effectively support static recursive route configuration.

Symptoms: ATM port mode, after adding xconnect statement under ATM interface, the interface continues flapping until shut/noshut the interface

Conditions: This symptom has been observed when provisioning xconnect statement under ATM interface.

Conditions: In customer network having MLPPP links CT3 SPA and Serial interfaces on CT3 and T3 SPAs with VRFs configured and having large no of BGP routes.

Further Problem Description: The symptom happening because of corrupted IPC packet coming from SPA. As part of solution Sanity checks and buffer corruption correction code added to avoid crashes. More details in the attached log mail.

Symptoms: During the OIR of SPA, the following syslog messages may appear and soon after the E5+ SIP 601 LC will reload.

SLOT 2:*Jan  5 18:50:13.984 UTC: %EE192-3-SPABRG_DRV: wwolf_handle_spa_removed: too 
many spas for grp: 0x0, 0x0, 0x0

-Traceback= 40A2751C 405E1ED0 405C437C 405CDBC0 40657C94 4074A360 406D0F64 406D2104 
4121901C 406D0798 406D2374 406CFC98 406CFF00

SLOT 2:*Jan  5 18:50:34.732 UTC: %EE192-3-SPABRG_DRV: wwolf_spa_init: queue group not 
found: 0x0, 0x0, 0x0

-Traceback= 40A2751C 405E1A78 405CD0AC 40657C74 4074956C 406D1A8C 4121901C 406D0798 
406D2374 406CFC98 406CFF00

Conditions: This symptom can occur on rare occasions of OIR of SPA's on E5+ SIP 601 LC.

Workaround: After each OIR of spa wait at least 2 minutes and then insert any new spa.

Symptoms: A Cisco 12410 with SIP-405/501/601 linecard with 5x1GE SPA and BFD configured and operating in RPR-Plus redundancy mode, break BFD sessions when redundancy force failover is initiated.

Conditions: Redundancy mode must be RPR-Plus BFD must be configured Force failover must be initiated

Workaround: Admin shut down the interface, wait for 15 seconds and then 'no shudown' the interface to recover the BFD session.

Symptoms: When sending 10 Gig traffic to a 2.5 Gig interface, the egress data rate as seen in the 2.5 gig pos spa interface is 9.4 G/s which is higher than the supported bandwidth.

Symptoms: On SIP-600, we are seeing the error message to be logged on the console as below.

SPA error interrupts are not yet

Symptoms: After issuing "hw-module slot x reload", the pseudowire vc stays down since there is no local label allocated

Workaround: Removed and then add the xconnect command. Shutting down and restarting the interface does not work.

Symptoms: After RPR+ forced switchover, traffic is no longer passed across the affected frame-relay subinterfaces on the CT3 SPA. Commands ran on SPA LC indicate the incorrect mapping of if_number.

Conditions: RPR+ switchover must occur, BGP needs to be running, and routes injected, we hit it with 180,000 routes.

Symptoms: Ce-ce ping/traffic fails, mip reg counter incremets on remote pe condition: Frompls, with both PE's reload simultaneously workaround: need to unprovison both atom vc;s on both Pe's, unprovisoning on one Pe doesn't work.

Symptoms: A PXF crash could happen if multiple combinations of features are present: input ACL, input QOS, output ACL, output QoS.

Conditions: The crash will happen only if the input policymap is first disabled on the input interface ('no service-policy input X') and then it is de-configured on the router ('no policymap X').

Symptoms: The packets might get punted to Tx slow-path. FRR switchover will not happen in 50ms time.

Conditions: This symptom has been observed when primary link goes down and backup tunnel takes over.

Symptoms: RP failover would cause SIP-601 core-facing line card to crash, and reload.

%RP-4-RSTSLOT: Resetting the card in the slot: 15,Event:linecard error report

%LINK-5-CHANGED: Interface GigabitEthernet15/0/0, changed state to

administratively down

%OSPF-5-ADJCHG: Process 2, Nbr 10.0.0.45 on GigabitEthernet15/0/0 from FULL to

DOWN, Neighbor Down: Interface down or detached

%LDP-5-NBRCHG: LDP Neighbor 10.0.0.45:0 (1) is DOWN (Interface not operational)

%LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet15/0/0, changed

state to down

%PIM-5-NBRCHG: neighbor 10.144.2.13 DOWN on interface GigabitEthernet15/0/0 non DR

%LCINFO-3-CRASH: Line card in slot 15 crashed

%MBUS_SYS-3-NOBUFFER: Message from slot 15 in stream 1 dropped

%MBUSFLASH-3-TIMEOUT: No response from slot 8 (type 5,time 2000)

%BGP-5-ADJCHANGE: neighbor 172.16.1.2 vpn vrf m1 Up

%BGP-5-ADJCHANGE: neighbor 172.16.2.2 vpn vrf m1 Up

%RP-3-EXEC_SLOT: Slot 15 not enabled

Conditions: This symptom has been observed on a Cisco 12000 node fully populated with E3 line cards, mvpn configuration, and large scale multicast routes and traffic.

Conditions: This symptom has been observed when the LC received the 255.255.255.255 broadcast packets.

Conditions: Traffic Generator -- G7/0/0 (SIP-600) GSR (SIP-601) G8/0/0 -- CRS-1(1) -- CRS-1(2)-- Same Traffic Generator

When generating 9.5Gbps of traffic the interface counters of Cisco 12000 populated with SIP-601 and SPA-1XTENGE-XFP showed the 5 minute output rate to be ~ 6.23 Gbps. CRS-1 interface connected to SIP-601 shows 5 minute input rate to be ~ 9.4 Gbps.

Symptoms: When a multilink bundle has one end connected to Cisco 12000 router and the other end connected to a non-Cisco-12000 router, the multilink interface receiver, at the non-Cisco-12000 router side, may drop all received packets due to packet fragment loss or out-of-order.

Conditions: This may happen when the first member link of the multilink bundle comes up immediately after all member links of bundle have gone down.

Shut down all member links of the multilink bundle using the shutdown command.

Bring up the member link interfaces one-by-one using the no shutdown command.

If the above workaround (option 1) does not resolve the symptom, perform the following steps:

Shut down all the links in the affected multilink bundle using the shutdown command.

Remove all the links from the bundle using the no multilink-group command until the bundle has no link.

After a few minutes (~5 min) add the links back to the same bundle using the multilink-group group number command.

Symptoms: If cleanup is not done on CIs used by L2TPV3, and these CIs are reused by other applications, then it will have extremely undesirable effects (like in case of MLP on SIP 400/2 or 4 Port CT3/DS0 SPA, all packets get dropped). To avoid this L2TPv3 codebase has to clean up explicitly.

Conditions: Configure L2TPv3. Configure MLP. Shutdown the L2TPv3 tunnel and then shutdown the MLP. Then do a no shut on the MLP bundle. If the MLP reuses, the CI used by L2TPv3, then packets will get dropped.

Conditions: Test script used for provision/reprovision an MLFR bundle including associated member links appears to trigger a memory leak on the route processor (PRP-2).

Symptoms: Customer has a 12000-SIP-601 that crashes and leaves no crashinfo file.

Conditions: This occurs if MBE (and SBE) parity errors occur in the SIP-601 memory.

Symptoms: When E4P/E6 drops MTU exceeded packets coming in with aggregate label in mpls vpn setup

Conditions: In MPLS VPN enviroment, when the core side interface of disposition router is E4P/E6 and QOS is configured on the interface then all packets received with aggregate label that are punted to RX CPU for processing (for example MTU exceed packets) are dropped by the cef switching code.

Symptoms: BFD is not supported on Bundle interface. When user tries to configure BFD on link bundle, the router crashes.

Conditions: This symptom has been observed when applying BFD configuration on Pos-channel and ether-channel.

Symptoms: SPA-2XOC48POS/RPR goes to Out Of Service after encountering a SPA BUS ERROR. TRANSCEIVER-6-REMOVED messages were followed by SCC failure resulting the SPA to go to Out Of Service.

Conditions: Lots of L1 errors are found (B2-BER) on the link and the interfaces flapped lot of times before the BUS ERROR.

Symptoms: Packets from E5 to E2 get dropped on E2. On E2, the show contr events command shows the counters for "Tx short encap entry" increase.

Resolved Caveats—Cisco IOS Release 12.0(32)S6

Cisco IOS Release 12.0(32)S6 is a rebuild of Cisco IOS Release 12.0(32)S. The caveats listed in this section are resolved in Cisco IOS Release 12.0(32)S6 but may be open in previous Cisco IOS releases. This section describes only severity 1, severity 2, and select severity 3 caveats.

Basic System Services

Symptoms: A Cisco router may generate export packets in which the first flow record contains incorrect data such as incorrect IP addresses.

Conditions: This symptom has been observed on a Cisco router that is configured for NetFlow and NetFlow Data Export.

Symptoms: The active RP may crash when traps are sent to a host to which an SNMPv3 user is assigned.

Conditions: This symptom is observed only when an SNMPv3 user is configured with security level noAuthNoPriv or authPriv, when the same SNMPv3 user is assigned to the host through the snmp-server host command, and when this command includes the priv keyword. This is an improper configuration.

For example, the symptom occurs when traps are triggered after the following software configurations has been applied:

snmp-server user TESTUSER TESTUSER v3

snmp-server group TESTUSER v3 priv notify *tv.FFFFFFFF.FFFFFFFF.FFFFFFFF.F

snmp-server host 10.1.1.10 version 3 priv TESTUSER

snmp-server enable traps

Symptoms: The following may be seen when attempting to boot a Cisco 12000 router that has a GRP-B RP:

Address Error (load or instruction fetch) exception, CPU signal 10, PC = 0x502B96C8

--------------------------------------------------------------------

Possible software fault. Upon recurrence, please collect

crashinfo, "show tech" and contact Cisco Technical Support.

--------------------------------------------------------------------

-Traceback= 502B96C8 50770F54

$0 : 00000000, AT : 55230000, v0 : 00000000, v1 : 0B0D0B0D

a0 : 0000000A, a1 : 5775A568, a2 : 00000003, a3 : 58374B58

t0 : 00000048, t1 : 58D9FE04, t2 : 58D9FE00, t3 : 58D9FDFC

t4 : 58D9FDF8, t5 : 58D9FDF4, t6 : 58D9FDF0, t7 : 58D9FDEC

s0 : 58374860, s1 : 58374864, s2 : 58374864, s3 : 58374860

s4 : 55A20000, s5 : 00000000, s6 : 00000000, s7 : 00000000

t8 : 0D0D0D0D, t9 : 00000000, k0 : 30408400, k1 : 30410000

gp : 5522BCE0, sp : 58374828, s8 : 00000000, ra : 502B9694

EPC  : 502B96C8, ErrorEPC : F6BF7BFF, SREG     : 3400BF03

MDLO : 3B5BF4D5, MDHI     : D253FCD7, BadVaddr : 0B0D0B35

Cause 00000010 (Code 0x4): Address Error (load or instruction fetch) exception

Conditions: This symptom is observed on a Cisco 12000 series router that runs the gsr-p-mz image of Cisco IOS Release 12.0(32)SY.

Further Problem Description: RSP emits DBUS-3-SW_NOTRDY and CBUS-3-CCBPTIMEOUT messages for all VIPs and VIPs do not boot up.

Interfaces and Bridging

Symptoms: Spurious memory access occurs when removing channel groups in the T1/E1 cards.

Conditions: This symptom has been observed with a PA-MC-8TE1+ port adapter on a Cisco 7500 router running Cisco IOS Release 12.0S.

IP Routing Protocols

Symptoms: A CE router that has L2TP tunnels in an MPLS VPN environment with about 1000 VRFs may crash and generate the following error message:

Address Error (load or instruction fetch) exception, CPU signal 10, PC = 0x50766038

Conditions: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.0(32)S and that functions as a CE router when BGP neighbors are unconfigured via the no neighbor ip-address command while the show ip bgp summary command is entered from the Aux console. The symptom is not release-specific and may also affect other releases.

Symptoms: Prefixes that are tagged with Site of Origin (SoO) values may not be filtered at the border.

Conditions: This symptom has been observed when SoO values are configured for a peer group. The peer group members may not correctly filter the prefixes that are based on the SoO value at the border.

Workaround: BGP supports Dynamic Update peer groups, which ensure that packing is as efficient as possible for all neighbors regardless of whether or not they are peer-group members.

Peer groups simplify configurations, but peer-templates provide a much more flexible solution to simplify the configuration than peer groups.

If the SoO configuration is applied directly to the neighbor or to a template, the symptom does not occur. Using templates to simplify the configuration is a better solution and Dynamic Update peer groups ensure efficiency.

Symptoms: A label mismatch may occur between the CEF table and the BGP table, and a new label may not be installed into the CEF table.

Conditions: This symptom has been observed after a BGP flap has occurred on a Cisco router that is configured or MPLS VPN but that does not function in an inter-autonomous system and that does not have multiple VRFs.

Workaround: There is no workaround. After the symptom has occurred, enter the clear ip route command for the affected VRF.

Symptoms: OSPF stub-router advertisement feature doesn't work with RPR+ and SSO when RP switchover took place. After RPR+/SSO switchover took place, the newly active RP never advertise max-metric router LSA.

However, if NSF doesn't configured on the switch, it is desirable to advertise max-metric router LSA.

–

The max-metric router-lsa command is configured for OSPF in router configuration mode.

Workaround: If RP redundancy mode is configured as RPR, then OSPF stub-router advertisement feature work correctly.

Symptoms: After giving the command traceroute ip x.x.x.x or traceroute x.x.x.x over an LDP-enabled network, the penultimate hop router shows a traceback message in the console log.

Conditions: This symptom has been observed when the mpls ip option is configured in the network and for the core trunks.

Workaround: The issue seems to be avoided if you instead enter the traceroute mpls ipv4 A.B.C.D/32 command.

Symptoms: The OSPFv3 cost onPortChannel interfaces calculated based on the interface bandwidth may not be correct if the auto-cost reference-bandwidth command is used under the ipv6 ospf router mode..

Conditions: This symptom has been observed when the auto-cost reference-bandwidth command is used under the ipv6 ospf router mode.

Workaround: Use the shutdown command followed by the no shudown command on the PortChannl interface.

Miscellaneous

Symptoms: A Cisco router may reload when you enter the show standby or show standby brief command.

Conditions: This symptom has been observed on a Cisco router when multiple HSRP groups are configured and unconfigured in a loop while traffic for the HSRP groups is being processed.

Note, however, that a stress scenario in which many HSRP groups are configured and unconfigured while the show standby or show standby brief command is being entered may be a rather uncommon scenario.

Workaround: Do not to enter the show standby or show standby brief command while configuration changes are being made.

Symptoms: When you enter the clear cef linecard command, IDBs may be hold on the line card. When you enter the clear cef linecard command, the number of IDBs that are used on the line card increases.

Conditions: This symptom has been observed on a Cisco platform when a GRE tunnel interface is used.

Further Problem Description: When many tunnel interfaces are used and when the clear cef linecard command is entered several times, the SWIDB number may reach its upper limit. If this situation occurs, you must reload the platform.

Symptoms: All packets may be dropped across a T1 or E1 link on which class-based shaping is configured.

Conditions: This symptom has been observed on a Cisco 7200 series that has an NPE-G1 and that runs Cisco IOS Release 12.2(28)SB.

Symptoms: Cisco IOS router supporting BFD (Bi-Directional Forwarding Detection) will be able to configure minimum receive interval as low as 1 ms.

Conditions: If BFD minimum receive interval is configured below 50 ms, it can cause undesirable effects on the router. Also minimum receive interval below 50 ms is not supported in Cisco IOS.

Symptoms: Dual RP GSR reached an unreachable state during maintenance window, while new interfaces were being commissioned.

Once in this state, most of the interfaces on the router were down/down, and could not be brought up by shutting down and restarting or by reloading the line card.

Conditions: The memory leak was found in a function call made by the write memory command when NTP is running and/or the clock calendar-valid command has been configured.

Workaround: The router was reloaded to recover from this state. (RP switchover was also found to recover the router).

Further Problem Description: For these instances, the show controller csar queues command showed that the Free Queue available are 0, and all F/Q have been allocated and not released.

Symptoms: The standby RSP crashes in loop just after config sync by TLB exception

Conditions: This symptom has been observed on a Cisco 7500 with dual RSP4+ running Cisco IOS Release 12.0(31)S2 and configured for SSO.

Symptoms: When performing the show policy-map interface MFR command, the counters do not increment.

The counters in show policy-map interface MFR do not increment for any type/class of service. Even the class-default shows 0 packets. The counters in show frame-relay pvc show the packets correctly.

Conditions: This symptom has been observed when a map-class is configured under an MFR (FRF.16) bundle (sub- interface). This map-class consists of both an input and output service-policy.

Symptoms: A FRR backup tunnel undergoes reoptimization, resulting in the teardown of the old lsp that is carrying traffic for primary lsps that have cutover to the backup tunnel.

Symptoms: A customer running a Cisco 12000 series router with a Cisco 12000-SIP-400 and SPA-4XCT3/DS0 and SPA-4XT3/E3.

Conditions: This symptom has been observed when a CE connected to the SPA-4XCT3/DS0 via a Multilink Frame Relay interface if the interface is shut down. When the interface is restarted, interfaces on the SPA-4XT3/E3 flap.

Symptoms: ATM PVCs that are configured for OAM may go down because OAM cells are not properly received on the ATM PVCs. When this situation occurs, the ATM interface remains in the up/up state, but the subinterfaces on which the PVCs are configured enter the down/down state. Subinterfaces on which PVCs are configured without OAM remain in the up/up state, but traffic does not pass.

Conditions: These symptoms are observed on a Cisco Catalyst 6500 series switch that runs Cisco IOS Release 12.2SXF. However, the symptom is platform- and release-independent.

Workaround: Reset the affected interface or reload the line card on which the affected interface is configured.

Symptoms: For local switching MFR configuration, on a GSR with a CT3 SPA, packets greater than 1500 bytes are not being forwarded.

Conditions: This symptom is observed on hardware-based E5 engine line cards on a Cisco IOS Release 12000 series that is configured for Local switching

Conditions: This symptom is observed when dMLP+QoS+IPHC is configured on a Cisco 7500 series router and tries to bring up the standby.

Symptoms: A router may crash when you enter the show policy-map interface command while an automated script completes the policy map and then removes the policy map during cleanup.

Conditions: This symptom is observed on a Cisco router when you enter the show policy-map interface command while, at the same time, the automated script removes the policy map.

Symptoms: The line card of a Cisco 12000 series router reloads without creating a crashinfo file. %LCINFO-6-ZERO_BYTE_CI_FILE error may be reported.

–

Line card reloads when it receives from the network an MPLS packet with explicit NULL label.

Workaround: Reconfigure explicit null (mpls ldp explicit- null) from all routers in the VRF.

Further Problem Description: Explicit NULL label advertising is disabled by default in Cisco IOS.

Symptoms: Interface counters are double counted for tag-tag switching. This is seen on Cisco 12000 series router.

Symptoms: A Cisco 12000 router with an 8OC03/ATM/TS-IR-B 8-port OC-3 ATM Line card will reload (or require a manual reload) after the line card console is flooded with %QM-4_STUCK and %PM622-2-SAR_POLL_TIMEOUT.

Conditions: This symptom has been observed when LSP paths change or when the core link flaps in the network.

Symptoms: When a VC is down, the output of the show connection command on the local side shows that the VC is up, even though the output of the show mpls l2 vc detail command shows that the VC is down. The output of the show connection command on the remote side shows that the VC is down.

Conditions: This symptom is observed on a Cisco router that is configured for AToM when the MTU mismatches the Virtual Private Wire Service (VPWS) circuit.

Symptoms: Engine 3 4 x 1 GigE line card stops receiving any traffic. All the packets ingressing on the Gigabit interface are counted as overruns (rfifo_full).

1) Misconfiguration in switched network caused broadcast storm to be flooded into the Gig port and the port entered into stuck state and did not forward any traffic. All other ports on the same LC were working fine.

2) In a lab scenario, the problem could be reproduced with line rate bidirectional traffic of small sized packets (64 bytes or less) with some ARP packets prevented the Gig port for receiving any further traffic with rfifo_full incrementing.

Further Problem description: This issue can be identified with following commands.

Check for increments in the following counters GigabitEthernet 0 denotes port "0" in the LC

LC-Slot3#show controllers Gigabitethernet 0 mac | inc frmLstIntRcvMacErr frmLstIntRcvMacErr: 0x22B92568CF

LC-Slot3#show controllers Gigabitethernet 0 mac | inc rfifo_full 0 risl, 0 riq, 0 rdrop, 0 rsupp, 0 rinvalid_encap, 149148306788 rfifo_full

The RX queue pointers does not change while executing the below command repeatedly for few times

Conditions: This symptom has been observed in SSO mode, not RPR+. %TAGCON-3-LCLTAG_ALLOC: Cannot allocate local tag

Symptoms: When customer uses one MPLS-TE tunnel for EoMPLS traffic and add and bring up new tunnel which does not use for traffic, unexpected traffic drop happened on EoMPLS.

Condition: This symptom has been observed when both TE-tunnel destinations are the same. Traffic drop happens at first tunnel up after changing pseudowire-class. No problem at subsequent flap.

Symptoms: MBUS ram agent fails to allow IDEEPROM on some cards to be updated - written from IOS, IDS (Manufacturing Diags) and likely IOX as well. Some or all attempts to write to the IDEEPROM will FAIL. This Affects Field Diagnostics, OBFL (Blackbox) , and any functions within IOS that update the IDEEPROM. The list of affected cards appears below.

Conditions: This symptom has been observed when running field diagnostics against any of the following products, The testing may complete successfully (or with errors, properly detecting faulty hardware) and then the Field Diags ceases to complete writing eeprom results back to CSC (or a ny fab card tested).

f-diag5#diag 16 prev

    Field Diag eeprom values: run 2 fail mode 0 (PASS) slot 16

       last test failed was 0, error code 0

    f-diag5#diag 16 verbose

    Running DIAG config check

    Redundancy available: testing permissable, 5 acvive xbars

    Executing Field Diagnostics on fabric cards may cause momemtary loss

    of traffic through fabric once testing has completed

    Verbose mode: Test progress and errors will be displayed

    Runnning Diags will halt ALL activity on the requested slot.[confirm]

    f-diag5#

    PID of f_diag_run is 192, set test_pid[16]

    gdb slot is 0

    Launching a Field Diagnostic for slot 16

    Removing fabric card 16 from active config

    Done removing UUT card from fab

    Test: 1 Iris Registers

    FDIAG_STAT_PASS, test 1 Iris Registers

    Test: 2 Register Test

    FDIAG_STAT_PASS, test 2 Register Test

    Test: 3 Fabric FPGA Registers

    FDIAG_STAT_PASS, test 3 Fabric FPGA Registers

    Test: 4 XCVR Loopback

    00:07:30: %MBUS-6-FABCONFIG: Switch Cards 0x1E (bitmask)

    Primary Clock is CSC_1

    Fabric Clock is Non Redundant

    Bandwidth Mode : 40Gbps Bandwidth

    FDIAG_STAT_PASS, test 4 XCVR Loopback

    FINAL STATUS, PASS

    Field Diagnostic ****PASSED**** for slot 16

    Field Diag results from eeprom before updating slot 16, run# 0x2 were 0x0

    previous field diag eeprom values: run 2 fail mode 0 (PASS)

       last test failed was 0, error code 0

    f-diag5#diag 16 prev

    Field Diag eeprom values: run 2 fail mode 0 (PASS) slot 16

       last test failed was 0, error code 0

f-diag5#diag 16 verbose

Sorry... Field diagnostics already running on requested board

73-10474-01   PCA,MBRD,PSC-1,GSR

73-8812-10    PCA,SUB,PRP2,GSR

73-9615-01    PCA,SUB,CSC160-OSP,GSR

73-9718-02    PCA,SUB,ENHANCED CSC160,GSR12810

73-9719-01    PCA,SUB,ENHANCED CSC256,GSR12816

73-9870-03    PCA,SUB,PRP1,GSR

73-9617-01    PCA,SUB,SFC256-OSP,GSR

73-9618-01    PCA,SUB,CSC256-OSP,GSR

This symptom was found as a result of testing for other DDTS related to Field Diags and EEprom results and related bugs.

Symptoms: In using E5 interface as a CE facing interface, when the ce facing interface is enabled for SNF on output direction, the line card stops forwarding traffic after close to 16k packets have been punted to CPU for SNF.

Conditions: This symptom has been observed when using an E5 interface as a CE facing interface.

Workaround: Remove the SNF configuration from the CE facing interfaces in output direction.

Symptoms: An OC48E/POS-SR-SC-B linecard is experiencing high CPU and soft drops after a Cisco IOS upgrade from Cisco IOS Release 12.0(31)S2 to Cisco IOS Release 12.0(32)SY. This is causing latency in traffic.

Conditions: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.2(28)SB3 and that is configured for L2TP and QoS.

Workaround: Remove the QoS configuration. If this is not an option, there is no workaround.

Symptoms: A policy-map with one class that matches on an extended ACL - used for rate-limiting pings in this case - cannot be applied to an interface on a SIP-600 nor a SIP-601. The following error message is displayed: "SLOT 0:Oct 2 12:49:34.502 EDT: %EE48-5-TM_PROC: Add Profile Mgr failure for intf Serial0/0/3.451: 1" This error message is followed by a traceback.

Conditions: This symptom has been observed when trying to apply the policy-map to an interface on the SIP-600 and SIP-601.

Workaround: Use the rate-limit command at the interface level in conjunction with an ACL to perform rate-limiting.

Further Problem Description: The service-policy will show up in the configuration; however, it does not function.

Symptoms: In a MPLS TE Fast Reroute environment, if a protected link were to be flapped ('down' followed by 'up'), all primary LSPs protected by a backup tunnel and going over the link, would undergo reoptimization (replacement of old FRR Active LSP with new LSP). For 0 bw primary TE tunnels (such as primary auto-tunnels), the new LSP ends up being protected by a suitable NHOP/NNHOP backup tunnel, but when the backup tunnel goes down later for some reason, the new primary LSP isn't re-evaluated and moved off the backup tunnel.

–

A PLR (MPLS TE FRR Point of Local Repair) is running Cisco IOS Release 12.0S or Cisco IOS Release 12.2S.

–

An event that requires the LSP for the backup tunnel protecting the primary's TE LSP, to be torn down.

Conditions: This symptom has been observed when applying policy on control plane of 2 LC of same router.

Symptoms: On ISE ATM LC, after vbr-nrt SCR rate is changed. The PVC needs to be bounced before the traffic gets policed by the new SCR rate.

Conditions: This symptom has been observed when vbr-nrt SCR is changed. Only police function is not working. All traffic in other class queues would use new SCR rate automatically.

Symptoms: After perform a RP switchover on a long idle(20 hrs above) GSR router loaded with 092806 nightly build image, all LCs got reset.

Conditions: This symptom has been observed when all line cards get reset, after RP switchover on a long idle router.

Symptoms: SNMP ifIndex does not exist for MFR subinterfaces on CHOC12/DS1-IR-SC ISE line card after removal and reinsertion (OIR) of the card.

Conditions: This symptom has been observed in Cisco IOS Release 12.0(32)SY1 when using a configuration with a large number of serial interfaces configured, including a 8xDS1 MFR bundle. The MFR interface had a corresponding point-to-point subinterface with one DLCI defined. The subinterface was pollable before removal and insertion of the card, but the ifIndex is missing afterward.

Workaround: By reloading the router, or removing the subinterface and readding it, the ifIndex will be recreated.

Symptoms: Port mode EoMPLS, after CE gigE interface is bounced, the PE Eng3 gigE interface does not stay up and keep on flapping until you bounce the PE gigE interface as well. This problem does not happen when VLAN is configured.

Symptoms: A memory leak and memory exhaustion may occur when QoS policies are updated on 40,000 sessions.

Conditions: This symptom is observed on a Cisco 10000 series but may also affect other platforms.

Symptoms: After removing and re-applying a policy-map with hierarchical class-based policer on a Vlan sub-interface on an Engine5/SIP-601 line card, all policer counters displayed in show policy-map interface are zero despite traffic being sent through the interface, and the policer no longer drops excess traffic when configured to do so.

This symptom could also affect other SIP-601 interfaces/SPA's and other ingress QoS configurations.

There is a hierarchical shaper configured in the outbound direction on the main interface (L3/nC/mD ISE QoS model).

Workaround: There is no workaround other than avoiding the above conditions. A line card reload temporarily fixes the problem, but once the configuration is toggled again, the symptom resurfaces.

Symptoms: Perform a line card reload on an Engine 0 CT3 or DS3 line card, upon recovery any frame-relay sub-interfaces on those line cards which have rate-limit output configured will no longer have this applied.

Workaround: Reload of the router or a reload of the ingress line card resolves the issue.

Conditions: MPLS pings and traceroutes are sent from tunnel headend to tail. They were seen failing at penultimate hop when the router had E4+ ingress line card and E5 egress line card. When E4+ is used both as ingress and egress line card, issue is not seen.

Conditions: 2xOC48 SPA and 2x1GE SPA are co-resident in same SIP601 LC. GE interfaces are configured for EoMPLS Pseudowires. To verify CSCek30312, the negotiation auto command was configured on GE interface. The GE interfaces start flapping. OC48 interfaces also start flapping. With no negotiation auto command, GE interfaces stop flapping but OC48 keeps flapping. GE interfaces do not resume forwarding traffic.

Symptoms: After OIR of the SPA-4XCT3/DS0 SPA, some of the frame-relay sub-interfaces no longer forward traffic.

Conditions: This symptom has been observed when an online removal and insertion is done on the SPA-4XCT3/DS0 SPA.

Symptoms: Ethernet frames with layer 2 header comprised of BEEFF00D are sent out of an MPLS TE tunnel during MPLS TE tunnel reconvergence.

Conditions: This symptom has been observed on Engine 3 gigabit ethernet line card of a Cisco 12000 series router. Problem was not present on POS line cards of the same Engine.

Symptom is observed between the time when reconvergence of all tunnels is triggered and the time when reconvergence of the particular tunnel is completed.

Conditions: This symptom has been observed while applying Hw-module ip load-sharing per-packet.

Symptoms: On Cisco 12816/E5 running Cisco IOS Release 12.0(32)S5, packets are lost after activating the standby CSC or a SFC.

Conditions: After shutting down and then activating the standby CSC or a SFC ( ex hw-module slot 16 shut / no hw-module slot 16 shut" there is traffic lost .

Symptoms: In a Cisco 7500 router, when we attach a Service policy to a MFR bundle that has the Serial Interface - member links configured for vip based fair-queueing, the VIP having the Serial Interfaces will crash.

Conditions: The crash is seen when attaching the Service Policy to a MFR bundle with traffic flowing the same.

Workaround: Disable the fair-queue command in the serial interfaces before adding them as member links to the MFR bundle.

Symptoms: For both 1xchoc12-ds1 and 4xchoc12-ds3, deleting the channel informations under Sonet controller, the definitions are removed on the primary RP but not the standby RP.

Symptoms: After an RPR+ switchover traffic ingress on a SIP-600 and egress on 4xGE ISE LC or 10xGE E4+ LC was being forwarded on the wrong VLAN.

Symptoms: On a Cisco 12816 running Cisco IOS Release 12.0(32)S5, the E6 card continues to be crashed to IPC timeout when deactivating the standby CSC and activating the CSC. The same problem occurs when deactivating a SFC and activating the SFC.

E6 card continues to be crashed to IPC timeout which was caused by continuosly incremental CRC errors in FIA from output of the show controller fia command until the whole Cisco 12816 chassis is reload.

Symptoms: GSR Engine 2 -8xOC3 ATM LC ports stops traffic forwarding with %QM-4-STUCK error.

Conditions: This symptom has been observed when there are bad packets hitting the GSR router, especially those packets with sizes that GSR's ATM SAR device can not handle. They include the sizes 1 byte to 4 bytes small. Unexpected large packets also might cause similar issues. L2VPN configuration exists on the Engine 2 ATM line-card and there are AAL0 VCs (encapsulation aal0). Please note that these packets are errored or corrupted packets and are not intentionally generated by regular traffic.

Symptoms: QM-STUCK or QM-BUFFER-SANITY messages are logged continuously on Engine 2 ATM Line card of Cisco 12000 series router. The traffic in the egress direction of this line-card is impacted and all packets are dropped on the Tx BMA.

It is very difficult to execute any command on the line card due to slow response and the above continuous messages.

Conditions: This symptom has been observed when there are bad packets hitting the ATM SAR device, especially those packets with sizes that ATM SAR device can not handle. They include the sizes 1 byte to 4 bytes small. Unexpected large packets also might cause similar issues. L2VPN configuration exists on the Engine 2 ATM line-card and there are AAL0 VCs (encapsulation aal0). Please note that these packets are errored or corrupted packets and are not intentionally generated by regular traffic.

Symptoms: Certain packet corruption which changes bufhdr length to 0 can result in a line card traffic disruption because of Alpha errors.

Conditions: This symptom has been observed with a Cisco 12000 router, Engine-3 line card, and multicast traffic.

Symptoms: Egress packet rate is doubled due to fragmentation. This can be seen in the output of the show ip mroute vrf vrf-name active command.

Conditions: This symptom has been observed only in multicast VPN environment for 1500 bytes packets when the egress linecard is based on Layer 3 Forwarding Engine 3 or 5.

Symptoms: MPLS OAM ping and traceroute to TE tunnel tail fail at penultimate hop

Conditions: This symptom has been observed when the penultimate hop P router has 4 port OC48 E4 line card. The egress line card E4+ is 4 port OC48. Move the egress fiber to another port on the ingress E4 line card and problem is gone.

Symptoms: A router with a DPT uplink is stuck in reload loop. Every time the router reloads a crash occurs.

Conditions: This symptom has been observed when some DPT uplinks when using images that have the latest version of the Rx FPGA (for example: 32.SY1, 32.S5).

Symptoms: The GSR systems incorrectly detect removal of all cards in the system, including the Active RP. If a Standby RP is present it will automatically assume Active role but it will also not see any other cards in the system. User-initiated or automatic system reload attempts will fail, the system being unable to come back up. Only a power cycle of the entire chassis will restore the system to normal operation - the only difference from the case in which a _permanent_ h/w failure affecting the MBUS CAN, not covered by this bug.

Conditions: This symptom has been observed in extremely rare occurrences, with no apparent reason. The only theory that could explain the symptoms and was successfully simulated in the lab is a temporary failure in the MBUS h/w circuitry.

Workaround: No workaround can presently prevent the service interruption. A manual power cycle of the chassis would restore normal operation.

Further Problem Description: The MBUS is the backbone of the GSR - a shared CAN bus connecting all cards in the system. A jam condition on this bus would affect communication between all cards, effectively bringing the system down with at least the symptoms described above. The jam condition can happen only if a permanent or temporary failure occurs in the MBUS circuitry. There is nothing the software can do if the failure is permanent, not even a chassis power cycle would correct the problem. Sequenced physical OIRs for all cards in the system would be the only way of identifying the bad one. But at least in some of the temporary failure cases there are chances of software-driven, automatic recovery, which would reduce or maybe even eliminate completely the service interruption. This bug was raised to track the implementation of this particular recovery solution.

Symptoms: An errmsg of type: IDBINDEX_SYNC-3-IDBINDEX_ENTRY_SET for an interface. And "show idb" shows an if-index value of -1 for one or more IDBs on either the Standby or Active RP.

If this happens on a Standby RP there is no affect on traffic. However if the RP switches over to become Active it will prevent traffic from flowing on the affected interfaces.

Conditions: This symptom has been observed when a platform has a bug such that OIR insertion notifications are synced to the Standby RP before the corresponding interface index values have been synced. The normal order is to always guarantee the index values arrive first.

Workaround: A workaround if this happens on an HA protected Active RP (which affects traffic) is to check whether the Standby RP has good if-index values for all interfaces by running the "show idb" EXEC command on the Standby RP. If so, then do an RP switchover, so the RP with good interface indexes becomes the Active RP.

If the Standby RP shows this symptom, the workaround is to reload the Standby RP and check that after it comes up it has good interface index values, which should happen in most cases.

Further Problem Description: This DDTS is to provide a platform-independent code workaround that allows the interface index values to self-recover after the correct if-index values are synced to the Standby RP.

If the condition is seen on an Active RP, this DDTS fix will allow it to recover following an OIR deletion/insertion rather than remaining in the error condition.

The root-cause of the incorrect syncing order will still need to be fixed by the platform that has this symptom. But this DDTS will lower the severity by allowing it to self-recover in most cases on its own without user intervention.

Symptoms: Submitting the clear counters command crashes the 6CT3-SMB linecard.

Conditions: This symptom has been observed when entering the clear counters command.

Symptoms: After RPR+ forced switchover, traffic is no longer passed across the affected frame-relay subinterfaces on the CT3 SPA. Commands run on SPA LC indicate the incorrect mapping of if_number.

Conditions: This symptom has been observed with RPR+ switchover, BGP running, routes injected and 180,000 routes set up.

Wide-Area Networking

Symptoms: When an attempt to move an interface from one multilink group to another fails due to platform-specific limitations, it leaves the interface in an invalid state. The multilink-group command still appears in the interface configuration, but the interface does not appear in the output of show ppp multilink.

Conditions: This symptom can occur on platforms that support distributed implementations of multilink (such as the Cisco 7500, Cisco 7600, Cisco 10000, or Cisco 12000), and the platform does not allow the interface to be added to a multilink group for some reason (e.g. resource constraints).

Workaround: Use the no multilink-group command to remove the interface from its current multilink group before adding it to a new one.

Resolved Caveats—Cisco IOS Release 12.0(32)S5

Cisco IOS Release 12.0(32)S5 is a rebuild of Cisco IOS Release 12.0(32)S. The caveats listed in this section are resolved in Cisco IOS Release 12.0(32)S5 but may be open in previous Cisco IOS releases. This section describes only severity 1, severity 2, and select severity 3 caveats.

Basic System Services

Conditions: This symptom is observed after you have entered the microcode reload command.

Workaround: Enter the shutdown interface configuration command followed by the no shutdown interface configuration command on the affected interface.

IP Routing Protocols

Symptoms: Mal-formatted MDT updates with 0 masklen are sent to the neigbor PE. It is denied, and PIM neigborship cannot be established properly over the MVPN GRE tunnel.

Symptoms: When a First Hop Router receives (S,G) stream for an Embedded RP group, the router crashes while trying to send register packets.

Miscellaneous

Conditions: This symptom is observed when Null0 interface is specified as the default output in local policy routing.

Symptoms: The Cisco IOS TACACS+ is not able to communicate with a TACACS+ server.

Conditions: This symptom occurs when no authentication and encryption key has been configured.

Conditions: This symptom has been observed when the following two configuration commands are active at the same time:

This symptom causes a large memory leak each time the configuration is written. If neither configuration or only one configuration command is present, then there will not be a memory leak. This symptom affects RPM-PR and RPM-XF platforms.

%SYS-2-NULLCHUNK: Memory requested from Null Chunk -Process= "Init", ipl= 3, pid= 3 
with an accompanying traceback.

Conditions: This symptom is platform- and release-independent and occurs when the router boots.

Workaround: There is no workaround. However, proper system operation is not affected.

Symptoms: A E5 or E5+ line card crashes when applying a 40K ACL which is used to filter the rate-limit statement.

Conditions: This symptom has been observed on a Cisco 12000 series Internet router which contains 40K ACL filters on E5 or E5+ CAR statements.

Symptoms: The counters for the show ip mroute vrf vrf-name count command and the show ip pim vrf vrf-name interface count command show huge counts in the decapsulation direction.

Conditions: This symptom has been observed on systems running Cisco IOS Release 12.0(32)S4 after the core facing interface on the local OR remote PE is flapped.

Workaround: The clear ip pim vrf vrf- name count command would clear this condition for the respective counters. For the show ip mroute vrf vrf-name count command counters, either the route should be cleared or timed out for these counters to reset.

Symptoms: Reloading a dual-rp router, where engine 0 6xCT3 or engine 3 1xChOC12 is in the hw-module shutdown state, may cause %IDBINDEX_SYNC-3- IDBINDEX_ENTRY_LOOKUP and some tracebacks.

Conditions: This symptom has been observed on a Cisco 12000 series dual-prp router running Cisco IOS Release 12.0(32)S.

Symptoms: Multicast states are created but Multicast routing instances are not found on the line card.

Conditions: This symptom has been observed when the hosts join the multicast group.

Symptoms: A Cisco 7200 series router that is running Cisco IOS Release 12.3(17) may experience an Output stuck condition on PVCs that are running on PA-A3-8T1-IMA. The condition results in all traffic over affected PVCs ceasing to pass.

Interface ATM1/ima0 VC 1/41

 Queueing strategy: fifo

 Output queue 40/40, 9156 drops per VC

Issue is reproducible in TAC Labs that are running Cisco IOS Release 12.3(17a).

Issue is not reproducible in TAC Labs that are running Cisco IOS Release 12.4(5a).

During the problem, after interfaces are wedged, doing the shut command followed by the no shut command on the logical IMA interface results in the interface showing down/down (disabled).

Condition appears in all Cisco IOS versions that contain the fix for CSCee20451.

Symptoms:The MFR bundle flaps after taking out all of the members and then adding them back.

Conditions: This symptom has been observed when taking out all of the members and then adding them back.

Symptoms: A 2OC192 Engine6 line card always samples less ingress netflow from total input packets.

Symptoms: After a router reloads, traffic with frame size of 1615 bytes may not pass through MLPPP bundle configured with Link Fragmentation and Interleaving (LFI).

Conditions: This symptom is observed on a Cisco 12000 series router that is running the c12kprp-p-mz image of Cisco IOS Release 12.0(32)SY.

Workaround: Remove LFI on both sides of the MLPPP link and re add LFI to both sides of MLPPP link simultaneously.

Symptoms: The hardware addresses of 4xGE are varying every time when attempting to perform RPR+ switchover.

Additional Notes: If we take out the standby RP and execute the show gsr chassis detail command , with single RP and still we are not able to read then the problem may be caused by a misprogrammed EEPROM, otherwise it should be CSCse48477.

Conditions: This symptom has been observed when applying a service policy on E4P Gigabit subinterface with Xconnect.

Symptoms: Adding uRPF to the main Frame Relay interface will break sub- interfaces added afterwards. Removing uRPF from the main interface does not clear the symptom.

Conditions: This symptom has been observed on a Cisco 12410 router with a SIP- 601/CT3 SPA line card running Cisco IOS Release 12.0(32)S3.

Workaround: uRPF should not be applied to a main interface that is supporting sub-interfaces, so avoiding this negative scenario will prevent this symptom. If this scenario is encountered, shutting down and restarting the sub- interface should clear the issue.

Symptoms: Incorrect Legacy QoS behaviour occurs on subinterfaces configured on Engine 0 and Engine 1 line cards. When this error occurs, QoS appears to be applied even though it hasn't been configured.

Conditions: This symptom has been observed when configuring a new subinterface on a line card after another subinterface with QoS was deleted.

Symptoms: Unexpected TE FRR status occurs. TE tunnel traffic is not forwarded on expected interfaces. FRR is active when it should have returned to Ready mode. The FRR state on the RP is not the same as the line cards, as seen by the following commands:

Conditions: This symptom has been observed on Cisco 12000 Series Internet Router distributed platforms with Interface or Hello events that activate TE FRR. With TE tunnels, manually configured tunnels are affected, but the symptom is more likely to be seen with auto-tunnel primary and auto-tunnel mesh.

Workaround: There is no workaround. The router must be reloaded, or each line card reset.

Further Problem Description: There is a data table in memory on the RP and each LC that contains the current status and forwarding information for FRR protection. The data should always be in sync, so that the RP and line cards have the same FRR state, but with this symptom, the tables on the line cards become out of sync with the RP, and contain stale information. The situation becomes worse over time as more stale entries are created. The tunnel items are not deleted from the TFIB/FRR database when a tunnel ID is reused by the RP to signal a tunnel to a new destination. These old entries may get chosen for FRR instead of the latest, correct ones, causing incorrect FRR forwarding and incorrect FRR status. Because the tunnels involve multiple destinations and multiple interfaces, with out-of-date forwarding, interface transitions on any TE interface may cause unexpected traffic on any other TE interface. Also, because it is not expected that multiple entries exist for a single tunnel ID, it is difficult to predict which entries will be used, and whether or not an interface event will result in incorrect forwarding.

Symptoms: SIP401, SIP501, SIP600, or SIP601 reloads after any of these conditions:

–

The symptom happens only in Cisco IOS Release 12.0(32)S release and above and is not applicable to Cisco IOS Release 12.0(31)S.

Workaround: Ensure that backup links are not E2, E4, E4+, or E6 under these FRR conditions.

Symptoms: Disabling scrambling on an OC48-POS SPA will be lost if a router reload occurs, resulting in a failed link.

Conditions: This symptom is observed with SPA-2xOC48POS/RPR in an Engine 5 Cisco 12000-SIP-601 or Cisco 12000-SIP-600 line card in a Cisco 12000 series router. Additional testing indicates that all supported POS SPA's are affected as shown in this list:

Symptoms: When you apply an input service policy to an AToM PVC, a router may reload and generate the following error message and traceback:

Unexpected exception to CPUvector 300, PC = 119B6D0

-Traceback= 119B6D0 118E2F8 5952270 118FDC4 11B7680 11B78EC 236988 24BDD4

2E95CC

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(32)S3 but is platform- and release-independent. The symptom occurs when you enter the following commands:

Router(config)#interface x/y.z point-to-point

Router(config-subif)# no ip directed-broadcast

Router(config-subif)# no atm enable-ilmi-trap

Router(config-subif)# pvc a/b l2transport

Router(cfg-if-atm-l2trans-pvc)# encapsulation aal5

Router(cfg-if-atm-l2trans-pvc)# xconnect a.b.c.d xy encapsulation mpls

Router(cfg-if-atm-l2trans-pvc-xconn)#

Router(cfg-if-atm-l2trans-pvc-xconn)#service-policy test

Symptoms: After failover of the primary RP to secondary RP, errors occurs on slot0 which is a 4 port Gigabit Ethernet Edge Engine 3 card on the Cisco GSR and the 4-port card stops forwarding outbound traffic.

Slot 0  type  = 4 Port ISE Gigabit Ethernet

        state = IOS RUN   Line Card Enabled

Conditions: This symptom has been seen with either a hardware OIR or a redundancy force-switchover command issued to failover the primary to secondary RP. The symptom seems to happen if there is traffic on the ports of the 4-port card.

Symptoms: The router may reload when process-level packets are sent to multiple interfaces with QoS enabled, and the user attaches or removes QoS policy from one of the interfaces under traffic.

Conditions: The symptom happens when there are continuous streams of process- switched or router-generated packets during the QoS configuration change.

Symptoms: Router may reload when both fastswitched and CEF switched packets and process-level packets are congesting an interface that has QoS enabled.

Conditions: The symptom is seen when the traffic consists of large packet size greater than 512 bytes and is reported on PA-4T/8T+ interface only. It is not seen on channelized interfaces.

EE48-3-QM_ERR_DECODE: FrFab QM_TX_ERRORS qm_oqdq_halted

This error will reset the forwarding and queueing ASICs resulting in small traffic disruption.

Conditions: This symptom occurs when the ports are dynamically channelized and/or unchannelized.

Workaround: Micro-reload the line card after channelization or unchannelization.

Symptoms: The output of the show policy-map interface interface-name vp vpi input command for an ATM interface does not show anything and states that the policy is not configured. However, the output of the show running-config command does show the service policy for the ATM interface.

Conditions: This symptom is observed on a Cisco router after an RP switchover has occurred twice.

Conditions: An LSP being reoptimized causes a label to get reused. This may cause the packet to be switched to an incorrect destination for a brief period. This brief period, usually about 1 or 2 seconds, is the time taken for the label for the old prefix to be withdrawn from other routers. It should be noted that the label reuse doesn't always occur.

Symptoms: Keep the T1 controller in the �up� state, and when reloading the router in the running configuration after the router comes up, the T1 controller has been shut down.

Conditions: This symptom has been observed on Cisco IOS Release 12.0(32)S and Release 12.0(32)SY.

Workaround: To bring the T1 controller up, use the no shutdown command on the controller. There is no workaround.

Symptoms: Incorrectly attaching a service policy under the ATM subinterface and then re-adding interfaces with service policies will cause the latter interfaces to not have QoS applied. This can be avoided by correctly adding service policies under the PVC configuration only.

Conditions: This symptom has been observed when applying service policies to the subinterface directly.

Workaround: The issue can be avoided by not applying service policies to the subinterface directly and instead applying the service policies under the PVC configuration. Shutting down and restarting the affected interfaces is required to clear the issue.

Symptoms: A CHSTM1 controller TUG-2 E1 remote alarm does not clear until the controller is shut down. This symptom happens only on an unframed E1.

Conditions: The receiver must be receiving a remote alarm, while the E1 framing is changed from framed to unframed.

Workaround: Enter the shutdown command and then the no shutdown command on the controller to clear the remote alarm condition.

Symptoms: A 4xOC3 ATM ISE line card stops process packets in the small buffers queue (80 byte queue). All traffic ceases and the ATM PVCs configured over the line card is set to a down/down state. Resetting the LC clears the buffers and resolves the issue, but it reoccurs. There appears to be a slow leak or alternately sporadic bursts of traffic with malformed 80-byte packets that fill the buffer and cause the issue to reappear.

Workaround: Micro-reload of line card solves the issue temporarily, but it reappears.

Symptoms: Output rate limit is not working for the new rate limit configured for the interface.

Conditions: This happens when an egress line card used does rate limit in the slow path (e.g.: Engine 0 line card). This occurs only when a new rate limit configuration is added after the router is brought up. If the rate limit on the egress is already present during the router bringup, then the issue will not be observed. When Engines that implement rate limit in hardware like E3/E5 are used as the egress, then the issue will not occur.

Workaround: The ingress line card would need to be reset for the new output rate limit configuration on the egress to take effect. If the rate limit on the egress is already present during router bringup, then the issue will not be observed.

Symptoms: The ACL actually applied on a line card interface is different than what is configured on the RP.

Conditions: This symptom has been observed when changing ACLs with the ip access group command applied to interfaces of a line card very quickly. The following examples shows the initial configuration and the configuration which has this symptom.

interface GigabitEthernet 2/0/0

   ip access-group 116 in

interface GigabitEthernet 2/0/1

   ip access-group 116 in

interface GigabitEthernet 2/0/2

   ip access-group 116 in

interface GigabitEthernet 2/0/3

   ip access-group 115 in

interface GigabitEthernet 2/0/0

   ip access-group 115 in

interface GigabitEthernet 2/0/1

   ip access-group 115 in

interface GigabitEthernet 2/0/3

   no ip access-group in

Workaround: Add a slight delay between interface configuration changes. This example shows the changed configuration which has no issues:

interface GigabitEthernet 2/0/0

   ip access-group 115 in

       <========== Couple of sec delay

interface GigabitEthernet 2/0/1

   ip access-group 115 in

       <========== Couple of sec delay

interface GigabitEthernet 2/0/3

   no ip access-group in

Symptoms: On a Cisco 12000-series Internet router running Cisco IOS Release 12.0(32)S or later, a SIP-600 or SIP-601 line card may exhibit the following error and software-switch packets:

*Sep 13 02:39:55 KST: %QM-2-TCAM_ERROR: TCAM pgm error(46): LC based QOS Mgr failed

*Sep 13 02:39:55 KST: %QM-4-SW_SWITCH: Interface GigabitEthernet3/0/0.36 routed 
traffic will be software switched in egress direction(s)

Conditions: This symptom occurs when the following are present on the same line card.

A policy-map configured on DOWN (but not shutdown) interface and the same policy-map configured on a SHUTDOWN interface.

If any UP interfaces also have the policy-map configured, the error may be seen.

Configure a separate policy-map (with different name) on each interface on that LC. The problem is only seen when interfaces are sharing the same policy.

Note

This may not be feasible for policies using complex class-maps due to increased resource utilization, especially when ACL matching is used.

Ensure SHUT interfaces never have a policy configured. Remove policies before shutting down any interface. Bring interfaces up before applying a policy which is also present on other interfaces.

Reload the line card. This needs to be done once each time the router is reloaded (after it comes up completely). If the LC has been reloaded since the most recent router reboot, the problem will not be seen.

If the error has already been seen on the router, possible modes of recovery are:

Reloading the LC will recover the LC and prevent any further problems while the router remains up. One of the other workaround should also be selected to prevent the symptom when the router is later rebooted.

Shut down all affected interfaces. Shut down any interfaces which are in the DOWN state and have the policy attached. Remove the policy from all the shutdown interfaces. Now the policy can be reapplied and interfaces brought back up.

Symptoms: The 4-port engine-3 based (edge engine) gigabit ethernet line card has missing RX side hardware entries when performing the show ip hardware-m rx vrf vrf-name group IP command.

TCP/IP Host-Mode Services

Symptoms: PAKPRIORITY flag is not set on the LDP TCP session. The mpls ldp tcp pakpriority command is configured and LDP session is restarted.

As result LDP TCP session may be prioritized over non important traffic. Consequently, under sustained traffic, LDP TCP packet might be dropped and LDP session reset.

The LDP session reset has been seen in specific conditions with E4+ and MQC classes matching the mpls experimental command (and not the ip precedence command).

Workaround: LDP TCP packets are marked with precedence-6 packets, therefore configuring a class matching on precedence 6 traffic will differentiate LDP traffic in a separate class from non important traffic.

Further Problem Description: MQC in combination with E4+ line cards do not allow matching both the mpls experimental command and the ip precedence command but only one of the two. This situation causes this symptom in the MPLS environment that requires MQC policies to be configured with mpls experimental to match indifferently on IP and MPLS transit traffic.

Resolved Caveats—Cisco IOS Release 12.0(32)S4

Cisco IOS Release 12.0(32)S4 is a rebuild of Cisco IOS Release 12.0(32)S. The caveats listed in this section are resolved in Cisco IOS Release 12.0(32)S4 but may be open in previous Cisco IOS releases. This section describes only severity 1, severity 2, and select severity 3 caveats.

Basic System Services

Symptoms: Spurious memory access errors and tracebacks may be generated on a Cisco AS5800.

Condition: This symptom is observed on a Cisco AS5800 that processes TCPclear calls.

IP Routing Protocols

Symptoms: When you modify an access control list (ACL) by entering the ip multicast boundary command, the command may not fully take effect.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(28)S4 or Release 12.0(32)S but appears to be platform- and release-independent.

Conditions: This symptom has been observed following a switchover from the primary RP to the secondary RP.

Conditions: This symptom is observed on a Cisco router that functions as a PE router when you apply an export map to a VRF and when the interface that connects the PE router to a CE router is configured for OSPF.

Workaround: Enter the clear ip ospf process command to enable the BGP Scanner to pick up the changes in the export map.

Symptoms: A router may delete the VPNv4 prefixes from the BGP table, even though the counters in the output of the show ip bgp command may indicate that the VPNv4 prefixes are present in the BGP table. This situation may cause loss of VPN connectivity.

Conditions: This symptom is observed on a Cisco router that is configured for MPLS VPN and that functions as a PE router.

Workaround: There is no workaround. When the symptom occurs, enter the clear ip bgp * command to restore proper operation of the router.

Symptoms: When a VRF route is redistributed into the MP-BGP cloud, a routing loop may occur for the prefix (that represents the VRF route) between the EIGRP cloud and the MP-BGP cloud.

Conditions: This symptom is observed on a Cisco router that functions as a PE router when the following conditions are present:

–

The router has a static VRF route that is redistributed into the configuration that is defined by the address-family vrf vrf-name command and that is part of the BGP routing process.

Workaround: There is no workaround. Applying a route map with a pre-bestpath option does not resolve the loop.

Symptoms: A route is not installed through an MPLS TE tunnel even though the tunnel mpls traffic-eng autoroute announce command is enabled.

Conditions: This symptom is observed on a Cisco router that also has the mpls traffic-eng multicast-intact command enabled.

Symptoms: A router crashes when you enter the show ip bgp pending-prefixes command.

Conditions: This symptom is observed on a Cisco router that runs an interim release for Cisco IOS Release 12.0(32)S and that is configured for BGP

Symptoms: A router may reload unexpectedly because of a bus error crash after you have removed a summary-prefix IPv6 OSPF command.

Conditions: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.2(18)SXF but may also occur in other releases. The symptom occurs only when the summary-prefix IPv6 OSPF command is configured without any redistribute commands.

Workaround: Configure a redistribute command under the IPv6 OSPF configuration.

Symptoms: If Spatial Reuse Protocol (SRP) is used, Enhanced Interior Gateway Routing Protocol (EIGRP) does not respond to the ring drop notification from the interface.

Symptoms: When a route map is configured, routes may not be filtered as you would expect them to be filtered.

Conditions: This symptom is observed on a Cisco router that is configured for BGP and that functions in an MPLS VPN environment.

Further Problem Description: The symptom does not occur for redistributed route maps.

Miscellaneous

Symptoms: A traceback and 100-to 600-ms traffic loss may occur on an Engine 3 line card while FRR rewrite and reoptimization occurs.

Conditions: This symptom is observed on a Cisco 12000 series that functions as a PE router and that has a multihop TE tunnel to a P router. The multihop TE tunnel is configured for FRR protection and VPNv4 traffic. The symptom may be platform-independent.

Symptoms: MPLS TE LSPs may not come up and may remain stuck in the RSVP signaling proceeding state.

Conditions: This symptom is observed on a Cisco router when the MPLS TE LSPs are processed over inter-autonomous system broadcast links on which the passive-interface command is enabled.

Further Problem Description: The passive-interface command contains the router ID of the remote Autonomous System Border Router (ASBR). A PATH message that leaves the passive interface of the Cisco router is sent to remote ASBR, causing an ARP request to be initiated for remote ASBR. However, there is no response to the ARP request (when there is no proxy configured for ARP), preventing the PATH message from reaching the remote ASBR.

Symptoms: There is no show command to view if an STS-1 in loopback mode on a CHOC12/DS1 line card.

Conditions: This symptom has been observed when an STS-1 in loopback mode on a CHOC12/DS1 line card.

Conditions: This symptom has been observed with nCmD configured on the main interface when running Cisco IOS Release 12.0(32)S. It is a timing issue and does not happen all of the time. The interface flaps causes nCmD configuration to update on the line card. Due to some timing issue, the update events sequence does not get to the LC properly. Therefore, LC resets on invalid memory access. The symptom usually occurs with a large nCmD configuration, since it generates more update events which would have a higher chance to get wrong orders to the LC. Interface flapping is not the only trigger. Modifying the QoS configuration, shutting down and restarting the main interface or sub interface, and any events triggering the nCmD configuration to be updated on the LC can cause this issue.

Conditions: Remove the RP when the main RP is present, do the switch over, and then replace the card.

Symptoms: An interface of a T3/E3 serial SPA passes traffic even though the output of the show controller command shows that there is a "Loss of Frame" alarm. When you enter the shutdown interface configuration command followed by the no shutdown interface configuration command on the interface of the SPA, the alarm is not cleared.

Conditions: This symptom is observed on a Cisco platform that is configured with a T3/E3 serial SPA.

Workaround: Enter the shutdown interface configuration command followed by the no shutdown interface configuration command on the interface at the remote end.

Further Problem Description: The symptom does not affect proper operation of the platform or the traffic. However, the incorrect alarm status may affect network management utilities.

Symptoms: PIM sessions fail to come up over a Data Multicast Distribution Tree (MDT).

Conditions: This symptom is observed on a Cisco router that functions as a PE router after you have entered the clear ip bgp * command.

Workaround: Enter the clear ip mds linecard linecard-slot-number command. For the linecard-slot-number argument, enter the core-facing slot.

Symptoms: In unidirectional mode, an Automatic Protection Switching (APS) switchover from a protect channel to a working channel may fail because the interface of the working channel remains in the up/down state.

Conditions: This symptom is observed on a Cisco 12416 that runs Cisco IOS Release 12.0(32)S3 and that has two redundant 1-port channelized OC-48 POS ISE line cards that are configured for APS.

Symptoms: On an Engine 3 CH/OC12 card configured for MLPPP, when MTU is mismatched between the two ends, shutting down and restarting causes MTU to be negotiated. However, this action causes buffer recarving, which is undesirable.

Conditions: This symptom has been observed when MLPPP configured mismatched MTU values, such that the new MTU value would be one that triggers a recarve.

Symptoms: Changing an MTU value on an interface may cause the RP to trigger buffer recarve unexpectedly.

Conditions: This symptom has been observed when changing the MTU value using the mtu bytes command. The new bytes value may or may not trigger the buffer recarve, and this behavior needs to be characterized.

Symptoms: The following error messages may be generated on a router that has IP ACL enabled:

Conditions: This symptom is observed on a Cisco router such as a Cisco 7200 series, Cisco 7304, and Cisco 7500 series when a Turbo ACL compilation is configured along with an ACL on an ingress interface and when traffic passes through the ingress interface. The symptom does not affect the Cisco 10000 series.

Symptoms: When MPLS-aware NetFlow is properly configured, it may not capture MPLS-labeled traffic.

Conditions: This symptom is observed on a Cisco router when MPLS-aware NetFlow is configured via the ip flow-cache mpls label-position label-position-1 global configuration command and when NetFlow is enabled on the interface via the ip route-cache flow interface configuration command.

When MPLS traffic passes over the interface that is configured to capture MPLS-aware NetFlow statistics, MPLS-aware NetFlow should capture label information for the traffic flow, but it does not because the MPLS switching vectors are not properly updated. You can verify this condition in the output of the show mpls interfaces privileged EXEC command. The interface that is configured to capture MPLS-aware NetFlow statistics should show "Feature" switching vectors, but instead shows "Fast" switching vectors:

Workaround: Re-initialize the switching vectors by flapping the interfaces. Enter the shutdown interface configuration command followed by the no shutdown interface configuration command on the interface that is configured to capture MPLS-aware NetFlow statistics.

First Alternate Workaround: Re-initialize the switching vectors by toggling MPLS. Enter the no mpls ip interface configuration command followed by the mpls ip interface configuration command on the interface that is configured to capture MPLS-aware NetFlow statistics.

Second Alternate Workaround: Re-initialize the switching vectors by toggling any MPLS debug mode. For example, enter the debug mpls packets command followed by the no debug all privileged EXEC command.

Symptoms: A router that is configured with a multilink bundle may reload unexpectedly with the following error message:

Conditions: This symptom is observed on a Cisco router when you attempt to remove a service policy from a multilink interface.

Symptoms: Packets with a size greater then 604 bytes are not received from an ISE line card. This situation is caused by the depletion of buffers, as indicated in the output of the show controller tofab queue command.

Conditions: This symptom is observed rarely on a Cisco 12000 series and may be related to corrupt or unidentified traffic.

Symptoms: On a Cisco 7500 router, the standby may reload with CCB PLAYBACK errors when the standby boots up.

Symptoms: An Engine 2 line card may crash when it receives multicast traffic that is not punted to the CPU of the line card but switched by the PSA memory of the line card.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(27)S or a later release when the Engine 2 line card is configured with the wrong PSA memory and when you enable multicast hardware acceleration by entering the hw-module slot slot-number ip multicast hw-accelerate command.

Symptoms: An Engine 3 or Engine 5 line card may crash while processing packets for Output Sampled NetFlow.

Conditions: This symptom is observed on Cisco 12000 series when Output Sampled NetFlow is enabled on the Engine 3 or Engine 5 line card and when packets are sampled that are not aligned to a 2-byte boundary.

Symptoms: A Cisco 12000 series line card may crash while processing packets that are not aligned to a 4-byte boundary.

Conditions: This symptom is observed when the packet is processed in the CPU of the line card.

Symptoms: MLPPP interfaces may fail to ping the far end after initial configuration of the bundle. Additionally, the IP address of the directly- attached MLPPP interface will not appear in the routing table.

Conditions: This symptom has been observed with multilink PPP interface(s) configured on the router.

Symptoms: The standby RP reboots when you perform an OIR of an active VIP that is installed in any slot of the router.

Conditions: This symptom is observed on a Cisco 7500 series that runs Cisco IOS interim Release 12.4(7.10) and that is configured for RPR, RPR+, or SSO. The symptom may also affect other releases.

Symptoms: Applying the rate-limit output command to an Engine 0 DS3 subinterface will cause the output bytes counter to not account for traffic passing across any subinterface on the interface that is not part of a VRF.

Conditions: This symptom has been observed when using the rate-limit output command.

Symptoms: There are 3 Multilink PPP (MLPPP) and 3 Multilink Frame Relay (MLFR) at 12 links per bundle and 11 HDLC serial interfaces on one SPA-1XCHSTM1/OC3. All MLPPP bundles are up/down on R1 when a cable is physically removed, an SSO switchover performed, and the cable inserted on R1. All MLPPP bundles are up/up on R2.

Conditions: This symptom has been observed when physically removing the cable, doing an SSO switchover, and inserting the cable with traffic running.

Workaround: Enter a shutdown and no shutdown command on the MLPPP bundles and the bundles will come up.

Symptoms: Some of the thousands of L2TPv3 sessions are not forwarding traffic. The number of affected VCs varies but is usually within a range of 5 to 10.

Conditions: This symptom has been observed with scaled L2TPv3 configuration, with up to 8000 L2TPv3 sessions.

Symptoms: When applying a policy map as in the following example, the system rejects the configuration.

%Error: Multi-action police command not supported on interface GigabitEthernet0/0/0

Conditions: This symptom has been observed when the police statement is entered on separate lines.

Workaround: Enter the police statement on a single line rather than on separate lines. For example:

Symptoms: A VIP may experience a reset due to a bus error (CPU signal 10) after an interface flap.

Symptoms: Loss of most of the configuration of a single Multilink PPP or Multilink FR bundle on an RPR+ or an SSO switchover except for the following commands:

Conditions: The symptom has been observed upon associating hardware with a previously configured multilink bundle (when adding the first link in the bundle), in a router which contains many interface bundles. The symptom is exacerbated when QoS policies are configured. The symptom has been seen when an RP switchover occurs shortly after the first link is added to one or multiple bundles.

Workaround: Manually reapply the configuration to the bundle in question. The symptom should be confined to one bundle only.

Symptoms: Serial interfaces associated with MFR bundles on CT3 SPA may not come back up in service after a physical online removal and insertion.

Conditions: This symptom is observed on a Cisco 12000 series router that runs the gsr-p-mz image of Cisco IOS Release 12.0(32)S2.

Symptoms: If a Cisco 12000 series Internet router is configured as a router in a large scale configuration, sometimes one or more line cards can reload silently during switchback from default standby PRP (now in active role) to default primary PRP (now in standby role).

Conditions: This problem is observed in Cisco IOS Release 12.0(32)S2 and later releases.

Workaround: There is no workaround. However, waiting until the CPU utilization has decreased to below 50% will severely reduce the possibility of this problem occurring.

Conditions: This symptom is observed on a Cisco 12000 series router that runs the gsr-p-mz image of Cisco IOS Release 12.0(32)S2.

Conditions: When configuring the frame-relay interface- dlci command and assigning or unassigning a mapclass to a data-link connection identifier (DLCI), which has service-policy, SPA connectivity is lost and the above message is displayed.

Symptoms: When you enter the no shutdown interface configuration command on an auto-template interface during deployment, some tunnels may be in the up/down state, and the tunnel mode may be GRE instead of the configured tunnel mode of MPLS.

Conditions: This symptom is observed on a Cisco router with about 70 primary MPLS TE tunnels. The symptom occurs when you first enter the no interface auto-template command, then you enter the tunnel mode mpls traffic-eng command, and finally you paste the template back.

Symptoms: If following steps are performed on 2xCT3 SPA and 10x1GE SPA located in E5 12000-SIP-601

–

Enter the hw-module slot shelf-id/slot-number shutdown command on 2xCT3 SPA.

–

Enter the no hw-module slot shelf-id/slot-number shutdown command on 2xCT3 SPA.

–

Enter the hw-module slot shelf-id/slot-number shutdown command on 10x1GE SPA. Before CT3 SPA comes up, the CT3 SPA may get in an out-of-service state.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS interim Release 12.0(32.1)S9

Workaround: Performing the no hw-module slot shelf-id/slot-number shutdown command on 10x1GE SPA, or the microcode reload command on the SIP-601 may fix the problem.

Symptoms: A Cisco 12000 Engine 3 card is used as an IP ingress to classify traffic to different classes using a set of IP standard and extended ACLs. After reaching a stable state, when the ACLs take a substantial part of the TCAM, adding a single line to an ACL used in the class-map may result in the TCAM Mngr merge process on the line card to take 99% CPU for a long time (over 60 seconds), and then crash the card with the following errors:

00:14:59: %QM-4-SW_SWITCH: Interface GigabitEthernet2/2 routed traffic will be

Conditions: This issue is observed with multiple interfaces on the same line card using service policies with long (over 1000 entries) ACLs.

Workaround: When restarting the line card with the policy-map configured, it boots correctly. Also, removing the QoS policy from all the interfaces, waiting for the TCAM to clear up, then changing the ACL and reapplying the QoS policy on the interface is a valid workaround.

Symptoms: Locally originated traffic is not prioritized correctly in specific MQC configuration for MPLS and Engine 4 plus used as the egress LC. Control plane protocols might flap as a result of oversubscription of the default class.

Conditions: This occurs if the policy-map includes a priority class, e.g. to carry voice traffic, and that there is no class mapping precedence 6 or 7 (i.e. the policy map defines mappings only for EXP 6 and 7).

Workaround: There is no workaround other than avoiding oversubscription of the default class.

Symptoms: After booting a Cisco 12000 router, Engine3 and Engine4+ line card may get stuck in the MRAM0230 state.

Conditions: This symptom is observed on a Cisco 12000 series router that runs the gsr-p-mz image of Cisco IOS Release 12.0(32)SY and may require physical removal and replacement of the LC if the symptom occurs.

Workaround: Remove automatic mbus agent upgrade by removing the service upgrade mbus-agent- rom command from the running configuration, and do a manual mbus agent upgrade by issuing the upgrade mbus agent command from the CLI.

Symptoms: A multichannel T1 or E1 port adapter may become deactivated when you enter the invert data or crc 16 interface configuration command.

Conditions: This symptom is observed on a Cisco 7500 series that is configured with an RSP4 that contains an R5000 microprocessor.

Symptoms: An interface of a 2-port OC-192 POS Engine 6 line card may stop sampling NetFlow.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(28)S6.

Workaround: Remove NetFlow from the affected interface and then re-apply NetFlow to the interface.

Symptoms: Reloading the SPAs in a 12000-SIP-601 line card may causes the 12000-SIP-601 to fail.

Conditions: This symptom is observed in a Cisco 12000-SIP-601 (which has 8xFE, 8xOC12 POS, 2xCT3, & 1xChOC3 SPAs) in a Cisco 12000 series router, running Cisco IOS interim Release 12.0(32.1)S11.

Symptoms: When upgrading the Cisco 12000 series Internet router to Cisco IOS Release 12.0(32)S2, the video streaming deployed through multicast with PBR configured results are heavily degraded with the image frequently frozen on the screen. Other services, like voice, do not seem to be affected.

Conditions: This symptom is observed on a Cisco 12000 series Internet router that is running Cisco IOS Release 12.0(32)S2 and has multicast service and PBR configured on it.

Symptoms: On a Cisco 12000 series router with SIP-601, there is a possibility to drop important packets, such as keepalives and hellos, as a result of an incorrect configuration.

Conditions: When traffic arrives to an SIP-601, if the egress interface has a glean adjacency, then all the important packets are dropped on the SIP causing protocol flapping.

Symptoms: MLP links on a Cisco GSR running Cisco IOS Release 12.0(32)S2 doesn't come up after ha switchover in SSO mode.

Conditions: This is seen on MLP links created off of serial interfaces on a frost bite card CHOC12/DS1-IR-SC= after ha switchover with mode set to SSO.

Workaround: Remove non-default router global configuration line aaa new-model from the router configuration and the problem will not happen.

If the aaa new-model command is configure, then one has to shutdown and enable the physical links to restore the MLP link

Symptoms: Transient and control traffic of packets that are sized 81-608B could be dropped.

Conditions: This symptom is observed when reloading an Engine 3 line card. Some queues are stuck.

Symptoms: The line protocol on E5 2xOC48 SPA, located in 12000-SIP-600, may not come up after performing the hw-module slot shelf-id/slot-number shutdown command followed by the no hw-module slot shelf-id/slot-number shutdown command on the SPA.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(31.4)SY1

Workaround: Performing the microcode reload command on the affected 12000-SIP-600 will fix the problem.

Symptoms: Periods of high latency may occur on a Multilink PPP interface, and finally the interface may lock up.

Conditions: This symptom is observed on a Cisco 7600 series when the Multilink PPP interface is configured on a SPA-8XCHT1/E1 that is installed in a SIP-200.

Workaround: Configure multilink interfaces on another line card that does not require insertion in a SIP.

Alternate Workaround: Configure IP load balancing by using two separate E1 links (that is, do not use multilink interfaces).

Conditions: The crash occurs if any FRoMPLS or Frame relay local switching connections are present.

Symptoms: Reloading a POS line card with interfaces configured for encapsulation frame- relay IETF and frame-relay subinterfaces, checking the ifIndex for the subinterfaces after the line card restarted it only shows: Incorrect ifIndex for POSxx/y.z

Conditions: The symptom is seen when a line card configured for encapsulation frame-relay IETF and frame-relay subinterfaces is reloaded while the RP remains running.

Workaround: Remove the subinterface configuration and then add it. The ifIndex will again show correct values.

Symptoms: On a Engine 5 line card, traffic is dropped on ingress direction without any error report.

Symptoms: Engine 0, 1 and 2 of a Cisco 12000 suffers from low free memory after upgrading the software image to Cisco IOS Release 12.0(32)S3. Large routing table (like 300,000 BGP routes) may cause memory allocation failure in the mentioned legacy engines, along with configuration of features such as input ACL.

Conditions: This symptom has been observed on a Cisco 12000 with 1GB RPR-1/2, populated with Engine 0, 1, and 2 line cards and when injecting more than 300,000 BGP routes, along with configuring large input security ACLs.

Symptoms: Reconfiguring a T1 on a CT3 SPA installed on a SIP401 with Cisco IOS Release 12.0(32)S3 cases the resulting frame-relay subinterfaces to reflect an incorrect interface number.

–

Configure associated main interfaces with frame-relay encapsulation then configure subinterfaces.

–

With all interfaces up and passing traffic, shut down the main interface and then remove the T1 from the T3 controller.

After reconfiguring the main and subinterfaces, the subinterface number still reflects the previous configuration.

Symptoms: Trap "ciscoRFSwacNotif" is not received on Network Management Station after an RPR+ switchover on a Cisco GSR.

Conditions: This symptom has been observed on a Cisco GSR platform running Cisco IOS Release 12.0(32)S3.

TCP/IP Host-Mode Services

Symptoms: Path MTU Discovery (PMTUD) may incorrectly select a higher MTU for an egress interface and may cause BGP to send packets that are larger than the size that the egress interface can support. When this situation occurs, packets are lost and the BGP session may be terminated.

Conditions: This symptom is observed when PMTUD is enabled over parallel links with different MTUs and when the paths in each direction use different links. Some other conditions may also apply, such as CEF and load-balancing being enabled.

Workaround: Enter the ip tcp mss command to configure the MSS to be less than the MTUs of all possible egress interfaces, or configure the MTUs of all possible egress interfaces to be same as the MSS.

Symptoms: When you enter the copy ftp disk command, the copy operation may fail and cannot be terminated, further copy commands may fail, and a TCP vty session for the purpose of troubleshooting the situation may fail and cannot be terminated.

Conditions: These symptoms are observed on a Cisco platform when the FIN flag is set in the initial ESTAB message from a neighbor. You must reload the router to recover from the symptoms.

Workaround: Do not enter the copy ftp disk command. Rather, enter the copy tftp disk command.

Wide-Area Networking

Symptoms: Frame Relay LMI crashes due to Address Error (load or instruction fetch) exception.

Conditions: The crash may occur when a Cisco type LMI packet is received which contains an invalid DLCI value.

Symptoms: On a Cisco router when a main interface is configured for frame-relay encapsulation with sub-interfaces, when a sub-interface is deleted and re-added, the DLCI information is not re-added to the running configuration and no error message is sent to the CLI to indicate an error.

Conditions: This only occurs if the main interface is shut down. If the main interface is administratively up, this does not occur in our testing. This symptom was introduced between Cisco IOS Release12.0(30)S and Release 12.0(31)S.

Workaround: Do not provision and rollback sub-interfaces on main interfaces that are shut down. This may not be possible without re-writing customer automated provisioning systems.

Resolved Caveats—Cisco IOS Release 12.0(32)S3

Cisco IOS Release 12.0(32)S3 is a rebuild of Cisco IOS Release 12.0(32)S. The caveats listed in this section are resolved in Cisco IOS Release 12.0(32)S3 but may be open in previous Cisco IOS releases. This section describes only severity 1, severity 2, and select severity 3 caveats.

Basic System Services

Symptoms: Intensive SNMP polling may cause the I/O memory of a router to be depleted.

IP Routing Protocols

Symptoms: The IGP metric is not updated to an eBGP peer when there is an IGP- metric change. The BGP peer is not advertising this change after its default timer of 10 minutes.

Conditions: This symptom occurs when a link goes down and the IGP metric to reach the BGP nexthop has changed. When used in a route-map, the set metric-type internal value should propagate any MED changes in updates to the BGP peers.

Workaround: Use clear ip bgp neighbor-address [soft [in | out]] on the respective router to get the correct metric changes and BGP best path.

Symptoms: A Cisco 12416 router running the Cisco c12kprp-k3p-mz.120-31.S1.bin image restarted.

Conditions: The symptom has been observed when using the Cisco c12kprp-k3p- mz.120-31.S1.bin image. The following message was shown:

Miscellaneous

Symptoms: When using the redundancy force-switchover command, a Cisco 12000 router may reload with bus error.

Conditions: This symptom has been observed on a Cisco 12000 router running Cisco IOS Release 12.0(26)S4 and with redundancy SSO enabled.

Conditions: This symptom occurs after loading the image and configuring SSO mode.

Symptoms: After configuring a feature bundle on an E3 4x1GE line card and reloading, the line card crashes when IPv6 traffic is processed by the line card.

Conditions: The symptom has been observed after the feature bundle is configured, the line card is reloaded, and as soon as IPv6 traffic is processed by the line card.

Symptoms: After a SPA-capable line card (LC) crash, some or all of the SPAs may occasionally fail to recover. The LC in question will display messages like these, one for every failed SPA (subslot) every 1 to 2 minutes:

Conditions: This symptom has been observed to occur after SPA-capable LC crashes, although such crashes are extremely rare.

Workaround: The following steps should be attempted, in this order, allowing a few seconds then checking the state between steps, until the problem is corrected and the SPA type is detected properly:

Further Problem Description: The MBUS transport is not 100% reliable. MBUS message loss is usually extremely rare, but not impossible. If the lost messages happen to be MBUS C2W programming messages, they may cause inconsistencies in the MBUS Agent C2W programming. If the inconsistency happens to affect a single bay's SPA EEPROM C2W device, then it becomes impossible to determine the type of the SPA inserted in that bay. The LC crashes cause very high peaks of MBUS activity, which apparently may cause MBUS message loss (suspecting buffer overrun or out-of-buffer conditions).

Symptoms: Continuous traceback with error messages is encountered on removal of an SPA-CH8TE1 card.

Conditions: This symptom has been observed when an SPA-CH8TE1 card is removed from the Cisco 12000 router in an SIP-601 jacket card.

Conditions: This symptom has been seen on a Cisco 7500 series router with a PA- 2FE-TX in a VIP4-80. The FE port is connected to a switch port. If the switch port is set for 10MB and the FE interface is unable to be manually configured with the speed command, the FE interface will be in an up/down state.

Workaround: Use the speed auto command on the switch port that connects to the FE interface on the router. By using this workaround, you will not be able to throttle the bandwidth to 10MB as desired. The interface will be up/up, and will pass traffic at 100MB.

Symptoms: Performing dual switchover with MDRR configured on an SPA interface shows MDRR queues as unallocated for the show policy-map command.

Conditions: This symptom has been observed when configuring MDRR on an SPA interface (POS or SRP) and doing a dual switchover (Main-to-standby and standby-to-Main).

Workaround: Remove the MDRR configuration from the interface and reapply. The MDRR queues reappear.

Conditions: The symptom has been observed intermittently on Cisco IOS Release 12.0(31)S2 with 10G SPAs on SIP-600 LC.

Symptoms: A VIP may reload unexpectedly when you change the encapsulation from Frame Relay to PPP.

Conditions: This symptom is observed on a Cisco platform that has a VIP that is configured for QoS.

Symptoms: When the controller of multichannel T3 port adaptor SPA-4XCT3 goes down for a short duration and an alarm occurs, the port adapter does not report the type of alarm.

Conditions: This symptom has been observed on Cisco 7600 series and Cisco 12000 series routers that are configured with a SPA-4XCT3. The port adapter should provide a history table of recent alarm conditions along with a corresponding time stamp to allow for proper troubleshooting.

Symptoms: An SIP601 line card crashes when shutting down an SPA-8XCHT1/E1 for RPR+ switchover.

Conditions: This symptom has been observed when shutting down an SPA-8XCHT1/E1 on bay 1/1 and performing an RPR+ switchover with traffic running.

Symptoms: Using the snmpwalk command to query the SNMP variable cbQosSetStats in cbQoS-MIB causes a memory leak. The leaked blocks are of 1K size for each single snmp get command.

Conditions: The leak only exists if a service-policy is attached under the interface and the service-policy performs a set action as shown in the example:

Conditions: This symptom has been observed on a Cisco 12000 E6 line card with MPLS/Multicast traffic.

Symptoms: The T3 line state may flap when there are no apparent alarms or problems on a channelized T3 to DS0 SPA.

Conditions: This symptom is observed on a Cisco 12000 series when the T3 links are in channelized mode, when the T1 links on the Cisco 12000 series are configured for ESF framing, and when the T1 links on the router at the far end are configured to send T1 FDL ANSI reports.

Resolved Caveats—Cisco IOS Release 12.0(32)S2

Cisco IOS Release 12.0(32)S2 is a rebuild of Cisco IOS Release 12.0(32)S. The caveats listed in this section are resolved in Cisco IOS Release 12.0(32)S2 but may be open in previous Cisco IOS releases. This section describes only severity 1, severity 2, and select severity 3 caveats.

Basic System Services

Symptoms: A VIP4-80 card may fail to load the Cisco IOS software image. When this situation occurs, the following error messages are generated:

%DBUS-3-WCSLDERR: Slot 2, error loading WCS, status 0x4 cmd/data 0xDEAD pos 97

%DBUS-3-WCSLDERR: Slot 2, error loading WCS, status 0x4 cmd/data 0xDEAD pos 99

Conditions: This symptom is observed on a Cisco 7500 series when you enter the microcode reload command.

Further Problem Description: The symptom may also occur because of improperly installed line cards. If this situation occurs, re-install the line cards.

Symptoms: An NTP broadcast client may fail to synchronize with an NTP broadcast server if the server cannot be reached from the client.

Conditions: This symptom is observed in Cisco IOS interim Release 12.2(12.11)T or a later release, including Release 12.3. However, the symptom may also occur in other releases.

Conditions: This symptom is observed on a Cisco router that is configured for SSO and that has the snmp mib notification-log default command enabled.

Conditions: This symptom is observed a Cisco 7500 series that runs Cisco IOS Release 12.0(31)S when packets are forwarded across a PA-A3 port adapter.

EXEC and Configuration Parser

Symptoms: The RP may crash with a watchdog timeout error for the IP input process.

Conditions: This symptom is observed on a Cisco 12000 series when you delete a subinterface that processes traffic. The symptom may be platform-independent.

Interfaces and Bridging

Symptoms: An "%ATMPA-3-CMDFAIL" error message and "cm622_vip_parse_cmd" traceback may be generated on the VIP console for an OC-12 ATM interface of a 1-port ATM Enhanced OC-12/STM-4 port adapter (PA-A3-OC12) that is installed in the VIP, and a "setup_vc failed" error message may be generated on the RSP console. This situation causes a VC that is configured on the OC-12 ATM interface to become inactive.

Conditions: This symptom is observed on a Cisco 7500 series when you perform the following steps:

After the new standby RSP comes up, you delete a subinterface from the new active RSP.

Workaround: There is no workaround to prevent the symptom from occurring. When the symptom has occurred, enter the shutdown interface configuration command followed by the no shutdown interface configuration command on the OC-12 ATM interface to enable the VC to become active.

Symptoms: A cBus Complex Restart may occur on a Cisco 7500 series when you leave the interface configuration mode after you have changed the encapsulation on a serial interface from HDLC to another encapsulation type such as PPP or Frame Relay.

The maximum datagram for an interface a of low-speed serial port adapter with HDLC encapsulation and an MTU of 1500 is 1528 because the overhead that is added to the MTU is 28. The maximum datagram for an interface a of high-speed serial port adapter with HDLC encapsulation and an MTU of 1500 is 1530 because the overhead that is added to the MTU is 30.

When the encapsulation type is changed, the maximum datagram size may change, which causes an internal MTU change. This situation may cause some packets to be dropped as giants.

Conditions: This symptom is observed after the first change to the type of encapsulation from the default of HDLC to another encapsulation type when you leave the interface configuration mode. Subsequent changes to the type of encapsulation do not cause the cBus Complex Restart.

Workaround for the cBus Complex Restart: There is no workaround for the cBus Complex Restart. An MTU change on a Cisco 7500 series results in a cBus Complex Restart, which usually means a router outage of 15 to 30 seconds or a minute and a half, depending on the Cisco IOS software image that the router is running.

Workaround for the packet drops: Reconfigure the MTU to prevent packet from being dropped as giants.

Further Problem Description: The fix for this caveat ensures the maximum datagram for an interface a of low-speed serial port adapter with HDLC encapsulation and an MTU of 1500 is 1608 to allow for an overhead to the MTU of 108. The maximum datagram for an interface a of high-speed serial port adapter with HDLC encapsulation and an MTU of 1500 is then 1610 to allow for an overhead to the MTU of 110.

Symptoms: The throughput is far below what you would expect on an MFR bundle that is configured on a 8-port multichannel T1/E1 PRI port adapter (PA-MC-8TE1+).

Conditions: This symptom is observed on a Cisco 7200 series that runs Cisco IOS Release 12.0(31)S2 when the MFR bundle has four T1 links, three of which are shut down. When you generate 2.5 Mbps of traffic to congest the one active link, a throughput of about 37 kbps to 59 kbps is observed. You would expect a throughput of about 1.5 Mbps.

IP Routing Protocols

Symptoms: Memory utilization in the "Dead" process grows gradually until the memory is exhausted. The output of the show memory dead command shows that many "TCP CBs" re allocated. Analysis shows that these are TCP descriptors for non-existing active BGP connections.

Conditions: This symptom is observed on a Cisco 7200 series that runs Cisco IOS Release 12.3(13), that has an NPE-G1, and that functions as a PE router with many BGP neighbors. However, the symptom is not platform-specific, nor release-specific.

Symptoms: High CPU usage may occur and the table versions of BGP peers are reset to zero.

Conditions: This symptom is observed when you update a complex policy on a Cisco router that has a complex configuration of BGP peers.

Symptoms: A router that is configured for BGP and that has the ip policy-list command enabled may unexpectedly reload because of a bus error or SegV exception.

Conditions: This symptom is observed when BGP attempts to send an update with a "bad" attribute.

ISO CLNS

Symptoms: When IS-IS and CLNS are configured, a router may enter a state in which only one adjacency is shown in the output of the show clns interface command, even though the show clns neighbors command may correctly display all the neighbors that are connected to the interface.

When this situation occurs and any one of the neighbors on the segment goes down, all routing updates may be lost. The single adjacency is torn down and despite the fact that the output of the show clns neighbors command still shows the neighbors, routing stops because there are no adjacencies.

Conditions: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.2(18)S1 or Release 12.3(9b) when an adjacency goes down while it is still in the INIT state. The symptom occurs because the adjacency counter is incorrectly decremented. The symptom may also occur in other releases.

Workaround: Enter the shutdown interface configuration command followed by the no shutdown interface configuration command on the interface that reports only one adjacency.

Alternate Workaround: Enter the clear clns neighbors command on the affected router.

Miscellaneous

Conditions: This symptom occurs when using T1 ATM (the IMA function is not used) on a PA-A3-8T1IMA.

Symptoms: Multicast packets are not fast-switched on an MDT tunnel interface, causing packets to be blackholed.

Conditions: This symptom is observed on a Cisco router that functions as a PE router in a Multicast VPN environment.

Symptoms: A line card reloads when traffic is sent through an ATM PVC with the QoS policy enabled.

Workaround: Add a dummy class to the policy-map with a bandwidth or shape feature as shown in the following example:

Symptoms: A router that is configured with redundant RPs that function in RPR+ or SSO mode may generate the following error message after the router has been reloaded:

After the error message has been generated, the standby RP may reload unexpectedly.

Conditions: This symptom is observed on a Cisco 12000 series that is configured for Multicast VPN and that runs the gsr-p-mz image of Cisco IOS Release 12.0(30)S, Release 12.0(31)S, or an interim release for Release 12.0(32)S.

Symptoms: Traffic does no pass through the main interface of an ATM Engine 2 line card after you have reloaded microcode onto the line card.

Conditions: This symptom is observed on a Cisco 12000 series that runs an interim release for Cisco IOS Release 12.0(32)S. The symptom occurs only for the main interface and not for any subinterfaces. The symptom may also occur on other platforms.

Workaround: Enter the shutdown interface configuration command followed by the no shutdown interface configuration command on the main interface of the affected ATM line card.

Conditions: This symptom has been observed on a scaled IP and L3VPN configuration in which E3 cards are installed and part of the data path.

Symptoms: When an SSO switchover occurs less than 10 minutes after the router has been reloaded or a preceding SSO switchover has occurred, the following error message and a traceback are generated:

Configuration: This symptom is observed on a Cisco router that is configured for MVPN.

Workaround: Wait at least 10 minutes after the router has been reloaded or a preceding SSO switchover has occurred before initiating an SSO switchover.

Conditions: This symptom is observed on a Cisco router when an access control list is configured on the egress interface to deny all IP packets.

Symptoms: A Cisco router may crash unexpectedly because of a bus error and/or display some spurious memory accesses.

Conditions: This symptom is observed when an interface that is configured for some form of fancy queueing (that is, anything besides FIFO queueing) actively forwards traffic.

Symptoms: Traffic may not fully recover when you enter the shutdown interface configuration command followed by the no shutdown interface configuration command on an interface that functions as one of two members of a link-bundling port channel.

Conditions: This symptom is observed on a 4-port Gigabit Ethernet ISE line card that is installed in a Cisco 12000 series. The router runs the c12kprp-p-mz image of Cisco IOS Release 12.0(32)S and functions in a scaled configuration with AToM connections that are configured on a link-bundling port channel. The symptom is more likely to occur after the router or line card has just been reloaded.

Symptoms: When you enter the show diag command, the output is continuously generated, alternating between the various slots of a line card.

Symptoms: The xconnect command does not function when AToM over MPLS (AToMoMPLS) is configured for VP mode. Although you can enter the xconnect command when AToMoMPLS is configured for VP mode, the command is not applied, and the output of the show running-config command does not show the Xconnect configuration for the ATM interface.

Conditions: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.0(32)S. Note that the symptom does not occur when AToMoMPLS is configured for VC mode.

Workaround: Configure AToMoMPLS for VC mode. If this is not an option, there is no workaround.

Symptoms: An interface of an Engine 3 ingress line card that functions in feature mode may become stuck, and all traffic may be dropped.

Conditions: This symptom is observed on a Cisco 12000 series that is configured with an Engine 3 ingress line card that has the hw-module slot slot-number np mode feature command enabled and that is configured with an egress line card that has a link bundle interface.

Workaround: Disable the hw-module slot slot-number np mode feature command on the Engine 3 ingress line card.

Symptoms: When a T3/E3 Serial SPA is configured in Kentrox mode with a small bandwidth between 22 kbps and 250 kbps, either in T3 or E3 mode, the firmware miscalculates the bandwidth allocation and allows up to 24M of traffic to pass through.

Workaround: Do not configure such a small bandwidth when the T3/E3 Serial SPA is configured in Kentrox mode. The minimal bandwidth on a T3/E3 Serial SPA that is configured in Kentrox mode is either 1500 kbps in T3 mode or 1000 kbps in E3 mode.

Symptoms: A SIP may generate an error message or crash when you first perform an OIR of an SPA-10X1GE that is installed in the SIP and you then enter the show tech-support command.

Further Problem Description: The symptom does not occur when a 1-port OC192/STM64 POS/RPR SPA is installed in the SIP.

Symptoms: If the sender and receiver for a multicast group are on the same line card on a PE router, the v flag may be set, causing traffic to be punted to the RP.

Conditions: This symptom is observed on a Cisco 12000 series when a hash collision occurs. The symptom may not be platform-specific.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(32)S when the MQC policy map is configured on an OC-12 subinterface of a 1-port channelized OC-48 ISE line card in the ingress direction.

Symptoms: The Total, Switched, and Drops counters in the output of the show ip mds stats switching command are merged together (there is no space between the values) when their values become large, preventing you from reading the values.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(28)S4 and that is configured with multiple Engine 3 and Engine 4+ line cards.

Symptoms: Tracebacks may be generated for a 1-port channelized STM-1/OC-3 SPA that has a multilink configuration. The multilink interface and the 1-port channelized STM-1/OC-3 SPA may not function properly.

Conditions: This symptom is observed when you reload the Cisco 12000 series in which the SPA is installed.

Symptoms: Routers may crash with MPLS VPN configuration and shutting down PE-CE link.

Conditions: The symptoms may occur when the router acts as a PE on the MPLS VPN topology and when one or more PE-CE links are shut down.

Symptoms: dCEF switching does not function when Frame Relay over L2TPv3 is configured on a 2-port OC-3 POS port adapter (PA-POS-2OC3) that is installed in a VIP 6-80.

Conditions: This symptom is observed on a Cisco 7500 series that runs Cisco IOS Release 12.0(30)S3 when an interface of the PA-POS-2OC3 faces the core of the network. When traffic from the core leaves the PA-POS-2OC3 to a CE router, dCEF switching functions fine. However, when traffic form the CE router leaves the PA-POS-2OC3 to the core, dCEF switching does not function and the VIP 6-80 punts the traffic to the RSP.

Symptoms: Counters in the output of the show policy-map interface mfr command do not increment for any type and/or class of service. Even the class default shows no packets. The counters in the output of the show frame-relay pvc command show the packets correctly.

Conditions: This symptom is observed when a map class is configured on a subinterface that is part of an MFR (FRF.16) bundle and when the map class consists of both an input and output service policy.

Symptoms: A router may incorrectly install IPv6 routes that are associated with an ATM interface, even though the ATM interface is down.

Conditions: This symptom is observed when a PVC configuration is applied to the ATM interface.

Symptoms: Traffic from an Engine 3 or Engine 5 line card to an egress line card that is installed in slot 0 stalls.

Conditions: This symptom is observed on a Cisco 12000 series when you insert a new line card and remove the line card before it reaches the "IOS RUN" state, for example, when the line card is still in the "IOS STRT" or "IOS UP" state.

Workaround: After you have inserted a line card, wait until the line card is in the "IOS RUN" state before you remove it. If the line card becomes stuck before reaching the "IOS RUN" state, remove the line card, and enter the show controller tofab queue 0 command on other Engine 3 or Engine 5 line cards in the chassis to check if the tofab queues towards slot 0 are uncorrupted. If the queues are corrupted, reload slot 0 to recover from the situation.

Symptoms: A Cisco 3640 or Cisco 3660 stops encrypting GRE packets, which are then sent in the clear.

Conditions: This symptom is observed on a Cisco 3640 and Cisco 3660 that run Cisco IOS Release 12.3(13), that are configured for CEF, and that have an interface (but not the interface with the crypto map) that has that has the ip tcp header-compression command enabled and physical-layer async command enabled.

Workaround: Enter the no route-cache cef command followed by the route-cache cef command.

Alternate Workaround: Delete the crypto map from the interface and re-apply the crypto map.

Symptoms: When a Turbo ACL classification table grows beyond a certain size, a memory allocation failure may occur or the router may crash.

If the router runs Cisco IOS Release 12.1E or 12.3, memory corruption may occur, causing the router to crash. If the router runs Cisco IOS Release 12.2S, an error message similar to the following may appear during a Turbo ACL compilation, the compilation will fail, and a recompilation is forced:

%SYS-2-CHUNKBADELESIZE: Chunk element size is more than 64k for TACL Block -Process= "TurboACL", ipl= 0, pid= 82

Conditions: This symptom is observed on a Cisco router that is configured for Turbo ACL. The Cisco 10000 series is not affected.

Workaround: Monitor the output of the show access-lists compiled command and force the Turbo ACL tables to be cleared if a table is at risk of growing large enough to trigger the symptoms.

The tables that have significant sizes are the first and third tables shown next to "L1:" and the first table shown next to "L2:". When the number after the slash for one of these tables is greater than 16384 for the "L1" tables or greater than 32768 for the "L2" table, the table is already too large and the symptom may occur any moment.

When the number is in the range from 10924 to 16384 inclusive for the "L1" tables or the range from 21846 to 32768 inclusive for the "L2" tables, the table size will be too large on the next expansion. An expansion occurs when the number to the left of the slash reaches 90 percent of the value to the right of the slash. When the value to the left of the slash approaches 90 percent of the value to the right, enter the no access-list compiled command followed by the access-list compiled command to disable and re-enable Turbo ACL. Doing so causes the tables to be cleared and, therefore, delay the expansion. This workaround may be impractical when there is a high rate of incoming packets and when entries are added frequently to the tables.

Alternative Workaround: Disable Turbo ACL by entering the no access-list compiled command.

Note that neither of these workarounds are supported on a Cisco 7304 that is configured with an NSE-100: there is no workaround for this platform.

Symptoms: An inconsistency may occur in the outlabel information that is used by BGP and MPLS forwarding.

Conditions: This symptom is observed when there are two route reflectors (RRs) that advertise the same route and when one of the routes is the best path. The symptom occurs when the following conditions are present:

–

The PE router that is the source restarts, causing the prefix to be readvertised with a new label.

–

The RR that forms the non-best path delays the withdrawal and readvertisement of the prefix, for example, because the RR has a heavy load.

This situation causes BGP to function with the new label but MPLS forwarding to function with the old label.

Symptoms: A Cisco 12000 series RP crashes when you enter the clear l2tun all command.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(30)S3 when the debug vpdn l2x-packets command is enabled on the router.

Workaround: Do not enter the clear l2tun all command when the debug vpdn l2x-packets command is enabled on the router.

–

For a session that is in the idle state, the output of the show l2tun session brief command may not show a circuit/interface name, that is, the output shows "-".

–

When the debug ssm cm error command is enabled, a "Two segs" error is consistently generated in the log.

Conditions: This symptom is observed when multiple L2TPv3 Xconnect links are configured and occurs only after you have reloaded the router.

Workaround: Disable the xconnect command for the idle session and then re-enable the same command.

Symptoms: When you enter the shutdown interface configuration command followed by the no shutdown interface configuration command on all serial interfaces of an MLP bundle, a ping fails because OSPF is stuck in the INIT state.

Conditions: This symptom is observed even while the MLP bundle comes up after you have entered the no shutdown interface configuration command.

Workaround: Enter the shutdown interface configuration command followed by the no shutdown interface configuration command on the main interface of the MLP bundle.

Symptoms: When you reload one line card, all other line cards in the chassis may reload unexpectedly.

Conditions: This symptom is observed on a Cisco 7500 series that runs Cisco IOS Release 12.0(32)S or an earlier release and on a Cisco 7600 series that runs Cisco IOS Release 12.2(18)SX.

Symptoms: A POS line card that is configured with third-party vendor Small Form-Factor Pluggable Interface Converters (SFPs) and that is installed in a Cisco 12000 series fails the security check.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(30)S3 or Release 12.0(31) during the initial boot process.

Note that when the router runs Release 12.0(28)S3 and you boot the router with the SFPs already installed, the symptom does not occur. However, when you reload the router and then remove and reinsert the SFPs, they do not pass the security check either.

Symptoms: Incorrect outgoing labels are installed for BGP-IPv4 Multipath prefixes.

Conditions: This symptom has been observed anytime that a label changes from a BGP-IPv4 Multipath peer.

Workaround: Clearing the BGP neighbor should allow the correct labels to be installed.

Conditions: This symptom is observed on a Cisco 12000 series that is configured for multicast traffic.

Symptoms: In an MVPN topology running sparse-mode in the core with data-MDTs, an interruption in traffic in the core (e.g. reloading a P router) may cause a PE router to stop sending traffic to the core.

Conditions: The MVPN traffic is riding a sparse-mode data-MDT when a P router is reloaded. The interruption to the traffic causes the traffic to fall back to the default-MDT and the data-MDT sparse group enters the registering state. If the traffic resumes before the data-MDT mroute times out, it will initially be forwarded over the default-MDT and then switch to the data-MDT but no register traffic is sent.

Workaround: Enter the clear ip mds line x where x is the Ingress LC on PE router. Note that Cisco recommends the use of SSM for data-MDTs and this will avoid the issue entirely.

Symptoms: A multicast RPF check fails when the maximum-paths eibgp command is configured as part of an IPv4 VRF.

Conditions: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.0(31)S2 and that is configured for L2TPv3.

The symptom occurs because RPF expects the gateway for the VPNv4 route to be either in the same VRF or in the global routing table context, which is not the case when L2TPv3 is configured.

Conditions: This symptom is observed on a Cisco router that is configured with a SIP-600 in which POS SPAs are installed.

Symptoms: Traffic forwarding stops on the Gigabit Ethernet modular baseboard (EPA-GE/FE-BBRD) of a Modular GbE Engine 4+ line card.

Conditions: This symptom is observed on a Cisco 12000 series when you add an ACL to the 3-port Gigabit Ethernet port adapter (EPA-3GE-SX/LH-LC) of the same Modular GbE Engine 4+ line card.

Workaround: Perform a microcode reload on the 3-port Gigabit Ethernet port adapter of the Modular GbE Engine 4+ line card. Alternately, reload the router. However, note that both workarounds interrupt the traffic flow.

Symptoms: Interface numbering is not complete when executing commands on a Cisco 12000 line card.

Conditions: This symptom has been observed when entering commands on a Cisco 12000 line card which requires multiple physical interfaces on a single controller.

Symptoms: The BGP table and CEF forwarding table may have mismatched labels for prefixes that are learnt from a remote PE router.

Conditions: This symptom is observed on a Cisco router that functions as a PE router when an eBGP session flap or route flap occurs on the remote PE router. A new label for the prefix is learnt from the remote PE router, but forwarding may not be updated properly.

Workaround: There is no workaround. When the symptom has occurred, and to correct the situation, enter the clear ip route vrf vrf-name network command on the PE router that has mismatched labels.

Conditions: This symptom is observed when a Cisco router boots with a line card that has an interface on which Frame Relay DLCI, a QoS service policy, and FRF.12 are configured.

Workaround: Ensure that FRF.12 is not configured when the router boots. After the router has booted, add FRF.12.

Conditions: This symptom is observed on a Cisco Catalyst 6000 series and Cisco 7600 series (in which a FlexWAN is installed) and on a Cisco 7500 series (in which a VIP is installed). The symptom occurs when these platforms are configured for VRF, MLP, and QoS.

Symptoms: All channels on a multichannel T3 port adapter may go down. The router may then reload unexpectedly due to a software forced crash. If not, all of the channels in the T3 may stay down until corrective action is taken.

CT3SW WatchDog not cleared, WatchDog = 2 CT3SW WatchDog not cleared, WatchDog = 3

Conditions: This symptom affects routers using two-port multichannel T3 port adapters, the PA-MC-2T3 and the PA-MC-2T3+. The symptom occurs when one or more of the T1's in either T3 sees framing errors. One-port multichannel T3 port adapters, the PA-MC-T3 and the PA-MC-T3+, are not affected.

Workaround: There is no workaround to prevent this problem. Possible corrective actions are listed below:

Simulate removal and reinsertion with these exec mode commands in sequence: hw-module slot slot- number stop hw-module slot slot- number start

Possible Corrective Actions for the Cisco 7500 router: 1. Remove and reinsert the VIP with the affected port adapter. 2. Use the configuration mode command: microcode reload 3. Reload the router.

Symptoms: Some CEF entries are missing from some VRFs, as shown in the output of the show ip cef inconsistency now command.

Conditions: This symptom is observed after an OIR or reload of a Cisco 12000 series GE ISE line card.

Workaround: There is no workaround. When the symptom has occurred, enter the clear ip cef epoch command to recover the CEF entries. If this does not recover the CEF entries, enter the clear ip route vrf command.

Further Problem Description: So far, the symptom is observed for local "receive" entries, such as /32 entries for a VRF loopback interface. However, the symptom may also occur for other types of VRF FIB entries.

Symptoms: When you load a Cisco IOS software image, the interface-level uRPF configuration may be lost.

Conditions: This symptom is observed on a Cisco 12000 series when the uRPF interface level configuration has the allow-self ping keyword enabled in a command such as the ip verify unicast source reachable-via allow-self-ping command before you the Cisco IOS software image.

A list of the affected releases can be found at http://www.cisco.com/cgi-bin/Support/Bugtool/onebug.pl?bugid=CSCsd03412. Cisco IOS software releases that are listed under the "All affected versions" link at this location are affected. Cisco IOS software releases that are listed in the "First Fixed-in Version" field at this location are not affected.

Workaround: To prevent the symptom from occurring, remove the allow-self ping keyword before you load the Cisco IOS software image. When the symptom has occurred, reconfigure each interface that lost the uRPF configuration and ensure that the allow-self ping keyword is not part of a command.

Symptoms: When reloading a router(lsnt-ap-pe1, Cisco 7500 platform) with Cisco IOS interim Release 12.0(31.4)S1 from any Cisco IOS Release 12.0(28)S4b image, several IDBINDEX_SYNC-3-IDBINDEX_ENTRY_LOOKUP and traceback occur in the standby log.

Conditions: This symptom has been observed on a Cisco 7500 router platform with MVPN.

Symptoms: When an RP switchover occurs after you have reloaded a line card that is configured for MFR and MLP, the MFR and MLP links may remain down.

Conditions: This symptom is observed on Cisco 12000 series that runs Cisco IOS Release 12.0(28)S4, that runs in SSO mode, and that is configured with a 6-port channelized T3 Engine 0 line card and a 1-port channelized OC-12 ISE line card.

Workaround: Enter the shutdown interface configuration command followed by the no shutdown interface configuration command on the affected MFR and MLP interfaces.

Further Problem Description: Note that the symptom does not occur when the router runs in RPR+ mode.

Symptoms: On a router that runs Multiprotocol Label Switching (MPLS), the "%SYS-3-OVERRUN:" and "%SYS-6-BLKINFO" error messages may be generated and a software-forced crash may occur on the router.

Conditions: This symptom is observed when you enter the show mpls ldp discovery command under the following condition:

–

The adjacencies between peers through this interface have not been fully established for some peers.

–

The unestablished LDP adjacencies are coming while you enter the show mpls ldp discovery command.

Workaround: Do not enter the show mpls ldp discovery command while multiple LDP adjacencies are coming up. Rather, enter the show mpls ldp neighbor [detail] command while multiple LDP adjacencies are coming up.

Symptoms: When multicast hardware acceleration is enabled, a wrong label stack may be imposed on packets that have an IP destination address below 16.x.x.x. This situation occurs, for example, when the IP destination address is 10.1.1.1 and when the ingress interface is an MPLS VPN VRF subinterface that is configured for 802.1q. Note that in this situation, the CEF forwarding information is correct, that is, it has the correct label stack.

Conditions: This symptom is observed on a Cisco 12000 series that is configured with a 3-port GRE Engine 2 line card but may occur on any Engine 2 line card that has VRF interfaces. The symptom occurs only when multicast hardware acceleration is enabled.

Symptoms: The CPU usage may remain at 99 percent for a long time when NMS polls the ipRouteTable via the SNMP protocol.

Conditions: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.0(28)S or Release 12.0(31)S when there is a large number of routes in the routing table.

Symptoms: When you enter the show tech-support cef or show cef linecard internal command, the router may crash.

Conditions: This symptom is observed on a Cisco router that is configured for CEF.

Symptoms: On a subinterface of a 4-port Gigabit Ethernet ISE line card (4GE-SFP-LC) that has an LDP neighborship with a neighbor that uses per-interface label space, when the LDP neighborship goes down, the subinterface stops forward traffic.

Conditions: This symptom is observed on a Cisco 12000 series. The symptom does not occur with a 3-port Gigabit Ethernet line card (3GE-GBIC-SC).

Conditions: The symptom is observed when you modify an existing access control list (ACL) that is attached to an interface.

Workaround: Do not modify an ACL that is attached to an interface. If you cannot remove the ACL from the interface, create a new ACL and apply it to the interface.

Symptoms: The ip helper-address command does not function on a 4-port Gigabit Ethernet ISE line card (4GE-SFP-LC).

Conditions: This symptom is observed on a Cisco 12000 series when the Gigabit Ethernet interface is configured for VRF forwarding.

Symptoms: The FIB becomes disabled when you bring down a member link of an MLP or MFR bundle.

Conditions: This symptom is observed on a Cisco 7500 series that is configured with an MLP or MFR bundle and that runs Cisco IOS Release 12.0(30)S, Release 12.0(31)S, or Release 12.0(32)S, all of which integrate the fix for caveat CSCeg57219. Other Cisco IOS software releases that integrate the fix for caveat CSCeg57219 are not affected.

Symptoms: An ISE line card crashes when the MPLS VPN-VRF Selection Based on Source IP Address feature is enabled.

Conditions: This symptom is observed on a Cisco 12000 series when the MPLS VPN-VRF Selection Based on Source IP Address feature is enabled in the configuration.

Workaround: Do not enable the MPLS VPN-VRF Selection Based on Source IP Address feature.

Symptoms: When only one T1 link in a MLP bundle is active while all other links are shut down, the bundle does not pass traffic unless you bring up at least one more T1 link in the bundle.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(31)S2.

Symptoms: When you first enter the router isis area-tag command followed by the address-family ipv6 command and then change the administrative distance for the address-family IPv6 configuration, the RP may crash.

Conditions: This symptom is observed on a Cisco 12000 series that runs the gsr-p-mz image of Cisco IOS Release 12.0(30)S5, Release 12.0(31)S3, or Release 12.0(32)S1. The symptom may also occur on other platforms.

Symptoms: The output of the show memory bigger command shows negative block sizes for 4 GB of RAM.

Conditions: This symptom is observed on a Cisco 12000 series when a block with a size greater than 2 GB is also present.

Symptoms: An Engine 4, Engine 4 plus, or Engine 6 line card may punt MPLS packets to the RP, causing the CPU usage of the RP to be high.

Conditions: This symptom is observed on a Cisco 12000 series that receives a large number of small MPLS packets that have their TTL set to 1.

Symptoms: A line card that is configured with a multilink bundle may reset when the connected serial T1 interface on the remote router is shut down while traffic is flowing.

Conditions: This symptom is observed on a Cisco 12000 series when the traffic that enters the multilink bundle exits over an L2TPv3 tunnel and when multilink fragmentation is applied at the remote end.

Workaround: Disable multilink fragmentation at the remote end. If this is not an option, there is no workaround.

Symptoms: In an MVPN topology, an Engine 5-based SIP (that is, a SIP-401, SIP-501, SIP-600, or SIP-601) in which one or more SPAs are installed may reset when you forward multicast traffic with packets that are larger than the MTU of the interface and therefore require fragmentation.

Conditions: This symptom is observed on a Cisco 12000 series, that functions as a PE router, and that is located in the network core when a SPA that is installed in the SIP interconnects the PE router with a P router.

Symptoms: When a Cisco 7500 series that runs in SSO mode reloads (for example, because of a "c7500 CCB PLAYBACK" error), the active RP comes up properly but the standby RP crashes. When the router reloads while running in RPR+ mode, the standby RP does not crash but an "%HA-3-SYNC_ERROR: CCB Playback error" message is generated.

Conditions: This symptom is observed on a Cisco 7500 series that is configured for SSO or RPR+.

Workaround: There is no workaround. The symptom does not occur when you disable SSO or RPR+.

Symptoms: An RP crashes when a Gigabit Ethernet interface of a SPA is shut down. When this situation occurs, the following error message and traceback is generated:

Unexpected exception to CPUvector 700, PC = 2CEE34 -Traceback= 2CEE34 4C40000 2D8958 2D8D2C 2C1164 14048C 2CFB4C

On a router that is configured with two RPs that function in RPR+ mode, when the RP crashes, a switchover occurs. However, the crashed RP does not come up and remains in standby mode.

Conditions: These symptoms are observed on a Cisco router when the recursive lookup on a static MPLS route does not specify a next hop interface. For example, the symptom occurs when the ip route destination-prefix mask next-hop1 command is enabled but does not occur when the ip route destination-prefix mask interface1 next-hop1 command is enabled.

Workaround: Ensure that the interface argument is included in the static route, as in the following command: ip route destination-prefix mask interface1 next-hop1.

Symptoms: Aggregated Output NetFlow may aggregate output traffic on an incorrect prefix (such as 0.0.0.0/0) on a PE router that functions in an MPLS VPN environment.

Conditions: This symptom is observed on a SIP-600 that is installed in a Cisco 12000 series that runs Cisco IOS Release 12.0(32)S when egress traffic streams are sent in different routing contexts (via the global routing table and via a VRF or different VRFs) and when Aggregated Output NetFlow is enabled using a Prefix-ToS aggregation scheme.

The symptom occurs for both IP-to-IP and tag-to-IP traffic (the latter one from the MPLS core). The symptom does not occur when the egress traffic is within the same routing context, that is, when all egress interfaces are either in the global routing table or in the same VRF).

Symptoms: On a GSR router configured for MVPN, the reload of any E3 Edge facing line card can cause VRF mroutes to be missing on the reloaded line card while they are present on PRP and on other line cards. This will cause punts to PRP CPU and, depending on number of missing mroutes, potentially high PRP CPU utilization due to IP Input.

Conditions: This symptom has been observed to happen in a scaled MVPN environment and is triggered only by linecard reload.

Workaround: There is no workaround. To recover from this state, issue clear ip mds linecard linecard-slot- number where linecard-slot-number is the slot number of the reloaded line card.

Symptoms: A router may crash when you unconfigure an MLP bundle that is configured for QoS.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(32)S when the MLP bundle is configured on a 1-port channelized OC-12 ISE line card.

Symptoms: When you reload a 2-port clear channel T3/E3 Engine SPA that is installed in a PE router that performs MVPN encapsulation, multicast traffic does not resume.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(32)S1 and that functions as a PE router in an MVPN topology.

Workaround: Enter the ip mds linecard linecard-slot-number * command. For the linecard-slot-number argument, enter the SIP and the slot number in which the SIP is installed.

Symptoms: When configuring output ACL in an interface of an E4+ EPA-GE/FE-BBRD card, the following error message is shown. However, the output of show tcam-mgr gen7 appl acl-out block-info shows that there are 964 free entries in TCAM.

Conditions: This symptom has been observed in an E4+ LC card when the line card does not find enough free sequential entries in TCAM at its current location because of ACL expansion and needs to relocate ACL to a different location within the TCAM.

An example configuration of access-list 102 and 103 applied on GigabitEthernet interface at 9/0/1.

Remove all of the ACL applied to that line card interfaces globally. config terminal no access-list number [number = 102, 103]

Remove all of the ACL configuration from the line card interfaces. config terminal interface GigabitEthernet 9/0/1 no ip access-group number out [number = 102, 103]

On line card confirm that the ACLs have been removed. show access-lists gen7-fm label

Re-apply the ACL configuration to the LC interfaces, the one's deleted in Step 2.

On the line card, confirm that the ACLs have been recreated properly. show access-lists gen7-fm label

Symptoms: When you reload a channelized T3 SPA that is installed in a PE router, all interfaces on a directly-connected channelized T3 SPA that is installed in a CE router remain in the down state, and the following error message is generated:

%SPA_CHOC_DSX-3-SPA_CMD_SEND_ERR: Failed to send ipc_send_rpc_blocked command to SPA

Conditions: This symptom is observed in a VPN topology on a Cisco 12000 series that runs Cisco IOS Release 12.0(32)S1, that functions as a PE router, and that is connected to another Cisco 12000 series that functions as a CE router. The channelized T3 SPA in the PE router is installed in a SIP-601; the channelized T3 SPA in the CE router is installed in a SIP-400.

Symptoms: A 12000 series may indicate a low I/O memory condition. The output of the show memory summary command shows the following:

Conditions: This symptom is observed on a 12000 series that is configured with a PRP2 and 4 GB of RAM.

SLOT 9:Mar 2 21:44:31.827 PST: %SYS-2-WATCHDOG: Process aborted on watchdog timeout, process = MDFS LC Process.

Conditions: This symptom has been observed when a large number of mroutes (20K) and egress multicast QoS are configured. The symptom can be triggered by removing and adding the ip multicast-routing distributed command.

Conditions: The symptom has been observed in a system scale testbed with both L3VPN and MVPN scaled configurations with the following:

Workaround: A possible recovery is to use proc-max-time of 20 msec. This recovery works with E4P and E5 as the Core facing line cards.

Symptoms: The ip2tag unicast traffic from an E3 line card to an E3 line card is being dropped on a GSR.

Conditions: This symptom has been observed on a GSR running Cisco IOS Release 12.0(32)S 060307 nightly image where the GSR is acting as a PE router. The router is configured with unicast VPN and multicast VPN and unicast and multicast traffic is flowing through the router.

Symptoms: After removing VRF, the following message with tracebacks is observed:

Conditions: This symptom has been observed on a Cisco 12000 router with Cisco IOS Release 12.0(32)S.

Symptoms: On a bulk sync (reload) of a standby route processor (RP) or during an initial router reload, all MDT configuration is erased on the standby. This erasure causes all MVPN traffic to fail following an RP switchover.

Conditions: This symptom occurs on a dual RP router in RPR+ or SSO mode with MVPN configuration. This symptom is only seen in conn_isp and the Cisco IOS Release 12.0(32)S throttle branch.

Workaround: Deconfigure and reconfigure mdt default and mdt data configuration statements within the VRF configuration.

Further Problem Description: This symptom occurs as a result of a bug in the initial CSCsd09324 commit.

Symptoms: Under some Cisco 12000 router configurations after a line card reload of a 4GE ISE line card or RPR+ switchover, some interfaces on a 4GE ISE line card can stop receiving traffic.

Conditions: This symptom has been observed on a Cisco 12000 router with Cisco IOS Release 12.0(32)S.

Workaround: There is no workaround. Using the shutdown and no shutdown commands on the main interface recovers from this symptom.

Symptoms: In MPLS VPN topology, where E3 4xOC12 ATM line card is a Core-facing line card, after performing a reload on any of Egress line cards, the ATM line card may reload.

Conditions: This symptom has been observed on a E3 4xOC12 ATM line card configured with MPLS Aware Netflow, on a Cisco 12000 series that runs Cisco IOS Release 12.0(32)S2 or 12.0(32.1)S4.

Symptoms: Decapsulation traffic stops forwarding if traffic for the encapsulation stream is stopped when MVPN traffic is flowing bi-directionally on a PE router and the two streams collide with each other and the encapsulation stream occupies the hardware hash table entry.

Conditions: This symptom has been observed when sending MVPN traffic in both directions on an E3 line card, ensuring that one stream collides with the other stream.

Workaround: Use the clear ip mroute [vrf vrf-name] {* | group [source]} to clear this issue.

Wide-Area Networking

Symptoms: Unexpected drops may occur in the Multilink Frame Relay (MFR) output hold queue. The drops persist under a very low (25 pps) transmit rate.

After you have disabled the traffic source or shut down the ingress interface, the MFR output hold queue may take as long as 15 minutes to "drain."

Conditions: These symptoms are observed on a Cisco router when you run multicast traffic over GRE tunnel interfaces that in turn use an MFR interface for transport.

Symptoms: A router crashes when you attempt to delete a Frame Relay-to-Ethernet connection.

Conditions: This symptom is observed when you first remove the Frame Relay interface via an OIR and then you attempt to delete the Frame Relay-to-Ethernet connection.

Workaround: Re-insert the Frame Relay interface before attempt to delete the Frame Relay-to-Ethernet connection.

Resolved Caveats—Cisco IOS Release 12.0(32)S1

Cisco IOS Release 12.0(32)S1 is a rebuild of Cisco IOS Release 12.0(32)S. The caveats listed in this section are resolved in Cisco IOS Release 12.0(32)S1 but may be open in previous Cisco IOS releases. This section describes only severity 1, severity 2, and select severity 3 caveats.

Miscellaneous

Symptoms: Multicast traffic stops leaving from a SPA that is installed in a SIP-400 after an RPR+ switchover is performed on the remote end.

Conditions: This symptom is observed on a Cisco 12000 series when the remote end is also a Cisco 12000 series in which a SIP-400 is installed.

Workaround: Enter the clear ip mds linecard linecard-slot-number command on the SIP-400 that is installed in the local Cisco 12000 series. Note that the linecard-slot-number argument represents the SPA and affected output port.

Symptoms: When you enter the clear cef linecard command on a SIP-400 or SIP-600 in which channelized T1 and T3 SPAs are installed that are configured with MFR and MLP interfaces, the MFR interfaces may flap. The symptom may also occur when a line card that has MFR and MLP interfaces configured is reloaded or when the router is reloaded.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(32)S, that functions as a PE router, and that has multilink interfaces configured. The symptom is reproducible when CPU-intensive applications are running on the router while a large number of multilink interfaces is brought up. Note that the symptom does not occur for non-SPA line cards.

Workaround: Shut down the multilink interfaces. When the router is stable (that is, after CEF is cleared or after the line or router is reloaded), bring up the multilink interfaces.

Alternate Workaround: Increase the bundle link parameters on the PE router and the connected CE router by increasing the hello and acknowledgement periods via the seconds arguments in the following commands for each of the serial interfaces that are part of the multilink bundle:

Symptoms: After you have configured VPN, when the default MTU on an interface of an Engine 5 SPA is changed, the Engine 5 SPA may generate tracebacks.

Conditions: This symptom is observed on a Cisco 12000 series that has a VPN configuration when the MTU is changed while traffic is flowing.

Symptoms: MFR interfaces that are configured on T1 and T3 SPAs that are installed in a SIP-601 may remain in the IDLE state.

Conditions: This symptom is observed on a Cisco 12000 series that runs the gsr-p-mz image of Cisco IOS Release 12.0(32)S when any action occurs that changes the MFR link state followed by an RPR+ switchover.

–

Entering the shutdown interface configuration command followed by the no shutdown interface configuration command on an MFR interface.

–

Entering the hw-module slot slot-number shutdown command followed the no hw-module slot slot-number shutdown command on the SIP in which the SPAs are installed.

Workaround: Enter the shutdown interface configuration command followed by the no shutdown interface configuration command on affected MFR interface.

Conditions: This symptom is observed on a Cisco 12000 series when a standby Route Processor (RP) is inserted or removed.

Symptoms: Traffic does not pass on some MFR interfaces after an RPR+ switchover has occurred.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(32)S.

Workaround: Reload the SIP (and consequently the SPA) to enable all serial interfaces to enter the up/up state.

Wide-Area Networking

Symptoms: When you enter the shutdown interface configuration command followed by the no shutdown interface configuration command on an MFR interface when the bundle links are down, the serial interfaces that are associated with the MFR interface remain in the IDLE state.

Conditions: This symptom is observed on a Cisco router that is configure for MFR.

Workaround: Enter the shutdown interface configuration command followed by the no shutdown interface configuration command on each serial interface that is associated with the MFR interface.

Resolved Caveats—Cisco IOS Release 12.0(32)S

All the caveats listed in this section are resolved in Cisco IOS Release 12.0(32)S. This section describes only severity 1, severity 2, and select severity 3 caveats.

Basic System Services

Symptoms: When you attempt to apply an access list for SNMP by entering the snmp-server tftp-server-list command, the access list is not applied. This situation makes it possible to copy the configuration to and from any server, regardless of the contents of the access list.

Conditions: This symptom is observed on a Cisco platform that is configured for SNMP.

The following sample configuration causes the platform to reject configuration file transfers via SNMP from all hosts except the TFTP server that is specified in access list 5:

access-list 5 permit 10.1.1.1
snmp-server community private RW 5
snmp-server tftp-server-list 5

Workaround: Apply a more general access list to restrict traffic to and from the affected platform.

Symptoms: Configuring nonexisting NTP peers repeatedly may cause a router or switch to reload unexpectedly.

Conditions: This symptom is observed on a Cisco router and Cisco switch that are configured for NTP.

Workaround: Do not add and delete nonexisting NTP peers in quick succession, for example by using a cut-and-paste operation.

Symptoms: A Cisco router may reload when there are two or more Telnet or console sessions to the router.

–

In one session, enter the show ip as-path-access-list acl-number command. The output pauses at the "--more--" prompt when there is more than one page output.

–

In another session, enter the no ip as-path access-list acl-number command in which the acl-number argument is the same one as in the show ip as-path-access-list acl-number command.

–

In the first session, type in "enter" or "space" in the first session to display the rest of the show command output.

Workaround: Do not enter the show ip as-path-access-list acl-number command when the no ip as-path access-list acl-number command is being configured.

Symptoms: When the slave RSP crashes, a QAERROR is observed in the master console, resulting in a cbus complex. The cbus complex will reload all the VIPs in the router.

Conditions: This symptom happens when the slave crashes in a period when there is a large number of packets going towards the RSP. A large number of packets go to the RSP when CEF switching is configured or when routing protocol updates are numerous.

Symptoms: An OER border component does not sent passive updates for OER prefixes, preventing the prefixes from being controlled. The prefixes cycle from the default state to the hold-down state back to the default state.

Conditions: This symptom is observed when NetFlow is configured and when the mode monitor is configured to be "passive" or "both".

Workaround: Configure the mode monitor to be "active". The functionality of the mode monitor is limited to the "active" mode only.

Conditions: This symptom is observed on a Cisco 7200 series that runs Cisco IOS Release 12.4(1), that is configured with an NPE, and that has the L3 cache disabled. The symptom may also occur in other releases.

Conditions: This symptom is observed under rare circumstances on a Cisco router that is configured for Cisco Discovery Protocol (CDP) and that has hundreds of subinterfaces when you enter the no interface port command followed by the interface port command.

Workaround: After you have entered the no interface port command, wait a few seconds before you enter the interface port command.

Symptoms: A spurious memory access may be generated on an RSP when a VIP that is in a disabled or wedged condition is recovered because of a Cbus Complex or microcode reload.

Conditions: This symptom is observed on a Cisco 7500 series that has a VIP that is in a disabled or wedged condition after the router has booted.

Symptoms: The active Supervisor Engine or Route processor (RP) may reload when the standby Supervisor Engine or RP is inserted.

Conditions: This symptom is observed on a Cisco switch or router when the following conditions are present:

Workaround: Do not configure SSO as the redundancy mode. Rather, configure RPR+ as the redundancy mode.

Alternate Workaround: Disable the SNMP MIB notification log by entering the no snmp mib notification-log default command.

Symptoms: A router may crash while loading the image for a secondary RSP from a disk during the boot process.

Conditions: This symptom is observed on a Cisco 7500 series that is configured with redundant RSPs when the hw-module slot slot-number image disk0: image command is configured.

Symptoms: An MSFC may reload when the standby Supervisor Engine 720 is inserted. Similarly, the active Route Processor (RP) may reload when the standby RP is inserted.

Conditions: These symptoms are observed on a Cisco Catalyst switch or router when SNMP and SSO are configured, and when the following configuration is present on the platform:

snmp mib notification-log default
snmp mib notification-log globalageout 600
snmp mib notification-log globalsize 1500

Alternate Workaround: Remove the SNMP MIB notification-log configuration by entering the no snmp notification-log default command.

Conditions: This symptom is observed on a Cisco router when BGP is being updated.

Workaround: Enable the new deterministic regular expression engine by entering the bgp regexp deterministic command and then enter the show ip regexp command. Note that enabling the new deterministic regular expression engine may impact the performance speed of the router.

Conditions: This symptom is observed on a Cisco 7500 series that is configured for SSO and that has the snmp mib notification-log default command enabled.

Symptoms: A Cisco 7500 series may log the following error message even if no VIP is installed in slot 0:

Conditions: This symptom is observed after a crash of another VIP has occurred. Sometimes the symptom occurs when a VIP is installed in slot 0 but most of the time there is no VIP in slot 0 when the symptom occurs.

Symptoms: A Cisco router or switch may crash when you enter the no snmp-server command.

Conditions: This symptom is observed after a switchover when you enter the no snmp-server command on the newly active RP or Supervisor Engine.

Conditions: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.0(26)S6, that is configured for BGP, and that has the bgp regexp deterministic command enabled.

IBM Connectivity

Symptoms: DLSW transparent redundancy does not function via a Fast Ethernet port adapter.

Conditions: This symptom is observed when you use a Fast Ethernet port adapter with a particular third-party vendor chipset.

Interfaces and Bridging

Symptoms: A VIP that contains a PA-A3-OC12 ATM port adapter may unexpectedly reload.

Conditions: This symptom is observed on a Cisco 7500 series that functions in an ATM LANE configuration.

Workaround: There is no workaround. The traffic on the VIP is disrupted until the VIP comes back up.

Symptoms: A Versatile Interface Processors (VIP) with a PA-MC-8TE1 port adapter may report its memory size as unknown even though the VIP appears to function normally, and Distributed Multicast Fast Switching (DMFS) may fail to function properly.

Conditions: This symptom is observed on a Cisco 7500 series when any of the following conditions are present:

–

The mode of the controller of the PA-MC-8TE1 port adapter is not set to T1 or E1 and you insert or remove another VIP with any port adapter via an OIR.

–

Irrespective of whether or not the mode of the controller of the PA-MC-8TE1 port adapter is set to T1 or E1, you insert or remove a standby RSP via an OIR.

Workaround: Enter the card type {t1 | e1} slot [bay] command on the PA-MC-8TE1+ port adapter and ensure that none of the controllers on this port adapter are shut down.

Symptoms: Packets that enter an interface that is configured for IP may not be switched via dCEF.

Workaround: Enter the shutdown interface configuration command followed by the no shutdown interface configuration command on the affected interface.

ALternate Workaround: If many interfaces are affected, reload all port adapters by entering the microcode reload command on the control plane of the RSP.

Symptoms: Spurious memory accesses occur on a Cisco 7200 series and ALIGN-3-SPURIOUS error messages are generated.

Conditions: This symptom is observed after you have configured a new MLP interface and a new EBGP neighbor.

Conditions: This symptom is observed on a Cisco 7206VXR that runs Cisco IOS Release 12.3(10a) and that is configured with an NPE-G1 and a couple of PA-MC-8TE1+ port adapters. The symptom may also occur in other releases.

Symptoms: The line protocol on the POS interface of a PA-POS-OC3 port adapter flaps continuously.

Conditions: This symptom is observed on a Cisco 7500 series that runs Cisco IOS interim Release 12.3(14.10) but may also occur in other releases.

Symptoms: A Cisco 7xxx series router may crash because of a bus error exception and may report CPUHOG message when you perform an OIR of an ATM PA-A3 or ATM PA-A6 port adapter.

Conditions: This symptom is observed on a Cisco 7xxx series router that runs Cisco IOS Release 12.3 when PVC auto-provisioning is enabled on the ATM PA-A3 or ATM PA-A6 port adapter and when many PPP sessions are in transition. The symptom may also occur in other releases.

Symptoms: POS interfaces may remain in the up/down state after the router has been reloaded.

Conditions: This symptom is observed on a Cisco Catalyst 6500 series, Cisco 7500 series, and Cisco 7600 series.

Workaround: Reload the FlexWAN or VIP in which the POS port adapter is installed.

Symptoms: CEF, dCEF, and fast-switching counters are not accurate on outbound serial E1 or T1 interfaces.

Conditions: This symptom is observed on a Cisco 7200 series when CEF, dCEF, and fast-switching are enabled on a serial E1 or T1 interface.

Symptoms: When you perform an OIR of a PA-MC-8TE1+ or PA-MC-8E1 port adapter, the following error message and traceback may be generated:

%ALIGN-3-SPURIOUS: Spurious memory access made at 0x60379D34 reading 0xD8 % ALIGN-3-TRACE: -Traceback= 60379D34 604F1CFC 60BD0664 6032B93C 6039A0CC 6010A908 6032AA7C 6032EBAC

Conditions: This symptom is observed on a Cisco router when the port adapter is configured for QOS on an egress serial interface and traffic is flowing through this interface.

Symptoms: When you enter the microcode reload command, an error message similar to the following and a traceback may be generated:

RSP-3-RESTART: interface Serial3/0/1/4:0, not transmitting 
-Traceback= 404436B4 4044DE10

Conditions: This symptom is observed on a Cisco 7500 that is configured with an E1, T1, E3, or T3 port adapter.

Symptoms: A channelized T3 port adapter cannot detect C-bit errors and does not shut down after continuous C-bit errors.

Conditions: This symptom is observed on a Cisco 7200 series that is configured with a channelized T3 port adapter.

Symptoms: A router may intermittently transmit corrupt PPP packets. When you enter the debug ppp nego and debug ppp errors commands, it appears that "protocol reject" packets are received from the remote end.

Conditions: This symptom is observed on a Cisco 7500 series that has only one OC-3 POS port adaptor per VIP and that is configured for PPP encapsulation.

Workaround: Configure an outbound policy on the interfaces of the OC-3 POS port adaptors.

Interface Serial0/0:63 changed state to down %CT3-3-LOVEFAIL: CT3-SW-PA-0/0: failed to send T3 line state change love letter %AMDP2_FE-5-LATECOLL: Ethernet0/0 transmit error

Conditions: This symptom is observed on a Cisco 7500 series that is configured with a channelized T3 port adapter.

Symptoms: A Cisco 7500 series may generate the following error message and traceback:

%IPC-2-ONINT: Invalid operation at interrupt level: IPC blocking send request 103000C -Traceback= 40857C4C 40291754 40291AC4 40295214 4028BD1C 4028BBA0 4026C5A8 4075FF30 407615F0

Conditions: This symptom is observed on a Cisco 7500 series that is configured with a PA-MC-xT1/E1 type of port adapter.

Conditions: This symptom is observed after you have entered the shutdown interface configuration command followed by the no shutdown interface configuration command on the peer interface, causing a link up/down event to occur.

Further Problem Description: The symptom occurs because the address filter entry is deleted during the link up/down event. You can verify that the symptom has occurred in the output of the show controller command and you can manually confirm the deletion of the OSPF MAC entry. When you reconfigure OSPF, the OSPF MAC entry is re-inserted in the address filter.

Symptoms: Unicast packets are not CEF-switched on a VIP but are fast-switched on the RP.

Conditions: This symptom is observed on a Cisco router that has a VIP only when the ingress interface is an ISL subinterface.

Symptoms: Interfaces of a serial port adapter fail and do not come into service, preventing you from establishing links or tunnels via these interfaces.

Conditions: This symptom is observed on a Cisco 7500 series that runs an interim release for Cisco IOS Release 12.0(32)S.

IP Routing Protocols

Symptoms: Selective Packet Discard (SPD) with an IP precedence of 6 and 7 may not function correctly. Packets may be treated as nonpriority packets. SPD is used when a queue is filling (for example, because of a flap or change) and routing traffic must be guaranteed a high priority and not dropped while the interface recovers. In this situation, routing traffic with an IP precedence of 6 and 7 is not given proper priority, and the recovery process may be delayed.

Conditions: This symptom is observed with incoming traffic on any interface of a Cisco 7200 series and may be observed on other Cisco platforms as well.

Symptoms: EIGRP-specific Extended Community 0x8800 is corrupted and shown as 0x0:0:0.

Conditions: This symptom is observed when EIGRP-specific Extended Community 0x8800 is received via an IPv4 EBGP session. This occurs typically in the following inter-autonomous system scenario:

Workaround: Disable propagation of extended communities across autonomous systems.

Symptoms: During BGP scalability testing, error messages and tracebacks similar to the following ones may be logged, indicating a difficulty with TCP and buffer usage:

%SYS-2-MALLOCFAIL: Memory allocation of 4692 bytes failed from 0x6076F714, align
Pool: I/O  Free: 11143248  Cause: Memory fragmentation 
Alternate Pool: None  Free: 0  Cause: No Alternate pool 
-Process= "Pool Manager", ipl= 0, pid= 6
-Traceback= 607FE10C 607FF1EC 6076F71C 6080C1D0 6080C400

%TCP-6-NOBUFF: TTY0, no buffer available
-Process= "BGP I/O", ipl= 0, pid= 139
-Traceback= 6098B4EC 609938C8 60993C1C 60D55CE4 60D0BEB0

%TCP-6-NOBUFF: TTY0, no buffer available
-Process= "BGP Router", ipl= 0, pid= 138
-Traceback= 6098B4EC 609938C8 60993C1C 60D55CE4 60D29858 60D2AF88 60D1B4BC

Conditions: This symptom is observed on a Cisco router that is in the processing of building BGP sessions for about 80,000 prefixes and about 1200 BGP peers.

Symptoms: The RP and SP MFIB tables are not synchronized, preventing the outgoing interfaces for multicast flows from matching.

Conditions: This symptom is observed when you reload a Cisco Catalyst 6000 series or Cisco 7600 series that are configured for multicast. The symptom may be platform-independent.

Symptoms: Calls may not complete because ResvConfirm messages are dropped. You can enter the debug ip rsvp messages command to track RSVP messages as they traverse routers.

Conditions: This symptom is observed when RSVP is configured for call admission control in a network with routers that do not have RSVP and a proxy ARP enabled. The symptom occurs because the RSVP-capable hop that sends the ResvConfirm messages uses the next RSVP-capable hop as the next IP hop for the packets and does not have the MAC address that is needed to encapsulate the IP packets for this next IP hop.

Workaround: Configure a static ARP entry that enables the router to properly encapsulate the packet by entering the arp ip-address hardware-address arpa command. The ip-address argument is the address of the next hop (that is visible via the RSVP debugs) for the ResvConfirm messages and the hardware-address argument is the MAC address of the interface of the next IP hop through which the ResvConfirm messages should be routed.

Conditions: This symptom is observed when the platform on which the ping originates and the OSPF neighbor interface are connected via an OSPF sham link that is used for interconnecting traffic between two VPN sites.

Conditions: This symptom is observed on a Cisco 7200 series that runs Cisco IOS Release 12.3(3) when the clear ip bgp neighbor-address soft out command is entered for one of the members of the peer group of which the Cisco 7200 series is a member and when some changes to the outbound policy are made to the same member of the peer group. This situation causes some prefixes to remain struck in the other members of the peer group. The symptom may also occur in other releases.

The symptom is a very old behavior of the BGP peer group functionality: when one member of a peer group is cleared via either a hard reset or a soft reset and a policy change causes some of the prefixes to be withdrawn, inconsistencies may occur in the routes on the other members of the peer group.

Workaround: For peer groups and neighbors that are members of a peer group, do not enter the BGP neighbor-specific clear ip bgp neighbor-address soft out command or the clear ip bgp neighbor-address command. Rather, enter the peer group-specific clear ip bgp peer-group-name soft out command or the clear ip bgp peer-group-name command.

Symptoms: A ping, Telnet traffic, FTP traffic, and trace route traffic across a VRF-aware NAT do not function.

Conditions: This symptom is observed on a Cisco router that is configured for VRF-aware NAT only when the router is not directly connected to a gateway.

Symptoms: During an NSF switchover on a RP, the convergence may be delayed up to five minutes.

Conditions: This symptom is observed when a DBD exchange error occurs while the adjacency is brought up.

Symptoms: A Cisco router may crash when it is reloaded with PIM traffic on the network.

Conditions: This symptom is observed on a Cisco 7200 series router with multicast enabled but is not platform dependent. Bootup is the most likely place where this will happen, but the router may crash anytime if an interface flap happens at the right time while receiving PIM traffic.

Symptoms: A router terminates 102,000 VPNv4 routes but route reflectors (RRs) report only a a subset of the total.

Conditions: This symptom is observed on a Cisco MGX RPM-XF that runs Cisco IOS Release 12.3(11)T4 when 204 routes are configured per VRF over 496 VPNs (one VPN has about 1000 routes). However, Cisco MGX RPM-PRs that function as RRs show that only 76245 routes are terminated on the Cisco MGX RPM-XF. The symptom is platform-independent and may also occur in other releases.

Symptoms: A router may crash when a subinterface on which OSPF is running is deleted.

Conditions This symptom is observed when the mpls ldp sync command is configured under OSPF.

Symptoms: When the ip bgp fast-external-fallover permit interface configuration command is enabled on the main interface of a 4-port Gigabit Ethernet ISE line card and on a subinterface of a connected BGP neighbor, and when you enter the shutdown interface configuration command on the main interface, the BGP session that is established on the subinterface remains up for about 150 to 180 seconds before the BGP hold timer causes the session to go down.

Conditions: This symptom is observed on a Cisco 12000 series in a per-interface fast external fallover configuration on a 4-port Gigabit Ethernet ISE line card. However, the symptom may also occur on other platforms that function in a BGP configuration.

Workaround: There is no workaround. Note that the ip bgp fast-external-fallover permit command is currently not supported on subinterfaces.

Symptoms: A stale non-bestpath multipath remains in the RIB after the path information changes, and BGP does not consider the stale path part of the multipath.

Conditions: This symptom is observed on a Cisco router that has the soft-reconfiguration inbound command enabled and occurs only when the BGP Multipath Loadsharing feature is enabled for three or more paths, that is, the number-of-paths argument of the maximum-paths number-of-paths command has a value of three or more.

Workaround: Disable the soft-reconfiguration inbound command for the neighbor sessions for which the BGP Multipath Loadsharing feature is enabled or reduce the maximum number of paths for the BGP Multipath Loadsharing feature to two.

Conditions: This symptom is observed on a Cisco router that is configured for mVPN.

Symptoms: Conditional advertisement of the default route via a route map does not work when you enter the neighbor default-originate command.

Conditions: This symptom is observed on a Cisco router that is configured for BGP.

Workaround: Disable the route map entirely. If this is not an option, there is no workaround.

Symptoms: When an OSPF neighbor comes back up after a very fast (sub-second) interface flap, OSPF routes that are learned via the interface that flapped may not be re-installed in the RIB.

Workaround: Enter the shutdown interface configuration command followed by the no shutdown interface configuration command on the interface that flapped.

Symptoms: A neighbor reloads when you enter the shutdown interface configuration command followed by the no shutdown interface configuration command on an interface of an LSP router that functions as a tunnel headend.

–

The tunnel headend sends a Path via RSVP to the neighbor but the Resv message is delayed.

–

At the neighbor, the cleanup timer for the Path expires before the Resv message arrives, causing the session to be terminated.

Symptoms: The Border Gateway Protocol (BGP) multicast distribution tree (MDT) subaddress family identifier (SAFI) could send faulty notifications to Protocol Independent Multicast (PIM) resulting in a corrupted PIM database.

Conditions: This symptom has been observed when the BGP MDT SAFI receives BGP MDT SAFI rd 2 style updates.

Conditions: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.0(30)S or a later release when the MVPNs function in a mixed network that has both VPNv4 RD2 prefixes for MDT updates and IPv4 MDT subaddress family identifier (SAFI) prefixes.

Conditions: This symptom is observed on a Cisco MGX8850 RPM-XF that runs Cisco IOS Release 12.3(11)T. However, the symptom is platform-independent and may also occur in other releases.

Symptoms: An OSPF router may update and originate a new version of an LSA when it should flush the LSA.

Conditions: This symptom is observed on the originating router when it receives a self-originated MaxAge LSA before it can flush this LSA from its database. This symptom may occur under a rare condition when a neighboring router calculates that is has a newer copy of the LSA from the originating router and bounces the MaxAge LSA to the originating router.

Symptoms: When you enter the show running-config or process-min-time command, a spurious memory access or crash may occur.

Conditions: This symptom is observed on a Cisco router that is configured for OSPFv3.

Symptoms: When an RSVP application (for example, the MPLS TE feature) sends an updated Path message to reflect a modification in its QoS request, the updated Path message may not be forwarded by a downstream RSVP-aware router.

Conditions: This symptom is observed when the downstream RSVP-aware router has two RSVP features configured: local policy and refresh reduction. The commands to configure these features are the ip rsvp policy local command and the ip rsvp signalling refresh reduction command, respectively.

When an RSVP reservation is established with a Path/Resv message handshake and the sender application subsequently transmits an updated Path message that the downstream router applies to an RSVP local policy, the router does not forward the modified Path message. This situation prevents the application from receiving the corresponding Resv message, and may cause the application to fail.

Workaround: If this is an option, unconfigure the local RSVP policy or refresh the reduction and then restart the RSVP application. If this is not an option, there is no workaround.

Symptoms: OSPFv3 may write zeros into single words of memory in the heap. Depending on what (if anything) is allocated at the address that is being cleared, the router may reload or fail in some other way.

Conditions: This symptom is observed only when an OSPFv3 process or area is unconfigured or when you enter the clear ipv6 ospf process command and when both of the following conditions are present:

–

The area that is removed or the process that is cleared contains one or more non-self-originated type-4 LSAs.

–

The router does not have an intra-area path to an ASBR that is described by the type-4 LSA.

Symptoms: A Cisco router that functions in a multicast VPN environment may crash.

Conditions: This symptom is observed when you check the unicast connectivity and then unconfigure a VRF instance.

Symptoms: MVPN traffic is limited to about 9 Mpps and the CPU usage on the egress line card is 100 percent.

Conditions: This symptom is observed on a Cisco router that functions as a PE router when MVPN performs decapsulation in the slow path instead of the fast path.

Symptoms: When the distribute-list route-map map-tag command is used under the OSPF router mode and when the route map is modified, OSPF does not update the routing table based on the changes in the route map.

Conditions: This symptom is observed when a route map that is referenced in the distribute-list route-map map-tag command is modified.

Workaround: Enter the clear ip ospf process id command or the clear ip route * command.

Symptoms: A Cisco router may reload when the RSVP MIB object is polled via SNMP.

Symptoms: A crash that is related to OSPF flooding may occur on a Cisco router that is configured for OSPF and MPLS traffic engineering.

Conditions: This symptom is observed when 1600 OSPF interfaces are configured in an OSPF area that is also configured for MPLS traffic engineering and when OSPF interfaces and OSPF adjacencies flap. A list of the affected releases can be found at http://www.cisco.com/cgi-bin/Support/Bugtool/onebug.pl?bugid=CSCef16096. Cisco IOS software releases that are not listed in the "First Fixed-in Version" field at this location are not affected.

Workaround: Reduce the number of OSPF interfaces in the OSPF area to 300 or less. You can check the number of OSPF interfaces by entering the show ip ospf or show ip ospf interface interface-type interface-number brief command. Note that all interfaces that are covered by network statements are counted.

Symptoms: A BGP speaker may fail to send all of its prefixes to a neighbor if the neighbor sends a refresh request to the BGP speaker at the same time that the BGP speaker is generating updates to the neighbor. This situation causes the neighbor to miss some prefixes from its BGP table.

A common scenario is that a VPNv4 PE router is reloaded and then fails to learn all prefixes from its route reflector (RR). In this configuration, the symptom occurs when the processing of a VRF configuration causes the PE router to automatically generate a route-refresh request to the RR, while the RR is still generating updates to the PE.

Conditions: This symptom is observed when OSPF NSF is configured on an ABR and when a prefix can be learned via an "Area 0" link or via a link through another area (that is, there are redundant paths).

Symptoms: Suboptimal routing occurs in an OSPF configuration or a routing loop occurs between two border routers that redistribute BGP into OSPF.

Conditions: These symptoms are observed when at least two border routers are connected via eBGP to another autonomous system, receive the same prefix over these connections, and redistribute the prefix into OSPF. Under certain conditions, for example when the eBGP session from the preferred BGP exit point to the eBGP peer flaps, the second router in the local autonomous system becomes the preferred path and redistributes the eBGP route into OSPF. When the eBGP session with the first router comes back up, the LSA should be flushed but this does not occur. This situation may create routing problems on other OSPF routers or, when BGP has a higher administrative distance than OSPF, routing loops between both border routers.

Symptoms: When general Bidirectional Forwarding Detection (BFD) functionality is enabled and when Border Gateway Protocol (BGP) is configured without BFD functionality, BFD sessions may be started with the BGP neighbors. This is not proper behavior: BFD sessions should not be started when BGP is configured without BFD functionality.

Conditions: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.0(31)S.

Symptoms: The number of networks in the BGP table and the number of attributes increases, and a slower convergence may occur for members of a BGP update group.

Conditions: This symptom is observed on a Cisco router when the members of a BGP update group go out of synchronization with each other in such a way that they have different table versions, preventing the BGP Scanner from freeing networks that do not have a path.

To check if the members of the BGP update group are in synchronization with each other, enter the show ip bgp update-group summary command and look at the table version for each member. If they have the same table version, they are in synchronization with each other; if they do not, they are out of synchronization with each other.

Workaround: To enable the members of the BGP update group to synchronize with each other, enter the clear ip bgp * soft out command. Doing so does not bounce the sessions but forces BGP to re-advertise all prefixes to each member.

Symptoms: A router that is configured for OSPFv3 may crash because of memory corruption or a CPUHOG condition.

Conditions: This symptom is observed rarely in a configuration with a large LSA with at least 44 links that have OSPFv3 enabled and with some links configured for broadcast mode when an adjacency with a peer router flaps.

Symptoms: When BGP receives an update that has a worse metric route than the previously received route for equal-cost multipath, the BGP table is updated correctly but the routing table is not, preventing the old path from being deleted from the routing table.

Conditions: This symptom is observed on a Cisco router that is configured for BGP multipath.

Symptoms: When you perform an SSO switchover, packets may be lost because of a "no route" condition. The loss of these packets is an indication of a prolonged SSO convergence time. The routes for these packets are restored automatically within a few seconds.

Conditions: This symptom is observed on a Cisco router with dual RPs that function in SSO mode.

Symptoms: One router (R2) may begin sending updates to another router (R1) before R2 has received the BGP prefix list from R1.

R1 does apply its inbound BGP prefix list so routes are denied if they need to be. However, R2 sends routes to R1 which are denied by R1.

Conditions: This symptom is observed when both routers have negotiated a BGP outbound route filter (ORF) and when R1 sends its BGP prefix list to R2.

Symptoms: When an SSO switchover occurs, the newly active Supervisor Engine or RP generates a series of CPU Hog messages in the PIM Process, generates tracebacks, and finally crashes because the watchdog timer expires.

Conditions: This symptom is observed on a Cisco switch that has redundant Supervisor Engines and on a Cisco router that has redundant RPs when Auto-RP is configured and when regular multicast traffic runs for a few hundred multicast routes.

Symptoms: A Rendezvous Point is down but multicast routing continues to function.

Conditions: This symptom is observed when a statically mapped Rendezvous Point is defined as an interface address and when the interface is in the down/down state. In this situation, the router still attempts to become the Rendezvous Point for the defined group or groups.

Workaround: Do not use a a statically mapped Rendezvous Point. Rather, configure Auto-RP or BSR to configure a dynamic Rendezvous Point.

Symptoms: A router that is configured for OSPFv3 may crash because of memory corruption or a CPUHOG condition.

Conditions: This symptom is observed rarely in a configuration with a large LSA with 64 parallel links that have OSPFv3 enabled in broadcast mode when all adjacencies with a peer router flap.

Conditions: This symptom is observed when there is an interface in the global route table that has an IP address that matches the IP address of the OSPF sham link neighbor.

Workaround: Reconfigure the routers so that the IP address of the OSPF sham link neighbor does not match any IP addresses of interfaces in the global route table.

Alternate Workaround: Shut down the interface or change the IP address of the interface in the global route table.

Symptoms: The output of the show memory summary command may contain garbled characters in the "What" column.

Conditions: This symptom is observed when you configure OSPF with at least one network, and then unconfigure it.

Conditions: This symptom is observed rarely when all of the following conditions are present:

–

There is a very brief (shorter than 500 ms) interface flap on a point-to-point interface such as a POS interface.

–

The flap is not noticed by the neighbor, so the neighbors interface remains up.

–

The OSPF adjacency goes down and comes back up very quickly (the total time is shorter than 500 ms).

–

OSPF runs an SPF during this period and, based on the transient adjacency information, removes routes via this adjacency.

–

The OSPF LSA generation is delayed because of LSA throttling. When the LSA throttle timer expires and the LSA is built, the LSA appears unchanged.

Workaround: Increase the carrier-delay time for the interface to about 1 second or longer.

Alternate Workaround: Use an LSA build time shorter than the time that it takes for an adjacency to come up completely.

Symptoms: When an inter-area, external, or Not-So-Stubby Area (NSSA) route is learned via a link state update that follows the initial database synchronization, the route may not be added to the routing table by a partial shortest path first (SPF) computation even though the LSA is installed in the link state database. A subsequent full SPF computation causes the route to be added.

Conditions: This symptom is observed on a Cisco router and is most likely to occur when a large number of type 3, type 5, or type 7 LSAs are advertised and withdrawn.

Symptoms: A router reloads unexpectedly when a continue statement is used in an outbound route map.

Conditions: This symptom is observed on a Cisco router that is configured for BGP.

Conditions: This symptom is observed on a Cisco router that runs a Cisco IOS software image that integrates the fix for caveat CSCei47926. A list of the affected releases can be found at http://www.cisco.com/cgi-bin/Support/Bugtool/onebug.pl?bugid=CSCei47926. Cisco IOS software releases that are not listed in the "First Fixed-in Version" field at this location are not affected.

Symptoms: BGP does not advertise all routes to a peer that sends a route-refresh request.

–

The router is in the process of converging all of its peers and has updates ready in the output queue for the peer.

–

The peer sends a route-refresh request to the router. This may occur when the clear ip bgp * soft in command is entered on the peer or when a VRF is added to the peer.

–

The router processes the route-refresh request from the peer while the router still has updates in the output queue for the peer.

In this situation, all of prefixes that are advertised by the unsent updates in the output queue for the peer are lost.

Workaround: There is no workaround. When the symptom has occurred, enter the clear ip bgp * soft out command on the router to force the router to send all updates to its peers.

ISO CLNS

Conditions: This symptom is observed when IS-IS is used to advertise IP prefixes and when you enter a distance command that changes the overall configuration but keeps a subset of the prefixes at the same distance as in the previous configuration. The routes for which the distance does not change may be removed from the routing table.

The following two examples show configurations in which the symptom occurs. When the distance configuration for IS-IS is 115 ip and you enter one of the following command sequences, the symptom occurs:

Workaround: For all prefixes, configure distances that differ from the distances that were initially configured.

Symptoms: IPv6 routes that are learned from other IPv6 routers are not installed in the RIB.

Conditions: This symptom is observed on a Cisco router that is configured for Multi-topology IS-IS in transition mode. This symptom does not occur when the router is configured for Multi-topology IS-IS without the transition mode.

Workaround: Use the default IS-IS metric on the interfaces that are configured for IPv6 IS-IS.

Symptoms: A Cisco device running IOS and enabled for Intermediate System-to-Intermediate System (IS-IS) routing protocol may reset with a SYS-2-WATCHDOG error from a specifically crafted malformed IS-IS packet. The IS-IS protocol is not enabled by default.

Conditions: The IS-IS crafted malformed IS-IS Packet that requires processing will not be forwarded across a Level 1/Level 2 boundary. The specifically crafted malformed IS-IS packet would require local attachment to either a Level 1 or Level 2 router. A Cisco device receiving the malformed IS-IS packet will forward the malformed packet to its neighbors, and may reset.

Workaround: There is no workaround. Enabling IS-IS Authentication is seen as a best practice, and can be leveraged as a mitigation technique.

–

An SSO switchover occurs on the router or the router loses and then regains connectivity to the level-2 topology.

Workaround: Trigger a change that causes the router to regenerate its level-2 LSP.

Symptoms: A 30 to 40 ms interruption in traffic forwarding may occur when you modify the tunnel mpls traffic-eng bandwidth command for an MPLS traffic-engineering tunnel.

Conditions: This symptom is observed on a router that is configured for MPLS traffic engineering with IS-IS as the associated IGP.

Symptoms: A route that fails remains in the routing table with its old metric, preventing an alternate route from being used and causing a routing loop.

Conditions: This symptom is observed in a network that is configured for IS-IS and iSPF when the IP routes that are advertised in an LSP (irrespective of whether or not the LSP is fragmented) do not age-out during a rerouting failure.

Workaround: Remove iSPF from the IS-IS process by entering the router isis command followed by the no ispf command.

Symptoms: When an IPv4 prefix list is used in a redistribution command for the IS-IS router process, a change in the prefix list is not immediately reflected in the routing tables of a router and its neighbor. The change may take up to 15 minutes to take effect.

Conditions: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.0(28)S.

Workaround: To have a change take effect immediately, enter the no redistribute route-map command followed by the redistribute route-map command for the IS-IS router process.

Symptoms: A router may reload unexpectedly when you remove the IS-IS configuration at the interface or router level.

Conditions: This symptom is observed on a Cisco router when the following conditions are present:

–

The isis protocol shutdown interface configuration command is enabled on the interface.

–

You enter an interface configuration command that enables IS-IS such as an isis command, a clns command, or the ipv6 router isis before you enter the a router configuration command such as the net command.

When you now remove the IS-IS configuration at the interface or router level, the router may reload.

Workaround: Remove the isis protocol shutdown interface configuration command before you remove IS-IS from the interface or router level.

Miscellaneous

Symptoms: Temporary performance degradation may occur on a router, and the following error message is generated:

Conditions: This symptom is observed on a Cisco 7200 series that is configured with a large number of PVCs.

Symptoms: Traffic is dropped for up to 15 seconds while a recursive prefix is being resolved.

Conditions: This symptom is observed when a new recursive prefix is learned by CEF and when a less-specific prefix already exists. Traffic that would have been forwarded using the less-specific prefix is dropped for up to 15 seconds while the new recursive prefix is being resolved.

Symptoms: High CPU use may occur at the interrupt level on an ingress port adapter or line card that is configured for hardware multicast when there is a high multicast traffic rate.

Conditions: This symptom is observed when policy-based routing (PBR) matches the multicast traffic and when a switchover to another interface occurs.

Workaround: Change the deny statement in the PBR configuration so traffic for multicast destination addresses is denied earlier.

Alternate Workaround: For a short while, remove the PBR configuration from the ingress interface to enable multicast traffic hardware forwarding to be established.

Further Problem Description: PBR should not influence multicast traffic and it does not when traffic is switched in the hardware. When a switchover to a new interface occurs, multicast packets are initially forwarded in the software until hardware forwarding can take over. PBR interferes with the initial software-switched packets and prevents hardware entries from being created.

Symptoms: The atm abr rate-factor interface configuration command cannot be configured on an interface.

Conditions: This symptom is observed when an available bit rate (ABR) connection is added to a Route Processor Module-PRemium (RPM-PR) card on a Cisco MGX 8850 Processor Switch Module (PXM1) card that has a Cisco WAN Manager (CWM) carrier module (CM). The symptom may also occur on other platforms.

Workaround: Use the command-line interface (CLI) to add an ABR connection to the RPM-PR on the Cisco MGX 8850 PXM1 card.

Symptoms: A Cisco 7200 series may reload unexpectedly during a service-policy configuration.

Conditions: This symptom is observed when you attach a level 2 policy map as a child of a level 1 policy map and when the level 1 policy map is already attached to an interface.

Symptoms: A 4-port serial enhanced port adapter (PA-4T+) may receive packets, even though the status of the serial interface is "down/down."

Conditions: This symptom is observed on a PA-4T+ that is installed in a Cisco 7200 series router and that is connected to a 1-port serial WAN interface card (WIC-1T) that is installed in a Cisco 2600 series. The serial interfaces of both routers are connected with a CSU/DSU.

The input packet counter of the serial port of the PA-4T+ increments even though the status of the serial interface is "down/down." However, the 2600 series functions properly, and the input packet counter of its serial interface does not increment.

Symptoms: An Engine 4 or Engine 4+ line card in a Cisco 12000 series that switches multicast traffic may reload.

Conditions: This symptom is observed on a Cisco 12000 series when the ip multicast-routing distributed command is configured.

Workaround: Configure an access control list (ACL) to block ingress traffic with a source address of 0.0.0.0, as in the following example:

Note that this ACL does not have any effect on multicast traffic because multicast traffic with source address 0.0.0.0 is dropped anyway by the Reverse Path Forwarding (RPF) check.

Symptoms: An LSP ping reports that an LSP is fine although the LSP is unable to carry MPLS payloads such as VPN traffic.

Conditions: This symptom is observed on a Cisco router when MPLS echo request packets are forwarded from untagged interfaces that are directly connected to the destination of the LSP ping and when the IP time-to-live (TTL) value for the MPLS echo request packets is set to 1.

Symptoms: An inter-AS TE tunnel continues to resignal a path that is rejected instead of switching to a second path.

Conditions: This symptom is observed when the RSVP local policy on the ASBR rejects the path message because it does not meet the policy.

Symptoms: A VIP in which a PA-2FE port adapter is installed may reload because of memory corruption that is caused by a hardware issue of the PA-2FE port adapter.

Conditions: This symptom is observed when the VIP and port adapter function under stress, when the VIP is unable to serve memory read/write requests from the port adapter, and when there are PCI retry timeouts.

Symptoms: A Cisco 7500 series that has a multichannel E1 port adapter that is configured with PPP multilink groups on several E1 channels may drop packets:

–

When dCEF is enabled, packets that are larger than 1492 bytes are dropped. Note that this particular symptom is addressed and fixed via CSCin73658.

–

When the router has dCEF disabled, packets that are larger than 1500 bytes are dropped.

Conditions The symptom is observed on a Cisco 7500 series that runs Cisco IOS Release 12.0(28)S.

Workaround: Configure only one interface as the member of the PPP multilink group. Note that the symptom does not occur in Release 12.0(26)S3, 12.0(27)S2, and 12.1(19)E1.

Conditions: This symptom is observed on a Cisco 7500 series that functions in an SSO configuration when you perform an OIR of the primary RSP.

Conditions: This symptom is observed on a low-end router that uses a Compact Flash (CF) device when the root directory is nearly full with file entries.

Workaround: Keep the number of file entries in the root directory relatively small (less than 100).

Symptoms: APS does not function correctly on a channelized OC-48 ISE line card and the output of the show aps command shows that the line card is down.

Symptoms: When you enter the shutdown interface configuration command on the outgoing interface of a Traffic Engineering (TE) Label Switched Path (LSP), the Resv state should be removed immediately. However, the Resv state remains until a PathTear arrives or a timeout causes the TE LSP to be torn down.

When the TE headend is a Cisco router, the PathTear is sent very quickly and the state is removed.

Conditions: This symptom is observed on a Cisco router that runs a Cisco IOS software image that contains the fix for caveat CSCec26563 when the router has MPLS TE tunnels enabled.

Symptoms: IPv6 packets may be incorrectly forwarded by a Cisco 10720 or packet forwarding performance may be degraded significantly. Traceroute packets may be misrouted or high CPU use may occur when IPv6 packets are punted to the Route Processor.

Conditions: These symptoms are observed on a Cisco 10720 that runs Cisco IOS Release 12.0(27)S1.

Symptoms: The following error message and spurious memory access may be generated on a Cisco 7500 series or Cisco 7600 series that is configured for dMLFR.

%ALIGN-3-SPURIOUS: Spurious memory access made at 0x418FC0E0 reading 0x8 %ALIGN-3-TRACE: -Traceback= 418FC0E0 4026B644 40699284 40699A3C 40699368 40E80B84 40E7215C 4068A8AC

Conditions: This symptom is observed immediately after an MFR interface is created, after a switchover has occurred, or when a link flaps continuously.

Symptoms: When you send multicast traffic over an IPSec tunnel, a memory leak may occur on a router.

Conditions: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.3T when both IP CEF and hardware encryption are configured. The symptom may also occur in other releases.

Workaround: Switch to software encryption for a while and then switch back to hardware encryption.

Conditions: This symptom is observed on a Cisco platform while BFD sessions flap.

Symptoms: Packet drops occur in the ingress direction on a dMLP or dMLFR link with traffic at 95-percent of the line rate and when the number of packets with a small size is high.

Conditions: This symptom is observed on a Cisco 7500 series that functions as a provider edge (PE) router, that is configured for L2TPv3 L3VPN, and that has dMLP or dMLFR links to a customer edge (CE) router.

Symptoms: The performance of a router may be severely degraded (at approximately 90 percent of the line rate) when large packets are processed, when the MLP bundle link flaps, and when the router does not recover the MLP sequence numbers of the packets.

Conditions: This symptom is observed on a Cisco 7500 series and Cisco 7600 series that are configured for dMLP only when large packets are processed.

Symptoms: Shortly after a Cisco IOS software boot loader image has been downloaded, a PRP-2 may crash and does not reload.

Conditions: This symptom is observed on a Cisco 12000 series that runs the boot loader image of Cisco IOS Release 12.0(30)S.

An 8-port OC-48c/STM-16c POS Engine 6 line card may report the error following messages, indicating that the Rx power has exceeded the alarm levels for the port:

SLOT X: %LC_SFP-2-SFP_RX_POWER_HI_ALARM: RX power high alarm
SLOT X: %LC_SFP-4-SFP_EVENT_TX_FAULT: TX FAULT signal raised on port Y
SLOT X: %LC_SFP-4-SFP_EVENT_TX_FAULT: TX FAULT signal cleared on port Y

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(27)S2 or Release 12.0(31)S.

Symptoms: After a line card crashes, the crashinfo file may be saved to bootflash with a size of 0 bytes.

Workaround: There is no workaround. To troubleshoot the crash, enter the show bootflash: all to see the logs of the event.

Symptoms: You cannot ping with packets of certain sizes after an RPR+ switchover or after an interface flap on a multilink interface that has members of non-channelized port adapters when the multilink interface is configured with fragmentation and interleaving.

Conditions: This symptom is observed on a Cisco 7500 series and Cisco 7600 series.

Symptoms: When you change the MTU on a serial interface, the standby Route Processor (RP) generates tracebacks, enters the STANDBY COLD-CONFIG state, and reboots.

Conditions: This symptom is observed on a Cisco router when you change the MTU size to 1400 or a lesser value.

Symptoms: The ATM interface route cache counters do not increment after you have entered the ip cef command.

Conditions: This symptom is observed on a Cisco router that runs a Cisco IOS interim release for Release 12.0(31)S, that functions as a PE router, and that has L2 transport VCs configured.

Symptoms: A router may take a very long time to establish LDP sessions with its peers and advertise its label bindings. In some cases, the LDP sessions may flap.

Conditions: This symptom may occur when a Cisco router that uses LDP for label distribution has a large number (greater than 250) of LDP neighbors and several thousand label bindings to advertise.

Workaround: The time required to establish the neighbor sessions and advertise the label bindings when TDP is used in place of LDP may be substantially less. Using TDP in place of LDP will result in an acceptable convergence behavior.

Symptoms: L2TPv3 sessions are re-established after their L2TP class has been deleted.

Conditions: This symptom is observed when you enter the no l2tp-class global configuration command to delete the class that is used by existing Xconnect paths via a pseudowire class.

Conditions: This symptom is observed on a Cisco 7200 series and Cisco 7500 series that are configured for MPLS but may be platform-independent.

Symptoms: Bidirectional PIM DF election does not occur correctly when a PIM neighbor expires.

Conditions: This symptom is observed when the PIM neighbor that expires is the designated forwarder (DF) for multiple RPs. The DF election is triggered only for the first RP on the list and does not occur for all the other RPs.

Symptoms: A CSC OIR may cause a 6-port channelized T3 line card that is configured for FRoMPLS to fail.

Conditions: This symptom is observed on a Cisco 12000 series that runs a Cisco IOS interim release for Release 12.0(31)S and that has its redundancy mode set to RPR.

Symptoms: Traffic may not fully converge after you have reloaded a line card with a scaled configuration or after an HA switchover.

Conditions: This symptom is observed on a Cisco 12416 that runs the c12kprp-p-mz image of a Cisco IOS interim release for Release 12.0(31)S and that is configured with dual PRPs and a scaled configuration.

Symptoms: On a Cisco XR 12000 series hardware-based forwarding line card, the receive counters in the output of the show mpls l2transport vc command do not work in any images for AToM.

Conditions: This symptom is observed on all hardware-based engine line cards on a Cisco XR 12000 series that is configured for AToM and Sampled NetFlow on the core-facing line cards.

Symptoms: A 1-port channelized OC-12/STM-4 (DS1/E1) ISE line card on a CE router may crash when many (168) MLP interfaces are deleted and reconfigured via TFTP on a directly-connected PE router.

Conditions: This symptom is observed on a Cisco 1200 series that functions as a CE router.

Symptoms: Packets that flow to VPNv4 destinations may be dropped for up to one second when the next-hop router clears its IS-IS overload bit after having been rebooted.

Conditions: This symptom is observed in a MPLS-TE network with one-hop TE tunnels.

Symptoms: When the working link flaps with two to three second intervals on CHOC12 Internet Services Engine (ISE) line cards that are configured for automatic protection switching (APS), some T1 links may remain down.

Symptoms: High jitter occurs on a 1-port channelized OC-12/STM-4 (DS1/E1) ISE line card when when many (168) MLP interfaces are congested with real-time and normal data traffic.

Symptoms: When you configure MPLS Traffic Engineering AutoTunnel Mesh Groups by entering the following commands, many FIB messages and tracebacks are generated:

mpls traffic-eng auto-tunnel mesh
mpls traffic-eng auto-tunnel mesh tunnel-num min 10000 max 20000

Depending on the configuration of he router and the topology, the symptom may also occur when you enter the following sequence of commands:

no mpls traffic-eng auto-tunnel mesh
mpls traffic-eng auto-tunnel mesh
no mpls traffic-eng auto-tunnel mesh
mpls traffic-eng auto-tunnel mesh

Conditions: This symptom is observed on a Cisco 12000 series that runs the gsr-p-mz image of a Cisco IOS interim release for Release 12.0(31)S.

Workaround: When you want to disable and then re-enable the mpls traffic-eng auto-tunnel mesh command, save the configuration and reload the router after you have disabled the command and before you re-enable the command, as in the following example:

no mpls traffic-eng auto-tunnel mesh
...
copy run start
reload
mpls traffic-eng auto-tunnel mesh

Similarly, when you want to change the range of unit numbers (that is, tunnel IDs) that is used by the mpls traffic-eng auto-tunnel mesh command, disable the command, change the range, save the configuration, reload the router, and then re-enable the command, as in the following example:

no mpls traffic-eng auto-tunnel mesh
mpls traffic-eng auto-tunnel mesh tunnel-num min 10000 max 20000
...
copy run start
reload
mpls traffic-eng auto-tunnel mesh

Symptoms: Changing the encapsulation of a POS interface from HDLC to PPP or the other way around causes the link to go down.

Conditions: This symptom is observed when you change the encapsulation of a POS interfaces on an Optical Services Module (OSM) that is installed in a Cisco Catalyst 6000 series or Cisco 7600 series. However, the symptom is platform-independent.

Workaround: Power-cycle the OSM by entering the no power enable module slot command followed by the power enable module slot command.

Symptoms: An inter-AS TE LSP fails to send a signal after a router is rebooted as an ASBR.

Conditions: This symptom is observed when there are parallel links between ASBRs with a combination of point-to-point and broadcast interfaces that are configured with the MPLS Traffic Engineering—Inter-AS TE feature and (passive) link flooding.

Symptoms: A line card or port adapter may crash repeatedly when the router in which the line card or port adapter is installed comes up after a software-forced crash has occurred on the router.

Conditions: This symptom is observed on a Cisco router that is configured for IPv6 MPLS and that has redundant RPs that are configured for SSO. The symptom is more likely to occur when the ipv6 unicast-routing command is enabled.

Workaround: To diminish the chance that the symptom occurs, disable the ipv6 unicast-routing command.

Symptoms: A PRP switchover causes "%SYS-2-NOTQ" and "%SYS-2-LINKED" error messages and some tracebacks to be generated on a 1-port channelized OC-12c/STM-4 (DS1/E1) ISE line card, the serial interfaces of the line card flap, and eventually the line card resets.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(28)S or Release 12.0(30)S, that is configured with two PRPs, and that has the redundancy mode set to SSO.

Symptoms: In the a tag switching-to-IP switching scenario with an ISE ingress line card and an Engine 4 plus (E4+) egress line card, the following bad packets may be forwarded to the E4+ line card:

These bad packets cause packet corruption and a "TX192-3-PAM_PIM" error message on the E4+ line card and may even cause the E4+ line card to reset.

Further Problem Description: The fix for this caveat enables the ISE line card to drop the above-mentioned bad packets.

Symptoms: When an attachment circuit is configured for AToM pseudowire redundancy, an MPLS core network failure on the primary pseudowire may not cause a switchover to the redundant (or backup) pseudowire.

Conditions: This symptom is observed on a Cisco 7500 series that runs Cisco IOS Release 12.2(27)SBA and on a Cisco 12000 series that runs a Cisco IOS interim release for Release 12.0(31)S.

Symptoms: After removing a large number of Frame Relay subinterfaces, the following log is displayed:

SYS-3-CPUHOG: Task ran for 38160 msec (3/2), process = MDFS LC Process, PC = 41129150

Conditions: This symptom is observed on a Cisco 12000 series that is configured for Multicast VPN.

Symptoms: Links flap on a 1-port channelized OC-12/STM-4 (DS1/E1) ISE line card after an RP switchover has occurred.

Conditions: This symptom is observed on a Cisco 12000 series that has two PRPs and that runs Cisco IOS Release 12.0(31)S.

Symptoms: On a Cisco 12000 series router with a 1xChOC12/DS1 ISE line card configured with multilink MFR protocol and a MQC policy, after a reload the QoS does not get applied to the bundle. The QoS goes to the suspend mode.

Conditions: The bundle loses its QoS policy when the router is reloaded. This problem is observed when running Cisco IOS Releases 12.0(28)S1, 12.0(30)S, or an interim release for Release 12.0(31)S.

Symptoms: After you perform an OIR of a PA-SRP-OC12 port adapter on a Cisco 7200 series, the router may not show any nodes in the SRP ring and may stop forwarding traffic.

Conditions: This symptom is observed on a Cisco 7200 series that runs Cisco IOS Release 12.3(13) or Release 12.3(11)T3. The symptom may also occur in other releases.

Symptoms: Some %GSRSPA-3-PORT_IF_INDEX & %EELC_QOS_RES_MGR-3- HW_IDB_INDEX_NOT_FOUND messages may be seen on Engine 5 Gigabit Ethernet (GE) Shared Port Adapters (SPAs) when an MPLS traffic engineering tunnel is enabled on the interface.

Conditions: This symptom is observed on an Engine 5 GE SPA in a Cisco 12000 series router that is running Cisco IOS Release 12.0(31)S.

Symptoms: An Engine 3 line card sends unlabeled traffic after it has been toggled from explicit routing to default routing. The symptom is related to the handling of a default-route on an Engine 3 ingress line card that functions in an IP-to-MPLS path.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(30)S1 or any other image that includes the fix for caveat CSCsa64782, which is a preliminary requisite for default-route handling on an Engine 3 line card. The symptom occurs in the following scenario:

You configure BGP to advertise the target address, so the target address is directly known in the routing table.

You remove the advertisement from BGP and return to default routing, with the same source for the next hop as the platform that was the BGP next hop.

You enter the clear ip route network command, with the address of the BGP next hop for the network argument.

After the transition from non-default routing to default routing, entering the clear ip route network command, with the address of the next hop for the network argument, causes an inconsistency, and traffic is forwarded as unlabeled.

Workaround: To restore proper operation, enter the clear ip route 0.0.0.0 command.

Symptoms: In a tag switching-to-IP switching scenario with an ISE ingress line card and an Engine 4 plus (E4+) egress line card, the following bad packets may be forwarded to the E4+ line card:

These bad packets cause packet corruption and a "TX192-3-PAM_PIM" error message on the E4+ line card and may even cause the E4+ line card to reset.

Further Problem Description: The fix for this caveat enables the ISE line card to drop the above-mentioned bad packets.

Symptoms: The following incorrect error message is generated when an invalid QOS policy is applied to an L2 ATM interface before a valid policy is applied:

Conditions: This symptom is observed only on a Cisco router that is configured with a secondary RP and ATM L2VPNs that function in the VP mode.

Conditions: This symptom is observed on a Cisco 12000 series that is configured for multicast traffic.

Symptoms: When the router is configured with a new area, the links that are configured for TE are not flooded in the new area.

Conditions: This symptom is observed when you configure an area by entering the mpls traffic-eng area number command as part of the router OSPF configuration.

Symptoms: Fault recovery may fail to recover memory contents on ECC faults for certain FSRAM memory locations.

Conditions: This symptom is observed on a Cisco router that runs a Cisco IOS software image that includes the fix for DDTS ID CSCeg74562. A list of the affected releases can be found at http://www.cisco.com/cgi-bin/Support/Bugtool/onebug.pl?bugid=CSCeg74562. Cisco IOS software releases that are not listed in the "First Fixed-in Version" field at this location are not affected.

Symptoms: When you remove one CSC or one SFC, all traffic stops because there is insufficient fabric bandwidth. However, when you re-insert the CSC or SFC, all interfaces remain down.

Conditions: This symptom is observed when a Cisco 12000 series that functions as a PE router is located between the two CE routers.

Workaround: Reset the PIM neighbor for the CE router that has the lower IP address of the two CE routers.

Symptoms: A PE router that is configured for MPLS VPN—Carrier Supporting Carrier drops decapsulation traffic in the direction of a CE router. Encapsulation traffic works fine and is not affected.

Conditions: This symptom is observed when MPLS VPN--Carrier Supporting Carrier is configured with Label Distribution Protocol (LDP) as the protocol between the PE router and a CE router. In some circumstances such as a BGP peer flap or a route flap, LDP may free the local label that is allocated by BGP while BGP still uses the label. The same label may be allocated later for a different prefix, causing multiple prefixes to use the same local label, and, in turn, causing connectivity for the affected prefixes to fail.

Symptoms: When a router is reloaded with a scaled L2VPN configuration, the secondary RP may reload unexpectedly because of a "BFRP HA Chkpt send" failure.

Conditions: This symptom is observed on a Cisco 12000 series that runs the c12kprp-p-mz image of a Cisco IOS interim release for Release 12.0(31)S.

Symptoms: When an HA switchover occurs in an L2VPN scaled configuration, some of the subinterfaces on a 4-port Gigabit Ethernet ISE line card may use the incorrect MAC address for the destination MAC address, which you can see in the output of the show adjacency detail command.

Conditions: This symptom is observed on a Cisco 12000 series that runs the c12kprp-p-mz image of a Cisco IOS interim release for Release 12.0(31)S.

Workaround: Enter the shutdown interface configuration command followed by the no shutdown interface configuration command on the affected subinterfaces.

Symptoms: When traffic flows from an Engine 6 line card to a link-bundle interface (a POS interface or PortChannel interface), the Engine 6 line card cannot load-balance traffic between the physical ports that are part of the link bundle. This situation may cause traffic to be lost and may prevent the interface from being used to its full capacity.

Conditions: This symptom is observed on a Cisco 12000 series that runs a Cisco IOS interim release of Release 12.0(31)S and occurs for IP-to-IP traffic and when MPLS is globally disabled

Symptoms: An RP switchover may cause a 6-port channelized T3 Engine 0 line card to fail.

Conditions: This symptom is observed on a Cisco 12000 series that runs a Cisco IOS interim release for Release 12.0(31)S, that has two PRPs, and that functions in RPR+ redundancy mode.

Symptoms: An Engine 4 plus (E4+) line card that functions in an IP-to-tag switching scenario may generate "TX192-3-PAM_MODULE" and "%TX192-3-PAM_PIM" error messages and tracebacks or may crash.

Conditions: This symptom is observed on a Cisco 12000 series when the ingress interface is an Engine 2 line card that has an input ACL and when an external LDP flap occurs that affects the Engine 4+ line card.

Symptom 1: After you have disabled MVPN on a VRF interface, the CPU use for the PIM process increases to 99 or 100 percent and remains at that level.

Condition 1: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.2SB, Release 12.2SX, or a release that is based on these releases. The symptom may also occur in other releases.

Workaround 1: Before you disable MVPN on the VRF interface, enable and then disable multicast routing by entering the ip multicast-routing vrf vrf-name global configuration command followed by the no ip multicast-routing vrf vrf-name global configuration command.

Symptom 2: A router that functions under stress and that is configured with a VRF interface may crash when an MDT group is removed from a remote PE router.

Condition 2: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.2SB, Release 12.2SX, or a release that is based on these releases, and occurs only when there are frequent link flaps or other multicast topology changes that affect the VRF interface. The symptom may also occur in other releases.

Symptoms: A router or line card may reload when you enter any of the following commands:

Conditions: This symptom is observed when you enter above-mentioned commands on the console of a Cisco 7304 or on the console of a line card on a Cisco 7600 series or Cisco 12000 series when these routers are configured with any of the following SPAs (some SPAs are specific to the Cisco 7304 and some to the Cisco 12000 series):

Symptoms: After a Route Processor or line card has reloaded, the queue limit that is set for the class default is not properly programmed.

Conditions: This symptom is observed on a Cisco 12000 series 1-port channelized OC-12/STM-4 (DS1/E1) ISE line card that has an egress policy applied to a serial interface.

Symptoms: A Engine 3 ISE line card may not properly handle incoming bad IP packets but may generate a traceback and a transient error message:

%GSR-3-INTPROC: Process Traceback= 400E10B4 400FBA2C
-Traceback= 4047917C 405E5274 400F4B58
%EE48-3-BM_ERRS: FrFab BM SOP error 40000
%EE48-3-BM_ERR_DECODE: FrFab SOP macsopi_bhdr_pkt_len_zero_err

%GSR-3-INTPROC: Process Traceback= 400E1090 400FBA2C
-Traceback= 4047917C 405E5274 400F4B58
%LC-4-ERRRECOVER: Corrected a transient error on line card.

Conditions: These symptoms are observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(26)S1 or Release 12.0(26)S5a.

Symptoms: An interface stops traversing traffic through an Engine 6 2xOC192 POS line card.

Conditions: This symptom occurs when AToM is configured on an Engine 6 2xOC192 line card (facing the edge).

Symptoms: The no form of a configuration command such as the no shutdown controller configuration command may not be properly synchronized to the standby RP.

Conditions: This symptom is observed on a Cisco 12000 series that functions in SSO redundancy mode and that is configured with a 6-port channelized T3 line card. When you enter the no shutdown controller configuration command on the T3 controller of the 6-port channelized T3 line card, the no shutdown configuration does not synchronize to the standby RP, and after an SSO switchover has occurred, the T3 controller remains in the adminDown state on the newly active RP.

Workaround: Manually enter no shutdown controller configuration command on the T3 controller after the RP switchover has occurred.

Symptoms: When configuring service policies on any interface, the console erroneously displays a message indicating that Rate Limit and Policing can only be configured together on 4 port ISE Ethernet cards. More importantly, it prevents the service policy from being applied to the interface.

Conditions: This symptom has been observed on startup or anytime the configuration is entered.

Symptoms: When Policy Based Routing (PBR) is configured with the set interface command, packets continue to be forwarded to an interface when that went down, causing packets to be dropped. When the ip local policy route-map command is enabled, all locally-generated packets are impacted.

Conditions: This symptom is observed on a Cisco router and only applies to packets that require process-switching.

Workaround: Do not enter the set interface command. Rather, enter the set ip next-hop command.

Symptoms: When a line card has reloaded because you reloaded the router, the line card crashed, or you entered a command to reload the line card, the following message may appear on the console:

%MDS-2-RP: MDFS is disabled on some line card(s). Use "show ip mds stats linecard" to view status and "clear ip mds linecard" to reset.

This message may be generated because MDFS is erroneously disabled on the reloaded line card. Erroneous disabling of MDFS may unnecessarily extend network convergence time.

Conditions: This symptom is observed on a distributed router or switch such as a Cisco Catalyst 6000 series, Cisco 7500 series, Cisco 7600 series, Cisco 10000 series, and Cisco 12000 series. The symptom occurs when the router has the ip multicast-routing distributed command enabled for any VRF and when a line card is reloaded more than 50 seconds into the 60-second MDFS flow-control period.

Workaround: The symptom corrects itself after 60 seconds. Alternatively, you can enter the clear ip mds linecard slot number command.

Symptoms: A Cisco router crashes when performing an FPD image upgrade operation.

Conditions: This symptom can occur if the sending of the FPD image from RP to a line card fails because the target card is reloaded during this time window.

Workaround: Do not reload or physically remove the target upgrade line card when performing an FPD image upgrade.

Conditions: This symptom occurs whenever the operator enters sub-interface mode with an xconnect statement on the EPA-GE/FE-BBRD fixed port on the modular GE/FE card.

Workaround: While in the sub-interface mode, the operator can enter a shutdown command followed by a no shutdown command which allows the subinterface to resume forwarding.

Each time the operator enters the sub-interface on the fixed port, the workaround will need to be applied.

Symptoms: An RPR+ switchover may cause a VIP or line card to pause indefinitely.

Conditions: This symptom is observed when a high load of traffic passes through interfaces of a VIP or line card when these interfaces are configured for QoS.

Symptoms: When you enter the clear cef linecard command, CEF may be disabled on a POS ISE line card and does not recover by itself.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(28)S3.

Symptoms: If an existing file is extended, an ATA file system may become corrupted. When this situation occurs, the output of the dir command or of a show command does not list the files because the files are corrupted.

Conditions: This symptom is observed when you enter any command that extends a file such as the show interfaces ethernet | append disk0:file command.

Symptoms: When Multiprotocol Label Switching (MPLS) is enabled on a router, one or more LDP sessions may be disrupted during periods of extremely high CPU use.

Conditions: This symptom is observed when the CPU use of the router temporarily increases to more than 90 percent for several tens of seconds and when one or more high-priority processes are frequently active but do not necessarily use many CPU cycles.

For example, high CPU use may occur when a peer router is reloaded or when an interface with several hundreds of numbered IP subinterfaces comes up, which causes many processing changes on the router because of the "Tagcon Addr" process.

On a Cisco 12000 series, high CPU use may occur because of the "Fabric ping" high-priority process, which is frequently active.

Workaround: To increase the length of the hello adjacency holdtimes, enter the mpls ldp discovery hello holdtime command on the affected router. You may need to enter this command on all platforms in the network in order to provide full protection.

Symptoms: When an RP switchover occurs, the standby RP crashes, causing the switchover downtime to be longer than expected and traffic to be affected.

Conditions: This symptom is observed on a Cisco 12000 series that is configured for HA when you perform an RP switchover, when the active RP runs Cisco IOS Release 12.0(28)S4, and when the standby RP runs a Cisco IOS interim release later than Release 12.0(31)S.

Symptoms: In the outputs of the show ip psa-cef and show ip cef commands for an Engine 2 ingress line card, the "Local OutputQ (Unicast)" information may point to another and incorrect slot than the slot that the global CEF table points to.

When this symptom occurs, packets that are destined for these specific IP address are dropped.

Conditions: This symptom is observed on a Cisco 12000 series when an Engine 2 line card is used as an ingress line card for traffic that is directed to a default route.

Symptoms: An RP may crash continuously when you reload all the line cards in a dual-RP router that has the redundancy mode is set to SSO.

Conditions: This symptom is observed on a Cisco 12000 series that is configured with two GRPs or two PRPs that are configured for SSO and occurs only when a 1-port channelized OC-48 ISE line card, a 4-port channelized OC-12 ISE line card, or 16-port channelized OC-3 ISE line card is present in the router.

Symptoms: On a channelized OC-48 ISE line card with APS configured, a "Signal Failure" condition remains after the line card has been reloaded or after you enter the shutdown command followed by the no shutdown command.

Conditions: This symptom is observed on a Cisco 12000 series and affects only a channelized OC-48 ISE line card with an APS configuration.

Symptoms: A 3-port Gigabit Ethernet (3GE-GBIC-SC) line card that is configured for Fast-Path Multicast Forwarding may reset when receiving specific packets. However, it is not necessary that the line card will crash all times. The resulting action on these packets could result in a simple drop as well.

Conditions: This symptom is observed on a Cisco 12000 series when a packet with an IP destination address from the reserved multicast range (224.0.0.xxx) and a TTL larger than 1 is received on the 3GE-GBIC-SC line card and when multicast hardware acceleration is enabled.

Normally, the TTL should be 1 if the destination address is part of the reserved multicast range.

Workaround: Enter the no hw-module slot slot-number ip multicast hw-accelerate source-table size 16 offset 0 command.

Symptoms: Class-default counters (that is, the total number of packets) do not increment.

Conditions: This symptom is observed on a Cisco router when a class map matches an OAM that is applied to a policy.

Symptoms: MVPN PIM neighbors that are associated with both a 1-port channelized OC-48 ISE line card and a 1-port channelized OC-12 (DS1) ISE line card bounce when you perform a microcode-reload of a 1-port channelized OC-12 (DS1) ISE line card.

Symptoms: If a 4-port Engine 3 Gigabit Ethernet (GE) line card is fast reroute (FRR) headend, and only the RX cable is pulled out, convergence time is greater than 50ms.

Conditions: This symptom occurs when the 4-port Engine 3 GE line card is FRR headend, and only the RX cable is pulled out.

Symptoms: After an OIR of a PA-MC-E3 port adaptor that is installed in a VIP6-80, the serial interfaces do not transmit. The message "not transmitting" is generated, followed by "output frozen." After these messages, a Cbus Complex occurs.

Symptoms: An EERP-3-INVALID_LAYER error message may be generated followed by continuous tracebacks that flood the screen, causing some tracebacks to be dropped.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS 12.0(31)S and that is configured with an ATM VC and VP when a policy map of the following type is attached and when OAM traffic is sent on the VC or VP:

Second Alternate Workaround: Configure an alternative policy such as the following one:

Symptoms: The PPP protocols flap on a 64K-port of an interface of a 1-port channelized OC-12/STM-4 (DS1/E1) ISE line card that is configured for CoS and that is congested.

Conditions: This symptom is observed when two Cisco 12000 series are connected back-to-back, when the routers are connected via 1-port channelized OC-12/STM-4 (DS1/E1) ISE line cards, and when you send real-time traffic.

Workaround: Change the QoS output policy to prevent traffic from being for longer than 10 seconds and enable keepalive packets to time out.

Further Problem Description: The symptom does not occur when you do not send real-time traffic.

Symptoms: When a router has a large VRF configuration and a lot of routing information, the following error messages may be generated during an SSO switchover:

%SYS-3-CPUHOG: Task is running for (2000)msecs, more than (2000) msecs 
(272/145),process = IPC LC Message Handler. 
-Traceback= 40EAF5D8 411DBE94 411DBFB8 411DC5D0 411DEFEC 411DEE90 411E0200 41093100 410932B8

After the FIBDISABLE error messages has been generated, the router may no longer function properly.

Conditions: This symptom is observed on a Cisco 7600 series but is platform-independent.

Symptoms: When the active rate for a destination PE router is evenly distributed at 4 pps for 20 flows and the active rate for a destination CE router is evenly distributed at 4 pps for 19 flows, one flow is reported at twice that rate (that is, 8 pps).

Conditions: This symptom is observed on a Cisco 12000 series that functions in a MVPN VRF-Lite environment with 20 multicast streams that have a single sustained cell rate (SCR) and that have the pps rate evenly distributed across all streams.

Symptoms: Engine 3 and Engine 5 line cards in a Cisco 12000 series may fail continuously.

Conditions: This symptom is observed on a Cisco 12000 series that runs a Cisco IOS interim release for Release 12.0(32)S when control-plane policing is configured on the line cards in the router.

Router(config)#policy-map foo Router(config-pmap)#class match_prec1
Router(config-pmap-c)#band per 2 Service-policy is not supported on interface Multilink100.

Conditions: This symptom occurs when a service policy that is attached to a multilink interface with no members is modified.

Symptoms: When a ToFab FCRAM single-bit error (SBE) occurs on an Engine 5 line card, the following error message is generated, and the line card resets:

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(31)S and occurs only on an Engine 5 line card.

Symptoms: When you remove a multilink bundle by entering the no ppp multilink command under a serial link configuration, a serial link that is configured in the multilink bundle remains in the "line protocol down" state and does not recover.

Conditions: This symptom is observed on a Cisco 7600 series that is configured with a SIP1 line card in which a SPA-CT3 is installed and on a Cisco 12000 series that is configured with a SIP-400 or SIP-600 in which a SPA-CT3 is installed.

Workaround: Reload the SPA by entering the hw-module subslot slot subslot reload command.

Symptoms: A router that is configured for MPLS VPN cannot accept a Route Distinguisher with the value 0:0.

Conditions: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.0S or Release 12.4 and that integrates the fix for CSCeh12594. A list of the affected releases can be found at http://www.cisco.com/cgi-bin/Support/Bugtool/onebug.pl?bugid=CSCeh12594. Cisco IOS software releases that are not listed in the "First Fixed-in Version" field at this location are not affected.

Workaround: Use a Route Distinguisher with another value. If this is not an option, there is no workaround.

Symptoms: The tunnel interface does not come up when you configure an MPLS TE tunnel for IS-IS.

Conditions: This symptom is observed on a Cisco 12000 series ATM ISE line card that is configured for IS-IS. The symptom does not occur when the line card is configured for OSPF.

Symptoms: Using the show ssm id command on a line card after a switchover may crash the line card.

Conditions: This symptom is observed in Cisco IOS Release 12.0(31)S on dual RP platforms that are running high availability (HA). After a switchover, if the show ssm id command is issued on a line card, the line card may crash.

Symptoms: All traffic with a 0 label plus another label is dropped by a 3-port Gigabit Ethernet egress ISE line card.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(31)S.

Symptoms: The active PRP pauses indefinitely after it changes from standby to active.

Conditions: This symptom is observed when the redundant PRPs are configured for RPR+ mode, the router has two APS-protected CHOC12 line cards, the router has mVPNs configured, and the router runs Cisco IOS Release 12.0(28)S3.

Symptoms: When there are some incomplete xconnect configurations or rapid succession of unprovisioning and provisioning of xconnect, memory leaks may be observed on line cards in Segment Switch Manager (SSM).

Conditions: This symptom occurs in Cisco IOS Release 12.0(31)S or later releases.

Conditions: This symptom is observed on a Cisco router that is configured for data MDT groups.

Symptoms: When traffic enters an Engine 6 ingress interface and flows in the direction of equal-cost multiple egress interfaces, all traffic is lost.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(27)S5, Release 12.0(28)S3, or Release 12.0(31)S when the traffic destination is present in the routing table as a recursive route (for example, learned via iBGP) and when the router has the IP Source Tracker feature configured. The symptom occurs only when the ingress interface is an interface of an Engine 6 link card and when multiple equal-cost egress interfaces are used.

Workaround: Enter the clear ip route * command to enable the traffic to reach its destination.

Symptoms: High memory usage, high CPU usage, or both may occur on a router that is configured for Xconnect.

Conditions: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.0S or Release 12.2S when one of the following conditions occurs:

–

The PW redundancy feature set is configured and the provisioning of the data plane fails for one of the redundancy members.

Workaround: If MFR data plane provisioning fails, there is no workaround. If the PW redundancy feature set is configured, there is a workaround: configure a large value for the "enable" and "disable" timeouts.

Symptoms: A router may crash while displaying the output of the show ip pim mdt bgp command.

Conditions: This symptom is observed when withdraws for a MDT source group are received by PIM from BGP while you enter the show ip pim mdt bgp command.

Workaround: There is no workaround. To reduce the chance of the router crashing, change the screen-length argument in the terminal length screen-length command to 0. Doing so prevents the router from pausing between multiple output screens. (The default of the screen-length argument is 24.)

Symptoms: A standby RP crashes when you add or remove classes from a QoS policy or when a policy is applied to one line rather than to multiple lines.

Conditions: This symptom is observed on a Cisco 12000 series but is platform-independent.

Symptoms: A PE router that is configured with many (100 or more) Multicast VRFs (mVRFs) may create multiple MDT tunnels for one mVRF.

Conditions: This symptom is observed when you reload a Cisco router that functions as a PE router and that is configured for MVPN.

Symptoms: When a multilink bundle is configured on a 6-port channelized T3 line card, the delay for traffic in the priority queue may be 12 to 14 milliseconds more than what you would expect.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0S when a multilink protocol such as MFR or MLP is configured and when congestion occurs.

Symptoms: When a link failure occurs between two provider (P) routers, the LSP that is protected via Fast Reroute (FRR) for a primary tunnel and the directed LDP session within the tunnel are fast-rerouted onto an assigned backup tunnel. However, when the backup tunnel goes down, VPN prefixes that are protected by backup TE LSP entries in the LFIB become "Untagged." This situation causes packet loss for AToM and L3VPN traffic that uses the link between the P routers as its primary path.

Conditions: These symptoms are observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(31)S, that functions as a P router, and that is connected to another P router via a 4-port OC-3 ISE line card.

There are one-hop primary tunnels between every pair of routers that is listed above, and each link is protected by an NHOP backup tunnel LSP.

The symptom occurs when you pull the TX fiber cable from the 4-port OC-3 ISE line card that forms the protected link between the two P routers, when the protected LSP is fast-rerouted onto the backup tunnel, and when the backup tunnel is torn down. One P router may show "Untagged" entries in its LFIB, especially for a loopback interface to a provider edge (PE) router. This situation breaks the forwarding for all of the L2 and L3 VPNs that depend on that PE router.

The symptom is timing-dependent. The symptom does not occur all the time and does not seem to be Cisco 12000 series line card-dependent, nor is it specific to a link between the two P router because the symptom may also occur when you pull the TX fiber cable of a line card that forms the link between a PE router and a P router.

Symptoms: When QOS parameters are changed, traffic does not pass via an L2TPv3 link.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(31)S when QOS or HQoS parameters are changed and affect the traffic flow.

Symptoms: You cannot link a CoS slot table definition to a line card by entering the rx-cos-slot global configuration command. When you attempt to do so, the following error message is generated:

Conditions: This symptom is observed on a Cisco 12000 series that runs the gsr-p-mz image of a Cisco IOS interim release later than Release 12.0(31)S.

Symptoms: When you reload all the line cards on a Cisco 12000 series or when you initiate an RP switchover, a %PRP-3-CHP_DESCQ_FULL error message may be generated and some of the line cards may fail.

Conditions: This symptom is observed on a Cisco 12000 series that runs a Cisco IOS interim release later than Release 12.0(31)S when one of the clock and scheduler cards (CSCs) is shut down.

Symptoms: ISE line cards crash when a Cisco 12000 series receives 640,000 multicast streams.

Conditions: This symptom is observed when the memory of the ISE line cards becomes exhausted when the thousands of multicast streams are received.

Symptoms: Interface configuration parameters are not applied to the running configuration after an RPR+ switchover.

Conditions: This symptom is observed intermittently on a Cisco 12000 series that is configured with a 1-port CHOC-48 ISE line card but may also occur with other line cards.

Symptoms: An AToM VC that is configured on a Fast Ethernet or Gigabit Ethernet interface may not be functional and packets that are received on the interface are dropped.

Further Problem Description: When the symptom occurs, the output of the show hardware pxf cpu statistics interface interface-name detail command shows that the packets that are received on the interface are dropped because of an "mpls undefined port" condition.

Symptoms: Performance drops to 90 percent when the "N flag" is set incorrectly for the MDFS process.

Conditions: This symptom is observed on a Cisco 12000 series that is configured for mVPN, that uses an Engine 3 line card for imposition, and that uses an Engine 4+ line card for disposition.

Symptoms: When a Gigabit Ethernet (GE) link goes down between two 4-port GE ISE line cards, the link detection mechanism is inconsistent.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(31)S.

Symptoms: The standby PRP crashes when you apply an IPv6 ACL on an interface of an ISE line card.

Conditions: This symptom is observed on a Cisco 12000 series that is configured with redundant PRPs.

Symptoms: A router that is configured for Pseudowire Redundancy may and generate the following stack trace:

Conditions: This symptom is observed on a Cisco router that has one AToM and one non-AToM segment.

Symptoms: Multilink Frame Relay (MFR) interfaces that are configured on a channelized T3 SPA continue to flap.

Conditions: This symptom is observed on a Cisco 12000 series when you perform an online insertion and removal (OIR) of the 12000-SIP-400 in which the channelized T3 SPA is installed, or when the router reloads.

Symptoms: New subinterfaces and duplicate IP addresses are unexpectedly created for member interfaces of a port-channel subinterface.

Conditions: This symptom is observed on a Cisco 12000 series that is configured with dual Route Processors that function in RPR mode when a clock and scheduler card (CSC) is shut down before an RPR switchover occurs.

Symptoms: After a manual switchover occurs in RPR+ mode, a VPN that is configured on a Frame Relay subinterface fails to recover and CEF may be disabled on line cards.

Conditions: This symptom is observed on a Cisco 12000 series that runs the gsr-p-mz image of a Cisco IOS interim release for Release 12.0(30)S2.

Symptoms: A standby RP resets when you add a channel group by entering the channelized [mode t1 | e1] command on the controller of a T3 port that had the no-channelized command enabled before you made the change.

Conditions: This symptom is observed on a Cisco 12000 series that has two RPs and that is configured with a 4-port channelized T3 to DS0 Shared Port Adapter (SPA) that is installed in a 2.5G ISE SPA Interface Processor (SIP).

Symptoms: When you run an SNMP get or walk on the ifOperStatus object on a 1-port CHOC-12 OC-3 ISE line card, the status for the STS-1 path interface shows down although the channel is up.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(28)S3.

Symptoms: The entry for a tunnel is missing from the mplsOutSegmentTopLabel column of the MPLS-LSR-MIB.

Conditions: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.4 when a mibwalk is performed on the mplsOutSegmentTopLabel object. The symptom may also occur in other releases.

Symptoms: An MPLS LER does not impose labels for traffic that follows the default route, causing traffic to be forwarded via IP.

Conditions: This symptom is observed on a Cisco 12000 series when the default route has two equal paths, when the ingress line card is an ISE line card, and when the default router is learned via OSPF. The symptom may also occur for other protocols.

Workaround: Use a single path for the default route. If this is not an option, there is no workaround.

Symptoms: A 1-port OC-192 Engine 4+ line card (OC192E/POS) or a Modular GbE Engine 4+ line card (EPA-GE/FE-BBRD with EPA-3GE-SX/LH-LC) may crash when an SSO switchover occurs or when the router reloads.

Conditions: This symptom is observed on a Cisco 12000 series that runs a c12kprp-p-mz image of Cisco IOS Release 12.0(31)S and that has two RPs.

Symptoms: When a service policy is attached to an ATM PVP as an output policy that contains a queue limit, the policy is rejected with the error message "No ATM VC associated with this service policy."

Conditions: This symptom is observed only for ATM VP connections when a policy is attached as an output policy with a queue limit.

Conditions: This symptom is observed on a Cisco 12000 series after you have reloaded the router.

Symptoms: A 4-port GE ISE line card that is configured with 30 or more VLAN subinterfaces may fail.

Conditions: This symptom is observed on a Cisco 12000 series that runs a Cisco IOS release that is later than Release 12.0(31)S. The symptom may also occur on a 5-port GE Engine 5 SPA.

Symptoms: Line cards may enter the WAITRTRY state when the primary CSC is shut down.

Conditions: This symptom is observed only on a Cisco 12008 that runs an interim release for Cisco IOS Release 12.0(32)S when the active RP is installed in slot 2 of the chassis.

Workaround: Ensure that the active RP is not installed in slot 2 of the chassis.

Symptoms: MPLS may fail on a router when you reload a Gigabit Ethernet (GE) line card on which Link Bundling is enabled.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(31)S or an interim release for Release 12.0(32)S.

Workaround: Re-enable MPLS by entering the mpls ip global configuration command.

Symptoms: When Cisco IOS software is secured using "secure boot" commands and after formatting the disk, the show disk command will not display the secured image and the corresponding configurations in the output.

Conditions: This symptom occurs when securing the Cisco IOS software using the secure boot-config and the secure boot- image commands and formatting the disk.

Symptoms: PIM neighbors do not recognize each other via a VRF tunnel interface because multicast does not receive MDT updates from BGP. The output of the show log command shows the following debug message:

%BGP-3-INVALID_MPLS: Invalid MPLS label (3) received in update for prefix 2:55:1111:192.168.31.1/32 from 192.168.31.1

Conditions: This symptom is observed on a Cisco router and is not platform-dependent. The symptom occurs when a VRF instance is configured with BGP as the Exterior Gateway Protocol (EGP).

Symptoms: A nondefault configuration becomes lost for a serial interface on a channelized OC-48 ISE line card or on a 4-port OC-12 ISE line card.

Conditions: This symptom is observed on a Cisco 12000 series after you have reloaded the router.

Symptoms: A router crashes when you remove an Embedded Event Manager (EEM) applet.

Conditions: This symptom is observed on a Cisco 12000 series that runs an interim release for Cisco IOS Release 12.0(32)S. This symptom occurs under the rare occasion that the EEM applet is removed while EEM is attempting to trigger the applet for execution.

Before you remove the EEM applet, disable EEM applet scheduling by entering the event manager scheduler applet suspend command.

After you have removed the applet, re-enable EEM applet scheduling by entering the no event manager scheduler applet suspend command.

Symptoms: When a buffer leak occurs, the RP crashes because of the starvation of buffers.

Conditions: This symptom is observed on a Cisco 7500 series that has a VIP in which a channelized T1/E1 port adapter is installed and on Cisco 7600 series that has a FlexWAN in which a channelized T1/E1 port adapter is installed.

Symptoms: A policy on a main Ethernet interface does not properly match packets for one of its subinterfaces.

Conditions: This symptom is observed on a Cisco 12000 series Ethernet ISE line card when one of its subinterfaces is configured for Xconnect and has a layer 2 VPN configured and when the following events occur:

Symptoms: A Cisco 12000 Engine 3 line card might be affected by a memory leak in the "CEF LC IPC Backg" process that is caused by IPv6 route changes.

If all the line card route memory is consumed, the following error messages might be seen:

%SYS-2-MALLOCFAIL: Memory allocation of 65556 bytes failed
from 0x400DCE94, alignment 32
Pool: Processor  Free: 533440  Cause: Memory fragmentation
Alternate Pool: None  Free: 0  Cause: No Alternate pool 
-Process= "CEF LC IPC Background", ipl= 0, pid= 71
-Traceback= 40030CBC 400E22FC 400E6A38 400DCE9C 404ADB0C 404ADD80 4042354C 40425464 4114 4B08 411444AC 411443A4 4114B68C 410ECB6C 410ECEB4 410EDC3C

%EE48-3-IPV6_TCAM_CAPACITY_EXCEEDED: IPv6 pkts will be soft ware switched.
To support more IPv6 routes in hardware:
Get current TCAM usage with: show controllers ISE <slot> tcam
In config mode, reallocate TCAM regions e.g. reallocate NetFlow TCAM to IPv6
hw-module slot <num> tcam carve rx_ipv6_1 <prefix> <v6-percent>
hw-module slot <num> tcam carve rx_top_nf <nf-percent>
Verify with show command that sum of all TCAM regions = 100%
Reload the linecard for the new TCAM carve config to take effect
WARNING: Recarve may affect other input features(ACL,CAR,MQC,Netflow)

Doing the recarve, as suggested, will not fix the problem, except that by reloading the line card, the leaked memory will be recovered temporarily.

Conditions: This symptom occurs when continuous IPv6 routes change on an Engine 3 line card.

To disable [2], just use the "no" form of the command to get back to the default setting:

The default is 35% for NetFlow (RX_TOP_NF_REGION) and 4% for IPv6 (RX_IPv6_144b_REGION 128).

exec slot <slot number> sh controllers tofab alpha tcam carve | i IPv6_128|Total|Mask|RX_TOP_NF

Beware: With this workaround, all IPv6 packets will be process-switched on the line card, so make sure you do not have too much IPv6 traffic, or you will get 100% CPU usage on the line card and probably other problems.

Symptoms: An interface of a clear channel T3/E3 SPA that is installed in a 12000-SIP-400 is in a down state after you have entered the redundancy force-switchover command.

Conditions: This symptom is observed on a Cisco 12000 series that is configured with two PRPs that function in RPR+ mode.

Workaround: Enter the shutdown interface configuration command followed by the no shutdown interface configuration command on the affected interface to bring the interface back up.

Conditions: This symptom is observed on a Cisco 12000 series that has a channelized ISE ingress line card when packets are replicates to a VRF interface on the ingress line card, to a VRF interface on another line card, and to a core interface on a third line card. This symptom occurs only after some redundancy switchovers.

Symptoms: A bad checksum error, bad LLS TV length error, or both are reported on a router that is configured for OSPF and BGP. These protocols or other configured protocols may flap during the errors, and data packets that are sent to the PRP may be lost.

Conditions: These symptoms are observed on a Cisco 12000 series that is configured with a PRP-1 when the following conditions are present:

–

OSPF, BGP, and other control protocols are configured with scaled routes and peers.

–

Congestion occurs on the PRP-1 because control packets are targeted to the PRP-1 or because other packets are sent to one of the IP addresses of the router and are terminated on the PRP-1. This situation occurs, for example, when IP ping packets are directed towards one of the loopback addresses and are terminated on the PRP-1.

Symptoms: Local switching traffic is dropped from the ToFab queue on an egress port of an 2-port T3/E3 serial shared port adapter (SPA) that is installed in a 2.5G ISE SPA Interface Processor (12000-SIP-400).

Conditions: This symptom is observed on a Cisco 12000 series that functions as a PE router in an L2VPN environment.

Symptoms: CEF may become disabled on an Engine E4+ line card because of a MALLOC failure.

Conditions: This symptom is observed on a Cisco 12000 series when you enter the no mpls ip global configuration command immediately followed by the mpls ip global configuration command.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(28)SA4 and that is configured with redundant PRPs that run in RPR+ mode during a Large Scale Network Test (LSNT). The router has two channelized OC-12 line cards that are configured for APS and that each have 280 DS1 ports, 121 DS0 ports, and 42 MLP groups.

Symptoms: When you reload a Cisco 12000 series, the RP remains stuck for 30 minutes. No traceback or spurious memory access is generated after the parser is released.

Conditions: This symptom is observed on a Cisco 12000 series that runs an interim release for Cisco IOS Release 12.0(32)S when the QoS: Enhancements to Single QoS Policy Definition for a Physical Interface (L3/1C/nD) feature is enabled on a T1 interface that is configured for Frame Relay and that has several Frame Relay subinterfaces. The symptom occurs after the following events:

You save the configuration to NVRAM by entering the copy running-config startup-config command.

Symptoms: When an APS switchover occurs on 1-port channelized ISE line cards, traffic may not recover although the controllers may be in up state. The interfaces may remain indefinitely in the up/down state.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(31)S or an earlier release when a significant amount of traffic is being processed when the APS switchover occurs.

Workaround: There is no workaround. However, the built-in failure detection mechanism detects that the ingress data path is locked up and automatically resets the PLIM of the affected line card.

Symptoms: A router that is configured for Multiprotocol Label Switching (MPLS) Label Distribution Protocol (LDP) may crash when LDP is configured or removed from an interface or globally.

Symptoms: When you reload a line card or router and then remove a policy, the output of the show policy-map interface command shows that no policy is applied but the output of the show controller frf queue command does show that the queues are not fully released. This situation prevents traffic from flowing correctly.

Conditions: This symptom is observed on a Cisco 12000 series that is configured with a Gigabit Ethernet Engine 5 SPA that has an "L3/1C/nD" policy map attached to an interface.

Symptoms: On a router that is configured for Multiprotocol Label Switching (MPLS) Label Distribution Protocol (LDP), the CPU usage may increase considerably for an extended period of time when a large number of label bindings are withdrawn or released at the same time.

Conditions: This symptom is observed on a Cisco router only when LDP (as opposed to TDP) is used and when a large number (more than 250) of LDP neighbors and a large number of IP prefixes become unreachable at the same time.

Symptoms: Traffic drops may occur when traffic is sent over MFR or Frame Relay links.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(28)S1 or a later release and that is configured for software forwarding.

Symptoms: The output of the show ip hardware-cef tofab prefix command may display incorrect information for Engine 5 line cards.

Conditions: This symptom is observed on a Cisco 12000 series that functions in a scaled routing environment with more than 100,000 routes.

Symptoms: Multicast traffic does not resume fully after you have removed the active PRP from the router.

Conditions: This symptom is observed on a Cisco 12000 series that runs the c12kprp-p-mz image of Cisco IOS Release 12.0(28)S4 and that is configured with redundant PRPs that function in RPR+ mode. The router has two channelized OC-12 line cards that are configured with mVPNs.

Symptoms: The standby RP on a Cisco 12000 series may fail to come up and crash during initialization. The primary RP may generate the following error message:

Conditions: This symptom is observed only when there is a large number of files for the standby RP on the flash disk (for example, when a 1 GB flash disk is about half full) and when the average file size is also large.

Symptoms: On an Ethernet over AToM link, an abnormal queue depth may occur for Time-Based WRED and Byte-Based-WRED.

Conditions: This symptom is observed on a Cisco 7500 series but is platform-independent.

Symptoms: Fast Reroute (FRR) fails on a remote provider (P) router, causing packet loss.

Conditions: This symptom is observed on a Cisco 12000 series that functions as a P router in the following topology:

–

One P router (P1) connects to another P router (P10) via a primary tunnel that is configured on a 1-port 10-Gigabit Ethernet Engine 5 SPA. (P10 functions as the tunnel head-end.)

–

There is a backup next-hop (NHOP) FRR protection tunnel via an LSP path between P1 and P10.

–

P1 connects also to another P router (P2) that connects to yet another P router (P3), that, in turn, connects to P10, forming a second backup NHOP FRR protection tunnel via an LSP path.

When the link between P1 and P10 breaks at P1, the secondary backup tunnel does not come up immediately, causing packet loss of around 200 ms (about 90 percentile) when the link goes down and 5 seconds when the link finally comes up again.

Workaround: Enter the mpls traffic-eng topology holddown sigerr 0 command on the affected tunnel head-end (P10) to prevent the backup LSP from being hold down.

Further Problem Description: This symptom is only observed with 1-port 10-Gigabit Ethernet Engine 5 SPAs.

Conditions: This symptom is observed when you enter the clear ip mroute * on a connected PE router. The messages do not recover by themselves.

Workaround: To restart Auto-RP messages, enter the clear ip mds linecard command.

Alternate Workaround: To restart Auto-RP messages, debug the VRF Auto-RP by entering the debug ip pim vrf vrf-name auto-rp.

Symptoms: A Cisco 12000 series may crash when 1000 pseudowires are configured and one PVP is configured with an existing VCID.

Conditions: This symptom is observed in a L2VPN environment that is configured with 1000 pseudowires when a new configuration overwrites the old configuration with different VCIDs but one of the PVPs has the same VCID as a VCID in the old configuration.

Symptoms: When multicast traffic is being sourced from different sources, and one of the sources is removed, the show ip mroute vrf command for the VRFs still shows that source as active.

Conditions: This symptom is observed when a source is no longer active when using the show ip mroute vrf command.

Symptoms: The output of the show ip mroute vrf vrf-name active command shows an incorrect entry or rate for decapsulated traffic.

Conditions: This symptom is observed on a Cisco 12000 series that is configured for mVPN.

Symptoms: The Queue Manager may not converge on a 4-port OC-12 ATM ISE line card or a 1-port channelized OC-12 ISE line card, and tracebacks may be generated.

Conditions: This symptom is observed on a Cisco 12000 series that runs the c12kprp-p-mz image and occurs after the router has reloaded.

Symptoms: An MTU change on a multilink bundle interface takes no effect for SPA interfaces. This situation may cause the traffic to be dropped.

Conditions: This symptom is observed on a Cisco 12000 series when you change the MTU on a multilink bundle interface and when you change the traffic generation accordingly.

Symptoms: An Engine 4 ingress line card may enter an incorrect carving state in which it sends all packets that are larger than 608 bytes to the buffer size pool (freeq) of the wrong egress line card, causing all packets that are larger than 608 bytes to be dropped. The symptom is especially noticeable when the egress line card is an Engine 2 line card.

Symptoms: Multicast traffic does not reach a CE router that is connected via static IGMP joins to a PE router.

–

Traffic from the PE router that performs encapsulation flows towards the CE router.

–

The PE router that performs decapsulation has an egress line card with one interface connected to the core of the network and another interface connected to the edge of the network.

–

The PE router that performs decapsulation has static joins that are configured for Source Specific Multicast (SSM).

Symptoms: In an ATM L2VPN configuration, the protocol of some ATM interfaces on a customer edge (CE) router may be in the DOWN state, causing the corresponding PVCs to be in the DOWN state too.

Conditions: This symptom is observed rarely when an RP switchover occurs on the connected provider edge (PE) router.

Workaround: Enter the shutdown interface configuration command followed by the no shutdown interface configuration command on the affected ATM interfaces to enable the PVCs to enter the UP state.

Symptoms: High CPU usage occurs on an Engine 6 line card that processes multicast traffic.

Conditions: This symptom is observed on a Cisco 12000 series when the Engine 6 line card is an ingress line card that processes SSM multicast traffic.

Workaround: Lower the traffic rate, configure a mode other than SSM, or replace the Engine 6 line card with an Engine 4 line card.

Symptoms: A CE router that is configured for VRFLite does not receive Auto-RP mappings.

Conditions: This symptom is observed when MDS is enabled on the multilink interface that connects the CE router and the PE router.

Workaround: Configure process switching on the multilink interface that connects the CE router and the PE router by entering the no ip mroute-cache interface configuration command.

Symptoms: An interface of an Engine 6 line card is no longer shut down after an RP switchover occurs.

Conditions: This symptom is observed on a Cisco 12000 series when the following events occur:

The interface of the Engine 6 line card is configured with the no shutdown interface configuration command in the startup configuration.

After the new RP comes up, the interface appears no longer shut down and the interface comes up again.

Workaround: After you have entered the shutdown interface configuration command on the interface followed by the write memory command, reload the router.

Symptoms: A multicast group takes a long time to converge on an Engine 6 line card. The output of the show ip mroute command shows the multicast group but it does not pass traffic for several minutes. After 8 to 10 minutes, the multicast group passes traffic. The output of the show ip hardware-mds spd command shows hardware SPD entries for the multicast group, but the output of the show ip mds spd command does not show any software SPD entries for the same multicast group.

Conditions: This symptom is observed on a Cisco 12000 series when traffic from many multicast groups enters the line card before the multicast group Mstate is present, triggering the creation of many SPD entries.

Workaround: Wait for 10 minutes for the extra hardware SPD entries to be cleared up.

Conditions: This symptom is observed on a Cisco 12000 series after you have configured a deny option for an ACL.

Conditions: This symptom is observed on a Cisco 12000 series when you reload a SIP-600 in which two channelized OC-3 SPAs are installed (one in bay 2 and one in bay 3).

Symptoms: A primary SR-APS physical interface may flap for several minutes in a scaled configuration with 1000 VCs that are configured while the router boots.

Conditions: This symptom is observed on a Cisco 12000 series that runs the c12kprp-p-mz image, that is configured with a 4-port OC-3 ATM ISE line card or 4-port OC-12 ATM ISE line card, and that has SR-APS enabled.

Symptoms: A service policy on an Engine 4 + or Engine 6 line card is incorrectly rejected with the following error message:

%E4P and E6 LC requires to configure POLICE and SET %command in every class if either of these two commands %is configured in class-default class

Conditions: This symptom is observed on a Cisco 12410 that runs Cisco IOS Release 12.0(28)S3, that is configured with dual Performance Route Processors (PRP-1s) that operate in SSO mode, and that has multiple E4+ and/or Engine 6 line cards.

Symptoms: Data corruption may occur on a disk when directory entries are read by more than one process simultaneously.

Conditions: This symptom is observed on a Cisco platform that has an ATA file system when, for example, the dir disk0: command is entered on one vty connection and simultaneously, and for the same disk, the copy disk0: command is entered on another vty connection.

Symptoms: When you reload a 1-port channelized OC-12 ISE line card, all traffic over the line card may be dropped, and an error message and traceback similar to the following may be generated:

%IPCGRP-3-SYSCALL: System call for command 14 (slot7/0) : ipc_send_rpc_blocked timed-out (Cause: timeout) 
-Traceback= 1F9F20 1FA028 491DC4 49291C 492D08 2E17EC

Conditions: This symptom is observed on a Cisco 12000 series that runs an interim release for Cisco IOS Release 12.0(32)S and that functions in a scaled configuration.

Workaround: Reload the router. If this is not an option, there is no workaround.

Symptoms: When you reload a router, the secondary line card in an SR-APS configuration may stay in the "STRTIOS" state for about 20 minutes and then reset to reach the "IOS RUN" state.

Conditions: This symptom is observed on 4-port OC-12 line card that is installed in a Cisco 12000 series and that is configured for SR-APS. The symptom may also occur with an OC-3 ATM ISE line card.

Symptoms: The following error message is generated on a channelized OC-12 ISE line card and traffic is lost:

Conditions: This symptom is observed on a Cisco 12000 series that functions in an MVPN configuration, that has MDFS disabled, that has an Engine 4 Plus line card facing the core of the network, and that has a channelized OC-12 ISE line card facing the edge of the network.

Symptoms: A router or a VIP crashes during the hqf_dp_normalize_class_weights process.

Conditions: This symptom is observed on a Cisco router that is configured for Low Latency Queuing (LLQ).

Symptoms: When you change the Cisco IOS software image from any release to a release later than Release 12.0(27)S2, the redundancy mode is unexpectedly changed from SSO or RPR+ to RPR.

Conditions: This symptom is observed on a Cisco 7500 series. The symptom does not occur when you change the Cisco IOS software image from any release to a release earlier than Release 12.0(27)S2. For example, the symptom occurs in the following situations:

Workaround: After you have changed the Cisco IOS release, manually change the redundancy mode back from RPR to SSO or RPR+.

Symptoms: CEF errors occur when the protection line card is reset, a GRP switchover occurs, or the router is reloaded.

Conditions: This symptom is observed on a Cisco 12000 series that has line card redundancy configured on ATM ISE line cards when you shut down the primary line card of the redundancy pair.

Symptoms: After you reload a PE router that functions in an MVPN topology and that is configured for sparse mode and Auto-RP, the router may not learn the Auto-RP that is advertised by both a local and remote CE router, preventing traffic from resuming to flow.

Conditions: This symptom is observed on a Cisco 12000 series that runs an interim release for Cisco IOS Release 12.0(32)S and that functions as a PE router. The symptom may also occur in other releases of Release 12.0S.

Symptoms: A Layer 2 Protocol Tunnel (L2PT) command does not function when the core-facing line card is an ISE line card. This situation causes the protocol address to be transferred as it is on the pseudowire.

Conditions: This symptom is observed on a Cisco 12000 series when you enter the l2protocol {cdp|stp|vtp} tunnel command on an interface of the core-facing ISE line card.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(31)S3 and that has the mpls ldp protocol tdp command configured on the interface on which TDP peering cannot be established. The peer router has the mpls ldp protocol both command configured.

Workaround: Enter the mpls ldp protocol tdp command on the peer router. Note that this workaround may not be plausible for routers that run a legacy Cisco IOS software that only supports TDP.

Symptoms: In a multicast VPN (MVPN) environment, when a Stateful Switchover (SSO) occurs on a PE router, the multicast traffic in the MVRF does not recover because the neighboring PE router fails to re-establish its PIM neighbor relationship. Note that the symptom does not occur for unicast traffic.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(31)S2 or an interim release for Release 12.0(32)S and that functions as a PE router (PE1) in the following topology:

When an SSO occurs on PE1, PE2 does not re-establish its PIM neighbor relationship with PE1 in the MVRF. PE1 and PE2 are global PIM neighbors.

Further Problem Description: When the symptom occurs, PE1 still shows PE2 as its PIM neighbor in the MVRF. Clearing the multicast route in the MVRF does not help to resolve this issue.

Symptoms: Data traffic that is received by a provider edge (PE) router on an Ethernet port may not be forwarded over an L2TPv3 tunnel.

Conditions: This symptom is observed on a Cisco 12000 series when the line card that faces the customer edge (CE) router is one of the following line cards:

Symptoms: When you enable IPv6 on a 4-port GE ISE line card, the line card crashes when it processes Virtual Private LAN Services (VPLS) traffic.

Conditions: This symptom is observed on a Cisco 12000 series when the 4-port GE ISE line card faces the VPLS core.

Workaround: Stop the VPLS traffic. If this is not an option, there is no workaround.

Symptoms: When 336 MLP bundles with one link per bundle are configured on four 1-port channelized STM-1/OC-3 Engine 5 SPAs, the following traceback and error messages are generated when you reload one of the Engine 5 SPAs:

-Traceback= 406A2188 406A3670 406AA4E8 40339B28 4033C1D0 4033C374 40158D78 40159758 eelc_config_intf_tx_q(): EE_QM_QOS_INTERNAL_ERRORarg=3

%SPA_PLIM-3-HEARTBEAT: Subslot 0 has experienced an heartbeat failure Current Sequence 14 received Sequence 8 Time since last keep 440ms

Conditions: This symptom is observed on a Cisco 12000 series when no traffic is processed on the Engine 5 SPA that is reloaded.

Conditions: This symptom is observed on a Cisco 12000 series that is connected to a BGP peer over a link bundle interface.

Conditions: This symptom is observed when a high rate of packets are punted to the CPU of the line card that is configured for low-priority Raw Queue (RawQ).

Workaround: Identify the reason for the high rate of packets that are punted to the CPU of the line card and correct the situation.

Symptoms: A port adapter in a router or FlexWan module in a switch may crash when an SSO switchover occurs on a Route Processor or Supervisor Engine.

Conditions: This symptom is observed when the port adapter or FlexWan module is configured with a QoS policy.

Conditions: This symptom is observed on a Cisco 10000 series that runs an interim release for Cisco IOS Release 12.0(30)S when you run multiple tests without cleaning and occurs during the second test after the first test passes. The symptom is platform-independent.

Load the boot image, load the minimum configuration, and save the configurations.

Workaround: There is no workaround. Note that the symptom does not occur when you run the tests with cleaning.

Symptoms: If a new multilink bundle is configured, the slave Route Switch Processor (RSP) reloads.

Conditions: This symptom is observed on a Cisco 7500 series that runs Cisco IOS Release 12.0(24)S6 and that has dual RSPs.

Symptoms: A Cisco router that runs Cisco IOS Release 12.3T may reload when the ATM interfaces are swapped.

Conditions: This symptom is observed when an ATM IMA port adaptor is removed and a PA-A3 port adaptor is inserted in the same slot and when there is at least one PVC configured that has the inarp enabled. The symptom may also occur in Release 12.3 or Release 12.4.

Symptoms: QoS information disappears from a FlexWAN module or VIP that is configured with a distributed MFR interface.

Conditions: This symptom is observed after the FlexWAN module or VIP resets or after the interface flaps.

Workaround: Remove the service policy from the interface and reapply it to the interface.

Conditions: This symptom is observed when MLP member links flap while traffic is being processed.

Symptoms: An active SP becomes stuck with an "slcp process" error when you enter the test crash command on the active RP.

Conditions: This symptom is observed on a Cisco Catalyst 6000 series that is configured with an ATA file system but is not platform-specific. The symptom occurs because of an error in the ATA file system.

Conditions: This symptom is observed on a Cisco router that is configured with an ATA file system when the memory is corrupted. The router is unable to save MALLOC requests to a disk because the memory on the disk is corrupted, causing the router to hang.

Workaround: Configure the router in such a way that the crashinfo is written to bootflash memory. Ensure that there is sufficient space in the bootflash memory for the crashinfo.

Symptoms: Packets may be dropped as reassembly drops on a distributed (dMLP) ingress interface that has interleaving configured.

Conditions: This symptom is observed on a PA-MC-STM-1 port adapter when more than two DS0 members are part of an dMLP bundle that is configured for interleaving.

Symptoms: A VIP that has a dMLFR configuration may crash when you enter the microcode reload global configuration command.

Conditions: This symptom is observed on a Cisco 7500 series when traffic flows through the VIP.

Symptoms: When the standby RSP on a Cisco 7500 series boots while a Versatile Interface Processor (VIP) or other interface processor on the router reloads, the standby RSP reloads unexpectedly.

Conditions: This symptom is observed on a Cisco 7500 series that runs Cisco IOS Release 12.0S, Release 12.2S, Release 12.2SB, Release 12.3T, Release 12.4, or Release 12.4T.

Symptoms: After an OIR of a VIP on a Cisco 7500 series, MLP traffic causes a very heavy CPU load on the RP, in turn causing failures in the IPC configuration and memory allocation (malloc) failures.

Conditions: This symptom is observed on a Cisco 7500 series that is configured with a large number of distributed MLP bundles.

Symptoms: A VIP crashes at the "free_wred_stats" function during an RPR+ switchover.

Conditions: This symptom is observed on a Cisco router that is configured with a VIP that has a configuration with about 12 MLP bundles with two T1 members when QoS is applied while traffic is flowing.

Symptoms: On a Cisco 7500 series that is configured for dMLP, the txacc values of member interfaces may be wrongly credited to other member interfaces, causing RSP-3-RESTART messages, and finally causing traffic to stop.

Conditions: This symptom is observed when the member links flap continuously for some time while traffic is being processed.

Symptoms: A Cisco 12000 series may crash because of a bus error when you configure a loopback on one of the E3 interfaces on a 6-port E3 (6E3-SMB) or 12-port E3 (12E3-SMB) line card.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(28)S.

Symptoms: A VIP or FlexWAN module in which a PA-POS-2OC3 port adaptor is installed may crash.

Conditions: This symptom is observed rarely and at random on a Cisco 7xxx series router or Cisco Catalyst 6000 series switch.

Symptoms: Multi-Layer Switching (MLS) CEF may stop functioning when an interface status changes. Ping and connectivity problems may also occur.

Conditions: This symptom is observed on a Cisco Catalyst switch and Cisco router when you shut down an interface or change VRF routes and as a result no other interfaces can be provisioned.

Symptoms: IPC messages may be generated on a 1-port channelized OC-12 (DS3) line card and the line card may be disabled and reload.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(28)S or Release 12.0(28)SW1 when OC-3 subinterfaces are configured on the 1-port channelized OC-12 (DS3)line card, when these OC-3 subinterfaces are configured for Frame Relay, when the rate-limit command is enabled, and when L2TPv3 traffic is being processed.

Workaround: There is no workaround because the rate-limit command is not supported in a configuration in which L2TPv3 traffic is being processed.

Symptoms: A number of AToM virtual circuits may not stay up after a configuration changes for a large number of AToM virtual circuits.

Conditions: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.0(31)S or Release 12.2SB.

Symptoms: At random, subinterfaces loose the ability to ping a directly-connected peer.

Conditions: This symptom is observed on a Cisco 12000 series that is configured with two 3-port Gigabit Ethernet line cards.

Note that although regular and extended pings do not work, pings that use the record option do work.

Symptoms: Cells loss occurs on a single ATM link of PA-A3-8T1IMA or PA-A3-8E1IMA port adapter.

Conditions: This symptom is observed on a Cisco 7500 and 7200 series when one of the T1 or E1 member interfaces of an IMA group that is configured on a PA-A3-8T1IMA or PA-A3-8E1IMA port adapter is disconnected or when you enter the shutdown command on one of these T1 or E1 member interfaces. The symptom is not platform-specific and may also occur in other releases.

Symptoms: The following error message is generated on a Cisco 7200 series that has Multilink PPP (MLP) configured on serial interfaces of a PA-MC-STM-1 port adapter:

Conditions: This symptom is observed on a Cisco 7200 series that runs Cisco IOS Release 12.3(11)T3 only when MLP is configured on the serial interfaces. The symptom may also occur in other releases.

Symptoms: During a high bit error rate (BER) condition, the controller of a PA-MC-8TE1+ port adapter remains up, which is not in compliance with the E1 and T1 standard.

Conditions: This symptom is observed when BER data is injected into an E1 or T1 port of a PA-MC-8T1E1+ port adapter. The state of the controller does not change to DOWN after 10 seconds of a continuous severely errored seconds (SES) condition.

Conditions: This symptom is observed on a Cisco router that is configured for MPLS VPN when an IPv4 BGP neighbor that is configured to exchange MPLS labels goes down.

Symptoms: An IPC failure occurs and an OC-12 line card that is configured for Frame Relay over MPLS resets.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(27)S1.

Further Problem Description: The IPC failure and the line card reset occur after a depletion of the elements in the FrFab 608 byte queue for the line card. Consecutive outputs of the show controllers slot-number frfab queue command show a consistent and rapid leak of these buffers.

Symptoms: An RP crashes during large interface configuration changes when interfaces and QoS policies are added or deleted.

Conditions: This symptom is observed on a Cisco 12000 series when the configuration changes involve ATM and serial interfaces.

Symptoms: A Cisco switch or router may crash when you install a 1-port multichannel STM-1, single mode port adapter (PA-MC-STM-1SMI) in a FlexWAN or VIP.

Conditions: This symptom is observed when you first power down the switch or router, install the PA-MC-STM-1SMI, and then boot up the switch or router.

Symptoms: An LSP ping (or traceroute packet) is incorrectly sent from an unlabeled interface, preventing the LSP ping to detect LSP breakages when a one-hop label switched path is pinged.

Conditions: This symptom is observed on a Cisco router that is configured for MPLS OAM.

Symptoms: A crash that is related to MPLS TE bandwidth management may occur on a Cisco router which is configured for OSPF and MPLS Traffic Engineering.

Conditions: This symptom is observed on a Cisco router that integrates the fix for caveat CSCef16096 when the following conditions are present:

–

There are more than 300 OSPF interfaces (in any state, including administratively down) in the OSPF area that is configured for MPLS TE.

You can check the number of interfaces by entering the show ip ospf or show ip ospf interface brief command. Note that all interfaces that are covered by network statements are included in the command output, even those that are in the administratively down state.

Symptoms: The data path on a 3-port Gigabit Ethernet Engine 2 (3GE-GBIC-SC) line card may be reset because of a corrupted packet that is found in the Tx SOP SRAM. This situation causes packet loss and the routing protocol sessions to flap.

Conditions: This symptom is observed on a Cisco 12000 series that runs a Cisco IOS software release that includes the fix for caveat CSCef06121. A list of the affected releases can be found at http://www.cisco.com/cgi-bin/Support/Bugtool/onebug.pl?bugid=CSCef06121. Cisco IOS software releases that are listed in the "First Fixed-in Version" field at this location are affected.

Workaround: There is no workaround. The symptom causes a disruption of service, but service is restored.

Further Problem Description: When the symptom occurs, the following messages are generated in the log:

%RP-3-FABRIC_UNI: Unicast send timed out  (1)
CORRUPT PACKET DUMP:
000005C000000000 0200000000000000 0000000101000000 00062AD9B40A0003 
A09D008208004500 0000000000000000 0000000000000000 0000000000000000 
0000000000000000 0000000000000000 0000000000000000 0000000000000000

%RPGE-6-AUTONEG_STATE: Interface GigabitEthernet1/0: Link OK - 
autonegotiation complete
%RPGE-6-AUTONEG_STATE: Interface GigabitEthernet1/2: Link OK - 
autonegotiation complete
%RPGE-6-AUTONEG_STATE: Interface GigabitEthernet1/1: Link OK - 
autonegotiation complete
%LCGE-3-SOP_BAD_PACKET: Found corrupt pkts in tx-sop-sram. Data path was 
reset.

%OSPF-5-ADJCHG: Process 1, Nbr 10.142.65.38 on GigabitEthernet1/0 from 
LOADING to FULL, Loading Done
%OSPF-5-ADJCHG: Process 1, Nbr 10.142.65.44 on GigabitEthernet1/2 from 
LOADING to FULL, Loading Done

Conditions: This symptom is observed when the filename argument of the tftp-server command has a length of more than 67 characters.

Workaround: Ensure that the length of the filename argument does not exceed 67 characters.

Symptoms: An interface of a PA-T3+ port adapter remains up during an Unavailable Seconds (UAS) condition that occurs because of a high C-bit or P-bit error rate.

Conditions: This symptom is observed on a Cisco 7200 series that is configured with a PA-T3+ port adapter.

Symptoms: A 6PE router crashes during an IPv6 ping to another PE router at the far side of the network.

Conditions: This symptom is observed when you enter the no mpls ipv6 source-interface command followed by no interface type number command in which the type number argument represents the IPv6 source interface that was configured in the mpls ipv6 source-interface command.

Workaround: When you want to disable the IPv6 source interface, first enter the no interface type number command in which the type number argument represents the IPv6 source interface that is configured in the mpls ipv6 source-interface command and then enter the no mpls ipv6 source-interface command.

Symptoms: A large configuration in NVRAM on a primary or secondary RSP may become corrupted and the router may generate relevant warning messages during the execution of a copy system:running-config nvram: startup-config command.

When you erase NVRAM by entering the erase nvram command and then enter the copy system:running-config nvram: startup-config command, the router may crash.

Conditions: This symptom is observed on a Cisco 7500 series but is platform-independent.

Workaround: If the configuration file is significantly large, place a copy of the configuration file on a flash card or disk with ample space and enter the boot config slot0:startup-config command to force the startup configuration file to be read from the flash card.

When you enter the copy system:running-config nvram: startup-config command, the current running configuration is saved to the flash card or disk and the configuration is auto-synchronized to the corresponding flash card on the secondary RSP.

Caution: Do not remove the flash card while the boot config slot0:startup-config command is being executed.

Symptoms: Traffic to a network core is dropped from a link-bundle interface of an Engine 3 line card.

Conditions: This symptom is observed when the network core is a Cisco 12000 series that runs Cisco IOS Release 12.0(26)S or a later release, that functions as a PE router, that is configured for MPLS VPN, and that has L3 loadbalancing enabled on an egress path through a link-bundle interface.

Further Problem Description: The symptom occurs because there is incorrect FCR information in the Engine-3 hardware rewrites that point to the link-bundle interface.

Symptoms: In some scalability cases with a large number of tunnels, SVIs, or VLANs, FIB tracebacks occur after an SSO switchover.

Conditions: This symptom is observed because traceback recording for the general event log and the interface event log is on by default.

Workaround: There is no workaround. Note, however, that there is no functional impact.

Further Problem Description: The fix for this caveat turns off traceback recording for the general event log and the interface event log.

Symptoms: When you boot a Cisco 12000 series, some Layer 1 and CoS command are rejected with the following error messages:

Command "pos threshold sd-ber 9" not allowed on link-bundle member interface POS1/0 Command "tx-cos TEST" not allowed on link-bundle member interface POS1/0

Conditions: This symptom is observed on a Cisco 12000 series when a POS interface of an Engine 0 or Engine 2 line card has the tx-cos command enabled and is a member of a port channel or POS channel.

Symptoms: Unicast traffic that travels over an ATM subinterface between a PE router and a CE router stops.

Conditions: This symptom is observed on a Cisco 12000 series that functions as a PE router and that is configured with an ATM ISE line card when the following conditions are present:

Workaround: Enter the shutdown interface configuration command followed by the no shutdown interface configuration command on the ATM subinterface or remove and re-add the ATM subinterface configuration.

Symptoms: A Cisco 12000 series that is configured for L2TPV3 may continuously log the following CM_ERROR message, causing the syslog server to be flooded:

%SW_MGR-3-CM_ERROR: Connection Manager Error - unprovision segment failed %SW_MGR-3-CM_ERROR: Connection Manager Error - unprovision segment failed

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(30)S1 when a Multilink Frame Relay (MFR) interface is shut down in the configuration of the connect command, causing proper provisioning to fail, unprovisioning to occur, and the error message to be generated.

Possible Workaround: Enter the no shutdown interface configuration command on the MFR interface.

Symptoms: When you send a high rate of bidirectional unicast and multicast traffic, the PXF complex can crash and then recover on its own.

Conditions: This symptom is observed on a Cisco 10720 when a high rate of bidirectional unicast and multicast traffic is sent between GE ports across an SRP uplink.

Symptoms: No error condition is detected when a properly structured IPv4 packet has an invalid version value in the IP header. For example, IPv4 packets that have a version value other than 4 are forwarded without an error.

Conditions: This symptom is platform-independent and occurs under normal operating conditions.

Symptoms: When you send traffic with the full bandwidth of an IMA bundle, cell loss occurs. For example, cell loss occurs when you send traffic with more than 5 Mbps on an 8-link T1 IMA group or with 6.4 Mbps on an 8-link E1 IMA group.

Conditions: This symptom is observed on a Cisco 7500 series that runs Cisco IOS Release 12.0(30)S and that is configured for ATM L2TPV3, cell-packing, and multiple VP configurations. There is no cell loss with a single PVC without an L2TPv3 configuration.

Symptoms: When VBR ATM traffic is sent through a Cisco 12000 series 4-port ATM OC-3 ISE line card via an L2TPv3 IP tunnel to another 4-port ATM OC-3 ISE line card on another Cisco 12000 series, the VBR ATM traffic passes at lower rates than what is configured on the routers, and cell loss occurs.

Conditions: These symptoms are observed on a Cisco 12000 series that is connected back-to-back via an OC-192 POS link to another Cisco 12000 series.

Symptoms: A FIBNULLIDB error may occur on a Cisco platform that is configured for CEF.

Conditions: This symptom is observed under several conditions such as deleting a subinterface or performing an OIR of a new line card.

Symptoms: A 2.5G ISE SPA Interface Processor (SIP) in which a 4-port channelized T3 to DS0 Shared Port Adapter (SPA) is installed crashes when the controller is reconfigured and VRFs are present or have been present on the associated interfaces.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(31)S.

Further Problem Description: The SIP and SPA do not recover on their own; you must reload the router to recover the SIP and SPA.

Symptoms: Traffic man continue to pass for a period of time on an AToM session even when the LDP session between two PE routers is disrupted.

Conditions: This symptom is observed on a Cisco router that functions as a PE router when it receives an LDP DOWN event.

Conditions: This symptom is observed on a Cisco router that is configured for Multilink Frame Relay (MFR) over L2TPv3/AToM when there are services with incomplete MFR over L2TPv3/AToM configurations and when the router has run for a long period of time.

Symptoms: Packets do not switch through a core interface of a line card that has hardware acceleration enabled.

Conditions: This symptom is observed on a Cisco 12000 series when the line card that contains the core interface has also a VRF interface that is shut down.

Symptoms: An ATM subinterface on an ATM port adapter that is installed in a FlexWan module or VIP may transition to the down/down state.

Conditions: This symptom is observed on a Cisco Catalyst 6500 series, Cisco 7500 series, and Cisco 7500 series when you remove a QoS service policy from the ATM subinterface.

Symptoms: An "Output Hold Queue Wedge" condition may occur on PVCs that are defined on DS1 ports that are not configured for IMA.

Conditions: This symptom is observed on a Cisco 7200 series that runs the c7200-ik9s-mz image of Cisco IOS Release 12.3(13), that is configured with a PA-A3-8T1-IMA port adapter that is configured for DSL aggregation, and that terminates hundreds of UBR VCs on a DS1 interface. The "Output Hold Queue Wedge" condition occurs on idle subinterfaces or when multiple point-to-point subinterfaces are "spawned" from a single subinterface by entering a PVC range command such as the following:

–

Enter the shutdown interface configuration command followed by the no shutdown interface configuration command on the physical interface.

–

Enter the no pvc-in-range command followed by the pvc-in-range command on a wedged VC.

–

Enter the shutdown interface configuration command followed by the no shutdown interface configuration command on an uplink interface.

Symptoms: Multicast traffic may be incorrectly dropped if it is filtered by an IPv6 ACL that matches on source and destination address.

Conditions: This symptom is observed when a Cisco 12000 series that runs Cisco IOS Release 12.0(31)S and that performs IPv6 ACL filtering.

Workaround: Ensure that the any keyword is configured for the ACE instead of a source address, that is, enter the permit protocol any multicast-ipv6-prefix/prefix-length sequence value command.

Symptoms: When a class map that contains an access control list (ACL) that is too large and complex to fit in memory is applied to an MQC policy map on a Cisco 10720, the router pauses indefinitely while compiling the ACL and generates a MALLOCFAIL error. The router should report an out-of-memory situation.

Conditions: This symptom is observed when the ACL contains 2000 lines and is complex.

Symptoms: When the Any Transport over MPLS (AToM) feature is enabled on a router, AToM virtual circuits to a peer may not be re-established after an interface flap or after being reconfigured, because the required targeted Label Distribution Protocol (LDP) session is not re-established.

Conditions: This symptom is observed when LDP is not configured on any interfaces via the mpls ip interface configuration command, which is typically the case when MPLS Traffic Engineering (TE) tunnels are used to transport AToM traffic between endpoints and when the mpls ip interface configuration command is not enabled on any TE tunnels.

The symptom occurs in Cisco IOS software releases that include the fix for caveat CSCec69982 when any form of one of the following commands is configured on the router and appears in the running configuration:

Workaround: Enter the mpls ip command on a TE tunnel interface or temporarily on a physical interface to force LDP to be re-established.

Symptoms: An IPv6 ACL configuration may be lost or incorrect after an SSO switchover.

Conditions: This symptom is observed when a Cisco 12000 series performs IPv6 ACL filtering and when the ACL is modified.

Symptoms: When a high rate of multicast and unicast bi-directional traffic is sent into multiple access ports across an SRP uplink interface, the PXF engine may restart unexpectedly. Functionality is restored automatically.

Conditions: This symptom is observed on a Cisco 10720 that processes a high rate of multicast and unicast traffic.

Symptoms: A router misses an entry in its label forwarding table, which is shown in the output of the show tag-switching forwarding-table EXEC command for the missing entry and in the output of the show ip cef detail EXEC command for the prefix.

Conditions: This symptom is observed on a Cisco router that is configured for Multiprotocol Label Switching (MPLS) and that learns its routes through iBGP from redundant route reflectors (RRs) when BGP labeling is not enabled.

Workaround: There is no workaround. However, when you enter the clear ip route EXEC command for the affected prefix, the prefix is reinstalled in the label forwarding table.

The Cisco IOS Stack Group Bidding Protocol (SGBP) feature in certain versions of Cisco IOS software is vulnerable to a remotely-exploitable denial of service condition. Devices that do not support or have not enabled the SGBP protocol are not affected by this vulnerability.

Cisco has made free software available to address this vulnerability for affected customers. There are workarounds available to mitigate the effects of the vulnerability.

Symptoms: On a 6PE router, an IPv6 ACL that is configured on an egress interface that faces a CE router does not filter any traffic.

Conditions: This symptom is observed on a Cisco 12000 series that is configured for 6PE when the egress interface is an interface of a line card that cannot process packets in its hardware, for example, an Engine 0 or Engine 2 line card.

Symptoms: After a Cisco 12000 series is rebooted, the interfaces that are associated with a 6-port channelized T3 (T1) line card may not come up.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(31)S.

Symptoms: All VIPs or FlexWAN modules reload unexpectedly on a platform that is configured for Modular QoS CLI (MQC).

Conditions: This symptom is observed on a Cisco 7500 series (with VIPs) and a Cisco 7600 series and Cisco Catalyst 6500 series (both with FlexWANs) when the following steps occur while the physical interface is in the UP state:

An input policy and output policy map are already attached to an ATM or Frame Relay PVC. When you attach the same policy map to the main interface, an error message is generated and the configuration is rejected.

You remove the policy map from the PVC and attach the same policy map to the main interface.

At this point, all VIPS or FlexWAN modules reload, even though no traffic is being processed during the above-mentioned steps.

Symptoms: When IPv4 multicast packets have a resultant IP checksum of "FFXX", the checksum becomes corrupt.

Conditions: This symptom is observed on a Cisco 10720 that runs Cisco IOS Release 12.0(31)S.

Symptoms: A serial interface that is configured for CRC-16 may revert to CRC-32 when a router reloads.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(31)S when you configure a serial interface on a 4-port channelized T3 to DS0 SPA that is installed in a SIP-400 for CRC-16. When the configuration is saved and the router reloads, the interface comes up configured for CRC-32.

Symptoms: A Cisco 12000 series that is configured with dual PRPs, that has more than one 10G Engine 5 SPA Interface Processor (12000-SIP-600), and that has a 10-port Gigabit Ethernet (SPA-10X1GE) installed in each 12000-SIP-600 may not load one of the SPA modules after a cold boot.

Conditions: The symptom is observed only when the Cisco 12000 series is powered off and powered back on. The symptom does not occur on a Cisco 12000 series that is configured with a single PRP.

Further Problem Description: The symptom is related to a race condition that is only observed on the Cisco 12000 series. The symptom is more likely to occur when timing becomes an issue, for example, in a configuration with a large number of interfaces as described in the Symptoms above. However, the root cause of this race condition is platform-independent and relates to the interface IfIndex synchronization. This is the reason why the fix for this caveat is integrated in releases that do not support the Cisco 12000 series.

Symptoms: The startup configuration in NVRAM is not loaded onto line cards when the router is manually reloaded.

Conditions: This symptom is observed on a Cisco 12000 series that functions as a multiservice edge (MSE) router when the ATM Cell Relay over MPLS feature is configured on 500 connections. The symptom may also occur on other platforms.

Workaround: After the router has been reloaded, cut and paste the initially rejected configuration onto the line cards.

Symptoms: After you have send linerate traffic via an IMA interface for a while, a ping fails in a packed cell relay configuration via ATM over L2TPv3 pseudowires.

Conditions: This symptom is observed in a scalable packed cell relay configuration on an IMA interface of a PA-A3-8E1IMA or PA-A3-8T1IMA port adapter that is installed in a Cisco 7200 series or Cisco 7500 series.

Conditions: This symptom is observed on a Cisco 12000 series that functions in a scaled VPN environment when an Engine 6 line card faces the core of the MPLS network.

Symptoms: An snmpwalk for cmplsFrrFacObjects for the FRR-MIB fails to show entries for a tunnel headend.

Conditions: The symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(26)S6, Release 12.0(28)S3, or Release 12.0(31)S.

Conditions: This symptom is observed on a Cisco router after a duplicate BGP MDT extended community message is received that specifies a different Route Descriptor (RD) for an MDT that already exists for the specified MDT source and group address.

Symptoms: A line card or port adapter may generate SYS-6-STACKLOW error messages and reload because of a software forced crash. The crashinfo file shows that the crash is caused by the CEF Scanner process that is related to recursive calls:

The output of the show cef events command for the line card or port adapter shows that a CPU hog condition occurs after the CEF Scanner process:

...
Process Scanner event loop enter
CPUHOG  -1ms XDRtyp 8=control len=11 Hex:0F0000079C00FA00
CPUHOG  2044ms XDRtyp 8=control len=13 Hex:08000000038E2A541A01
Flag    FIB switching running set to yes
+3d04h  CPUHOG  -4ms XDRtyp 69=TFIB_FRR_UNPROTECT_TRANSIT len=15
Hex:020001F40000000400000000

Conditions: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.0(31)S.

Symptoms: An MFR link flaps when you remove and add a VLAN interface back on an SPA that is installed in an SPA Interface Processors (SIP).

Conditions: This symptom is observed on a Cisco 12000 series that is configured with a channelized T3 (to DS0) SPA that has both VLANS and an MFR configuration with Frame Relay subinterfaces. The SPA is installed in a SIP-400.

Symptoms: A Cisco router may report high CPU usage and memory depletion under a specific MPLS VPN configuration with static routes.

Conditions: This symptom is observed when equal cost static routes to a subnet point to a next hop address and there is a summary route that covers one of the next hops pointing to Null0.

If the directly connected route to the next hop is terminated because the interface goes down, the original route recurses to Null0 while the route recursing through the interface that is still up remains in the routing table. The end result is that the route now points to both Null0 and to a valid interface that is up, causing an MPLS recursion problem that results in high CPU usage and memory depletion.

ip route a.a.a.a b.b.b.b y.y.y.y
ip route a.a.a.a b.b.b.b z.z.z.z
ip route y.y.y.y  mask  Null0

If the directly connected route y.y.y.y is removed the a/b subnet recurses through the y.y.y.y/mask route to Null0.

Workaround: Use routes that point to both a next hop and an egress interface, as in the following example:

ip route a.a.a.a b.b.b.b interfaceY y.y.y.y
ip route a.a.a.a b.b.b.b interfaceZ z.z.z.z

Symptoms: When you enter the redundancy force-switchover command, an Engine 4 line card may crash.

Conditions: This symptom is observed on a Cisco 12000 series that runs the c12kprp-p-mz Cisco IOS software image and that has two RPs that function in SSO mode.

Conditions: This symptom is observed on a Cisco 7200 series that is configured with a PA-POS-2OC3 port adapter.

Symptoms: During a high CPU load, the IPC ports on the RP are not opened, preventing CEF from communicating with a line card and causing a FIBDISABLE error message to be generated.

Conditions: This symptom is observed only when the router functions under high stress (that is, there is a high CPU Load on the RP and line cards) during bootup or when you perform an OIR of a line card, RP, or SP.

Further Problem Description: Caveat CSCsb83521 resolves an issue that may occur if CSCsb44220 is integrated in an image. The issue concerns a scheduler error message.

Symptoms: A standby route processor (RP) may crash in the "CEF LC IPC Background" process.

Conditions: This symptom is observed on a Cisco platform when an SSO switchover occurs.

Symptoms: Cell loss occurs when bursty VBR ATM traffic is sent through a Cisco 12000 series 4-port ATM OC-12 ISE line card via an L2TPv3 IP tunnel to another 4-port ATM OC-12 ISE line card on another Cisco 12000 series and when the VBR traffic is sent at rates lower than what is configured on the routers (that is, at about 50 percent of the OC-12 line rate).

Conditions: These symptoms are observed on a Cisco 12000 series that is connected back-to-back via an OC-192 or OC-48 POS link to another Cisco 12000 series.

Symptoms: When you shut down an SRP interface on which the egress L2 priority is set to high by entering the no shutdown interface configuration command, the PXF engine of a downstream router may crash.

Workaround: Do not enter the no shutdown interface configuration command. Rather, force a ring wrap by entering the srp ips forced-switch command.

Symptoms: When you shut down an interface that is configured uRPF in VRF verification mode, the VRF drop count increments.

Conditions: This symptom is observed only on a distributed Cisco platform when the ip verify unicast vrf vrf-name permit|deny command is enabled on the interface and when the uRPF VRF drop counter is non-zero when the interface is shut down.

Symptoms: An IMA port adapter may fail to receive data on a VC that is configured for cell packing with AAL0 encapsulation. The "ignored" counter in the output of the show interface command increments and the "rx_cell_throttle" count in the output of the show controllers command also increments.

Conditions: This symptom is observed when the Maximum Number of Cells Packed (MNCP) parameter is changed for a large number (around 100) of VCs.

Conditions: This symptom is observed when a service policy attached to a T1 or E1 ingress interface on one of the following port adapters:

Symptoms: An Engine 3 or Engine 4+ line card may be stuck in the "request reload" state and CEF may be disabled on the line card, although the CEF table is up, as is shown in the output of the show cef linecard command:

Conditions: This symptom is observed on a Cisco 12000 series after an RPR+ switchover has occurred. However, the symptom is platform-independent and may also occur on another platform that is configured for CEF when an RPR+ switchover has occurred.

Symptoms: The mpls ldp router-id interface force command is not accepted by the router.

Conditions: This symptom is observed on a Cisco 7200 series and Cisco 7500 series when the interface in the interface argument of the mpls ldp router-id interface force command is not yet configured. The symptom may also affect other platforms.

Workaround: First, configure the interface that you intend to use for the LDP router ID. Then, enter the mpls ldp router-id interface force command.

Symptoms: A newly created channelized interface may show packet and byte counts before any traffic passes through the interface.

Conditions: This symptom is observed on a Cisco 12000 series. When a channelized interface is deleted, the interface index is released. This interface index may be re-allocated when a new channelized interface is created. The counters that are associated with the index need to be cleared when an interface is deleted so that they are properly initialized if the index is subsequently re-allocated to a new interface.

Workaround: There is no workaround. Although you can clear the interface counters via the CLI, doing so does not prevent the symptom from occurring because but there is an internal counter that is used in the Tx byte and packet counts and that may cause errors in the calculations.

Symptoms: A standby RP crashes and reloads when you apply an ATM QoS configuration.

Conditions: This symptom is observed on a Cisco 7304 that has two RPs and ATM line cards when an HA switchover occurs and when a QoS configuration is applied or changed.

Conditions: This symptom is observed on a Cisco router that is configured with multilink bundles.

Symptoms: A Cisco 10720 that functions as a transit router for MPLS applications such as MPLS VPN or AToM drops MPLS packets.

Conditions: This symptom is observed when the MPLS packets have multiple labels, when the egress interface on the Cisco 10720 has the ip mtu bytes command enabled, and when the MPLS packet size is greater than the value for the bytes argument.

Symptoms: An Engine 5 SPA enters the "FAILLC" state and resets when you shut down the main interface that is processing fragmented multicast traffic.

Conditions: This symptom is observed on a Cisco 12000 series when fragmented multicast traffic is forwarded across 500 VLANs (250 per port on two ports) and when the no service auto-reset command is enabled.

%SCHED-3-STUCKMTMR: Sleep with expired managed timer 55BE2914 time 0x1CD561 
(00:00:00 ago). 
-Process= "IPC LC Port Opener" ipl= 6 pid= 166

Conditions: This symptom is observed on a Cisco 12000 series that is configured for High Availability (HA).

Symptoms: The counters on a PA-MC-E3 port adapter may provide incorrect information. For some interfaces of the port adapter, the counters are always zero, and for others interfaces, the counters do increase but very slowly.

Conditions: This symptom is observed when you enter the show interfaces type slot command for a PA-MC-E3 port adapter.

Note that the symptom does not occur when you enter the show interface type number stats command or the show interfaces type slot accounting command. Also, when you enter the show interfaces type slot command for the VIP in which the PA-MC-E3 port adapter is installed, the counters provide correct information.

Workaround: Enter the show interface type number stats command to retrieve the correct information.

Symptoms: When you perform an OIR of an alarm module or fan module, the inventory serial number (SN) may become lost after the new module is inserted, and the output of the show inventory command may not show the SN.

This situation prevents you from keeping track of the inventory and affects the operation of the Component Outage On-Line (COOL) feature.

Symptoms: When an GE Engine 5 SPA forwards multicast traffic via multiple VLANs of one GE port, the content of IPv4 multicast packets becomes corrupted.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(31)S.

Symptoms: A software-forced crash may occur on a Cisco 7206VXR because of a watchdog timeout.

Conditions: This symptom is observed on a Cisco 7206VXR that has a low-speed Mueslix-based serial port adapter such as a PA-4T+, PA-8T-V35, PA-8T-X21, or PA-8T-232 port adapter and that runs a Cisco IOS image that integrates the fix for caveat CSCec63468.

The symptom occurs only for low-speed port adapters such as the PA-4T+, PA-8T-V35, PA-8T-X21, and PA-8T-232 port adapters. The symptom may also affect port adapters in adjacent slots, and not only the port adapters in physically adjacent slots, but also the port adapters that are logically adjacent in the initialization path. This memory corruption occurs in the PCI/IO memory space.

Workaround: There is no workaround. Note that high-speed or unchannelized serial port adapters are not affected.

Further Problem Description: The following error messages and tracebacks are generated just before the crash occurs:

%SYS-2-BADSHARE: Bad refcount in datagram_done, ptr=3, count=0
-Traceback= 6074F79C 601BB3AC 601BC72C

%MUESLIX-1-HALT: Mx serial: Serial2/0 TPU halted: cause 0x3 status 0x0043404F shadow 0x630FB864

%ALIGN-3-SPURIOUS: Spurious memory access made at 0x6074F388 reading 0x1F
%ALIGN-3-TRACE: -Traceback= 6074F388 601BB3AC 601BC72C 00000000 00000000 00000000 00000000 00000000
%ALIGN-3-TRACE: -Traceback= 6074F7C0 601BB3AC 601BC72C 00000000 00000000 00000000 00000000 00000000

%SYS-2-WATCHDOG: Process aborted on watchdog timeout, process = Per-Second Jobs.
-Traceback= 607E0078 607E44AC 607DACD0 601B0CD4 601B1A04 601ADEA8 603E2C2C 607CF128 6076E2EC

Symptoms: When you enter the no rd command, the subsequent configuration or unconfiguration of the rd command for the VRF fails.

Conditions: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.0S or Release and 12.4 (no other releases are affected) when the router bgp and address-family vpnv4 commands are not enabled and when the fix for caveat CSCeh12594 is integrated in the release. A list of the affected releases can be found at http://www.cisco.com/cgi-bin/Support/Bugtool/onebug.pl?bugid=CSCeh12594. Cisco IOS software releases that are not listed in the "First Fixed-in Version" field at this location are not affected.

Workaround: Enter the router bgp command followed by the address-family vpnv4 command.

Further Problem Description: The symptom occurs because a flag is set when you enter the no rd command for a VRF. Resetting the flag is essential to complete the process and occurs in a service routine that is registered only if the VPNv4 address family is configured for BGP. The fix for this caveat checks whether or not the VPNv4 address family is configured for BGP, and does not reset the flag if the VPNv4 address family is not configured for BGP.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(30)S1 during a smartbits test with random packet sizes.

Workaround: There is no workaround. Note that the symptom does not occur in Cisco IOS Release 12.0(30)S.

Symptoms: There is no traffic or traffic forwarded to an incorrect interface based upon the DSCP value of the IP packet.

Conditions: This symptom occurs when interfaces are deleted and added back with MQC.

Symptoms: When Virtual Router Redundancy Protocol (VRRP) is configured on a Cisco 12000 series and the FIB becomes disabled, VRRP becomes active in the groups for which it is configured and responds to ARP requests for the virtual address.

Conditions: This symptom is observed on a Cisco 12000 series when VRRP is configured and dCEF is disabled as a consequence of a memory allocation failure in the FIB.

Symptoms: A router may fail when you reload a Gigabit Ethernet (GE) line card or port adapter that has link-bundling enabled.

Conditions: This symptom is observed on a Cisco router when dot1q is configured on a GE interface of the line card or port adapter and when MPLS is enabled on an uplink.

Symptoms: The MQC may not take effect after you have applied a policy to an interface of a 4-port OC-48 POS Engine 4+ line card.

Conditions: This symptom is observed on a Cisco 12000 series that runs the gsr-p-mz image of an interim release for Cisco IOS Release 12.0(32)S.

Symptoms: Source Border Gateway Protocol Policy Accounting (BGPPA) counters do not increment.

Conditions: This symptom is observed on a Cisco 12000 series when you enter the show cef interface type number policy-statistics command.

Symptoms: In Cisco IOS software that is running the Multiprotocol Label Switching (MPLS) Label Distribution Protocol (LDP), the router could reload while trying to access a bad virtual address.

Conditions: This symptom may be observed when LDP is being used. It will not be observed with TDP. It may happen when LDP receives a protocol message larger than 512 bytes right after receiving several Label Mapping messages smaller than 25 bytes. This problem is likely to be accompanied by the presence of one of the following error message:

Address Error (load or instruction fetch) exception, CPU signal 10, PC = 0xD0D0D0D

The above error message may be preceded by one of the following four error messages:

%ALIGN-1-FATAL: Corrupted program counter 
pc=0xD0D0D0D, ra=0x61164128, sp=0x64879B98

This problem may be seen in releases that include the fix for CSCeg74562 but do not have the fix associated with this defect.

Symptoms: A GE Engine 2 line card that is configured for EoMPLS and the Carrier supporting Carrier feature with IPV4 BGP labels on two different dot1q subinterfaces may reset when you enter the no hw-module slot slot-number shutdown command on the line card.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(31)S or a later release.

Symptoms: When a PE router has an output policy with the mpls experimental number command enabled on an egress interface that is configured for LDP, incoming IP packets that match the number argument are not selected and handled as default packets.

Conditions: This symptom is observed on a Cisco 10720 that functions as a PE router with a single output policy and no input policy.

Workaround: On the egress interface that is configured for LDP and that has the output policy, configure also an input policy.

Symptoms: The following error messages and tracebacks are generated on the RP and line card of a Cisco 12000 series that processes IPv6 multicast traffic:

%SYS-2-MALLOCFAIL: Memory allocation of 3996 bytes failed from 0x50FFC814, alignment 0 
Pool: Processor  Free: 300276  Cause: Memory fragmentation 
Alternate Pool: None  Free: 0  Cause: No Alternate pool

-Process= "MFIB", ipl= 0, pid= 171
-Traceback= 501E0B58 50295620 5029A848 50FFC81C 50F78C20 50F78DE0 50FFDC30 50F78020 50F781F4 50F761D0 50F7588C 50F75BCC 50FFF194 50289CE4 50289CD0

-Process= "TAG Stats Background", ipl= 0, pid= 79
-Traceback= 40030E1C 400E3798 400E68D8 412A0B10 412A0F70

%SYS-2-CFORKMEM: Process creation of OBFL Coalesce failed (no memory).
-Process= "Logger", ipl= 0, pid= 25
-Traceback= 40030E1C 4011334C 401134F8 409F0868 409EFD78 409F15FC 409F175C 4002EFDC 4003170C

Conditions: This symptom is observed on a Cisco 12000 series that runs the gsr-p-mz image of an interim release for Cisco IOS Release 12.0(32)S.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(31)S1 and only affects Engine 5 line cards.

Workaround: Create an SNMP View that excludes the Fabric MIB (CISCO-FABRIC-C12K-MIB).

Symptoms: A Cisco 12000 series that is configured with a link bundle interface may generate a "FIB-3-LB_INTNOTEXIST" error message and traceback.

Conditions: This symptom is observed when any of the following conditions occur:

–

A link-bundle member that contains subinterfaces is re-added to a port channel.

–

A link-bundle member that contains subinterfaces is moved from one port channel to another.

–

A normal Gigabit Ethernet interface that contains subinterfaces becomes a link-bundle member.

Further Problem Description: The root cause of the symptom is any operation that involves the recreation of a subinterface re-creation with a link bundle.

Symptoms: In a VPN topology, when you enter the hw-module slot x qos interface queues 8 command on a 1-port channelized OC-12 (DS3) ISE line card, traffic does not go through MLP links.

Conditions: This symptom is observed on a Cisco 12000 series that runs an interim release for Cisco IOS Release 12.0(32)S.

Symptoms: An Engine 5 SPA continuously crashes when 2000 unique egress policies are applied to VLAN subinterfaces.

Conditions: This symptom is observed on a Cisco 12000 series and is related to a timing issue.

Symptoms: In an MVPN topology, an Engine 4+ POS line card that is located in the network core may punt packets with a size of 1477 bytes or more to its CPU.

Conditions: This symptom is observed on a Cisco 12000 series that runs an interim release for Cisco IOS Release 12.0(32)S.

Symptoms: When you send multicast traffic at or above 32 Kpps, the PXF buffers may deplete until they are exhausted.

Conditions: This symptom is observed on a Cisco 10720 when ACL logging is configured to deny multicast traffic at a rate of 32 Kpps and above.

Symptoms: After an SSO switchover, interfaces of a channelized T1/E1 SPA go down and do not recover.

Conditions: This symptom is observed on a Cisco 12000 series that is configured with two PRP2 processors that function in SSO mode when you enter the redundancy force-switchover command.

Workaround: Enter the shutdown command followed by the no shutdown command on the controller of the SPA to bring up the interfaces.

Symptoms: A SIP-600 in which a 10-port Gigabit Ethernet SPA is installed resets because of an IPC timeout.

Conditions: This symptom is observed on a Cisco 12000 series when the SIP-600 is configured for VPLS and faces the core of the network. Note that the symptom does not occur with a 4-port Gigabit Ethernet ISE line card.

Conditions: This symptom is observed on a Cisco 12000 series when you enter the service download-fl command.

Symptoms: An L2 overhead specification for shaping on a Gigabit Ethernet ISE line card is saved with the wrong syntax, and after you reload the router, this configuration is rejected.

Conditions: This symptom is observed on a Cisco 12000 series that runs an interim release for Cisco IOS Release 12.0(32)S.

Symptoms: You may not be able to ping across a serial T1 link that is configured on a channelized T3 SPA that is installed in a SIP-400.

Conditions: This symptom is observed on a Cisco 12000 series that runs the gsr-p-mz image of an interim release for Cisco IOS Release 12.0(32)S and that has the ip routing external overload signalling command enabled.

Workaround: Disable and then re-enable the ip routing external overload signalling command enabled.

Symptoms: When the router at the opposite site is reloaded, a Dynamic Packet Transport (DPT) line card crashes because of a "Bus Error exception."

Conditions: This symptom is observed on a Cisco 12000 series when the router at the opposite site is also configured with a DPT line card.

Symptoms: MPLS may fail when you remove Link Bundling from a Modular GbE Engine 4+ line card.

Conditions: This symptom is observed on a Cisco 12000 series that runs an interim release for Cisco IOS Release 12.0(32)S and that is configured for VPLS.

Workaround: Re-enable MPLS by entering the mpls ip global configuration command.

Symptoms: When multicast QoS is configured on an egress interface of a POS or GE ISE line card and you reload the router, %EE48-3-ALPHAERR error messages are generated and traffic may not resume on the line card.

Conditions: This symptom is observed on a Cisco 12000 series that runs an interim release for Cisco IOS Release 12.0(32)S and that functions in an MVPN topology.

Conditions: This symptom is observed on a Cisco 12816 that runs Cisco IOS Release 12.0(28)S1 and that is configured for QoS.

Symptoms: When you send IPv6 multicast traffic without hardware routes, the IPv6 multicast traffic is punted to the CPU of the line card without any hardware rate-limiting. This situation may cause the CPU of the line card to become overloaded.

Conditions: This symptom is observed on a Cisco 12000 series that is configured with Engine 3 and Engine 5 line cards.

%IPC-5-INVALID: NACK Source Port=0x403F0000
%MCC192-3-CPU_PIF: Error=0x4
%MCC192-3-CPUIF_ERR: Packet Exceeds Programmed Length.
%GSR-3-INTPROC: Process Traceback= 40D32E5C 406D8CE0
...

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(28)S2.

Symptoms: When IS-IS IPv6 routes flap, a memory leak occurs on an ISE line card, eventually causing dCEF to be disabled when no more memory is available.

Conditions: This symptom is observed on a Cisco 12000 series that is configured with 180,000 BGP routes and 6000 IS-IS routes, 10 percent of which flaps each 30 seconds. The symptom occurs only when IS-IS flaps, not when BGP flaps. The symptom does not occur either when IPv6 routing is not configured.

Workaround: There is no workaround. If this is an option, remove IPv6 routing by entering the no ipv6 unicast-routing global configuration command. When the symptom has occurred and dCEF is disabled, you must reload the line card to restore its memory.

Symptoms: Multicast Distributed Switching (MDS) may become disabled on one or more line cards, and the following error messages may be generated:

%SYS-2-WATCHDOG: Process aborted on watchdog timeout, process =
TCAM Mngr merge process.
-Traceback= 40030EFC 40112C80 401176A0 4010CB14 405FB2E0 405FC234 4069F47C 4069F6C4 406A12B4 406A2AA0 404E3950 404E57E8 404E59B0 404E5AD4 404F23A8 404FC7E4
--More-- 
%RP-4-RSTSLOT: Resetting the card in the slot: 1,Event: linecard error report

%QM-2-BAD_TLV: Error in internal messaging - bad tlv 0
%LINK-5-CHANGED: Interface POS1/0/0, changed state to administratively down
%LCINFO-3-CRASH: Line card in slot 1 crashed

The output of the show ip mds stats linecard command shows the MDS status. To reset the lMDS on the line card(s), enter the clear ip mds linecard command.

Conditions: This symptom is observed on a Cisco 12000 series that runs an interim release for Cisco IOS Release 12.0(32)S, that is configured with one or more Engine 5 line cards, and that has the following MQC outbound policy attached to an interface:

Symptoms: A router that is configured for Multiprotocol Label Switching (MPLS) Label Distribution Protocol (LDP) may crash when LDP is configured globally or on an interface.

Conditions: This symptom is observed when you enter the show mpls ldp neighbor command while LDP sessions are coming up or going down.

Symptom 1: A switch or router that is configured with a PA-A3 ATM port adapter may eventually run out of memory. The leak occurs when the FlexWAN or VIP that contains the PA-A3 port adapter is removed from the switch or router and not re-inserted.

The output of the show processes memory command shows that the "ATM PA Helper" process does not have sufficient memory. The output of the show memory allocating-process totals command shows that the "Iterator" process holds the memory.

Cisco IOS software releases that are not listed in the "First Fixed-in Version" fields at these locations are not affected.

Workaround 1: Either do not remove the PA-A3 ATM port adapter from the FlexWAN or VIP or re-insert the PA-A3 ATM port adapter promptly. The memory leak stops immediately when you re-insert the PA-A3 ATM port adapter.

Symptom 2: A switch or router that has certain PIM configurations may eventually run out of memory.

The output of the show processes memory command shows that the "PIM process" does not have sufficient memory. The output of the show memory allocating-process totals command shows that the "Iterator" process holds the memory.

Cisco IOS software releases that are not listed in the "First Fixed-in Version" field at this location are not affected.

Workaround 2: When the ip multicast-routing command is configured, enable at least one interface for PIM. When the ip multicast-routing vrf vrf-name command is configured, enter the ip vrf forwarding vrf-name command on at least one interface that has PIM enabled.

Symptoms: When you create a VRF, a router generates the following error message for a link bundle that does support MPLS VPN and that is configured on a 3-port Gigabit Ethernet (GE) Engine 2 line card:

When you apply the VRF to an interface of the 3-port GE Engine 2 line card by entering the ip vrf forwarding command, the 3-port GE Engine 2 line card crashes.

Conditions: These symptoms are observed on a Cisco 12000 series that is configured with a port channel for link bundling.

Symptoms: A POS ISE line card crashes when you enter the hw-module slot slot-number np mode feature command.

Conditions: This symptom is observed on a Cisco 12000 series when you enter the above-mentioned command after you have configured and unconfigured a subinterface on the POS ISE line card.

Symptoms: A channelized T3 ISE line card in a Cisco 12000 series resets and when it comes back up, the MFR bundle link and the serial interfaces that belongs to the MFR bundle link remain in the down state.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(31)S and that functions as a CE router in a MVPN topology when an interruption of service (such as an RP switchover) occurs on the neighboring router. This interruption of service causes the channelized T3 ISE line card to reset.

Workaround: To bring the interfaces up, enter the microcode reload command on the affected line card.

Conditions: This symptom is observed on a Cisco 12000 series that runs an interim release for Cisco IOS Release 12.0(32)S and that is configured for VPLS when the SIP-600 processes traffic.

Symptoms: When Bit error rate testing (BERT) is configured on a timeslot or channel group, the entire T1 line goes down.

Conditions: This symptom is observed on a Cisco 7600 series and Cisco 12000 series that are configured with a channelized T3 to DS0 SPA.

Symptoms: After an RPR+ switchover has occurred, MFR interfaces that are configured on T1 and T3 SPAs that are installed in a SIP-601 may go down.

Conditions: This symptom is observed on a Cisco 12000 series that runs the gsr-p-mz image of Cisco IOS Release 12.0(32)S.

Symptoms: All interfaces of the SPAs that are installed in a SIP-600 may be administratively down after an RPR+ switchover has occurred.

Conditions: This symptom is observed on a Cisco 12000 series that runs an interim release for Cisco IOS Release 12.0(32)S and that is configured with redundant RPs.

Symptoms: A standard ingress ACL for transit traffic does not function on an interface that is configured for MFR.

Conditions: This symptom is observed on a Cisco 7500 series that runs Cisco IOS Release 12.3(11)T8 and that has an MFR bundle that is configured on a PA-MC-8TE1 port adapter. The symptom may also occur in other releases.

Symptoms: When you regularly poll the ifHCInOctets MIB counter on a subinterface of a Modular GbE Engine 4 line card, the counter restarts at zero after the value 274651394019 (around 2^38) has been reached. The ifHCInOctets MIB counter is supposed to restart at zero after the value 2^64 has been reached.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(31)S2.

Workaround: There is no workaround. Note that the symptom does not occur in Release 12.0(28)S5.

Symptoms: An 1-port 10-Gigabit Ethernet Engine 4+ line card may crash when you perform an OIR by entering the hw-module slot slot-number shutdown command followed by the no hw-module slot slot-number shutdown command or when you enter the microcode reload slot-number command.

Conditions: This symptom is observed on a Cisco 12000 series that runs the gsr-p-mz image of Cisco IOS Release 12.0(32)S

Symptoms: When the mpls ping and traceroute commands are configured, the specified destination address does not take effect, nor is the 127.0.0.1 default address used when the destination address is not specified. Instead, the target FEC is used as the destination IP address in IP header of the outgoing packet.

When you specify a range of destinations for troubleshooting, the target IP address is always used. This situation prevents you from using the mpls ping and traceroute commands to troubleshoot an equal-cost multipath (ECMP) configuration.

Conditions: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.0S.

Symptoms: A 1-port channelized OC-3/STM-1 Engine 5 SPA may crash when you delete an MLP bundle that has 12 links and that is configured on the SPA.

Conditions: This symptom is observed on a Cisco 12000 series when you perform an OIR of the SPA, enter the hw-module reload for the SPA, and then delete the MLP bundle from the SPA.

Symptoms: A 2-port OC-192 POS Engine 6 line card (the 2OC192/POS-SR-SC and the 2OC192/POS-IR-SC) may stop forwarding traffic after running properly for a while. When this situation occurs, the POS interface is in the UP/UP state, but a ping to the directly connected POS interfaces fails. No error messages are generated for the affected line card.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(28)S2 or Release 12.0(28)S5, irrespective of whether or not an ACL is configured on the line card.

Workaround: There is no workaround. You must reset the line card to recover from the symptoms.

Symptoms: Traffic does not flow on some ports (ports 6 through 9) of a 10-port Gigabit Ethernet Engine 5 SPA.

Conditions: This symptom is observed on a Cisco 12000 series that is configured with a SIP-601 that has 10-port Gigabit Ethernet Engine 5 SPAs in Bay 0 and Bay 1.

Symptoms: After an SSO switchover, traffic does not fully recover on a 3-port GE Engine 2 line card that is configured for EoMPLS.

Conditions: This symptom is observed on a Cisco 12000 series that runs the c12kprp-p-mz image of Cisco IOS Release 12.0(28)S5.

Workaround: Enter the clear cef linecard slot-number command to recover from the symptom and enable traffic to pass properly.

Symptoms: All line cards on a Cisco 12000 series may reload when you configure a named ACL.

Workaround: There is no workaround. Named ACLs are not supported on Cisco 12000 series line cards.

Symptoms: Line cards reset continuously and the following error messages are generated:

%FIB-2-FIBDISABLE: Fatal error, slot 1: IPC Failure: timeout %RP-4-RSTSLOT: Resetting the card in the slot: 1,Event: CEF failure

Conditions: This symptom is observed on a Cisco 12000 series when you boot the router with the c12kprp-p-mz image of an interim release for Cisco IOS Release 12.0(32)S.

Symptoms: A Cisco 12000 series may report incorrect ifIndex values in the NetFlow Data Export (NDE) packets that are sent from a Gigabit Ethernet (GE) ISE line card. Flows that arrive via VLAN subinterfaces may be reported as zero with the SNMP ifIndex or as the ifIndex of the physical GE interface.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(30)S3 or a later release after you have reloaded the GE line card.

Workaround: There is no workaround. When the symptom has occurred, re-apply the configuration of the subinterfaces.

Symptoms: In an MVPN topology, a PE router that performs decapsulation may not forward multicast packets to a CE router.

Conditions: This symptom is observed on a Cisco 12000 series that runs an interim release for Cisco IOS Release 12.0(32)S.

Symptoms: Multicast traffic is dropped from an egress interface of an Engine 3 (ISE) and Engine 5 line card.

Workaround for Engine 3 (ISE) line cards: Enter the microcode reload slot-number command.

Conditions: This symptom is observed on a Cisco 12000 series when you apply QoS policies on the SIP-600. The symptom occurs with any QoS policy (WRED, color, time stamping, and so on).

Symptoms: When you reload the core router in an MVPN topology, multicast traffic may not be switched via an ingress Engine 3, Engine 4+, or Engine 5 line card that is installed in the core router.

Conditions: This symptom is observed on a Cisco 12000 series runs Cisco IOS Release 12.0(32)S and that functions as a core router in an MVPN topology.

Workaround: Enter the clear ip mds linecard linecard-slot-number command, in which the linecard-slot-number argument represents the affected line card.

Symptoms: A MAC address is not learned as expected after a microcode reload procedure is performed on a line card that is configured for VPLS.

Conditions: This symptom is observed on a Cisco 12000 series that functions as a core router.

Symptoms: After performing a manual switchover in RPR+ mode, the following error message may be generated continuously on an ISE line card and traffic is no longer forwarded:

Conditions: This symptom is observed on a Cisco 12000 series that runs the gsr-p-mz image of an interim release for Cisco IOS Release 12.0(32)S when the following conditions are present:

–

The ISE line card is installed on the Cisco 12000 series and interconnects the router with a P router.

Symptoms: When you reload a channelized SPA, the following error message and traceback are generated:

Conditions: This symptom is observed on a Cisco 12000 series that is configured with an SPA that has MLP interfaces on which a service policy is attached.

Symptoms: When you configure a class policer in a child policy by entering the police cir percent command, the police rate calculation for the class may be incorrect.

Conditions: This symptom is observed on a Cisco 12000 series that has an hierarchical Modular QoS CLI (MQC) output service policy on the Ethernet interface of a shared port adapter (SPA).

Symptoms: The following traceback and error message may be generated when you reload an SPA or line card on which Multilink PPP (MLP) bundles are configured:

-Traceback= 406A2188 406A3670 406AA4E8 40339B28 4033C1D0 4033C374 40158D78 40159758 eelc_config_intf_tx_q(): EE_QM_QOS_INTERNAL_ERRORarg=3

%SPA_PLIM-3-HEARTBEAT: Subslot 0 has experienced an heartbeat failure Current Sequence 14 received Sequence 8 Time since last keep 440ms

Conditions: This symptom is observed on a Cisco 12000 series that is configured for MLP and occurs irrelevant of whether or not the SPA or line card is processing traffic.

Symptoms: You cannot ping via some MFR interfaces and there is only unidirectional traffic through these MFR interfaces.

Conditions: This symptom is observed on a Cisco 12000 series that is configured with a 1-port channelized STM-1/OC-3 Engine 5 SPA and occurs after you have performed a microcode reload of the SIP in which the SPA is installed.

Workaround: There is no workaround. To recover from the symptoms, enter the shutdown interface configuration command followed by the no shutdown interface configuration command on the affected MFR interfaces.

Symptoms: The RP may crash when you change the MTU while the interfaces are coming up after you have reloaded an SPA or line card. This situation may cause the line protocol of all interfaces on the SPA or line card to remain down.

Conditions: This symptom is observed on a Cisco 12000 series that is configured for Multilink Frame Relay (MFR).

Symptoms: Traffic does not recovers after an HA switchover, and the following ifIndex syslog message and traceback are generated:

%EERP-3-INVALID_INDEX: slot 0, port 1, vc -1 (type 68): index= 4096, maximum= 32 -Traceback= 1FB244 1FB34C 84C5CC 84D524 84D628 7FDD0C 7DC62C 329FBC 32ABFC 32AC88 2DAD9C 27E484 2E2C8C

Conditions: This symptom is observed on a Cisco 12000 series that is configured with the following features: IP, L3VPN, MLP, MR-APS, and mVPN.

Symptoms: MVPN traffic that traverses in the decapsulation direction is punted to the slow path an Engine 4+ line card that faces the core.

Conditions: This symptom is observed on a Cisco 12000 series that runs an interim release for Cisco IOS Release 12.0(32)S.

Symptoms: Traffic may not go through on VLAN-to-ATM and VLAN-to-Frame Relay links in AToM circuits that are configured for local switching.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(32)S.

Conditions: This symptom is observed on a Cisco router that has two PRPs that function in RPR+ redundancy mode.

Workaround: There is no workaround. If all line cards are affected, enter the clear cef linecard command to recover from the symptom. If a specific line card is affected, enter the clear cef linecard slot-number command to recover from the symptom. Enter these command from a console that is attached to the RP or RRP.

TCP/IP Host-Mode Services

Symptoms: A TCP session does not time out but is stuck in the FINWAIT1 state and the following error message is generated:

Conditions: This symptom is observed on a Cisco 12000 series that is configured for BGP and that is connected to a third-party vendor router after the BGP authentication password is changed on the Cisco 12000 series.

Conditions: This symptom is observed when an active TCP connection is established and when data is sent by the Cisco router to the remote router at a much faster rate than what the remote router can handle, causing the remote router to advertise a zero window. Subsequently, when the remote router reads the data, the window is re-opened and the new window is advertised. When this situation occurs, and when the Cisco router has saved data to TCP in order to be send to the remote router, the Cisco router may drop the TCP connection.

Workaround: Increase the window size on both ends to alleviate the symptom to a certain extent. On the Cisco router, enter the ip tcp window-size bytes command. When you use a Telnet connection, reduce the screen-length argument in the terminal length screen-length command to 20 or 30 lines.

Further Problem Description: BGP in Cisco IOS Release 12.0S and Release 12.4 is not affected because the retransmit timeout is disabled for BGP in these releases.

Wide-Area Networking

Symptoms: Some members of a multilink bundle remain inactive, while others are active.

Conditions: This symptom is observed when the interfaces are configured with the ppp chap hostname or ppp multilink endpoint command. Very high speed interfaces may come up and join the multilink bundle faster than the configuration can be processed, which causes them to use the host name of the router (instead of the configured user name or endpoint value) as the Endpoint Discriminator during Link Control Protocol (LCP) negotiations. This situation causes a mismatch between these links and those that come up after the configuration command is processed.

Workaround: Enter the shutdown interface configuration command followed by no shutdown interface configuration command on the active links to enable the links to renegotiate LCP with the correct Endpoint Discriminator value.

Symptoms: Pings fail when translational bridging and ATM DXI encapsulation are configured.

Conditions: This symptom is observed on a Cisco router that runs Cisco IOS Release 12.0S, Release 12.2S, or a release that is based on Release 12.2S.

Workaround: Do not configure ATM DXI encapsulation. Rather, configure HDLC, PPP, or Frame Relay encapsulation.

Conditions: This symptom is observed on a Cisco router that is configured as an LNS that has several tunnels to different LACs.

Symptoms: InvARP packets on multiple MFR bundle interfaces may be dropped, causing traffic to fail after you have reloaded microcode onto a line card that processes a high load of traffic over many PVCs on MFR interfaces.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(31)S when 42 MFR bundles are configured over 336 full T1s and when egress MQC is configured on the 42 MFR bundle interfaces. However, the symptom is not platform- and release-specific.

Symptoms: The Route Processor (RP) of a Cisco 7613 may crash when stress traffic is processed on all WAN links of FlexWANs in which channelized port adapters are installed and when interfaces of the channelized port adapters flap.

Conditions: This symptom is observed when the channelized port adapters are configured for MFR. The symptom may not be platform-specific.

Symptoms: Incoming MPLS packets with IETF Frame Relay encapsulation are process-switched.

Workaround: Do not configure IETF Frame Relay encapsulation. Rather, configure Cisco Frame Relay encapsulation.

Symptoms: With automatic protection switching (APS) configuration on CHOC12 Internet Services Engine (ISE) cards, flapping the working link within a 2- to 3-second time interval may result in some of the T1 links staying down.

Conditions: This symptom has been observed with APS configuration on a CHOC12 ISE card.

Workaround: Enter a shutdown command and then a no shutdown command to clear the problem.

Symptoms: A serial interface on a channelized port adapter may stop forwarding traffic through the router but traffic to and from the router over the interface may still go through. The Tx accumulator "value" counter in the output of the show controllers cbus Exec command does not exceed the value 2, as is shown in the following example:

Router#sh controllers cbus | include Serial5/1/0.1/2/6/2:0
Serial5/1/0.1/2/6/2:0, txq E8001B40, txacc E8000412 (value 2), txlimit 26

Conditions: This symptom is observed on a Cisco 7500 series that runs Cisco IOS Release 12.0S when QoS is configured on at least one interface on the VIP in which the channelized port adapter is installed. The symptom occurs after the affected interface has flapped very frequently because of OSI layer 1 errors.

Symptoms: AToM PVCs on an MFR interface that has keepalives disabled do not pass traffic after the router is rebooted.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0S.

Symptoms: The L2TP management daemon process may leak memory if the parsing of some L2TP control messages fails.

Conditions: This symptom is observed on a Cisco router that is configured or VPDN L2TP tunnels when PPPoE sessions were brought up and down after a period of time.

Symptoms: A router that is configured for PPP Multilink reloads because of a bus error.

Conditions: The exact conditions which caused the router to reload are unknown. It is believed to be triggered by a lack of free packet memory in the router.

Symptoms: When you enter the redundancy force-switchover command on a router that is configured for PPP encapsulation, the IS-IS neighbor comes up in the INIT state.

Conditions: This symptom is observed on a Cisco router that is configured with two RPs that run in in SSO redundancy mode.

Workaround: To bring up the IS-IS state in the IS-IS neighbor, enter the isis protocol shutdown interface configuration command followed by the no isis protocol shutdown interface configuration command on interface that provides the connection to the IS-IS neighbor.

Symptoms: A virtual-access interface fails to come up after you have configured virtual templates.

Conditions: This symptom is observed on a Cisco router that is configured for MFR.

Conditions: This symptom is observed on a Cisco 7500 series that is configured with two RPs that function in RPR+ mode.

Resolved Caveats—Cisco IOS Release 12.0(31)S6

Cisco IOS Release 12.0(31)S6 is a rebuild of Cisco IOS Release 12.0(31)S. The caveats listed in this section are resolved in Cisco IOS Release 12.0(31)S6 but may be open in previous Cisco IOS releases. This section describes only severity 1, severity 2, and select severity 3 caveats.

Basic System Services

Symptoms: IPC communication to an Engine 4 line card may be lost, the line card may reset, and an error message such as the following may be generated on the console of the RP:

%MDX-1-DAEMSGSNDFAILED: FAILED to send IPC message of TYPE MDX_DAE_PULL_REQ to slot 4 on the DAE, FAIL REASON = timeout

When the line card is in this state, the execute-on and attach commands do not function for the line card.

Conditions: This symptom is observed on a Cisco 12000 series that runs the c12kprp-p-mz image of Cisco IOS Release 12.0(27)S5 or Release 12.0(28)S5.

IP Routing Protocols

Symptoms: A Cisco router may reload when you enter the show ip route vrf vrf-name EXEC command.

Workaround: There is no workaround. However, if a single route must be displayed, enter the show ip route vrf vrf-name prefix EXEC command. Doing so may decrease the possibility that the router reloads. When you enter the show ip route vrf vrf-name | in prefix EXEC command, you may increase the possibility that the router reloads.

Symptoms: MVPN traffic is limited to about 9 Mpps and the CPU usage on the egress line card is 100 percent.

Conditions: This symptom is observed on a Cisco router that functions as a PE router when MVPN performs decapsulation in the slow path instead of the fast path.

Workaround: Enter the clear ip ospf process command to enable the BGP Scanner to pick up the changes in the export map.

Conditions: This symptom is observed rarely when all of the following conditions are present:

–

There is a very brief (shorter than 500 ms) interface flap on a point-to-point interface such as a POS interface.

–

The flap is not noticed by the neighbor, so the neighbors interface remains up.

–

The OSPF adjacency goes down and comes back up very quickly (the total time is shorter than 500 ms).

–

OSPF runs an SPF during this period and, based on the transient adjacency information, removes routes via this adjacency.

–

The OSPF LSA generation is delayed because of LSA throttling. When the LSA throttle timer expires and the LSA is built, the LSA appears unchanged.

Workaround: Increase the carrier-delay time for the interface to about 1 second or longer.

Alternate Workaround: Use an LSA build time shorter than the time that it takes for an adjacency to come up completely.

Symptoms: A route is not installed through an MPLS TE tunnel even though the tunnel mpls traffic-eng autoroute announce command is enabled.

Conditions: This symptom is observed on a Cisco router that also has the mpls traffic-eng multicast-intact command enabled.

Conditions: This symptom is observed on a Cisco router when an RSVP session is set up.

Miscellaneous

Symptoms: A VIP in which a PA-2FE port adapter is installed may reload because of memory corruption that is caused by a hardware issue of the PA-2FE port adapter.

Symptoms: Removing a policy that has shape and bandwidth in the same class (in that same order) may cause a router to crash.

Conditions: This symptom is observed when the router functions under a traffic load.

Symptoms: When you configure MPLS Traffic Engineering AutoTunnel Mesh Groups by entering the following commands, many FIB messages and tracebacks are generated:

mpls traffic-eng auto-tunnel mesh
mpls traffic-eng auto-tunnel mesh tunnel-num min 10000 max 20000

Depending on the configuration of he router and the topology, the symptom may also occur when you enter the following sequence of commands:

no mpls traffic-eng auto-tunnel mesh
mpls traffic-eng auto-tunnel mesh
no mpls traffic-eng auto-tunnel mesh
mpls traffic-eng auto-tunnel mesh

Conditions: This symptom is observed on a Cisco 12000 series that runs the gsr-p-mz image of a Cisco IOS interim release for Release 12.0(31)S.

no mpls traffic-eng auto-tunnel mesh
...
copy run start
reload
mpls traffic-eng auto-tunnel mesh

no mpls traffic-eng auto-tunnel mesh
mpls traffic-eng auto-tunnel mesh tunnel-num min 10000 max 20000
...
copy run start
reload
mpls traffic-eng auto-tunnel mesh

Symptoms: MPLS TE LSPs may not come up and may remain stuck in the RSVP signaling proceeding state.

Conditions: This symptom is observed on a Cisco router when the MPLS TE LSPs are processed over inter-autonomous system broadcast links on which the passive-interface command is enabled.

Further Problem Description: The passive-interface command contains the router ID of the remote Autonomous System Border Router (ASBR). A PATH message that leaves the passive interface of the Cisco router is sent to remote ASBR, causing an ARP request to be initiated for remote ASBR. However, there is no response to the ARP request (when there is no proxy configured for ARP), preventing the PATH message from reaching the remote ASBR.

Symptoms: A 4-port OC-48 POS Engine 4+ line card crashes repeatedly after you have entered the microcode reload command for all line cards in the router in succession.

Conditions: This symptom is observed on a Cisco 12000 series that runs an interim release for Cisco IOS Release 12.0(32)S and that functions as a PE router in an option 3 MPLS-(M)VPN Inter-AS environment when the affected line card has a core-facing interface.

Workaround: Perform a forced SSO switchover of the RP. Doing so stops the line card from crashing.

Symptoms: A 4-port OC-3 ATM ISE or 4-port OC-12 POS ISE line card on a PE router may crash when one of its egress interfaces that faces a P router is shut down while traffic is flowing.

Conditions: This symptom is observed on a Cisco 12000 series that functions as a PE router in an MPLS VPN environment, that is configured with a 4-port OC-3 ATM ISE or 4-port OC-12 POS ISE line card that faces the P router in the MPLS core, and that is configured with a 1-port OC-48 POS ISE line card that faces a CE router.

%SYS-2-NULLCHUNK: Memory requested from Null Chunk -Process= "Init", ipl= 3, pid= 3 with an accompanying traceback.

Conditions: This symptom is platform- and release-independent and occurs when the router boots.

Workaround: There is no workaround. However, proper system operation is not affected.

Symptoms: The following error message may be generated when you configure a police statement in a policy map:

Conditions: This symptom is observed on a Cisco router that functions in a L2VPN configuration with QoS features.

Symptoms: Traffic loss may occur during reoptimization on a Cisco router that functions as a transit node for zero-bandwidth MPLS TE label switched paths (LSPs). The traffic loss stops when the TE tunnel headend switches traffic over to the new LSP.

Conditions: This symptom is observed on a Cisco router when reoptimization is triggered on the headend either periodically, manually, or as a result of a topology change.

Symptoms: An Engine 6 line card may forward traffic to an incorrect destination port.

Conditions: This symptom is observed on a Cisco 12000 series that functions in a scaled configuration with 32,000 adjacencies, 770,000 single-path routes, and 99,000 4-path multi-path routes.

Symptoms: OSPF adjacencies may be reset twice after an RPR+ switchover has occurred.

Conditions: This symptom is observed on a Cisco 12000 series when, after the OSPF adjacencies have been reset as expected because of the RPR+ switchover, the OSPF adjacencies are reset again because the core-facing interface reports an APS change state even without an APS configuration.

Symptoms: Failures may occur on multiple DS3 interfaces of a 1-port CHOC-48 ISE line card. The line protocol on these interfaces may not come up, even not to an internal controller loop of the entire OC-48 circuit.

Conditions: These symptoms are observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(28)S4 when dynamic provisioning is active.

Symptoms: Duplicate processes are created for SONET alarm and Path Trace Buffer (PTB) handling on an Engine 4+ line card.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(31)S or a later release.

Conditions: This symptom is observed on a Cisco 12000 series when you perform a basic operation such as shutting down the interface, reloading the line card, or performing an OIR of the line card.

Workaround: There is no workaround to prevent the symptom from occurring. When the symptom has occurred, re-attach the service policy to the interface.

Symptoms: Multicast packets with certain ranges of packet sizes may be dropped. by an ISE line card.

Conditions: This symptom is observed on a Cisco 12000 series that functions as a PE router and that is configured with an ISE line card. The ISE line card has feature mode enabled and has an interface that connects to a CE router.

Symptoms: PIM sessions fail to come up over a Data Multicast Distribution Tree (MDT).

Conditions: This symptom is observed on a Cisco router that functions as a PE router after you have entered the clear ip bgp * command.

Workaround: Enter the clear ip mds linecard linecard-slot-number command. For the linecard-slot-number argument, enter the core-facing slot.

Conditions: This symptom is observed on a Cisco 12416 that runs Cisco IOS Release 12.0(32)S3 and that has two redundant 1-port channelized OC-48 POS ISE line cards that are configured for APS.

Symptoms: A VIP crashes at the "free_wred_stats" function during an RPR+ switchover.

Symptoms: In an MVPN topology, an Engine 4+ POS line card that is located in the network core may punt packets with a size of 1477 bytes or more to its CPU.

Conditions: This symptom is observed on a Cisco 12000 series that runs an interim release for Cisco IOS Release 12.0(32)S.

Conditions: This symptom is observed on a Cisco 12816 that runs Cisco IOS Release 12.0(28)S1 and that is configured for QoS.

Symptoms: In an MVPN topology in which routers in the core are configured for PIM sparse mode with data MDT groups, an interruption in traffic in the core such as a reload of a P router may cause a PE router to stop sending traffic to the core.

Conditions: This symptom is observed when the traffic interruption causes the traffic to fall back to the default MDT and PIM sparse mode to enter the registering state. When the traffic resumes before the multicast route for the data MDT group times out, the traffic is initially forwarded over the default MDT and then switched to the data MDT, but no PIM registration control packets are sent.

Workaround: Enter the clear ip mds linecard linecard-slot-number in which the linecard-slot-number argument is the ingress line card on the affected PE router.

Further Problem Description: We recommend the use of Source Specific Multicast (SSM) for data MDTs. Using SSM, you can avoid the symptoms entirely.

Symptoms: An Engine 2 line card may crash when it receives multicast traffic that is not punted to the CPU of the line card but switched by the PSA memory of the line card.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(27)S or a later release when the Engine 2 line card is configured with the wrong PSA memory and when you enable multicast hardware acceleration by entering the hw-module slot slot-number ip multicast hw-accelerate command.

Symptoms: An Engine 3 or Engine 5 line card may crash while processing packets for Output Sampled NetFlow.

Symptoms: A Cisco 12000 series line card may crash while processing packets that are not aligned to a 4-byte boundary.

Conditions: This symptom is observed when the packet is processed in the CPU of the line card.

Conditions: The symptom is observed intermittently on a Cisco 12000 series that runs Cisco IOS Release 12.0(31)S2 and that is configured with 1-port 10GE SPAs that are installed in a SIP-600.

Symptoms: The IPC link between a SIP and its SPAs may fail, an "RX IPC FIFO FULL" error message and heartbeat failure error messages may be generated, and all interfaces on the SPAs that are installed in the SIP may go down.

Conditions: This symptom is observed when a large number of channel interfaces are defined on the SPAs and when traffic with small packet sizes is passing at a high rate (near line rate) through the SIP.

Workaround: There is no workaround. To recover from the symptoms, reload the affected SPAs or reload the SIP.

Symptoms: Multicast packets may be lost intermittently on a Cisco 12000 series. On average, one packet may be lost per minute per multicast stream.

Symptoms: A traffic interruption in an MVPN core such as a reload of a P router may prevent a connected PE router that functions as a PIM neighbor from re-establishing a connection with other PE routers.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0(31)S2.

Workaround: Enter the shutdown interface configuration command followed by the no shutdown interface configuration command on the interface of the P router that has been reloaded and that connects to the PIM neighbor.

Symptoms: When you configure FRoMPLS on a 4-port CHOC-12 ISE line card, the line card may reload continuously.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco IOS Release 12.0S or Release 12.0(32)SY.

Symptoms: A Cisco 7200 series that has a service policy on a channelized interface may reload unexpectedly.

Conditions: This symptom is observed on a Cisco 7200VXR that runs Cisco IOS Release 12.0(31)S4 or Release 12.0(32)S and that is configured with an NPE-G1 and a PA-MC-8TE1+ port adapter. The symptom may also affect other releases of Release 12.0S. The symptom occurs only under the following conditions:

–

A CBWFQ service policy is applied to a channelized interface of the PA-MC-8TE1+ port adapter.

–

The router is booted with the service policy applied to the interface (that is, the service policy is defined in the startup configuration).

Workaround: After the router comes up, remove and re-apply the service policy to the channelized interfaces.

Alternate Workaround: Do not boot the router with the service policy applied to the interface but apply the service policy after the router has booted.

Symptoms: A header buffer leak may occur on a Cisco 7200 series that has a service policy on a channelized interface. You can verify the leak in the output of the show buffers | i Head command by looking at the total number of header buffers that are in use:

–

A CBWFQ service policy is applied to a channelized interface of the PA-MC-8TE1+ port adapter.

–

The router is booted without the service policy applied to the interface (that is, the service policy is not defined in the startup configuration).

conf t
access-list 60 permit 10.0.7.3
access-list 60 permit 10.0.1.5
access-list 60 permit 10.0.2.6
access-list 60 permit 10.0.3.7
access-list 60 permit 10.0.5.1
access-list 60 permit 10.0.6.2
access-list 60 permit 10.0.8.12

interface Auto-Template1
ip unnumbered Loopback0
no ip directed-broadcast
tunnel destination access-list 60
tunnel mode mpls traffic-eng

access-list 60 permit 10.0.7.3
access-list 60 permit 10.0.1.5
access-list 60 permit 10.0.2.6
access-list 60 permit 10.0.3.7
access-list 60 permit 10.0.5.1
access-list 60 permit 10.0.6.2
access-list 60 permit 10.0.8.12

Symptoms: A multichannel T1 or E1 port adapter may become deactivated when you enter the invert data or crc 16 interface configuration command.

Conditions: This symptom is observed on a Cisco 7500 series that is configured with an RSP4 that contains an R5000 microprocessor.

Symptoms: An interface of a 2-port OC-192 POS Engine 6 line card may stop sampling NetFlow.

Conditions: This symptom is observed on a Cisco 12000 series that runs Cisco OS Release 12.0(28)S6.

Workaround: Remove NetFlow from the affected interface and then re-apply NetFlow to the interface.

Resolved Caveats—Cisco IOS Release 12.0(31)S5

Cisco IOS Release 12.0(31)S5 is a rebuild of Cisco IOS Release 12.0(31)S. The caveats listed in this section are resolved in Cisco IOS Release 12.0(31)S5 but may be open in previous Cisco IOS releases. This section describes only severity 1, severity 2, and select severity 3 caveats.

Basic System Services

Symptoms: Intensive SNMP polling may cause the I/O memory of a router to be depleted.

Symptoms: A spurious memory access may be generated on an RSP when a VIP that is in a disabled or wedged condition is recovered because of a Cbus Complex or microcode reload.

Conditions: This symptom is observed on a Cisco 7500 series that has a VIP that is in a disabled or wedged condition after the router has booted.

Interfaces and Bridging

Symptoms: POS interfaces may remain in the up/down state after the router has been reloaded.

Conditions: This symptom is observed on a Cisco Catalyst 6500 series, Cisco 7500 series, and Cisco 7600 series.

Workaround: Reload the FlexWAN or VIP in which the POS port adapter is installed.

When the encapsulation type is changed, the maximum datagram size may change, which causes an internal MTU change. This situation may cause some packets to be dropped as giants.

Workaround for the packet drops: Reconfigure the MTU to prevent packet from being dropped as giants.

Symptoms: A Cisco 7200 series may reload unexpectedly when an Automatic Protection Switching (APS) switchover occurs on Packet over SONET (POS) interfaces that are configured for redundancy.

IP Routing Protocols

Symptoms: The IGP metric is not updated to an eBGP peer when there is an IGP- metric change. The BGP peer is not advertising this change after its default timer of 10 minutes.

Workaround: Use clear ip bgp neighbor-address [soft [in | out]] on the respective router to get the correct metric changes and BGP best path.

Symptoms: A router may delete the VPNv4 prefixes from the BGP table, even though the counters in the output of the show ip bgp command may indicate that the VPNv4 prefixes are present in the BGP table. This situation may cause loss of VPN connectivity.

Conditions: This symptom is observed on a Cisco router that is configured for MPLS VPN and that functions as a PE router.

Workaround: There is no workaround. When the symptom occurs, enter the clear ip bgp * command to restore proper operation of the router.

Miscellaneous

Symptoms: A Cisco 7500 series that has a multichannel E1 port adapter that is configured with PPP multilink groups on several E1 channels may drop packets:

–

When dCEF is enabled, packets that are larger than 1492 bytes are dropped. Note that this particular symptom is addressed and fixed via CSCin73658.

–

When the router has dCEF disabled, packets that are larger than 1500 bytes are dropped.

Conditions The symptom is observed on a Cisco 7500 series that runs Cisco IOS Release 12.0(28)S.

Workaround: Configure only one interface as the member of the PPP multilink group. Note that the symptom does not occur in Release 12.0(26)S3, 12.0(27)S2, and 12.1(19)E1.

Symptoms: Spurious memory accesses occur on a Cisco 7200 series and ALIGN-3-SPURIOUS error messages are generated.

Conditions: This symptom is observed when there is traffic through the serial interface. Workaround: There is no workaround.

Symptoms: A linecard reloads when traffic is sent through an ATM PVC with the QoS policy enabled.

Workaround: Add a dummy class to the policy-map with a bandwidth or shape feature as shown in the following example:

Symptoms: An MTU change on a multilink bundle interface takes no effect for SPA interfaces. This situation may cause the traffic to be dropped.

Conditions: This symptom is observed on a Cisco 12000 series when you change the MTU on a multilink bundle interface and when you change the traffic generation accordingly.

Symptoms: An interface of an Engine 3 ingress line card that functions in feature mode may become stuck, and all traffic may be dropped.

Workaround: Disable the hw-module slot slot-number np mode feature command on the Engine 3 ingress line card.

Symptoms: A SIP may generate an error message or crash when you first perform an OIR of an SPA-10X1GE that is installed in the SIP and you then enter the show tech-support command.

–

Issue each command separately that makes up the show tech set of commands. Doing so will avoid the command show hw-module subslot all brief due to which the crash happens.

–

Shut down SIP-600 and then OIR the spa. This will result in the other spa in the SIP-600 getting reloaded as well.

Further Problem Description: The symptom does not occur when a 1-port OC192/STM64 POS/RPR SPA is installed in the SIP.

Symptoms: If the sender and receiver for a multicast group are on the same line card on a PE router, the v flag may be set, causing traffic to be punted to the RP.

Conditions: This symptom is observed on a Cisco 12000 series when a hash collision occurs. The symptom may not be platform-specific.

Symptoms: Routers may crash with MPLS VPN configuration and shutting down PE-CE link.

Conditions: The symptoms may occur when the router acts as a PE on the MPLS VPN topology and when one or more PE-CE links are shut down.

Conditions: This symptom is observed only when the multilink interface is oversubscribed and does not occur under normal traffic conditions.

Symptoms: A VIP or FlexWAN module in which a PA-POS-2OC3 port adaptor is installed may crash.

Conditions: This symptom is observed rarely and at random on a Cisco 7xxx series router or Cisco Catalyst 6000 series switch.

Symptoms: CEF may not be updated with a new path label that is received from the BGP peer.

If a router configured for BGP IPv4+labels multipath receives a BGP update that only changes the MPLS label for a non-bestpath multipath, the router fails to update the forwarding plane. This results in dropping or mis-branding the traffic.

Conditions: In a IPv4+labels multipath setup, if a label is changed for the non-bestpath multipath and that is the only change in the new update received from the neighbor, the new label will not be programmed in forwarding, hence there will be label inconsistency between the BGP and the forwarding tables.

Symptoms: An inconsistency may occur in the outlabel information that is used by BGP and MPLS forwarding.

–

The PE router that is the source restarts, causing the prefix to be readvertised with a new label.

–

The RR that forms the non-best path delays the withdrawal and readvertisement of the prefix, for example, because the RR has a heavy load.

This situation causes BGP to function with the new label but MPLS forwarding to function with the old label.

Workaround: Enter the clear ip route network command for the affected prefix.