|
|
Table Of Contents
Troubleshooting Cisco Emergency Responder
Troubleshooting Phone-Related Problems
Phone Sometimes Disappears in Cisco Emergency Responder
Wrong ERL is Used for a Shared Line
802.11b Endpoints Using Wrong ERL
Troubleshooting Emergency Call Problems
Emergency Calls are Not Being Intercepted by Cisco Emergency Responder
ELIN not Transmitted to the PSAP
ELIN For Default ERL Used For Calls From Other ERLs
Emergency Calls Not Routed to the Correct PSAP
Emergency Callers Sometimes Get Busy Signal and Emergency Calls Are Sometimes Not Routed
Onsite Alert Personnel Are Not Getting Telephone Alerts
Onsite Alert Phone Does Not Ring When Emergency Call is Placed
Prompts for Phone Alerts Not Getting Played
Onsite Alert Personnel Not Getting Email (or Paging) Notifications
Incorrect Location Information Sent To Onsite Alert Personnel
Emergency Call History Problems
Failed to Establish Communication with Cisco Emergency Responder Phone Tracking Engine
Lost Communication with Cisco Emergency Responder Phone Tracking Engine
Failed to Send Unlocated Phone Details to Remote Cisco Emergency Responder Server Group
Emergency Call Could Not be Routed
Calling Party Modification Failed
Troubleshooting Cisco Emergency Responder System and Administration Problems
Troubleshooting Login Problems
Using Cisco Unified Operations Manager
Troubleshooting Cisco Emergency Responder Switch and Port Configuration Problems
Using The ERL Debug Tool to Verify Cisco Emergency Responder Configuration
Replacing the Publisher Server and Subscriber Servers
Using the Cisco Emergency Responder Admin Utility
Troubleshooting the Database and Enterprise Replication
Troubleshooting Cisco Emergency Responder System Problems
Troubleshooting Cisco Unified Communications Manager Configuration Problems
Identifying the Cisco Emergency Responder Groups and Servers in a Cisco Emergency Responder Cluster
Phones Moving Between Clusters
Starting and Stopping a Cisco Emergency Responder Server
Troubleshooting ALI Data Uploads
Editing NENA 2.0 and 2.1 File Formats
Collecting Trace and Debug Information
Enabling Detailed Trace and Debug Information for Cisco Emergency Responder
Integrating with Network Management Systems
Monitoring Cisco Emergency Responder Subsystem Status
Collecting Information from Syslog
Backing Up and Recovering Data
Troubleshooting the Data Migration Assistant
Troubleshooting Windows Upgrades
Troubleshooting Linux Upgrades
Troubleshooting Cisco Emergency Responder
These topics address problems you might encounter with Cisco Emergency Responder (Cisco ER), and provide ways to resolve them; also included are other tasks associated with problem identification and resolution.
•
Troubleshooting Phone-Related Problems
•
•
•
•
•
•
•
•
•
•
•
•
Troubleshooting Phone-Related Problems
These topics help you troubleshoot problems related to assigning phones to ERLs and managing the phones:
•
•
•
Undiscovered Phones
If Cisco ER is not discovering the phones homing to Cisco Unified Communications Manager, check that all Cisco Unified Communications Managers are SNMP-reachable and that the SNMP settings are correct. Cisco ER will log an event if Cisco Unified Communications Manager is SNMP-unreachable.
To verify the Cisco Unified Communications Manager SNMP settings, follow these steps:
Procedure
Step 1
utils network ping <ipaddress of CUCM>Step 2
•
•
Start > Settings > Control Panel > Administrative Tools >
Services Properties > SNMP > Properties > Security TabStep 3
utils snmp get <ccm ip-address/host name> <snmp-read-community-string> .1.3.6.1.2.1.1.2.0If the Cisco Unified CM is SNMP reachable, then the output of the above command should be similar to the following:
Variable = .1.3.6.1.2.1.1.2.0
value = OBJECT IDENTIFIER <sys-oid-of-ccm>Too Many Unlocated Phones
Cisco ER obtains a list of registered phones from Cisco Unified Communications Manager and tries to locate all phones. If Cisco ER cannot locate a phone behind a switch port or in any configured IP subnets, and the phone is not a configured synthetic phone, the phone will be placed in the list of unlocated phones.
If there are a lot of unlocated phones, first try running the switch port and phone update process to see if Cisco ER can resolve some of the problems automatically. See the "Manually Running the Switch-Port and Phone Update Process" section on page 5-45 for more information.
These are some things that can prevent Cisco ER from locating a phone:
•
•
•
•
•
–
–
–
Unreachable switches are not retried until Cisco ER runs the next full switch-port and phone update process, unless you run it against the individual switch (see below).
•
•
After fixing the problems that are preventing Cisco ER from locating phones, run the switch-port and phone update process on the affected switches, or on all switches:
•
•
Related Topics
•
•
Phone Sometimes Disappears in Cisco Emergency Responder
If Cisco ER is in the middle of a phone tracking process, and a phone is in the middle of homing to a different Cisco Unified Communications Manager cluster, no Cisco Unified Communications Manager cluster has a record of the phone. Thus, Cisco ER does not know the phone exists, and you will not be able to look up the phone in the Cisco ER interface. However, assuming the phone successfully connects to a Cisco Unified Communications Manager cluster, Cisco ER tracks the phone during the next incremental phone tracking process, and the phone should then appear in the Cisco ER interface.
This problem can also occur if phones are reconnecting to a primary Cisco Unified Communications Manager server from a backup server during the Cisco ER phone tracking process.
Wrong ERL is Used for a Shared Line
When two or more phones with a shared line appearance move from switches that are monitored by one Cisco ER group to switches that are monitored by a different Cisco ER group, then Cisco ER may assign an incorrect ERL to these phones during an emergency call. This can occur when the phones move to a different campus that has a different Cisco Unified Communications Manager cluster (although the moved phones are still registered with the original Cisco Unified Communications Manager cluster), and it can also occur when the phones move within a single large campus that is served by multiple Cisco Unified Communications Manager clusters.
Because the moved phones are still registered to their original Cisco Unified Communications Manager cluster, emergency calls from these phones are routed to the original Cisco ER group. In this case, the Cisco ER group detects that the calling phone is connected to a switch that is monitored by a different Cisco ER group, and the call is forwarded to the appropriate Cisco ER group through an H.323 inter-cluster trunk. Because the inter-cluster trunk does not pass the MAC address of the calling phone, the receiving Cisco ER group does not know the MAC address of the calling phone and must associate the phone to an ERL based on the calling party number.
In cases with a single phone connected to the switches monitored by the receiving Cisco ER group, this is not a problem. However, when multiple phones with a shared line appearance connect to switches monitored by the receiving Cisco ER group, then Cisco ER must guess which phone has placed the emergency call. If all of the phones with a shared line appearance are in the same ERL, the guess is correct. If the phones span multiple ERLs, then the guess might be incorrect.
Related Topics
•
•
802.11b Endpoints Using Wrong ERL
802.11b endpoints (such as Cisco Wireless IP 7920 Phones and Cisco IP SoftPhones running on 802.11b) are using switch port-based ERL instead of the configured subnet-based ERL.
Cisco Emergency Responder (Cisco ER) give a higher priority to switch port association for call routing. If Cisco ER finds a switch port mapping for any endpoint (including 802.11b endpoints), it uses the switch port mapping to route emergency calls. If the switch port mapping is not found or if the ERL is not configured for the corresponding switch port, Cisco ER 1.2 routes emergency calls using subnet-ERL configuration.
Be aware that Cisco ER 2.0 will locate 802.11b endpoints behind a switch port under the following conditions:
•
•
See the switch port screen or the ERL debug tool (see Using The ERL Debug Tool to Verify Cisco Emergency Responder Configuration) to check if the 802.11b endpoint is associated with a switch port.
It is recommended that you track 802.11b endpoints using subnet-based ERLs. Therefore, enable CDP on the switch port and the access points to route emergency calls from 802.11b endpoints using subnet-based ERLS.
Related Topics
•
Troubleshooting Emergency Call Problems
These topics help you troubleshoot problems related to the routing of emergency calls and the information supplied with the calls:
•
•
•
•
•
•
•
•
•
Emergency Calls are Not Being Intercepted by Cisco Emergency Responder
If Cisco ER is not intercepting emergency calls, there is probably a mistake in your Cisco Unified Communications Manager configuration or its representation in the Cisco ER configuration. Check these items (based on the names used in the examples in "Configuring Cisco Unified Communications Manager 5.0, 5.1, and 6.0 for Cisco Emergency Responder 2.0").
•
•
•
•
•
•
•
1.
2.
3.
ELIN not Transmitted to the PSAP
If the ELIN is not transmitted to the PSAP, and you are using a PRI connection to route emergency calls to the PSAP, check the configuration of the gateway. The PRI must be configured to send the real calling party number (which will be the ELIN) rather than a static number, such as the main site number. See the "Obtain CAMA or PRI Trunks to the PSTN" section.
ELIN For Default ERL Used For Calls From Other ERLs
If an emergency call is assigned an ELIN defined for the Default ERL rather than an ELIN assigned to the ERL whence the call was made:
•
•
If the route pattern for an ERL fails, Cisco ER uses the route pattern defined for the Default ERL.
Emergency Calls Not Routed to the Correct PSAP
If an emergency call is not routed to any PSAP, check whether the route patterns used for the ERL from which the call was made and for the default ERL are configured and use the correct partitions and calling search spaces (see the "Creating the Route Patterns for ELINs" section). Ensure that the partitions and calling search spaces for the gateways are correct (see the "Configuring the Calling Search Space for the Gateways Used to Connect to the PSAP" section).
If an emergency call successfully leaves your network but does not get routed to the correct PSAP, look at these possible points of failure:
•
•
See these topics:
–
–
•
•
–
–
–
Note
Emergency Callers Sometimes Get Busy Signal and Emergency Calls Are Sometimes Not Routed
If callers hear a busy signal when calling the emergency call number, or if emergency calls sometimes do not get routed, there is probably a problem with the configuration of your standby Cisco ER server:
•
•
PSAP Call Back Errors
You might encounter these problems if a PSAP operator tries to call back an emergency caller using the ELIN provided by caller ID:
Symptom PSAP could not reach the original emergency call extension.
Recommended Action Cisco ER caches a mapping between the caller's true extension and the ELIN you define for an ERL. If more calls get made than the number of ELINs you define for an ERL, Cisco ER must reuse these numbers and thus overwrites the original caller's extension. You can view the call history to determine the extension of the original caller. See the "What Happens When an Emergency Call Is Made" section.
If this is not the problem, check the configuration of the PSAP callback route point in Cisco Unified Communications Manager and Cisco ER (see the "Creating the Emergency Call Route Points" section and the "Configuring Group Telephony Settings For the Cisco Emergency Responder Server" section on page 5-18), and the ELIN translation patterns in Cisco Unified Communications Manager (see the "Creating the Translation Patterns for ELINs" section).
Symptom Onsite alert (security) personnel get callbacks from the PSAP.
Recommended Action Cisco ER routes PSAP callbacks to the onsite alert personnel for the default ERL if ELIN-to-extension mapping for the emergency call has expired from the cache. By default, this is three hours, although you can configure expiration to be a longer or shorter time. See the "Cisco ER Group Settings" section on page A-3.
Onsite Alert Personnel Are Not Getting Telephone Alerts
If the onsite alert personnel are not getting telephone alerts when an emergency call is made in an ERL they are covering, ensure that all phones and CTI ports (both device and line) are in the Phones partition and use the PhoneCSS calling search space. You can use additional partitions, but they must be set up with relationship to the Cisco ER partitions and calling search spaces in the same manner as these partitions in the examples described in the "Setting Up Cisco Emergency Responder to Handle Emergency Calls" section.
Also, ensure that the Cisco ER configuration for the Cisco Unified Communications Manager clusters is correct. The Cisco ER configuration should show the correct begin address for the telephony ports you defined as CTI ports in Cisco Unified Communications Manager, and the number of telephony ports should be the correct number and it must be greater than 0 for any calls to occur. Cisco ER uses this CTI ports to place the telephone calls to onsite alert personnel.
If the Event Viewer in the Cisco ER Serviceability web interface displays the error message "No port to place call," then there were not enough CTI ports defined to initiate all the calls to onsite alert personnel. Therefore, you must define additional ports. To access the Event Viewer, log in to the Cisco ER Serviceability web interface and select Tools > Event Viewer.
Onsite Alert Phone Does Not Ring When Emergency Call is Placed
You might encounter this problem if the onsite alert phone does not ring when an emergency call is placed:
Symptom The onsite alert phone does not ring when an emergency call is placed.
Possible Cause The onsite alert phone will not ring if the Do Not Disturb (DND) feature is enabled on the phone and if Cisco ER is configured with Cisco Unified Communications Manager 6.0.
Recommended Action Do not enable DND on an onsite alert phone.
Prompts for Phone Alerts Not Getting Played
You might encounter this problem if prompts for phone alerts are not getting played:
Symptom Prompts do not get played at the onsite alert phone when the call is initiated from the CTI ports.
Explanation This problem can occur when a single CTI port is configured with multiple lines. Prompts may not get played from one or more of these lines when the onsite alert notifications call is initiated through them.
Recommended Action To avoid this problem, configure only one line per CTI port in the Cisco Unified Communications Manager that is configured for Cisco ER.
Onsite Alert Personnel Not Getting Email (or Paging) Notifications
If the onsite alert personnel are not getting email, or email-based pages, even though you configure email addresses for them (see the "Onsite Alert Settings" section on page A-10), check the Cisco ER configurations SMTP settings. Ensure that the SMTP server address and source mail ID are correct (see the "Cisco ER Group Settings" section on page A-3), and that there is an account for the mail ID in the SMTP server.
Incorrect Location Information Sent To Onsite Alert Personnel
If your onsite alert (security) personnel are receiving incorrect location information for an emergency call, consider these potential problems:
•
•
•
–
–
•
–
–
–
•
Emergency Call History Problems
These are some issues you might encounter when viewing the emergency call history information (see the "Viewing the Emergency Call History" section on page 5-57):
Symptom Emergency call information does not show up in call history right away.
Recommended Action Cisco ER writes call history information to the database every 15 seconds. You should be able to view history information after 15 seconds.
Symptom The call history does not show the ELIN and route pattern used for a call.
Recommended Action If the call could not be routed to the PSAP, you will not see an ELIN or route pattern. Check to determine why the call could not be routed. See the "Emergency Calls Not Routed to the Correct PSAP" section.
Troubleshooting Email Alerts
These topics help you troubleshoot problems related to the email alerts that Cisco ER generates:
•
•
•
•
•
Emergency Call Alert
Whenever a user makes a 911(Emergency) call, Cisco ER generates an email alert. Cisco ER sends the email alert to all of the onsite alert (security) personnel whose email ids are configured for the ERL from which the call was made. (See the "Configuring a Cisco Emergency Responder Server Group" section on page 5-16.)
Security personnel are expected to respond to that user. For detailed call information, refer to the following URL:
http://<<CERServer HostName>>/ceruserreports
When a 911 call is made and the backup Cisco ER server handles the call, an alert similar to the following is sent:
Subject: Emergency Call Alert -- Extn # 332101 (Generated by Backup Cisco ER)Message: EMERGENCY CALL DETAILS (Generated by Cisco ER)Caller Extension:332101Zone/ERL :Z1Location :dddCall Time :June 2, 2003 3:47:30 PM ISTTransition Alert
When the standby Cisco ER server takes control and becomes the active server, a Transition Alert is sent to the Cisco ER administrator. This situation occurs under any of the following circumstances:
•
•
•
The administrator should diagnose the cause and fix the problem as soon as possible.
When the Cisco ER backup server takes control, an alert similar to the following is sent:
Subject: Transition Alert: Cisco ER Backup is activeMessage:Backup Cisco ER <<CERServer HostName>> has taken control as Active Cisco ER.Transition Time :June 2, 2003 3:57:12 PM ISTWhen the master Cisco ER server takes control, an alert similar to the following is sent:
Subject: Transition Alert: Cisco ER Master is activeMessage:Master Cisco ER <<CERServer HostName>> has taken control as Active Cisco ER.Transition Time :June 2, 2003 3:57:12 PM ISTTracking Failure
At the end of a switch-port and phone tracking process, if there are any devices that could not be tracked, Cisco ER sends a Tracking Failure email to the Cisco ER administrator.
The administrator should look at the event log on the Cisco ER server to find the list of devices that were not tracked. Then the administrator should check the following and make any required corrections:
1.
2.
3.
4.
Here is an example of a tracking failure alert.
Subject: CER Phone Tracking failed to track some devicesMessage:CER Phone Tracking could not get information [using SNMP] from 2 Cisco CallManager(s) and 1 Switch(es)Check Event Viewer on CER Server for details.Failed To Get Provider
Cisco ER sends a Failed to Get Provider Alert to the Cisco ER administrator if Cisco ER is not able register to one of the configured Cisco Unified Communications Manager clusters. Cisco ER will continue trying registration until it succeeds. Cisco ER sends the Failed to Get Provider email after a few retries.
The message provides information on how to clear the problem, as shown in the following example.
Subject: Failed to get JTAPI Provider for Cisco CallManager <<CCM IP/Host Name>> (Generated by Backup CiscoER)Message:Please check the following:1) Check if the Cisco CallManager is connected to the CER server.2) Check if the configured Call Manager is running a version supported by the CER server.3) Check if the given login credentials are correct:CTI Manager Host Name:<<CCM IP/HostName>>Failed to Establish Communication with Cisco Emergency Responder Phone Tracking Engine
Cisco ER sends this email alert to the Cisco ER administrator if the Cisco ER server fails to establish communication with the Phone Tracking Engine for some time. This can occur if the Cisco ER Phone Tracking Engine service is down. The administrator should perform the following steps:
1.
2.
Here is an example of a tracking failure alert.
Subject: CER Server failed to establish communication with CER Phone Tracking Engine.Message:CER Server could not communicate with CER Phone Tracking Engine.Lost Communication with Cisco Emergency Responder Phone Tracking Engine
Cisco ER sends this email alert to the Cisco ER administrator if the Cisco ER server loses communication with the CER Phone Tracking Engine. This is most liked to occur if the Cisco ER Phone Tracking Engine service goes down when the Cisco ER server is running.
The administrator should restart the Cisco ER Phone Tracking Engine service.
The following shows an example of a tracking failure alert.
Subject: CER Server lost communication with CER Phone Tracking EngineMessage:CER Server could not communicate with CER Phone Tracking Engine.Failed to Send Unlocated Phone Details to Remote Cisco Emergency Responder Server Group
If Cisco ER fails to send unlocated entries to a server group because it is already in the process of sending entries to that server group, this alert is sent.
This alert will occur very rarely. It can occur when a Cisco ER server is found in more than one Cisco ER server group. To resolve this problem, check to see which server group is an old configuration and remove that server group.
Subject: CER Server failed to send Unlocated Phones details to Remote CER Server Group.Message:CERServer failed to send Unlocated Phones to Remote CER Server Group. Please ensure that the CER servers are not found under more than one CER Server Group.CER Servers in Remote Server Group:<< CERServer HostNames >>Emergency Call Could Not be Routed
If the emergency call routing to some route patterns configured in the ERL fails, Cisco ER sends an email to the system administrator.
Subject: Emergency call could not be routed using some route patterns (CERServer:<server hostname>)
Message Body: Emergency call from :<Caller Extn> could not be routed using some Route Patterns. Check Event Log.
The Event Log displays the following message:
Emergency call from <extn> could not be routed using the following route patterns<RoutePattern1><RoutePattern2>*****************Call Routed to <RoutePattern-X>Please check the availability of the above routes. Also, check for the following error conditions:1. If FAC and/or CMC are configured on the route patterns used for Cisco ER, please disable them.2. If the "Calling Party Number Modification" flag on the CER user page in the CallManager is not enabled, please enable it.Solution
1.
2.
3.
Calling Party Modification Failed
If the calling party modification was not successful, Cisco ER sends the following email to the system administrator:
Subject: Emergency Calling Party Modification Failed (CERServer: <server>)
Message Body: Emergency call from :<Caller Extn> cannot be routed with calling party modification. Check Event Log.
The Event Log displays the following message:
Emergency Call from <Caller Extn> has been routed to default ERL because the calling party modification failed.Please make sure that the checkbox "Enable Calling Party Number Modification: is checked on the Cisco CallManager user page for the CER user. PSAP callbacks MAY NOT work correctly. The CER service will need to be restarted once the flag is checked on the CallManager User page.Solution
Troubleshooting Web Alerts
You might encounter this problem when receiving web alerts:
Symptom Web alert continues to refresh every 30 seconds. You can see this problem by checking the status in the browser. The status will display the seconds remaining before refresh if it is in this mode.
Recommended Action Check if there are other web alert screens open on the same client machine. Only one browser from a client machine can operate in the real-time mode. Remove any extra browsers.
Troubleshooting Cisco Emergency Responder System and Administration Problems
These topics help you troubleshoot problems related to the Cisco ER system and its administration, such as server and web server problems:
•
•
•
•
•
•
•
•
•
Cannot Validate Publisher
If the installation cannot validate the Publisher (Step 5 of the "Installing the Cisco Emergency Responder Subscriber" section), check the following:
1.
2.
3.
4.
Troubleshooting Login Problems
These are some issues you might encounter while logging into Cisco ER:
Symptom You cannot log in to the Cisco ER Administration website.
Recommended Action Log in to CLI and run the utils service list command. Check if the status "Cisco IDS" is STARTED. If not, start the service using the utils service start service name command.
Symptom You cannot open multiple Cisco ER sessions using Netscape Navigator.
Recommended Action Netscape/Mozilla Navigator uses the same session ID across multiple windows. This creates problems if you try to log into Cisco ER using different IDs. Normally, you can open multiple windows when logged in as system administrator. With Internet Explorer, if you open separate IE session by starting a new IE instance (rather than by opening a new window from an existing session), IE uses different session IDs, and you should be able to log in using separate IDs (for example, as a user and an administrator, or as LAN switch and ERL administrators).
Related Topics
•
Using Cisco Unified Operations Manager
Use Cisco Unified Operations Manager 2.01 to continuously monitor the health of the Cisco ER system.
For information on setting up Cisco ER to use Cisco Unified Operations Manager, see the "Configuring Test ERLs" section on page 5-34.
For information on installing and using Cisco Unified Operations Manager, see the documentation at:
http://www.cisco.com/en/US/products/sw/cscowork/index.html
Troubleshooting Cisco Emergency Responder Switch and Port Configuration Problems
Some issues that you might encounter while configuring switches or switch ports in Cisco ER are shown below:
Symptom Cisco ER is configured with Cisco Unified Communications Manager information, but no phones get discovered.
Recommended Action Ensure that the Cisco Unified Communications Manager servers are reachable on the network. Then, ensure that the SNMP read community strings are configured correctly for the switches and Cisco Unified Communications Manager servers (see the "Configuring the SNMP Connection" section on page 5-39.) Then, manually run the switch port and phone update process (see the "Manually Running the Switch-Port and Phone Update Process" section on page 5-45.) Use the CLI-based utils snmp command to determine if the Cisco Unified Communications Manager is SNMP reachable.
Symptom Cisco ER does not show the ports on a switch configured in Cisco ER.
Recommended Action If you add a supported switch to Cisco ER and run phone tracking on the switch after adding it, you should be able to view the list of Ethernet ports on the switch. If Cisco ER does not list the ports, check the SNMP settings in Cisco ER for the switch (see the "Configuring the SNMP Connection" section on page 5-39.) Also, verify that the switch is reachable over the network. Retry the selective phone tracking process on the switch (click Locate Switch Ports when viewing the switch details; see the "LAN Switch Details" section on page A-28.)
If the problem persists, ensure that the switch is supported (see the "Network Hardware and Software Requirements" section.) Also, check the Event Viewer for error messages.
Symptom Some phones do not appear in the switch port list.
Recommended Action Check if the phone is found under configured IP subnets or in synthetic phones. If it is not found in either of those places, then they will be place un unlocated phones. See the "Too Many Unlocated Phones" section for a list of reasons that a phone could not be located.
Symptom Cannot delete a switch from the Cisco ER configuration.
Recommended Action You cannot delete a switch when a phone tracking process is in progress. Retry the deletion after the process has ended. If this is not the problem, the Cisco ER server might not be running. Check the control center and restart the server (see the "Starting and Stopping a Cisco Emergency Responder Server" section.)
Symptom Import or export of the switch port details fails.
Recommended Action If a switch port import or export attempt fails, it might be due to these reasons: the first switch-port and phone update process has not yet ended (wait for it to finish); the Cisco ER server is not running (use the control center to restart it, see the "Starting and Stopping a Cisco Emergency Responder Server" section); the Cisco ER server is not completely initialized (wait for it to initialize).
Symptom The import of some switch port configurations fail.
Recommended Action To import switch port configurations, Cisco ER must already be configured with the switch and Cisco ER must first discover the ports on the switch using the switch-port and phone update process. If you try to import a configuration for ports not yet discovered in Cisco ER, the importation of those settings fails. See the "Manually Running the Switch-Port and Phone Update Process" section on page 5-45 for information on the process. Run it on the switches whose port configurations you could not import, then retry the import.
Symptom Phones moved from other Cisco ER groups to this Cisco ER group, and then moved back, are still showing up in the switch port details for the Cisco ER group.
Recommended Action This types of phones are not removed from the switch port details until the next full switch-port and phone update process is run. If this is an issue for you, you can run the process on the switch (or on all switches) manually. See the "Manually Running the Switch-Port and Phone Update Process" section on page 5-45.
Using The ERL Debug Tool to Verify Cisco Emergency Responder Configuration
The ERL Debug Tool takes a phone extension as the search criteria and displays the ERL(s) currently being used for routing emergency calls for the phone(s).
Use this diagnostic tool to verify the Cisco ER configuration during the ERL creation and the ERL assignment phase, and to troubleshoot calls directed to incorrect ERLs.
For example, you configured the phone in ERL_1 as a manually configured phone, however a misconfigured IP subnet matches this phone's IP address, and associates it with ERL_2. Now that you have found the configuration problem using the Debug Tool, you can correct it.
To use the ERL Debug Tool, follow these steps.
Procedure
Step 1
Cisco ER displays the ERL Debug Tool page.
Step 2
Cisco ER displays the ERL currently being used for routing emergency calls for the phone.
Step 3
Note
Replacing the Publisher Server and Subscriber Servers
If you need to replace a faulty Publisher server or a faulty Subscriber server, perform the appropriate procedure:
•
•
Replacing a Faulty Subscriber
To replace a faulty Subscriber, go to Cisco ER administration and delete the faulty Subscriber. Install a new Cisco ER Subscriber for the Publisher (see the "Installing Cisco Emergency Responder 2.0 on a New System" section).
Note
Replacing a Faulty Publisher
You can restore the Publisher only if you have backed up the Publisher using the Disaster Recovery System available as part of the Cisco ER. See the "Backing Up and Recovering Data" section.
To replace a faulty Publisher, follow these steps:
Procedure
Step 1
Step 2
Step 3
Using the Cisco Emergency Responder Admin Utility
You can use the Cisco ER Admin Utility tool to perform the following tasks:
•
•
•
This section describes the following topic:
•
How to Use the Cisco ER Admin Utility Tool
To use the Cisco ER Admin Utility tool, follow these steps:
Procedure
Step 1
Step 2
a.
b.
c.
Note
Step 3
Troubleshooting the Subscriber Database Setup
To configure the Publisher-Subscriber setup again if you have an issue with the Subscriber (apart from DB replication), follow these steps:
Procedure
Step 1
Step 2
Step 3
Step 4
This step might a take a while setup.
Troubleshooting the Database and Enterprise Replication
Use the following CLI commands for troubleshooting the Informix Dynamic Server (IDS) database:
•
•
•
•
•
Use the following CLI commands for troubleshooting Enterprise Replication:
•
•
•
For troubleshooting database problems using logs, download logs from the Cisco ER Serviceability website or through CLI.
The following logs provide information for debugging database related issues
•
•
•
•
Symptom Replication fails to start after the Subscriber is installed with DNS and the CLI command utils dbreplication status shows replication not working.
Possible Cause The .rhosts will have the Host Name for the Subscriber instead of FQDN (Fully Qualified Domain Name) of the Subscriber.
Recommended Action Use the CLI command utils dbreplication repair to repair the replication issue. This command tries to repair replication by rebuilding the corrupted .rhosts file.
Troubleshooting Cisco Emergency Responder System Problems
These are some issues you might encounter with general operation of the Cisco ER system and the configuration screens that involve the Cisco ER server, group, and cluster:
Symptom Cisco ER intra-cluster call routing fails or Cisco ER does not discover phones correctly.
Recommended Action Ensure that all the Cisco ER servers in a Cisco ER cluster can be found by their host name, and ensure that all are reachable on the network by all the other Cisco ER servers.
Recommended Action Ensure that all the Cisco ER servers can reach the Cisco ER cluster DB host and that the cluster DB password is the same across all servers in the cluster.
Symptom Cisco ER exits after starting.
Possible Cause You have configured Cisco ER to use a TCP port that is already in use.
Recommended Action Check the Windows Event Viewer for the message "Cisco ER could not open socket at port peer-tcp-port, Exiting." If you see this message, change the Cisco ER group configuration to use a different TCP port. See the "Configuring a Cisco Emergency Responder Server Group" section on page 5-16 for instructions.
Symptom The Cisco ER Groups in Cluster screen does not load, and exhibits the error "Cannot connect to cluster DB host."
Recommended Action Ensure that the cluster DB host can be found by host name.
Ensure that the specified cluster db host password is the same across all Cisco ER server groups in the cluster.
For more information, see the "Configuring the Cisco Emergency Responder Cluster and Cluster DB Host" section on page 5-23.
Related Topics
•
•
•
Troubleshooting Cisco Unified Communications Manager Configuration Problems
These are some issues that you might encounter with Cisco ER's communications with Cisco Unified Communications Manager. Additional problems with symptoms that involve emergency call failures are discussed in the "Troubleshooting Emergency Call Problems" section.
Symptom Cisco ER does not register with the route points and CTI ports configured for its use.
Recommended Action Ensure that the route points and CTI ports are associated with the Cisco Unified Communications Manager Cisco ER user (see the "Creating a Cisco Emergency Responder Cisco Unified Communications Manager User" section.) Ensure that the CTI Manager on the Cisco Unified Communications Manager server (or the DC Directory on a Windows-based Cisco Unified CallManager server) is running properly.
Symptom When trying to delete a Cisco Unified Communications Manager from the Cisco ER configuration, Cisco ER prevents me and displays the message "Phone tracking in progress."
Recommended Action You cannot delete a Cisco Unified Communications Manager server from the Cisco ER configuration while a phone tracking process is in progress. Retry the deletion after the process has ended.
Updating Cisco Emergency Responder After You Add Devices
You must create a Cisco Unified Communications Manager user for Cisco ER's use and CTI ports and route points that need to be assigned to the user before Cisco ER tries to create a provider with the Cisco ER cluster. Cisco ER only registers the CTI ports and route points that are associated with the user when the provider is created. Thus, any devices you add to the user after starting Cisco ER will not be registered by Cisco ER.
If you add devices to the Cisco ER user in Cisco Unified Communications Manager, you can force Cisco ER to recreate the provider using any of these techniques:
•
•
•
•
Identifying the Cisco Emergency Responder Groups and Servers in a Cisco Emergency Responder Cluster
If you are connected to the administrator's interface on a Cisco ER server, you can view the details of the server and the Cisco ER group's standby server by selecting System > Cisco ER Group Settings.
You can also identify the Cisco ER groups and their Cisco ER servers that are in the same Cisco ER cluster. To view the other Cisco ER groups in the cluster, select System > Cisco ER Groups in Cluster. From the Cisco ER Groups in Cluster page, select the group you want to view; and Cisco ER displays the Cisco ER servers that are in the group. To view the details for these servers, you must log in t to the Cisco ER Administration interface running on one of the servers, select System > Cisco ER Groups in Cluster, then select the group you want to view from the list of groups.
If you need to uninstall a Cisco ER group, first delete the group from the Cisco ER cluster using this page. You must log in as a system administrator to delete the group. Deleting the group from the cluster simply removes the entries for the group from the Cisco ER Cluster DB; it does not remove Cisco ER from the group's servers.
Related Topics
•
Phones Moving Between Clusters
The following scenario illustrates how Cisco ER clusters work and how Cisco ER treats phones moving between clusters:
•
–
–
•
–
–
•
–
–
•
Starting and Stopping a Cisco Emergency Responder Server
When you install Cisco ER, the Cisco ER server is set up to automatically start whenever the computer is powered up or rebooted. However, you can stop and then restart a Cisco ER server through the Cisco ER Serviceability web interface without powering down or rebooting the computer. You might find this helpful if you are trying to debug a problem.
To start or stop a Cisco ER server, follow these steps:
Procedure
Step 1
The Control Center Services page displays, showing all Cisco ER services and the current status of each one.
Step 2
Note
Note
Table 11-1 explains the meaning of the icons you see on the Control Center Services page.
Related Topics
Troubleshooting ALI Data Uploads
Periodically, you must export your ALI data and submit it to your service provider. The ALI data is used to route emergency calls from your network to the correct PSAP, and provide the PSAP with information about the location of the emergency call.
Cisco ER lets you export the ALI data in a variety of NENA formats. Ask your service provider which format you should use.
During the upload process, you might find that some ALI data records did not upload correctly. Your service provider should be able to provide you with a list of errors, or you might see these when using your service provider's data upload software. You must fix any mistaken records and resubmit the ALI data export file. To fix the records, you might need to manually edit the records in error.
These sections describe the general procedure for fixing ALI data records, and explain how to edit the various types of NENA formatted files:
•
•
Fixing ALI Data Records
To correct data errors you might receive when uploading ALI records to your service provider, follow these steps:
Before You Begin
Obtain NENA Doc 02-010, Recommended Formats and Protocols for Data Exchange, from NENA or your service provider. This document explains the various NENA formats in detail.
Procedure
Step 1
Step 2
Step 3
Step 4
Open the ALI export file in a text editor and change the function code for the records that you are fixing. Use an editor that will not add formatting or other extra characters. See these sections for details about editing the files:
•
•
Step 5
Editing NENA 2.0 and 2.1 File Formats
The NENA 2.0 and 2.1 file formats have these characteristics:
•
•
•
•
Use NENA Doc 02-010, Recommended Formats and Protocols for Data Exchange, to determine the byte location and length of each field. When you edit the file, ensure that you are not lengthening the records. Delete any extra spaces that get added. If the length of an item is less than the length of a field, pad the field with blanks. Depending on the field, padding might be on the right or the left.
The file contains one header and one trailer record. The ALI data records are contained between these records.
Table 11-2 describes the fields you are most likely to edit. You should use the Cisco ER web interface to change the other fields.
Editing NENA 3.0 File Formats
The NENA 3.0 file format has these characteristics:
•
•
•
–
–
–
•
•
Use NENA Doc 02-010, Recommended Formats and Protocols for Data Exchange, to determine tag name and values for each field. Ensure that your values do not exceed the maximum length for the field. You do not need to pad fields with extra blanks.
The file contains one header and one trailer record. The ALI data records are contained between these records.
Table 11-3 describes the fields you are most likely to edit. You should use the Cisco ER web interface to change the other fields.
Collecting Call History Logs
Cisco ER maintains extensive call history logs, which include entries for each emergency call handled. You can view call history information from the administration and user interfaces.
Cisco ER maintains in its database a history of the emergency calls that have been placed. When the primary Cisco ER server (Publisher) is not active, emergency calls are handled by the backup Cisco ER server (Subscriber). Through replication, the call history records on both these servers are synchronized when they are active. For this reason, the call history can be viewed on either of the Cisco ER servers.
To download these records, click on the Download button at the top of the table displaying the call history. These records are downloadable in Excel (.xls) format.
Collecting Trace and Debug Information
When you contact Cisco Technical Support for help with a problem that you are having with Cisco ER, Cisco might request that you collect trace and debug information.
Because collecting trace and debug information will affect Cisco ER's performance, you should only turn on tracing and debugging at Cisco's request. The generated information is for Cisco's use in resolving product problems.
Use the following sections to learn about:
•
Enabling Detailed Trace and Debug Information for Cisco Emergency Responder
To enable detailed trace and debug information for Cisco ER, follow these steps:
Procedure
Step 1
Cisco ER opens the Server Settings page.
Step 2
Cisco ER displays the settings for the server.
Step 3
The lists in each section are identical; make sure that you select the package in the list that Cisco requested. Packages selected in the Debug list generate trace information plus extra debug data. If Cisco requests that you select all packages, click Select All for the appropriate list.
The available packages include:
•
•
•
•
•
•
•
•
•
•
Step 4
Cisco ER begins generating the requested trace and debug information.
Note
Step 5
Related Topics
•
•
•
Enabling Syslog
To collect trace and debug information, you must enable syslog for Cisco ER).
To enable syslog for Cisco ER, see the "Collecting Information from Syslog" section.
Viewing Event Messages
You can view Cisco ER event messages to help diagnose problems with the software by using the Cisco ER Serviceability web interface.
For information on viewing Cisco ER events, see the "Using the Event Viewer" section.
For details on the Find and List Events page, see the "Event Viewer" section on page B-2.
Managing Performance
Refer to the Release Notes for Cisco Emergency Responder 2.0 for supported Cisco MCS Unified Communications Manager Appliance platforms and their Cisco ER scalability.
Cisco ER performance can be affected if Cisco ER is managing switches across a WAN link. Cisco ER must send SNMP requests to the managed switches, and WAN delays can lead to SNMP timeouts and increase the time needed to track phone and switch changes. You might need to tune the SNMP parameters. See the "Configuring the SNMP Connection" section on page 5-39 for more information.
Integrating with Network Management Systems
You can manage the status of the Cisco ER server remotely using CiscoWorks2000 or another SNMP-based network management system. CiscoWorks2000 is the standard Cisco network management system, but it is not bundled with Cisco ER. For more information about CiscoWorks2000, Campus Manager, and Topology Services, refer to the documentation, available at the following URL:
http://www.cisco.com/en/US/products/sw/netmgtsw/tsd_products_support_category_home.html
These topics provide information to assist you in integrating Cisco ER with network management systems:
•
•
Understanding CDP Support
Cisco ER uses the Cisco Discovery Protocol (CDP) to periodically send out CDP messages, on the active interface, to a designated multicast address. These messages contain information such as device identification, interface name, system capabilities, SNMP agent address, and time-to-live. Any Cisco device with CDP support can locate a Cisco ER server by listening to these periodic messages.
Using information provided through CDP, the CiscoWorks2000 Server can detect the Cisco ER server, and the Campus Manager application, Topology Services, can build topology maps displaying the Cisco ER server.
In addition to sending out CDP messages, the Cisco ER server uses CDP to locate phones that support CDP. You must ensure CDP is enabled on your switches so that Cisco ER can obtain this information through SNMP queries to the switches.
Table 11-4 shows the SNMP OIDs for the Cisco ER hardware platforms.
Monitoring Cisco Emergency Responder Subsystem Status
Cisco ER supports the SYSAPPL-MIB that allows you to use CiscoWorks2000 or a third-party SNMP browser to remotely access information about the following Cisco ER components:
•
–
•
–
•
The SYSAPPL-MIB uses SNMP. Cisco ER supports the following SYSAPPL-MIB tables:
•
•
•
•
Collecting Information from Syslog
You can configure Cisco ER to use the Cisco Syslog Collector. Cisco Syslog Collector and Cisco Syslog Analyzer are offered with CiscoWorks2000 as part of the Resource Management Essentials package. You can also adapt Syslog output from Cisco ER for use with other network management systems.
The Cisco Syslog Collector keeps common system logs of messages reported to Cisco ER.
The Cisco Syslog Analyzer controls and displays all events efficiently so they can easily be read, interpreted, and used for system maintenance and problem solving.
To install and configure the Cisco Syslog Collector, refer to the CiscoWorks2000 documentation.
To enable syslog, follow these steps:
Procedure
Step 1
Cisco ER opens the Cisco ER Group Settings page.
Step 2
Step 3
Step 4
Cisco ER immediately begins writing messages to syslog.
Related Topics
•
Backing Up and Recovering Data
Cisco ER 2.0 uses the Disaster Recovery System to backup and restore system data.
For information on using the Disaster Recovery System, see "Configuring the Cisco Emergency Responder 2.0 Disaster Recovery System."
Related Topics
•
Troubleshooting the Data Migration Assistant
The Data Migration Assistant (DMA) operates in two phases. In the first phase, Database, the following folders are backed up to a tar file:
•
•
•
•
In the second phase, the contents of the backed-up Cisco ER database are verified against the Cisco ER 2.0 database schema.
Symptom DMA backup and validation failed.
Recommended Action Go through the following check list:
–
–
–
Symptom DMA backup is successful but the validation failed.
Recommended Action Go through the following check list:
–
–
The DMA Logs are located in the following locations:
•
•
•
The validation log files are as follows:
•
•
•
•
Troubleshooting Windows Upgrades
If you receive an error stating "DMA backup tar cannot be located" during an upgrade from a Windows version to Cisco ER 2.0, perform the following checks:
•
•
•
•
Troubleshooting Linux Upgrades
You might encounter certain problems when upgrading to future versions of Cisco ER from Cisco ER 2.0. This section explains what could cause these problems and the provides recommended actions.
Symptom On the first page of the Install / Upgrade menu, after you enter the details for an upgrade patch, the error message "No valid upgrade options found" appears.
Recommended Action Verify that you are not trying to upgrade the Subscriber before upgrading the Publisher. When upgrading a Cisco ER servergroup, you must always upgrade the Publisher first.
Recommended Action Verify that the local/remote path that you have specified actually contains a valid, signed ISO image, having the extension .sgn.iso.
Symptom On the first page of the Install / Upgrade menu, after you enter the details for an upgrade patch at a remote location, the error message "Incorrect user name/password" appears.
Recommended Action Verify that the username and password entered for the remote SFTP/FTP location are correct.
Symptom After downloading the ISO image onto the Cisco ER server, the checksum values do not match.
Recommended Action Download a fresh ISO image from Cisco.com and try the upgrade again.
Symptom The upgrade was cancelled, but a warning message appears prompting you to reboot the system.
Recommended Action During the upgrade, certain services on the Cisco ER server could have been stopped, depending on when the upgrade was cancelled. In this case, it is highly recommended that you reboot the server.
Posted: Fri Jun 8 00:23:45 PDT 2007
All contents are Copyright © 1992--2007 Cisco Systems, Inc. All rights reserved.
Important Notices and Privacy Statement.
