Release Notes for the Cisco 7600 Wireless Security Gateway, Release 5.0
Open Caveats Prior to WSG Release 5.0
WSG Release 5.0—Resolved Caveats
Resolved Caveats Prior to WSG Release 5.0
Obtaining Documentation and Submitting a Service Request
This document explains features, requirements, and caveats for the Cisco 7600 Wireless Security Gateway (WSG), Release 5.0.
The WSG is a high-density IPSec gateway for mobile wireless carrier networks. IP Security (IPSec) is an open standards set. IPSec gives confidentiality, integrity, and authentication for data between IP layer peers. The WSG uses an IPSec-protected tunnel to connect outside endpoints.
The WSG runs on the Cisco Service and Application Module for IP (SAMI), a new-generation high performance service module for the Cisco 7600 series router platforms.
For more information about the WSG, see the Cisco 7600 Wireless Security Gateway Configuration Guide, Release 5.0. For more information about the Cisco SAMI, see the Cisco Service and Application Module for IP User Guide.
WSG Release 5.0 supports the following features:
For hardware requirements, such as power supply and environmental requirements, as well as hardware installation instructions, see the Cisco Service and Application Module for IP User Guide.
WSG Release 5.0 software application ships preloaded on the Cisco SAMI processors with fully-functional default settings. During an image upgrade, the image is automatically loaded onto each SAMI processor.
WSG Release 5.0 requires the following hardware and software:
Cisco Supervisor Engine 32 with a Multilayer Switch Feature Card 2A (MSFC2A) or Policy Feature Card 3 (PFC3B) running Cisco IOS Release 12.2(33)SRC or later.
The Cisco Supervisor Engine 32 requires LCP ROMMON version 12.2[121] or later on the
Cisco SAMI.
For details on upgrading the Cisco IOS release running on the SUP, refer to the “Upgrading to a New Software Release” section in the Release Notes for Cisco IOS Release 12.2SR.
For details on verifying and upgrading the LCP ROMMON image on the Cisco SAMI, refer to the Cisco Service and Application Module for IP User Guide.
To find out which IOS version your system is running, log in to the SUP and enter the show version command.
To find out which WSG version your system is running, establish a session with a PPC, and enter the show version command.
Caveats are unexpected behavior in Cisco software releases. Severity 1 caveats are the most serious caveats, Severity 2 caveats are less serious, and Severity 3 caveats are moderate. Typically, only Severity 1, Severity 2, and select Severity 3 caveats are included in the caveats lists.
If you have an account with Cisco.com, you can use the Bug Navigator II tool to find the current list of caveats of any severity for any software release. To access the Bug Navigator II tool, go to http://www.cisco.com/support/bugtools.
The following caveat is unresolved in this release:
Description: Cannot disable Old CRL file feature.
This condition occurs only with active tunnels, in other words, when the existing tunnel is using CRL file.
Workaround: Clear existing tunnel before un-configuring the crypto pki enable old-crl-file CLI command.
The following caveats are unresolved in this release:
Description: PPC4 alone will be missing when a snmpwalk is done for “hrProcessorLoad” on WSG.
This issue is specific to Active-Active scenario wherein the primary Active PPC4 SAMI is reloaded, and once it comes back after the SYNC is successful between the Active PPC4 and Standby PPC4, the issue is seen with the standby PPC4.
This condition occurs on WSG with Active-Active scenario with below mentioned conditions:
Now, PPC4 will not report to SNMP polling for “hrProcessorLoad”.
Workaround: Reloading only the affected WSG.
Description: The gateway address should be verified as routable before being accepted. If it fails then an error code should be returned.
Description: Following configuration issues observed in crypto ipsec-fragmentation configuration.
– User unable to know the default path MTU since both following configuration are consider as default but not in running configuration, though the configuration successful.
– After crypto ipsec-fragmentation none configured and saved to start configuration, then after sami reload, then new tunnel established, the path MTU is 1400 instead 0.
– Path MTU is changed in CLI output but function does work for existing tunnel.
Description: WSG doesn't provide ipv6 interface and address information via SNMP get or snmpwalk. This is seen with ipv6 interface and address information.
Description: This issue may be observed when the above mentioned CLI is executed on the WSG instance. Upon execution:
– It allows sending the traffic up to size 1300.
– Traffic gets dropped and no error is reported when the traffic size is between 1300 and 1400.
– “Destination un-available (Fragmentation needed) error is reported when the traffic size exceeds 1400
Workaround: It is possible to mitigate this issue by reconfiguration.
Description: This issue may be observed when WSG is in the idle state and no profile is selected. In some scenarios following trace is observed:
[7ffc9d00] [40005cc0] show_stack+0x58/0x198 (unreliable)
[7ffc9d50] [40287488] yield+0x54/0x68
[7ffc9d60] [401abe2c] netlink_broadcast+0x32c/0x35c
[7ffc9db0] [401ac45c] nlmsg_notify+0x60/0xa0
The following caveats are resolved in WSG Release 5.0:
Description: Certificate from the cache is not removed when it is un-configured using the CLI command no crypto pki wsg-cert
This condition occurred only in site-to-site (S2S) deployment when 2 or more profiles were activated.
Description: If redundancy failed for any reason, the show command show ha info showed different IP range rather than the IP configured under HA interface.
This condition occurred on redundancy failure scenarios.
Description: The WSG deleted the older tunnel that was created by an IKE-PEER if that same PEER sends additional tunnel request with INITIAL-CONTACT set.
Description: "SA import failed, out of memory" error message was seen on the standby card and IPSEC SA didn’t sync from ACTIVE to STANDBY card.
Description: In site-to-site scenarios, WSG didn't use PPC 5-8. The HM messages on un-used PPCs had to be disabled.
Description: QNX io-net process crashes and SAMI reloads.
This condition occurs when io-net core is present in LCP core directory (which can be checked after the SAMI is brought up).
Description: QNX qconn process crashed resulting in SAMI card reload.
The qconn core could be seen in the LCP core directory after the SAMI comes up.
Description: DHCP relay function was not working on SAMI card.
This condition occurred when VRF was configured on the WSG.
Description: In rare instances, SAMI card reloaded unexpectedly.
This condition could be seen in LCP core directory. A file by the name ixp#.txt contains the QNX kernel core dump.
The following caveats were resolved in WSG Release 4.4.6:
Description: Cannot use more than 10 service VLAN interfaces - loopback interfaces. There was no such limitation in WSG 3.x. While upgrading to WSG 4.x, the VLANs exceeding 10 service VLAN interfaces are placed in shutdown state after configuration migration. The exceeding VLANs were not saved in startup-config.
The following error is seen if "no shutdown" the service VLAN interface is attempted:
This condition was seen while upgrading from WSG 3.x to 4.x and, when more than 10 VLAN interfaces with loopback /32 ip addresses are used before migration.
Description: After removing an interface with no service interface vlan <x>, all the subsequent interfaces are also deleted. However, the IP addresses of the deleted interfaces responds to the ping and the VPNs using them as source works fine.
The following caveat was resolved in WSG Release 4.4.5:
Description: All sub-commands under service interface are lost after reload.
This condition occurs if VRF/MTU configuration exists under service interface vlan.
The following caveats were resolved in WSG Release 4.4.4:
Description: SAMI card reset occurs after command execution.
This condition occurs when at the SAMI LCP CLI prompt, the command show logging internal facility is executed.
Description: WSG encodes v1/v2 Trap OID values incorrectly.
All Enterprise trap OIDs were affected by this issue.
The following caveats were resolved in WSG Release 4.4.3:
Description: On January 27, 2015, a buffer overflow vulnerability in the GNU C library (glibc) was publicly announced. This vulnerability is related to the various gethostbyname functions included in glibc and affect applications that call these functions. This vulnerability may allow an attacker to obtain sensitive information from an exploited system or, in some instances, perform remote code execution with the privileges of the application being exploited. This vulnerability is documented in CVE-2015-0235.
A Cisco Security Advisory has been published to document this vulnerability at:
http://tools.cisco.com/security/center/content/CiscoSecurityAdvisory/cisco-sa-20150128-ghost
Description: When EnB requests for PSK profile, WSG picks up RSA profile instead of PSK leading to failure in tunnel negotiation.
Description: Core Dump File will have Proc Memory Info details.
LCP Sysmgr had crashed due to Proc Memory Info corruption.
The following caveat was resolved in WSG Release 4.4.2:
Description: WSG CPU on SAMI gets overwhelmed with abrupt terminations of script, and thereby hogging up the CPU.
This issue is seen on WSG running 4.4.
The following caveat was resolved in WSG Release 4.4.1:
Description: Feature support to Configure the Facility level parameter in the syslog messages.
By default, facility will be set as per the process.
Example: The syslogs related to IPSec will be set to the facility level 4 since all of them are related to the Authentication/Security.
The following caveats were resolved in WSG Release 4.4:
Description: WSG data path might crash intermittently. This condition might occur with IPv6 over IPv4 traffic.
Description: The CMP command UPDATE and ENROLL will fail if HTTP is used as transport Protocol.
This condition occurs when CMPv2 is executed from Exec mode.
Description: WSG resets the ESP sequence number when AP changes it's Port Number.
This issue caused the WSG to put wrong sequence number in the ESP packets and the AP to drop the packets, since the ESP sequence number did not fall in the Anti-replay window.
Description: Authentication failure is seen when sha2 is used as a signature algorithm.
This condition occurs when signature algorithms like sha256/512/384 is used while signing the cert.
Description: IXP mecore files (qnx_1_mecore_ucdump) are not present in crashinfo_collection.
Description: Card might crash and reload itself with HM Datapath failure event.
The crash info file will be collected at LCP core: directory
Different packet type takes different processing time in the Lookup leading to packet reaching late to Resequencer ME. In case if the specific sequence number packet does reach the resequencer before it's buffers is filled up completely, it can cause the resequencer's threads to hang, leading to WSG crash due to HM failure. Resequencer ME is not robust enough to handle such situations.
Note CSCup74321 was resolved in WSG Release 4.3.3.
The following caveats were resolved in WSG Release 4.3.3:
Description: IPsec SA creation fails in WSG with certificates.
WSG IPsec SA fails to pick correct private key when WSG has two certificates with the same subject name, but signed by two different rootCAs.
Description: Card might crash and reload itself with HM Datapath failure event.
The crash info file will be collected at LCP core: directory
Different packet type takes different processing time in the Lookup leading to packet reaching late to Resequencer ME. In case if the specific sequence number packet does reach the resequencer before it's buffers is filled up completely, it can cause the resequencer's threads to hang, leading to WSG crash due to HM failure. Resequencer ME is not robust enough to handle such situations.
Description: Whenever HM failure occurs, Core dumps are not being collected for the IXPs in the crash-info tar of LCP.
When “exception ixp <no>” is not present in the LCP configuration, core dumps are not being collected.
Workaround: Configure “exception ixp <no>” in the LCP
The following caveat was resolved in WSG Release 4.3.2:
Description: WSG is not able to retrieve DNS server IP from DHCPv4 server.
While IPSec tunnel creation, FAP expects DNS IP from WSG. However, WSG is not able to retrieve DNS server IP from DHCPv4 server.
The following caveat was resolved in WSG Release 4.3.1:
Description: When the AP has two different certs with the same subject name and tries to create two different tunnel with WSG, Authentication failure is seen.
The following caveats were resolved in WSG Release 4.3.0:
Description: Packet drops in traffic of Site to Site tunnels with overlapping addresses in traffic selectors.
This condition occurs when overlapping traffic selector addresses are present on WSG configuration.
Description: Session to LCP from SUP may not work at times showing the following message:
If the sysmgr cores in LCP, without this DDTS fix, core will not be copied to dir core: In LCP and sysmgr, core will not result in SAMI reload. Also, sysmgr will keep generating core files in /var/tmp directory leading to /var/tmp getting full.
Description: Removed the DHCPv6 configs, de-activated and also removed from the profile, however, new address-pool config fails with the cause “Remove DHCP server configuration before adding address pools”.
When the DHCPv6 config is removed and the profile de-activated, configure the address-pool config and activate the profile. Profile should get activated.
Description: Address pool configs cannot be removed even after de-activating the profile.
This condition is seen when an address pool is configured with an active profile. WSG wont allow un-configuring address pool when the profile is in active state.
Description: WSG blocks user from configuring 0.0.0.0 as access-permit IP address
This condition occurs when user tries to configure 0.0.0.0 as access permit under any profile.
Description: During the unexpected reload of any of the SAMI processors possibly due to a software defect, the SAMI LCP attempts to collect debug information that will help determine the cause of the unexpected reload. On rare occasions, the LCP itself can undergo an unexpected reload during such information collection with the following error message:
Description: Core Dump File includes Proc Mem Info Details. LCP Sysmgr crashes due to Proc Memory Info Corruption.
The following caveats were resolved in WSG Release 4.2:
Description: In this scenario there are 16k and 8 k IPv6 tunnels on different profiles and VRF. When WSG initiated P2 re-keys fail due to TS lookup failure on any of the profile. The active WSG IpSec process hangs after 99% completion, after execution of clear cry ips sa – command. Then it again initiates the set-up of 16 k IPv6-VRF tunnels for both profiles. It also requires to load both the active as well as stand-by WSG for recovery.
Description: This issue is observed when a single SNMP getnext request tries to query two objects from the ipAddress Table. The OID of the second result is incorrect in most cases. This issue is observed only when there are more than two queries in a single getnext request.
Workaround: Whenever the SNMP getnext is used, the request should be split into single OID queries.
The following caveats were resolved in WSG Release 4.0.3:
After a phase-2 re-key, a small number of IPSec tunnels (approx. 10-20) may be deleted.
This has been seen with large number of tunnels (8500 tunnels on each PPC) established at a high rate and the lifetime was set to a short value in an engineering environment.
Description: After removing a root certificate from the WSG configuration, the removal appears to be successful. However, the root certificate is not deleted from the WSG database.
– Attempting to remove a root certificate while crypto profiles are active.
– Attempting to update an existing root certificate while crypto profiles are active.
Workaround: Deactivate all crypto profiles when deleting or modifying root certificates.
Description: With a BGP related configuration for the WSG Reverse Route Injection (RRI) feature, the removal of the BGP configuration off the WSG may result in an abnormal SAMI reload.
Workaround: Block the BGP neighbors from sending any routes to the WSG. Remove the BGP neighbor configurations one by one before removing the BGP configuration.
Description: This occurs if a SNMP getnext is the first operation performed on the objects after the tunnel is established.
Workaround: Do not perform getnext immediately after establishing the tunnel. If the getnext operation is the first one performed, the returned values will be correct after the tunnel has been established for at least 15 minutes.
Description: This occurs after connectivity is lost with the BGP neighbor for more than 3 minutes.
Workaround: Remove and re-add the BGP neighbor configuration.
Description: Normally, the “copy start run” command is used at the beginning of setup. In a case where the user used this command after the tunnels were created, we observed all of the tunnels were torn down (e.g. the start and running configurations were the same). This bug was filed to find a way to avoid it.
Description: The values returned by snmpwalk on ceipSecGlobalStats and some cikeGlobalStats objects are not accumulated across all PPCs.
Snmpwalk on CISCO-ENHANCED-IPSEC-FLOW-MIB stops.
The HA switchover occurs during the snmpwalk.
Workaround: Re-run the snmpwalk after the switchover is complete. The required statistics are not available immediately after the switchover.
Description: Output from crypto debugs is displayed incorrectly. SAMI resets after enabling crypto debugs and debug messages are displayed. WSG is configured with a timezone consisting of at least four characters.
Workaround: Ensure that the configured timezone is less than four characters.
The following caveats were resolved in WSG Release 3.0:
The CMP initialize command returns error when executed.
This occurs when the access method (as indicated in the URL) to the CA server is TCP.
Workaround: Use the HTTP access method (http://...) in the CMP initialize command to communicate with the CA server.
sh run in entity-all fails on one or another secondary PPC. snmpd does not respond to configuration request and times out with SAPS-->28 error.
This condition occurs when you execute snmpwalk on UDP-MIB in continuos loop with sleep 200 secs in between two successive iterations. In entity-all mode execute the show running-config command. snmpd on one or more secondary PPCs timeouts with SAPS-->28 error.
Workaround: On the bash shell of the secondary that got stuck, issue killall -9 snmpd command. This causes snmpd to re-spawn again.
The WSG deletes tunnels after an HA switchover.
This conditions rarely occurs when there are 100,000 remote-access tunnels established. With 100,000 tunnels established, 12 were deleted.
SNMP walk may periodically fail to poll MIB instances/elements on secondary WSGs.
SNMP walk fails to poll data from secondary WSG/PPC periodically. This situation occurs when CPU utilization on primary WSG increases considerably. However, the CPU utilization on secondary WSG always remain normal. High CPU utilization situation remains for very brief time period. The whole issue is not observed on tunnels established on primary WSG.
In the rare instance where a tunnel is not fully imported on the standby card, the standby cannot recover the tunnel from this issue.
To see if the standby card has a tunnel in this state, issue the show crypto isakmp summary command and show crypto ha db info command. This will show if a tunnel count mismatch has occurred.
Workaround: Reboot the standby card to force a new sync of the tunnels.
Observed SNMP crashinfo on SUP related to primary WSG/PPC3.
This condition occurred when we configured an SNMP related configuration on primary WSG/PPC. Leave these commands configured on the primary WSG for overnight tests. You may observe SNMP crashinfo files (on SUP disk0) due to SNMP process crash and re-initialization.
WSG Initiated IKEv2 Phase 2 rekey happens only for one child SA.
IKE SA with multiple child IPSec SAs, with Phase 2 rekeys initiated by WSG (client Phase 2 rekey lifetime > the Phase 2 lifetime configured on WSG).
Workaround: Initiate rekeys from the client side (configure client Phase 2 rekey lifetime < WSG Phase 2 lifetime).
The UDP source port is not correctly displayed using the show crypto ipsec sa remote-ip command.
This problem occurs under the following conditions:
– IPSec tunnel is established via a device performing PAT.
– A condition triggers a source port change (for example, a timeout).
Workaround: Use the show crypto isakmp sa command to display the UDP source port.
The ipsecpm process failed after tunnel failure with auto-initiate.
The ipsecpm failure is observed with the following conditions:
– IKE ID for the remote peer does not match the DN, and Certificate does not include a subjectALtName extension.
Workaround: Configure the remote peer to use DN as the ID.
Authentication sometimes fails when there is more than one trust point configured.
If you have two trust points configured, two entries of get certificate request in IKE_SA_INIT all point to the certificate in the first trust point.
Workaround: One trust point works.
The following caveat was resolved in WSG Release 2.2.2:
Description: IPSec tunnel establishment fails when certificate based authentication is used. The messages "Certificate Path Construction Failed" or "No Certificate Found Anywhere" appear in the WSG event log. This symptom occurs when the certificates in use have an entry for "CAIssuers" in the AIA extension.
Workaround: Use certificates that do not use “CAIssuers” in the AIA extension.
The following caveats were resolved in WSG Release 2.2.1:
Description: The WSG sends DHCP release prematurely under certain conditions:
1. Access Point reboot on an existing IPSec tunnel.
2. The reboot cycle occurs before the WSG can delete the IPSec tunnel via dead-peer-detection.
The WSG sends the DHCP release when the old tunnel is deleted.
Workaround: The client IP address is re-assigned via DHCP when the IKE SA is rekeyed, provided
the address is still available. Otherwise, the tunnel is torn down, requiring re-establishment by the Access Point.
Description: WSG allows tunnels to be established with remote peer ID not matching the ID in the remote peer certificate. This issue is seen in 2.X images.
The following caveats were resolved in WSG Release 1.2:
Description: Some tunnels are deleted by WSG. The corresponding IKE error counters do not show any errors.
This happens in multiple circumstances:
1. Immediately after a large number of tunnel creation (For example, when creating tunnels at 90 tunnels per second).
2. If the client does not respond to the INFORMATIONAL messages (and probably the IKE timeout happens).
Description: The snmpwalk utility returns zero values for all instance statistics (per tunnel in/out packets and octets) for the cisco-enhanced-ipsec-flow-mib table.
Workaround: Use the snmpget utility to see valid statistics for each specific instance.
Description: Crashinfo files for syslogd are created and saved to the SUP. No other problems are observed with syslog after the SAMI comes up. The files are saved to the SUP after the SAMI is reset. This could occur after upgrading the software image (a reset is required to complete the upgrade), or simply resetting the SAMI.
Description: Adding new trustpoint (i.e. root) certificates to the WSG configuration requires deactivation of all crypto profiles. Additional root certificates are required when tunnels are already established.
Description: Multiple access-permit statements cannot be configured in remote access crypto profiles.
Description: The show crypto ipsec sa command does not display all of the traffic selector information. Only the first traffic selector data is displayed. Occurs when:
– Remote access crypto profile configuration on WSG.
– Remote access tunnels are established using multiple traffic selectors.
Description: Attempting to configure the remote secret results in the message, “Configuration failed in ipsecpm.” Occurs when the user attempts to configure the remote secret parameter while a crypto profile is active.
Workaround: Deactivate all profiles prior to configuring the remote secret.
Description: Traffic lost on a particular WSG or the show arp command displays the wrong MAC address. Occurs when HSRP WSG active/standby is configured.
Workaround: Reload the WSG exhibiting this problem.
Description: WSG has no configuration after a reboot. The startup configuration is corrupted. Occurs after these steps:
– An error condition such as SAP->611 is encountered which prints an error message into the running configuration when it is displayed using the show running-configuration command.
– User saves the running configuration.
Workaround: If an error message is displayed during the execution of the show running-configuration command, do not save the running configuration.
Description: The standby WSG will sporadically usurp the MAC address of the active WSG and announce itself causing ARP resolution to fail. This issue can also cause a failure to pass traffic on newly added routes. This issue is seen intermittently in a redundant HA setup when there is no traffic for a period longer than 5 minutes.
Workaround: Configure mac mac-address table timeout to 4 hours on the appropriate VLAN. Adding new routes requires a WSG reload to take effect.
Description: ESP packets are dropped on the WSG. This issue occurs when the VLAN currently configured on a PPC was previously configured on a lower numbered PCC, and the SAMI was not reset since that last configuration change. The information pertaining to the previously configured VLAN is not removed from memory when the configuration changes.
Description: Traffic fails on an IPSec tunnel. The tunnel does not pass traffic until a subsequent IPSec SA (phase-2) rekey occurs on the same tunnel.
– Remote access tunnels are terminated on WSG.
– Tunnels are terminated on two adjacent PPCs (e.g. 3 and 4, or 7 and 8).
– Multiple traffic selectors are configured or negotiated for the tunnels terminating on the higher numbered PPC.
– A phase-2 rekey occurs on a tunnel terminating on the higher numbered PPC. This triggers the traffic loss on a tunnel terminating on the lower numbered PPC.
Workaround: If remote-access tunnels with multiple traffic selectors are needed, do not terminate tunnels on an adjacent, lower numbered PPC.
Description: Traffic does not pass through the WSG even though the IPSec tunnel is successfully established. This issue has been observed after configuration changes to VLAN interfaces on different WSG instances (i.e. PPCs) on the same SAMI. Typically, the VLANs are reused but moved to interfaces on different WSG instances.
Description: The following message may appear in the system log. However, the system does not reload:
This process is not supposed to initiate a system reset. It is spawned upon demand and will be re-created as necessary.
For more detailed installation and configuration information, see the following publications:
http://www.cisco.com/public/sw-center/netmgmt/cmtk/mibs.shtml
For information on obtaining documentation, using the Cisco Bug Search Tool (BST), submitting a service request, and gathering additional information, see What’s New in Cisco Product Documentation.
To receive new and revised Cisco technical content directly to your desktop, you can subscribe to the What’s New in Cisco Product Documentation RSS feed. The RSS feeds are a free service.