Image Information and Supported Platforms
Guest OS persistent logging through reload
Guest OS file system corruption detection and recovery
Plug and Play Agent (PnP) support over 4G/Ethernet
AutoSim and Firmware Based Switching
Bidirectional Forward Detection support on the IR8X9
Logrotate of IR8x9 Guest-OS logs
CG-Mesh RF and PLC Dual-PHY WPAN
OverlayFS for the rootfs on the IR829
The following release notes support the Cisco IOS 15.7(3)M1 release. These release notes are updated to describe new features, limitations, troubleshooting, recommended configurations, caveats, and provide information on how to obtain support and documentation.
Note You must have a Cisco.com account to download the software.
Cisco IOS Release 15.7(3)M1 includes the following Cisco IOS images.
This bundle contains the following components:
– IOS: final version 15.7(3)M1
The IR807, IR809, and IR829 do not have feature parity. Consult the Software Configuration Guides for the IR800 and IR807 devices for additional information.
Note When discussing features in these release notes, it is important to note that the IR807 does not have VDS, GOS, or Hypervisor.
The latest downloads for the IR807, IR809 and IR829 can be found at here:
https://software.cisco.com/download/navigator.html?mdfid=286287045&flowid=75322
Click on the 829, 809, or 807 link to take you to the specific software you are looking for.
The IR809 and IR829 will have selections for Software on the Chassis:
The IR829 also includes downloads for the AP803 Access Point Module:
The IR807 runs a single IOS image.
Note The ir800-universalk9-bundle.SPA.157-3.M1 bundle can be copied via Trivial File Transfer Protocol (TFTP) or SCP to the IR800, and then installed using the bundle install flash:
<image name> command. The ir800-universalk9-bundle.SPA.157-3.M1.bin file can NOT be directly booted using the boot system flash:/image_name.
Detailed instructions are found in the Cisco IR800 Integrated Services Router Software Configuration Guide.
Note The cipher dhe-aes-256-cbc-sha (which is used with the commands ip http client secure-ciphersuite and ip http secure-ciphersuite) is no longer available in IOS 15.6(3)M and later as part of the weak cipher removal process. This cipher was flagged as a security vulnerability.
For details on the CGR1000 installation, please see:
http://www.cisco.com/c/en/us/td/docs/routers/connectedgrid/cgr1000/ios/release/notes/OL-31148-05.html#pgfId-998856
The latest image file for the CGR 1000 Series Cisco IOS image is:
https://software.cisco.com/download/navigator.html?mdfid=284165761&flowid=75122
This release has the following limitations or deviations for expected behavior:
Caveat CSCvf76265 crosses over several different IOS software releases, and is a platform driver code issue. It is included here as a known limitation with the IR800 and CGR Industrial Routers.
On both the CGR1000 and IR800, the core dump fails to write into the local flash. The IOS is running as a virtual machine and then hypervisor is running underneath. The local flash is provided by the hypervisor as a virtual disk. When a crash occurs, this virtual disk is no longer available therefore copying to flash will fail. The workaround is to use an ftp server to copy the core dump to.
Caveat CSCvi17033 pertains to the CGR1000 series. The details of the limitation are as follows:
Guest OS Interfaces have an MTU of 1492.
Symptoms : Symptoms of this limitation can be TCP timeout or sessions closing on applications using an MTU of 1500 running on the GOS. The ifconfig command in GOS will show:
This release includes the following enhancements:
Log files related to the Guest OS file system are stored on the /var/log directory of IOx. This is a volatile location because they may be lost when the IOS or IOx receives a reload command. for this reason, the caf.log, daemon.log, tpmc.log and syslog files from /var/log are now moved to a persistent storage location under /software/downloads (i.e. /dev/sdb filesystem) and the data in it will be restored upon multiple reloads. On reinstallation, the files under /software/downloads will be removed.
The command is persistent across IOS reloads unless a new GOS image is loaded or a bundle install to the new GOS image.
The Guest OS running on the IR800 series have had a higher likelihood for file system corruptions after an abrupt power failure. Now, upon Guest OS start or restart, a mandatory FSCK is performed on the rootfs and the datafs in order to attempt file system recovery.
This feature can be enabled or disabled using the config command iox recovery-enable <timeout>, where timeout specifies the TPMS timer timeout value in minutes. If unspecified, the default value is 5 and maximum is 15. If no registration request is received from TPMC before the timer expires, the Guest OS will be reinstalled. By default, the feature is disabled so that the customers who do not use Guest OS will not run into a situation where the Guest OS is reinstalled because networking is not configured correctly for Guest OS. The command is persistent across IOS reloads.
Plug N Play Cloud Service is a Cisco hosted service for customers to configure devices shipped from Cisco. Configurations include specifying a Controller (APIC-EM) and a Configuration file. An option was added to the bundle install command:
bundle install <bundle_image_name> rom-autoboot
When this option is specified, the IOS system image to boot will NOT be written into the running-config. Instead, it will be set into the rommon BOOT variable (BOOT=<system_image>) ONLY.
After bundle install <bundle_image_name> rom-autoboot and write erase commands, when the device reloads it will automatically boot up the IOS image saved in rommon BOOT. This also ensures the device does not have any startup configuration when it boots up so it will allow PNP to start up.
PNP can be started either using Ethernet or cellular 4G. If connected to both, Ethernet will take precedence over Cellular 4G.
Note Not available on the IR807.
1. Specifying OPTION 43 on DHCP ROUTER
Example: option 43 ascii 5A1D;B2;K4;I<APIC-EM_IP_ADDRESS>;J80
2. Specifying DNS on DHCP ROUTER
3. Specifying CCO’s address by configuring devicehelper.cisco.com on DHCP ROUTER
PNP using 4G cellular can be done by configuring the device information (Serial number, PID and controller profile-APIC-EM) on CCO.
Once PNP is completed, issue a write mem command to save the configuration. PNP pushes the configuration but does not save it. The configuration must be saved after PNP is successfully completed.
To verify if PNP is completed or not, verify with the sh run command. At the bottom of the command output, there should be a pnp profile and the APIC EM address. This means the device was redirected to APIC-EM and the initial PNP was successfully done. Now once the configuration file is pushed from APIC-EM, verify this using the sh pnp task command and verify the Config-Upgrade Task should have Result: Success.
Note The device should not be interrupted until PNP is completed. If the device is interrupted, PNP will stop. If at any point something goes wrong, reload the router without saving the configuration and PNP will start once again. Once PNP is completed it is necessary to save the configuration by issuing the write mem command.
A configuration register behavior has been emulated on the IR800 and CGR1000. The user or the PnP server can change the configuration register value from the IOS CLI using the config commands config-register <value> or using cfgreg <value> from rommon1 on the IR800 and rommon2 on the CGR1000. The value can be viewed by using the show version exec command in IOS. The default value of the config-register is 0x102.
The Format for the configuration registers is 0 x _ _ _ _ (4 bytes)
0x102, 0x2102, 0x2142, 0x142, 0x101, 0x2101
Table 1 shows the configuration register 1st byte values and descriptions.
Table 1 Configuration Register 1st byte
Table 2 shows the configuration register 2nd byte values and descriptions.
Table 2 Configuration Register 2nd byte
Table 3 shows the configuration register 3rd byte values and descriptions.
Table 3 Configuration Register 3rd byte
Table 4 shows the configuration register 4th byte values and descriptions.
The AutoSim feature will identify the SIM card of the Carrier inserted and correspondingly load the correct modem firmware. The advantages of the AutoSim feature are:
Auto-SIM is supported in Sierra wireless firmware Version 02.20.03. A new CLI is added in the cellular controller to enable/disable Auto-SIM. The modem in Auto-SIM mode selects the right carrier firmware after a SIM slot switch and an automatic modem reset. Auto-SIM is supported on the MC7455, MC7430, EM7430, and EM7455 modems. During bootup, if the Auto-SIM configuration on the modem doesn’t match to the IOS configuration, the corresponding Auto-SIM or manual mode is pushed to the modem.
After an Auto-SIM configuration change, the modem is automatically reset. Once it is up, issue a modem power-cycle for auto-sim to take effect. The default is “auto-sim” enabled.
Note After enabling auto-sim, wait for 5 minutes until the radio comes up. Once the radio is up, issue a modem power-cycle and wait for 3 minutes for the radio to come up again. Modem Power-Cycle is mandatory for auto-sim configuration to take effect.
Note After disabling auto-sim, wait for 5 minutes until the radio comes up. Once the radio is up, issue a modem power-cycle and wait for 3 minutes for the radio to come up again. Modem Power-Cycle is mandatory for auto-sim configuration to take effect.
If Auto-SIM is disabled and the modem is in manual mode, select a carrier with a new exec CLI:
The following CLI shows the firmware-index of the carrier in the modem:
For additional information, see the following guide:
Cisco 4G LTE and Cisco 4G LTE-Advanced Network Interface Module Software Configuration Guide
The IR809 is able to communicate with a Battery Backup device via the serial port on this release. There are Command Line Interface (CLI) commands to support its use.
Note This feature was introduced specifically to support a battery via the serial interface for our India AMI partner. Cisco does not offer its own BBU option for the IR809.
To configure the IR809 serial port:
To verify the BBU configuration:
Any syslog and/or snmp notifications are also displayed in the Field Network Director Events and Issues page.
Whenever there is status change reported by the BBU, the following syslogs (according the event received):
If current status = A/C, and the BBU reported power source=Battery, power status=Battery Low, the syslog message would be:
If current status = Battery and BBU reported power source=Battery. power status=Battery Cut-Off, the syslog message would be:
If current status = Battery and BBU reported power source=Mains DC - IN. power status=Mains Available, the syslog message would be:
The device uses the following SNMP trap states:
This release introduces some VDS troubleshooting/serviceability enhancements. This ensures that if there is any memory leaks or cpu load issues during ' bundle install ' image operation, the bundle install will timeout much sooner than the previous timeout of 15 minutes.
Bundle install will not proceed, and prevent any further memory issues or VDS lockups resulting in router reload due to a VDS communication error. If a bundle install failure or system reload occurs and the router resets due to a VDS communication error, the contents of flash: will have a snapshot of the output from 'top', '/var/log', ‘/tmp’, and ‘df –h’ information.
This will help determine if there was any issue with cpu or memory resources triggering the crash. The user is expected to execute fsck flash: in order to view this file (also indicated by a syslog message).
This release introduces support for Cisco's Bidirectional Forward Detection (BFD) on the IR809 and IR829. The only supported interfaces are the Layer 3 gigabit ethernet interfaces. There is no SVI or Vlan support. These platforms support BFD over static routes, mGRE and DMVPN as well.
The Command Line Interface for the gyroscope feature has been changed in order to be compatible with the CCP Express NMS.
From this release going forward, the format has been modified to:
After upgrading to this release, the router will have to be reconfigured.
The logrotate feature has been uniformly implemented across all logs in the Guest-OS /var/log path. If persistent-logging is enabled, the specific logs will be saved on /software/downloads and logrotate is implemented on those as well. By default, log-rotate takes effect every day at 7:30am.
The CG-Mesh Dual-PHY feature defines a topology where both IEEE 802.15.4g/e RF and IEEE 1901.2a Power Line Communication (PLC) WPAN interfaces are deployed. The IEEE 802.15.4g/e RF is the primary transmission medium, and the IEEE 1901.2 PLC is the secondary transmission medium.
CG-Mesh Dual-PHY is focused on MAC and PHY layer. It is transparent to 6LoWPAN adaptation layer and upper layers on the Dual-PHY mesh node. The Dual-PHY and Single-PHY nodes have the same behavior on 6LoWPAN adaptation layer and upper layers. The main advantage of Dual-PHY node comparing with Single-PHY node is that the Dual-PHY node can leverage the secondary PLC link when the transmissions on the RF link fails.
A new Overlay File System is created with a read-only /dev/sda1 as the lower directory and a tmpfs based Read-Write upper directory. The overlay file system will be mounted as the new root by the initialization scripts. Any changes to the files on the root directory will be written to tmpfs, thereby preserving the original state of /dev/sda1. When the Guest OS is rebooted, all of the changes from the tmpfs will be purged out and the Guest OS starts afresh.
Any changes done to /oldroot can be seen from the OverlayFS as long as there is NO copy of the file in the upperdir (/oldroot/ovfs-rw). Files in the upperdir (tmpfs) always take precedence. In order to create a fully functioning root file system with OverlayFS, the device mounts of /dev, /proc and other vital mount points have been created. This is accomplished through BindFS, and linkages are as follows:
By default, /dev/sda1 is mounted as a read-only file system. To remount the file system as read-write, use the following command:
In order to create persistent changes to the root file system, remount /oldroot as read-write using the command specified above, and then use the following command:
Note Changes applied to /software (/dev/sdb) are always persistent across reboots and reinstalls.
The following documentation is available:
https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/15-7m/release/notes/15-7-3-m-rel-notes.html
http://www.cisco.com/c/en/us/support/routers/800-series-industrial-routers/tsd-products-support-series-home.html
http://www.cisco.com/c/en/us/support/routers/1000-series-connected-grid-routers/tsd-products-support-series-home.html
Caveats describe unexpected behavior in Cisco IOS releases. Caveats listed as open in a prior release are carried forward to the next release as either open or resolved.
Note You must have a Cisco.com account to log in and access the Cisco Bug Search Tool. If you do not have one, you can register for an account.
For more information about the Cisco Bug Search Tool, see the Bug Search Tool Help & FAQ.
Running excessive traffic from an external host to the Guest OS on the IR800 can cause a system hang.
Conditions : Like all Cisco interfaces, when an overload of traffic is sent, the IOS CPU spikes high and the console will be inaccessible. Removing traffic will recover the console access.
Workaround : Removing the traffic will recover the console access. Apply qos policies to rate-limit traffic to ensure IOS CPU <65%.
NTPd is not available in the CGR1000 GOS images.
Network Time Protocol is not available in the CGR1000 GOS images (with CAF support) because the GOS time sync is now obtained through IOS via TPMC heartbeats.
When changing the ignition timer with ignition already enabled, a graceful shutdown does not happen at ~100s.
Symptoms : Change the ignition timer as in the example in conditions.
Conditions : Seen with the following steps:
show ignition, verify timer countdown kicks-in
show ignition, verify timer reset to new value and countdown kicks-in
The expected behavior is that the device should shutdown at ~100s like in all other scenarios.
Observation: Times down to ~0s
Workaround : Once ignition timer is changed, configure the following:
This should reset and gracefully shutdown at ~100seconds [to allow time for IOx, cellular processes to shutdown].
On the IR829, IOS not able to get AT command responses from modem after a modem firmware upgrade.
Symptoms : When starting from a fresh/no-config IR829, the AT-command session and commands work as expected. After a firmware upgrade the AT-commands from IOS do not provide modem responses. From Linux, the AT commands still yield modem responses.
Workaround : Power-cycle the modem or reload the system.
VDS may potentially lockup in the presence of cellular interface (even if the modem is not actively in use).
Symptoms : During IOx read/write operations, bundle install, during traffic scenarios.
Workaround : Link-recovery disable on cellular interface mitigates the issue. Router automatically auto-reloads in 30 minutes with reset reason: VDS communication error. No data is lost in the process.
MCU Bootloader version not updated from 28 to 30 as expected
Symptoms : Occasionally a mismatch may be observed in show version | inc MCU Bootloader version: It may show 28 or 30.
Workaround : None required, this is benign.
Some CGR1000's exhibit read_inode_bitmap errors.
Symptoms : On occasion you may see something like the following:
Conditions : This is possibly related to a failure in the physical flash.
Guest-os bringup could be 2 - 5 minutes.
Symptoms : Guest-OS Reboot time increase
Conditions : If disk storage is 0%, the bring-up time is less than 2 minutes. If disk storage is 100%, the bring-up time could be as great as 5 minutes.
Workaround : None, the increased time is due to fsck checks in progress. The fsck program helps detect file corruption.
Output from show ignition status sometimes reflects 'Bootloader'
Symptoms : On occasion the show ignition status CLI reflects 'bootloader' as ignition status and input voltage goes to 0.0V. For example:
Workaround : Re-executing the command will display the correct values. No functional impact has been found due to this behavior.
Packet Elements memory leaks on CGR1000
Symptoms : Basic router configuration with NAT may randomly trigger minor packet element leaks with not much functional impact.
On the IR809 platform only, the baud rate support for the WPAN interface is limited to 460.8kbps.
Symptoms : On IXM and IR809, the baud rate limit today is 460.8kbps of the allowed 961.2kbps.
IR800 crash: Couldn't initialize the marvell chip ERROR: ir800_esw_analyze_interface.
Symptoms : Router crash and recover
Conditions : Only occurs on severe read/write stress testing.
On the CGR1K, inconsistent RAT preference displays as WCDMA or GLW when UMTS is configured.
When RAT technology is selected to be 'auto', it may show up on some modem firmware as 'GWL' (ie, GSM, WCDMA, or LTE) as RAT preference under 'show cellular slot radio'. This is a cosmetic issue, and does not affect functionality.
GOS networking fails to come up
Symptoms : Even on a router reload, GOS IPv6 addressing never gets assigned. GOS networking is completely down.
Conditions : When encapsulation dot1q is configured on the GOS sub-interface and BVI attached to it, and then 'shut' the interface and change to another physical interface without dot1q.
Workaround : Default the interface with dot1q encapsulation, even if it is in 'shut' state.
Failed to find interface shows up when user logs into to IOS or IOx with no AAA login.
Symptoms : On a CGR router that is NOT configured to run AAA, random errors are displayed:
Workaround : These are simply debug messages and there is no impact on functionality.
High load on interface events.
Symptoms : On an idle device a syslog message appears:
Workaround : This is Day01 IPv6 issue. This message is benign, there is not a problem with the interface.
The following caveats are fixed with this release:
On the CGR1K, firmware upgrade fails with an error code 105. Seen on both the MC7430 and MC7455 modems.
Workaround : Reload the router.
IKEv2 tunnel fails to come up between Cisco routers after upgrading one router to 15.5(3)S5, 15.5(3)M5
Symptoms : IKEv2 tunnel negotiation between two Cisco routers fails in IKE AUTH exchange after upgrading one of the routers to 15.5(3)S5 or 15.5(3)M5.
1. KEv2 tunnel configured between 2 Cisco routers (IOS or IOS-XE)
2. IKEv2 Fragmentation enabled and IKEv2 IETF fragmentation being negotiated between the two peers.
3. One of the routers is upgraded to15.5(3)S5 or 15.5(3)M5.
4. IKE AUTH packet size exceeds the IKEv2 Fragmentation MTU and hence is fragmented at IKE layer.
Workaround : Disable IKEv2 Fragmentation, or, upgrade the peer as well to 15.5(3)S5 or 15.5(3)M5
Bundle install times out on IR800 and CGR1000 router left idle for a long period of time.
Symptoms : Bundle install timeout, or if a lot of cellular modem reset or traffic operation is there, the router would reload with reset reason message "VDS communication error".
Conditions : When the router was left idle for a long time, or if a lot of cellular modem related operations occurred. Root cause was cellular log files not rotating and flushing periodically. Due to increase in log size, there was not much memory eventually impacting memory intensive operations like 'bundle install', or, if cellular modem used extensively and modem reset too many times it would eventually run out of memory and reload and recover.
Workaround : Workaround would be to reload router, and reattempt bundle install. That would flush the logs automatically.
IR829 would always reload instead of powering down despite the lack of ignition signal.
Conditions : This problem was seen on the IR829 running 15.7(3)M or 15.6(3)M3. When the ignition management is enabled and the ignition is OFF, IR829 does not stay shut down when its ignition off-timer expires. It keeps reloading back to IOS, getting shut down again and the same cycle repeats. The battery will be eventually drained due to this repeated cycle
Workaround : Resolved. Customers not on this release are advised to use 15.6(3)M2 to avoid this problem.
CGR: Bundle-install timeout due to IOS extraction failure.
Conditions : The /var/log/cwan_modem0.log and /var/log/cwan_modem1.log were not getting rotated. As the result, file size grow bigger and VDS is running out of memory.
Conditions : The IR800 series router could report BIOS upgrade failure messages after a reload/power-cycle/software upgrade. The following message (or similar) could be displayed: