Information About PowerOn Auto Provisioning
PowerOn Auto Provisioning (POAP) automates the process of upgrading software images and installing configuration files on Cisco Nexus switches that are being deployed in the network for the first time.
When a Cisco Nexus Series switch with the POAP feature boots and does not find the startup configuration, the switch enters POAP mode and checks for a USB device containing the configuration script file. If it finds one, it checks that device to see if it also contains the software image files and the switch configuration file.
If the switch does not find a USB device, or if the USB device does not contain the needed image files or switch configuration file, the switch also locates a DHCP server and bootstraps itself with its interface IP address, gateway, and DNS server IP addresses. The switch then obtains the IP address of a TFTP server or the URL of an HTTP server from which it downloads the necessary configuration files.
Note |
The DHCP information is used only during the POAP process if any configuration files are unavailable on the USB device. If the backup configuration file does not have the administrative username and the password, POAP causes a console lockout after completion. It is a mandatory step to add the username and the password in the configuration file. |
Network Requirements for POAP
POAP requires the following network infrastructure:
-
A DHCP server to bootstrap the interface IP address, gateway address, DNS server, and log server
-
A TFTP or HTTP server containing the configuration script used to automate the software image installation and configuration process
-
One or more servers containing the desired software images and configuration files
POAP Configuration Script
We provide sample configuration scripts that are developed using the Python programming language. You can customize these scripts to meet the requirements of your network environment.
A complete POAP script can be found at https://github.com/datacenter/nexus9000/blob/master/nx-os/poap/poap.py
For information about customizing this script using Python, see the Cisco NX-OS Python API Reference Guide for your platform.
Using the POAP Script and POAP Script Options
Before using the POAP script, perform the following actions:
-
Edit the options dictionary at the top of the script to ensure all the relevant options for your setup are included in the script. Do not change the defaults (in the default options function) directly.
-
Update the MD5 checksum of the POAP script as shown using shell commands.
f=poap_nexus_script.py ; cat $f | sed '/^#md5sum/d' > $f.md5 ; sed -i "s/^#md5sum=.*/#md5sum=\"$(md5sum $f.md5 | sed 's/ .*//')\"/" $f
-
If the device has a startup configuration, perform write erase and reload the device.
The following is a comprehensive list of POAP script options and can be specified to alter the POAP script behavior. When downloading from a server, 'hostname', 'username', and 'password' are required. For every mode except 'personality', the 'target_system_image' is also required. Required parameters are enforced by the script and the script will abort if the required parameters are not present. Every option except 'hostname', 'username', and 'password' has a default option. If you do not specify the option in the options dictionary, the default as mentioned here will be used.
-
username
The username to use when downloading files from the server. This is not required when using USB.
-
password
The password to use when downloading files from the server. This is not required when using USB.
-
hostname
The name or address of the server to download files from. This is not required when using USB.
-
mode
Default is serial_number.
Use one of the following options:
-
personality
A method to restore the switch from a tarball.
-
serial_number
The serial number of the switch to determine the configuration filename. The format for the serial number in the configuration file is conf.serialnumber. Example: conf.FOC123456
-
hostname
The hostname as received in the DHCP options to determine the configuration filename. The format for using the hostname in the configuration file is conf_<hostname>.cfg Example: conf_3164-RS.cfg
-
mac
The interface MAC address to determine the configuration filename. The format for using the hostname in the configuration file is conf_<macaddress>.cfg Example: conf_7426CC5C9180.cfg
-
raw
The configuration filename is used exactly as provided in the options. The filename is not altered in any way.
-
location
The CDP neighbors are used to determine the configuration filename. The The format for using the location in the configuration file is conf_<host>_<intf>.cfg where <host> is the host connected to the device over the POAP interface, and <intf> is the remote interface the POAP interface is connected to. Example: conf_remote-switch_Eth1_8.cfg
-
-
required_space
The required space in KB for that particular iteration of POAP. Default is 100000. For multi-step upgrades, specify the size of the last image in the upgrade path of the target image or images.
-
transfer_protocol
Any transfer protocol such as http, https, ftp, scp, sftp or tftp that is supported by VSH. Default is scp.
-
config_path
The path to the configuration file on the server. Example: /tftpboot. Default is /var/lib/tftpboot.
-
target_system_image
The name of the image to download from the remote server. This is the image you get after POAP completes. This is a required parameter for every mode except 'personality'. Default is "".
-
target_image_path
The path to the image on the server. Example: /tftpboot. Default is /var/lib/tftpboot.
-
target_kickstart_image
The name of the kickstart image to download from the remote server. This is the kickstart image you get after POAP completes. Default is "".
-
destination_path
The path to download images and MD5 sums to. Default is /bootflash.
-
destination_system_image
The name for the destination system image file name. If not specified, the default will be the 'target_system_image' name.
-
destination_kickstart_image
The name for the destination kickstart image file name. If not specified, the default will be the 'target_kickstart_image' name.
-
user_app_path
The path on the server where the user scripts, agents, and user data are located. Default is /var/lib/tftpboot.
-
disable_md5
This is True if MD5 checking should be disabled. Default is False.
-
midway_system_image
The name of the system image to use for the midway system upgrade. By default, the POAP script finds the name of any required midway images in the upgrade path and uses them. Set this option if you prefer to pick a different midway image for a two-step upgrade. Default is "".
-
midway_kickstart_image
The name of the kickstart image to use for the midway system upgrade. By default, the POAP script finds the name of any required midway images in the upgrade path and uses them. Set this option if you prefer to pick a different midway image for a two-step upgrade. Default is "".
-
usb_slot
The USB slot number to use for USB POAP. Default is 1.
-
source_config_file
The name of the configuration file when 'raw' mode is used. Default is poap.cfg
-
vrf
The VRF to use for downloads and so on. The VRF is automatically set by the POAP process. Default is the 'POAP_VRF' environment variable.
-
destination_config
The name to use for the downloaded configuration. Default is poap_replay.cfg
-
split_config_first
The name to use for the first configuration portion if the configuration needs to be split. Applicable only when certain configuration requires a reload to take effect. Default is poap_1.cfg
-
split_config_second
The name to use for the second configuration portion if the configuration is split. Default is poap_2.cfg
-
timeout_config
The timeout in seconds, for copying the configuration file. Default is 120. For non-legacy images, this option is not used and the POAP process will timeout. For legacy images, FTP uses this timeout for the login process and not for the copy process, while scp and other protocols use this timeout for the copy process.
-
timeout_copy_system
The timeout in seconds, for copying the system image. Default is 2100. For non-legacy images, this option is not used and the POAP process will timeout. For legacy images, FTP uses this timeout for the login process and not for the copy process, while scp and other protocols use this timeout for the copy process.
-
timeout_copy_kickstart
The timeout in seconds, for copying the kickstart image. Default is 900. For non-legacy images, this option is not used and the POAP process will timeout. For legacy images, FTP uses this timeout for the login process and not for the copy process, while scp and other protocols use this timeout for the copy process.
-
timeout_copy_personality
The timeout in seconds, for copying the personality tarball. Default is 900. For non-legacy images, this option is not used and the POAP process will timeout. For legacy images, FTP uses this timeout for the login process and not for the copy process, while scp and other protocols use this timeout for the copy process.
-
timeout_copy_user
The timeout in seconds, for copying any user scripts and agents. Default is 900. For non-legacy images, this option is not used and the POAP process will timeout. For legacy images, FTP uses this timeout for the login process and not for the copy process, while scp and other protocols use this timeout for the copy process.
-
personality_path
The remote path to download the personality tarball from. Once the tarball is downloaded and the personality process is started, personality will download all files in the future from locations specified inside the tarball configuration. Default is /var/lib/tftpboot
-
source_tarball
The name of the personality tarball to download. Default is personality.tar
-
destination_tarball
The name for the downloaded personality tarball after it is downloaded. Default is personality.tar
-
compact_image
To copy the compact image set this value to "True" as shown below. The default value is "False".
"compact_image" : True
For more details on the compact image, see the Compact Image for Cisco Nexus 3000 section in Cisco Nexus 3000 Series NX-OS Software Upgrade and Downgrade Guide.
POAP Upgrade Path
On a Cisco Nexus 3000 Series switch, moving from one software image to another requires intermediate upgrade steps that are handled as part of POAP. You need to know the upgrade path you require to avoid confusions while POAP loads multiple software images.
For the N3K-C3048TP-1GE-SUP platform, if you are using software versions older than Cisco NX-OS Release 5.0(3)U5(1), upgrade to Cisco NX-OS Release 5.0(3)U5(1) first, then upgrade to Cisco NX-OS Release 6.0(2)U6(2a), and finally upgrade to 6.0(2)U6(7) or a latest release.
If you like to skip this extra step of upgrading to Cisco NX-OS Release 6.0(2)U6(2a), you need to use the midway_system_image and midway_kickstart_image options.
The required software images for each step are:
-
Cisco Release 5.0(3)U5(1) - n3000-uk9-kickstart.5.0.3.U5.1.bin, n3000-uk9.5.0.3.U5.1.bin
-
Cisco Release 6.0(2)U6(2a) - n3000-uk9-kickstart.6.0.2.U6.2a.bin, n3000-uk9.6.0.2.U6.2a.bin
-
Cisco Release 6.0(2)U6(7) - n3000-uk9-kickstart.6.0.2.U6.7.bin, n3000-uk9.6.0.2.U6.7.bin
For all other Cisco Nexus 3xxx platforms, if you are using software versions older than Cisco NX-OS Release 5.0(3)U5(1), upgrade to Cisco NX-OS Release 5.0(3)U5(1) first, then upgrade to 6.0(2)U6(7) or a latest release.
However, if you have a mix of N3K-C3048TP-1GE-SUP platform and other devices, you need to use the default upgrade path (no options needed) for the N3K-C3048TP-1GE-SUP platform, and a different script with the midway options set for your other devices. You can use the Vendor Class ID to distinguish between the type of devices.
POAP Upgrade Bootflash Size
For Cisco Nexus 3xxx devices having small bootflashes (< 1.6 GB) requiring multi-step upgrade process may have problems if bootflash is not sufficiently empty. The total amount of bootflash needed can be calculated using the following:
(size of images currently running) + (size of largest midway images) + (size of target images) + (18% of bootflash size)
For example, if you are on a N3K-C3048TP-1GE-SUP platform with 1.6GB of bootflash, running 5.0(3)U5.1.bin, and you are attempting to upgrade to Cisco Release 7.0(3)I4(5), you would require:
5.0(3)U5(1) - current image, 6.0(2)U6(7) - largest midway image, 7.0(3)I4(3) - target image, and 18% of 1.6GB bootflash
(126MB system + 24MB kickstart) + (198MB system + 37MB kickstart) + (665MB NX-OS) + (295MB free space) = 1.31GB
In this example, you have only 200MB on bootflash apart from your current software image.
Setting up the DHCP Server without DNS for POAP
Starting with NX-OS release 7.0(3)I4(5), tftp-server-name can be used without the DNS option. To enable POAP functionality without DNS on releases earlier than 7.0(3)I4(5) release, a custom option (150) must be used to specify the tftp-server-address.
To use the tftp-server-address option, specify the following at the start of your dhcpd.conf file.
option tftp-server-address code 150 = ip-address;
For example:
host MyDevice {
option dhcp-client-identifier "\000SAL12345678";
fixed-address 2.1.1.10;
option routers 2.1.1.1;
option host-name "MyDevice";
option bootfile-name "poap_nexus_script.py";
option tftp-server-address 2.1.1.1;
}
Downloading and Using User Data, Agents, and Scripts as part of POAP
Under the options dictionary, you can find the download_scripts_and_agents function. If you choose to download user scripts and data, uncomment the first poap_log line and then use a series of download_user_app function calls to download each application. Since older Cisco NX-OS versions do not support recursive copy of directories, such directories must be put into a tarball (TAR archive) and then unpacked once on the switch. The parameters for the download_scripts_and_agents function are as follows:
-
source_path - The path to where the file or tarball is located. This is a required parameter. Example: /var/lib/tftpboot.
-
source_file - The name of the file to download. This is a required parameter. Example: agents.tar, script.py, and so on.
-
dest_path - The location to download the file on the switch. Any directories that do not exist earlier will be created. This is an optional parameter. The default is /bootflash.
-
dest_file - The name to give the downloaded file. This is an optional parameter. The default is unchanged source_file.
-
unpack - Indicates whether a tarball exists for unpacking. Unpacking is done with tar -xf tarfile -C /bootflash . This is an optional parameter. The default is False.
-
delete_after_unpack - Indicates whether to delete the downloaded tarball after unpack is successful. There is no effect if unpack is False. The default is False.
Using the download functionality, you can download all the agents and files needed to run POAP. To start the agents, you should have the configuration present in the running configuration downloaded by POAP. Then the agents, scheduler, and cron entry, along with EEM, can be used.
Troubleshooting for POAP
The following is a list of known issues and suggestions while using POAP:
-
Issue: POAP script execution fails immediately with no syslogs or output except for a "Script execution failed" statement.
Suggestion: Use the python script-name command on the server and make sure there are no syntax errors. The options dictionary is a Python dictionary so each entry must be comma separated and have the key or option and the value separated by a colon.
-
Issue: A TypeError exception occurs at various places depending on the incorrectly used option.
Suggestion: Some options use integers (for example, timeouts and other numeric values). Check the options dictionary for numeric values that are enclosed in quotes. Refer to the options list for the correct usage.
-
Issue: POAP over USB is not finding the files that are present.
Suggestion: Some devices have two USB slots. If you are using USB slot 2, you need to specify that as an option. For more details, see the options under Using the POAP Script and POAP Script Options.
POAP Process
The POAP process has the following phases:
-
Power up
-
USB discovery
-
DHCP discovery
-
Script execution
-
Post-installation reload
Within these phases, other process and decision points occur. The following illustration shows a flow diagram of the POAP process.
Power-Up Phase
When you power-up a switch for the first time, it loads the software image installed at manufacturing and tries to find a configuration file to apply after the switch boots. When no configuration file is found, POAP mode starts.
During startup, a prompt appears asking if you want to abort POAP and continue with normal setup. You can choose to exit or continue with POAP.
Note |
No user intervention is required for POAP to continue. The prompt that asks if you want to abort POAP remains available until the POAP process is complete. |
If you exit POAP mode, you enter the normal interactive setup script. If you continue in POAP mode, all the front-panel interfaces are set up in Layer 2 mode, which ensures that the device does not participate in any Layer 2 forwarding.
USB Discovery Phase
When POAP starts, the process searches the root directory of all accessible USB devices for the POAP configuration script file (the Python script file, poap_script.py), configuration files, and system and kickstart images.
If the configuration script file is found on a USB device, POAP begins running the configuration script. If the configuration script file is not found on the USB device, POAP executes DHCP discovery. (When failures occur, the POAP process alternates between USB discovery and DHCP discovery, until POAP succeeds or you manually abort the POAP process.)
If the software image and switch configuration files specified in the configuration script are present, POAP uses those files to install the software and configure the switch. If the software image and switch configuration files are not on the USB device, POAP does some cleanup and starts DHCP phase from the beginning.
DHCP Discovery Phase
The switch sends out DHCP discover messages on all of the active interfaces (including the mgmt interface) soliciting DHCP offers from the DHCP server or servers. The DHCP client on the Cisco Nexus switch uses the switch serial number or its MAC address in the client-identifier option to identify itself to the DHCP server. The DHCP server can use this identifier to send information, such as the IP address and script file name, back to the DHCP client.
POAP requires a minimum DHCP lease period of 3600 seconds (1 hour). POAP checks the DHCP lease period. If the DHCP lease period is set to less than 3600 seconds (1 hour), POAP does not complete DHCP negotiation.
The DHCP discover message also mandates some of the options and solicits these options from the DHCP server after receiving the DHCP OFFER from the DHCP server.
-
Option 66 ( TFTP server name) or Option 150 (TFTP server address)—The DHCP server relays the TFTP server name or TFTP server address to the DHCP client. The DHCP client uses this information to contact the TFTP server to obtain the script file.
-
IP address
-
Default gateway
-
Option 67 (Bootfile name)—The DHCP server relays the bootfile name to the DHCP client. The bootfile name includes the complete path to the bootfile on the TFTP server. The DHCP client uses this information to download the script file.
When multiple DHCP offers that meet the requirement are received, an offer is randomly chosen. The device completes the DHCP negotiation (request and acknowledgment) with the selected DHCP server, and the DHCP server assigns an IP address to the switch. If there is a failure in any of the subsequent steps in the POAP process, the IP address is released back to the DHCP server.
If no DHCP offers meet the requirements, the switch does not complete the DHCP negotiation (request and acknowledgment) and an IP address is not assigned. The POAP process is reinitiated until it succeeds or you manually abort the POAP process.
POAP Dynamic Breakout
Beginning with Cisco NX-OS Release 7.0(3)I4(1), POAP dynamically breaks out ports in an effort to detect a DHCP server behind one of the broken-out ports. Previously, the DHCP server used for POAP had to be directly connected to a normal cable because breakout cables were not supported.
POAP determines which breakout map (for example, 10gx4, 50gx2, 25gx4, or 10gx2) will bring up the link connected to the DHCP server. If breakout is not supported on any of the ports, POAP skips the dynamic breakout process. After the breakout loop completes, POAP proceeds with the DHCP discovery phase as normal.
Note |
For more information on dynamic breakout, see the interfaces configuration guide for your device. |
Script Execution Phase
Once the device has bootstrapped itself using the information in the DHCP acknowledgement, the switch downloads the script file from the TFTP server or the HTTP server.
The switch runs the configuration script, which downloads and installs the software image and downloads a switch-specific configuration file.
However, the configuration file is not applied to the switch at this point, because the software image currently running on the switch might not support all of the commands in the configuration file. After the switch reboots, it begins running the new software image, if one was installed. At that point, the configuration is applied to the switch.
Note |
If the switch loses connectivity, the script stops, and the switch reloads its original software images and bootup variables. |
Post-Installation Reload Phase
The switch restarts and clears the existing configuration if any, and applies (replays) the configuration on the upgraded software image. Afterward, the switch copies the running configuration to the startup configuration.