Troubleshoot Smart License in Secure Web Appliance

Introduction

This document describes the steps to configure and troubleshoot Smart License in Secure Web Appliance (SWA).

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

• How Smart License works.
• Secure Web Appliance (SWA) administration.

Cisco recommends that you have:

• Physical or Virtual Secure Web Appliance (SWA) Installed.

• Administrative Access to the SWA.
• Access to Smart License portal.

Components Used

This document is not restricted to specific software and hardware versions.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Cisco Smart License 

Smart Licensing provides the ability to:

• Manage all of your product licenses from a central location
• Normalizes the process between Physical and Virtual Email Security Appliance (ESA) / Security Management Appliance (SMA) / SWA, with the use of one method to apply and manage licenses
• Easily apply a license to your ESA/SMA/SWA
• Receive Alerts related to license expiration
• Hardware model ESA/SMA/SWA, out of the box, have a 90-day Evaluation Period for all services

To register the SWA  with smart licensing, the owner of the appliance must have a Smart Account.

• Smart Accounts are issued one per domain.
• The administrator of the Smart Account can create sub-level Virtual Accounts that allow segregation of resources.
• Virtual Accounts can be used to restrict access to different Cisco Product Licenses, based on your needs.
• Access the Cisco Smart Software Manager (CSSM) to manage licenses and download tokens.

The links to resources provided by Cisco, include videos, guides, and explanations related to Smart Licensing:

• Create New Smart Account or Request to add a user to an existing account
• Smart Software Licensing Overview Cisco WebPage
• Smart Licensing Deployment Guide

• Cisco Smart Accounts Cisco Page

• Smart Software Manager Cisco Page

• Cisco Smart Software Manager (CSSM)
• Generate License Key File from PAK File for Email Security - Cisco

Smart Software Manager Satellite

Cisco Smart Software Manager satellite is a component of Cisco Smart Licensing.

CSSM Satellite works in conjunction with CSSM to manage product licenses, provide near realtime visibility and reporting of Cisco licenses in use.

For security reasons, if you do not want to manage the installed base with Smart Software Manager residing on Cisco.com, you can choose to install the Smart Software Manager satellite on premises.

For more information about Smart Software Manger Satellite, please visit this link : Cisco Smart Software Manager - Cisco .

Definitions Related to Smart License

License types:

• Classic License (CL) refers to the legacy methods used for both hardware and virtual licenses.
• Smart License (SL)

License Authorization Status: the status of a given license within the appliance.

• The ESA/SWA/SMA does not display the actual expiration date with the Smart Licenses page.
• Location: GUI > System Administration > Licenses.
• Location: CLI > license_smart > SUMMARY.

The status of a specific feature appears with one of these values:

• Eval:
  • SL Service has been enabled on a new (Hardware) ESA/SMA without token registration
  • SL Service has been enabled on an appliance with current CL installed
• Eval Expired: 90-Day Evaluation Smart License  has expired and the appliance has transitioned to the additional 30-day grace period
• In Compliance: The appliance has been registered with a token and currently the feature consumes a valid license
• Out of Compliance (Grace Period)can be observed in 2 scenarios:
  
  • One-click request for a temporary 30-day feature license is in use
  • A license has expired on the appliance and the 30-day grace period has initiated
• Out of Compliance (Expired): LIcense fully expired and the associated service ceases to function

Note: A Perpetual key indicates that there is no expiration period for that feature. A Dormant key indicates that the feature itself has an End User License Agreement (EULA) that must be accepted, or that the feature must be configured and enabled. Once completed, the feature moves to Active, and the expiration timer begins.

Configure Smart license in Secure Web Appliance 

You can Connect SWA to Smart license via both Graphical User interface (GUI) and Command Line Interface (CLI).

Before You begin

Caution: Enablement of the Smart License Feature on the ESA/SMA/SWA is permanent and does not permit the option to revert an appliance back to Classic License Mode.

• All hardware model SWA purchased include 90 days Evaluation Licenses for all features.
• All hardware models that migrate with current Classic Licenses (CL) to Smart license, receive 90 days Evaluation licenses.
• All Virtual SWA models require a basic Virtual License (VLN) which is an XML file, loaded to the appliance from Command Line Interface (CLI) loadlicense command, to link to the upgrade/update server.
• All Virtual SWA models, when created, do NOT include 90 days licenses and require registration by the Classic License VLN  (sometimes called XML) file.
• All Virtual SWA models that migrate with current Classic Licenses (CL) include 90 days Evaluation licenses.
• Currently SWA has Smart License in all release builds as an optional feature.
• Once Smart License is enabled, then Classic License is not used on that box.
• Classic License cannot be retained once Smart License is enabled.
• In SWA version 15.0 and newer release build, there is an option to directly enable Smart License without Classic License enablement. Here, the licenses need to be purchased and configured to use the SWA with Smart License.
• From 15.0 release build, there is a 30 days grace period for fresh installation and registration of Smart License without any Classic License activation.
• In the latest release build of 15.1 release, Smart License is mandatory to enable for SWA to function properly. Also, all Classic License-related features are removed in the latest release.
• Upgrade to Smart License mandate builds of SWA is restricted at the time of download if Smart Licese is not enabled in the base build.

Changes in CLI and GUI display

In CLI for Classic License four commands were used. Hence, in Smart License Mandate builds (15.1 and newer) those commands are removed.

List of CLI commands removed:

• loadlicense
• showlicense
• featurekey
• featurekeyconfig

In GUI for Classic License, two pages are mentioned in System Administration tab. Hence, in Smart License Mandate builds those, pages are removed.
List of GUI pages removed:

• Feature key settings
• Feature key

Reset and Reload

Reset Configuration in SWA is to perform factory reset where the entire configuration is wiped out and SWA reverts to its factory state.

With Smart License mandate build also same behavior is retained.

Reload is a CLI hidden command that wipes out the configuration data and removes the feature keys as well. If SWA was registered with Classic License and perform reload, load the license again.

If SWA was configured with Smart License, then after the reload, the Smart License is de-registered and disabled along with factory reset in current SWA behavior.

In SWA mandate build versions, Smart License never reverts to the disable state, hence the reload command wipes out all configuration.

The smart license stays in the registered state, therefore, request all licenses again.

Communication Requirements

Network or Proxy communication to smartreceiver.cisco.com on TCP port 443.

To test the connectivity from SWA, use these steps:

Step 1. Log-in to CLI.

Step 2. Type telnet and press Enter.

Step 3. Choose the interface you are expecting SWA connect to Smart License server.

Step 4. type smartreceiver.cisco.com and press enter.

Step 5. In the port section, type 443 and press enter.

Note: If you have configured Smart Software Manger Satellite, please add the Uniform Resource Locator (URL) or Internet Protocol (IP) address associated to that server in step 4.

Here is the sample of successful connection :

> telnet

Please select which interface you want to telnet from.
1. Auto
2. Management (10.48.48.184/24: management.swa1.cisco.com)
3. P1 (192.168.13.184/24: p1.swa1.cisco.com)
4. P2 (192.168.133.184/24: p2.swa1.cisco.com)
[1]> 4

Enter the remote hostname or IP address.
[]> smartreceiver.cisco.com

Enter the remote port.
[23]> 443

Trying 10.112.59.81...
Connected to smartreceiver.cisco.com.
Escape character is '^]'.

Here is the sample on failed connection: 

SWA_CLI> telnet

Please select which interface you want to telnet from.
1. Auto
2. Management (10.48.48.184/24: management.swa1.cisco.com)
3. P1 (192.168.13.184/24: p1.swa1.cisco.com)
4. P2 (192.168.133.184/24: p2.swa1.cisco.com)
[1]> 2

Enter the remote hostname or IP address.
[]> smartreceiver.cisco.com

Enter the remote port.
[23]> 443

Trying 10.112.59.81...
telnet: connect to address 10.112.59.81: Operation timed out
Trying 2a04:e4c7:fffe::f...
bind: Invalid argument

Note: To exit telnet, if ctrl+c does not work, hold Control and then press ] then type q and press enter.

Configure Smart License from GUI

Step 1. Log in to GUI and navigate to System Administration.

Step 2. Choose Smart Software Licensing.

Image- Choose Smart Software Licensing

Step 3. Choose Enable Smart Software Licensing 

Image- Choose Enable Smart Software Licensing

Step 4. Please read the instruction carefully and choose OK. 

Image- Read the instructions carefully

Caution: You cannot roll back from Smart License to Classic License, after you enable Smart License feature on your appliance.

Step 5. Commit changes.

Step 6. Pause, then refresh the Smart Licensing page. 

Step 7. Select Smart License Registration and click Confirm

Image- choose Smart License Registration

Step 8.(Optional ) If you have Smart Software Manager satellite in your network, add the URL or IP address of the server in Transport Settings.  

Step 9.  If you have separate routing table, but you have no access to https://smartreceiver.cisco.com/ from Management interface, choose Data from Test Interface section. 

By default, Management Routing table is selected.

Image - choose Routing table

Step 10. Choose Register to navigate to registration page.

Step 11. Log in to your Smart Software Manager portal ( Cisco Software Central ) or your Smart Software Manager satellite.

Step 12. Navigate to Inventory tab and, if you have no Token yet, generate a new Token, or else click the blue arrow to view your token. 

Image - Navigate to Inventory

Step 13. (Optional) to Create Registration Token, choose New Token, and fill the required fields. 

Step 14. Paste the token from Smart License portal to your SWA and choose Register. 

Image - Paste the Registration Token

Step 15.  (Optional) If the device has already been registered, you can re-register the device if you select the check box. 

Step 16. After a few minutes you can check the registration status.

Image - Registered Appliance

Verify The Integration

You can verify the integration from GUI, CLI or Smart License Portal 

Verify Smart License Status From GUI

Step 1. Log in to GUI and navigate to System Administration.

Step 2. Choose Smart Software Licensing.

Step 3. Check these items :

• Registration Status
• License Authorization Status
• Last Registration Renewal Attempt Status
• Last Authorization Renewal Attempt Status

Image - Verify Smart License in GUI

Step 4. From System Administration menu, choose Licenses .

Step 5. Check the desired licenses are In Compliance.

Image -License Status

Verify Smart License Status From CLI

Use these steps to verify the Smart LIcense status from CLI:

Step 1. Log in to CLI 

Step 2. Type license_smart press Enter

Step 3. Choose STATUS

Step 4. Check these items :

• Registration Status
• License Authorization Status
• Last Registration Renewal Attempt Status
• Last Authorization Renewal Attempt Status

Smart Licensing is : Enabled

License Reservation is: Disabled
Evaluation Period: Not In Use
Evaluation Period Remaining: 89 days 22 hours 40 minutes
Registration Status: Registered ( 04 Sep 2023 20:38 ) Registration Expires on:  ( 03 Sep 2024 21:03 )
Smart Account: XXXXXXXXXXXX18.cisco.com
Virtual Account: XXXXXXXXX
Last Registration Renewal Attempt Status: SUCCEEDED on 04 Sep 2023 21:07
License Authorization Status: Authorized ( 04 Sep 2023 20:38 ) Authorization Expires on:  ( 03 Dec 2023 20:03 )
Last Authorization Renewal Attempt Status: SUCCEEDED on 04 Sep 2023 21:07
Product Instance Name: wsa125to15.amojarra.calo
Transport Settings: Direct (https://smartreceiver.cisco.com/licservice/license)
Device Led Conversion Status: Started

Step 5.  From license_smart wizard, choose SUMMARY.

[]> SUMMARY

Feature Name                                                        License Authorization Status
----------------------------------------------------------------------------------------------------
Secure Web Appliance Cisco Web Usage Controls                          In Compliance
Secure Web Appliance Anti-Virus Webroot                                In Compliance
Secure Web Appliance L4 Traffic Monitor                                In Compliance
Secure Web Appliance Cisco AnyConnect SM for AnyConnect                In Compliance
Secure Web Appliance Secure Endpoint Reputation                        In Compliance
Secure Web Appliance Anti-Virus Sophos                                 In Compliance
Secure Web Appliance Web Reputation Filters                            In Compliance
Secure Web Appliance Secure Endpoint                                   In Compliance
Secure Web Appliance Anti-Virus McAfee                                 Not requested
Secure Web Appliance Web Proxy and DVS Engine                          In Compliance
Secure Web Appliance HTTPs Decryption                                  In Compliance

Step 6. Check the desired licenses are In Compliance.

Verify Device Status in Smart License Portal

Step 1. Log in to Smart Software Licensing Portal : Cisco Software Central

Step 2. Choose to Inventory tab.

Step 3. Choose Product Instances. 

Step 4. Verify your device is listed and click on the device name.

Image - Verify Device Status in Smart License Portal

Step 5.  Observe the Feature Keys and device status in General tab

Image - Verify the Feature Keys in Smart License Portal

Find VLN from CLI

To view your VLN from CLI, use smartaccountinfo command. Also, you can view some extra information such as Virtual Account Domain or ID and Product Instances. 

> smartaccountinfo

Smart Account details
---------------------

Product Instance ID       : 609XXXXXXXX-fXXXXXXXXX55
Smart Account Domain      : XXXXXXXXXXXXXXXXXXX18.cisco.com
Smart Account ID          : 111111
Smart Account Name        : XXXXXXXXXXXXXXXXXXX18.cisco.com
VLN                       : VLNWSA1111111
Virtual Account Domain    : WSA_XXXXX
Virtual Account ID        : 111111

Logging 

All the logs related to Smart License are collected in Smartlicense logs. This log is Enabled by default.

Use these steps to configure the Smart License log:

Step 1. Log in to GUI.

Step 2. From System Administration Menu choose Log Subscriptions.

Step 3. Scroll down and find Smartlicense logs.

Step 4. Click on the log name to edit the configuration. 

Tip: If you want to push the logs to your log collector server, it is advised to create a new Log Subscriptions and forward those logs, to have copy of logs locally on the SWA

Trouble Shoot Smart License  

Here are common Errors and the steps to resolve the issue. 

Sussessfull connection 

Here is the sample of smart_license logs with successful result:

Mon Sep  4 20:39:32 2023 Info: The product is registered successfully with Smart Software Manager.

Registration Failed 

Image - Registration Failed

If Registration Failed is returned, check the smart_license logs from CLI using these steps: 

Step 1. Log in to CLI.

Step 2. Type grep and press Enter.

Step 3. Find the number associated with smartlicense logs and type the number, and press Enter.

Step 4. Hit Enter until you see the logs.

Communication send error

If you see "Communication send error", check the connectivity between SWA and Smart license Server on port TCP 443.

Mon Sep 4 19:57:09 2023 Warning: The registration of the product with Smart Software Manager failed. The response from Smart Software Manager is: Communication send error. 

Tip: If you configured Smart Software Manager satellite, please check the connectivity to the configured Port number.

To check the connectivity, Use the steps provided in "Communication Requirements" section in this article. 

Also in display Alerts you can see the same Error message :

04 Sep 2023 20:19:29 +0200    The registration of the product with Smart Software Manager failed. The response from Smart Software Manager is: Communication send error.

Token is not valid

If the Token has expired or reached its maximum defined Used value, the warning log Token is not valid is returned.

You can verify the Error from either the displayalerts command or from smartlicense logs. 

Here is a sample of error from displayalerts in CLI:

04 Sep 2023 20:26:55 +0200    The registration of the product with Smart Software Manager failed. The response from Smart Software Manager is: Token is not valid

Here is a sample of log line from smartlcese logs from CLI:

Mon Sep 4 20:26:55 2023 Warning: The registration of the product with Smart Software Manager failed. The response from Smart Software Manager is: Token is not valid

To verify the token validity log in to your Smart License portal, navigate to inventory, check the expiration status and number of use.

Image - Expier Token

Failed To Renew Authorization

Communication Send Error

If you get Failed to renew authorization due to Communication send error, this could be due to the. connectivity issue, Make sure the correct Routing Table is selected and test the connectivity between SWA and smartreceiver.cisco.com TCP port 443 or your Smart Software Manager satellite server 

To check the connectivity, use the steps provided in "Communication Requirements" section in this article. 

You can verify the Error with either the displayalerts command or from smartlicense logs 

Here is a sample of error from displayalerts in CLI:

04 Sep 2023 22:23:43 +0200 Failed to renew authorization of the product with Smart Software Manager due to Communication send error..

Here is a sample of log line from smartlcese logs from CLI:

Mon Sep 4 22:22:58 2023 Warning: Failed to renew authorization of the product with Smart Software Manager due to Communication send error..

The Certificate is REVOKED

If the reason to renew authorization is the certificate is REVOKED, please check to see if the device has been removed from Smart License Portal.

Check "Verify Device Status in Smart License Portal" section in this article. 

Verify the Error using either the displayalerts command or from smartlicense logs 

Here is a sample of error from displayalerts in CLI:

04 Sep 2023 22:39:10 +0200 Failed to renew authorization of the product with Smart Software Manager due to Could not return the certificate for the given sn (111111111) since it is REVOKED..

Here is a sample of log line from smartlcese logs from CLI:

Mon Sep 4 22:39:10 2023 Warning: Failed to renew authorization of the product with Smart Software Manager due to Could not return the certificate for the given sn (1111111) since it is REVOKED..

To solve this issue, register the device again. 

No VLN in the device 

For physical devices, there is no VLN; the Virtual Licese Number is omly used in the virtual appliances.

If you are using a virtual SWA and there is no VLN in the output of smartaccountinfo in CLI, try to load the XML license file again with loadlicense command in CLI. 

Caution: The loadlicense command removes all existing feature key (includes evaluation keys) and the license file from the system prior to installs the new license file and keys.

Smart license agent service is unavailable

If you get this error message, this is due to a known Cisco bug ID "CSCwe36665" for SWA or Cisco Bug ID "CSCvo22855" for ESA. Please contact TAC to implement the workaround.

"Smart license agent service is unavailable. Please visit this page after some time. If you continue to see the same message, please contact Cisco Sales representative."

Failed to verify signature

If the  Smart License authorization fails with the error :

Tue Apr 22 09:46:27 2023 Warning: Failed to renew authorization of the product with Smart Software Manager due to Failed to verify signature..

[First test] This error could be due to a known Cisco bug ID CSCvx04164  .

The condition for this bug is the Virtual Account name on the Smart Licensing portal contains non-English characters, and the workaround for this issue is:

Rename the Virtual account and remove the non-English characters:

Step 1. Go to software.cisco.com.
Step 2. Navigate to Administration > Manage Smart Account > Virtual Accounts.
Step 3. Click on the virtual account in question.
Step 4. Define a new name and remove the non-English characters.

Note: User must have administrative privilege to be able to rename the virtual account.

[Second test]  If the Virtual Account name is correct, please make sure the device is listed in the Smart License portal inventory.

Use the steps provided in section "Verify Device Status in Smart License Portal" in this article.  

[Third test] If the device is listed in the Smart License Portal inventory, try to restart the SWA Smart License service from CLI:

Step 1. Log in to CLI.

Step 2. Run the diagnostic  command

Step 3. Choose SERVICES 

Step 4. Choose SMART_LICENSE 

Step 5. Choose RESTART

SWA_CLI> diagnostic

Choose the operation you want to perform:
- NET - Network Diagnostic Utility.
- PROXY - Proxy Debugging Utility.
- REPORTING - Reporting Utilities.
- SERVICES - Service Utilities.
[]> SERVICES

Choose one of the following services:
- AMP - Secure Endpoint
- AVC - AVC
- ADC - ADC
- DCA - DCA
- WBRS - WBRS
- EXTFEED - ExtFeed
- L4TM - L4TM
- ANTIVIRUS - Anti-Virus xiServices
- AUTHENTICATION - Authentication Services
- MANAGEMENT - Appliance Management Services
- REPORTING - Reporting Associated services
- MISCSERVICES - Miscellaneous Service
- OCSP - OSCP
- UPDATER - UPDATER
- SICAP - SICAP
- SNMP - SNMP
- SNTP - SNTP
- VMSERVICE - VM Services
- WEBUI - Web GUI
- SMART_LICENSE - Smart Licensing Agent
- WCCP - WCCP
[]> SMART_LICENSE


Choose the operation you want to perform:
- RESTART - Restart the service
- STATUS - View status of the service
[]> RESTART

smart_agent is restarting.

[Forth test] Generate a new Token in Smart License Manager Portal and re-register the device.

Smart_agent stuck in disabled state 

These errors can be seen on a ESA or SMA after upgrading the appliance (which the Smart License was enabled before upgrade) to version 14.1 or 14.0.

Note: This error seen on x195 or x395 devices.

Here is a sample of the message generated by the appliance 

08 Apr 2023 10:19:36 -0500 Initialization of smart agent service failed. Reason : Port 65501 is not available for smart agent service to run. Please try changing port for smart agent service through `license_smart setagentport` cli command or free the port from other service.

And in  smart_license logs you can see:

Mon Apr 8 09:02:36 2021 Warning: Smart License: Failed to change the hostname to esa.local for the product.

This error is due to a known Cisco bug ID  CSCvz74874 for ESA and Cisco bug ID CSCvx68947  for SMA. you need to contact Cisco support to resolve this issue.

Initialization of smart agent service failed

This error is mostly related to virtual appliances which are configured with more resources than expected.

Here is a sample of the log:

Thu Jun 23 16:16:07 2022 Critical: Initialization of smart agent service failed. Reason : Port 65501 is not available for smart agent service to run. Please try changing port for smart agent service through `license_smart setagentport` cli command or free the port from other service.
any attempts to swap ports using the defined command will fail.

To fix this issue, check the output of version command in CLI and make sure the number of CPUs and the allocated memory is set to expected.

If there are more cores to the appliance that are supported, correct the allocation. 

Invalid Response From Licensing Cloud

If the device has been removed from Smart License Manager Portal, older versions return this error, 

Thu Nov 15 13:50:20 2022 Warning: Failed to renew authorization of the product with Smart Software Manager due to Invalid response from licensing cloud..

To fix this issue, please re-register the appliance.

Also, please check Cisco bug ID CSCvr09743

No valid SSL certificate was sent

If you get this error from your appliance and you are unable to get the updates, refer to Field Notice: FN - 72502 for further information. 

21 Aug 2023 14:03:04 +0200    Unable to connect to the Cisco Aggregator Server.
Details: No valid SSL certificate was sent

Traditional VLN certificate files include a certificate created by Talos Keymaster for access to updates and upgrades. The old Keymaster certificate authority (CA) expired on January 13, 2023.

VLN certificate files with certificates issued prior to December 15, 2021, with a validity of more than 12 months, must be renewed and applied prior to January 13, 2023.

To solve this issue, please contact Cisco license support and ask for a new VLN file. 

Feature  has been moved to Out of Compliance

If you see some logs that one or some of your Features have been moved to Out of Compliance, please check:

1. You have a Valid License 
2. Appliance has connectivity with the Smart License server ( See "Communication Requirements " section in this article) 
3. Check the Smart License logs for further sight on the issue.
4. Contact Cisco Support for assistance 

Here is a sample log:

Mon Sep 4 20:41:09 2023 Warning: Secure Web Appliance HTTPs Decryption license has been moved to Out of Compliance successfully.
Mon Sep 4 20:41:10 2023 Warning: The Secure Web Appliance HTTPs Decryption is in Out of Compliance (OOC) state. You have 29 days remaining in your grace period.

Smart Agent is in Authorization Expired state

If you get the Critical error as "Smart Agent is in Authorization Expired state", please review the next lines to find the reasons for this state.

Here is a sample of error:

Fri Aug 18 15:51:11 2023 Critical: Web Security Appliance Cisco Web Usage Controls feature will stop working as Smart Agent is in Authorization Expired state. This can happen if there is no communication between the appliance and the Cisco Smart Software Manager (CSSM) for more than 90 days.

Please check the connectivity and make sure your device is registered in Smart License Portal.  

Reference 

Cisco Web Security Appliance Best Practices Guidelines - Cisco

BRKSEC-3303 (ciscolive)

User Guide for AsyncOS 14.5 for Cisco Secure Web Appliance - GD (General Deployment) - Connect, Install, and Configure [Cisco Secure Web Appliance] - Cisco

Cisco Smart Software Manager - Cisco