Troubleshoot APIC NXOS Style CLI Commands

Available Languages

Download Options

PDF (143.4 KB)
View with Adobe Reader on a variety of devices
ePub (167.8 KB)
View in various apps on iPhone, iPad, Android, Sony Reader, or Windows Phone
Mobi (Kindle) (106.7 KB)
View on Kindle device or Kindle app on multiple devices

Updated:October 17, 2022

Document ID:218326

Bias-Free Language

The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.

Introduction

This document describes steps to debug commands run from the APIC CLI.

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

ACI REST API
ACI Object Model

The reader must have a previous knowledge on how the works, also how the DME process logs their messages.

These documents explain in more detail about the ACI APIC and Object model:

https://developer.cisco.com/docs/aci/

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/policy-model-guide/b-Cisco-ACI-Policy-Model-Guide.html

Background Information

The Cisco Application policy Controller contains a single Northbound API which is used for Policy management.

Every policy based interaction with an APIC boils down to an HTTP/S API Request/Responded interaction. This holds true for the GUI, custom python scripts and the CLI Commands that exist on both the APICs and Switches.

This Image summarizes the way users and tools interact with the APICs API

Interacting with the APIC API

All show commands run on an APIC invoke the NXOS Style CLI. These commands trigger a series of python scripts which build out the API requests required to gather the information requested. When a response is received, it is parsed with python then delivered to the user in a pretty format.

NGINX

NGINX is an open source web server. Each APIC runs its own NGINX process which serves the RESTful API. On an APIC, NGINX includes logging via both the /var/log/dme/log/nginx.bin.log and /var/log/dme/log/access.log files.

The nginx.bin.log file shows the details of all API requests and DME interactions.

The access.log file logs every API request handled by NGINX.

As all API requests arel HTTP requests, standard HTTP response codes can be referenced for NGINX API Call handling:

200 code is OK
4XX codes are Client Errors
5XX Codes are Server Errors, which is the APIC in this case

The Decoy Service

The Decoy service serves a special API call via a python module hosted off of NGINX.

This service:

Accepts the requested CLI command
Identifies the python scripts requires to build out the REST API calls
Sends the API calls to the RESTful API
Accepts the response
Formats the response
Serves the formatted response to the user.

This service includes these log files:

/var/log/dme/log/decoy.log
/var/log/dme/log/decoy.error.log
/var/log/dme/log/decoy_server.log

The decoy.log file registers the CLI Commands that were run.

The access.log file from nginx uses the “POST /decoy/exec/cmd.cli HTTP/1.1” format along with the HTTP code associated to the request. . The file logs the REST API calls from the command.

Note: The nginx and decoy logs mentioned are collected within the APIC 3of3 Techsupport.

Troubleshoot the NXOS Style CLI

Trace the CLI Command through Logs

Run the CLI Command

For this example, the “show controller” is issued via the APIC CLI:

APIC-1# show controller
Fabric Name          : ACI-POD1
Operational Size     : 2
Cluster Size         : 3
Time Difference      : 0
Fabric Security Mode : PERMISSIVE

 ID    Pod   Address          In-Band IPv4     In-Band IPv6               OOB IPv4         OOB IPv6                        Version             Flags  Serial Number     Health
 ----  ----  ---------------  ---------------  -------------------------  ---------------  ------------------------------  ------------------  -----  ----------------  ------------------
 1*    1     10.0.0.1         0.0.0.0          fc00::1                    192.168.1.1                                      4.2(6h)             crva-  XXXXXXXXXXX       fully-fit
 2     1     10.0.0.2         0.0.0.0          fc00::1                    192.168.1.1                                      4.2(6h)             crva-  XXXXXXXXXXX       fully-fit
Flags - c:Commissioned | r:Registered | v:Valid Certificate | a:Approved | f/s:Failover fail/success
(*)Current (~)Standby (+)AS

The command returns a pretty formatted output.

Check decoy.log for CMD Execution

The decoy.log file can be checked to find the "show controller" CMD that was invoked:

APIC-1# tail /var/log/dme/log/decoy.log
...
...||AUTH COOKIE="XXXXXX"||...
...||CLI: {"option": "server", "loglevel": ["disable"], "cols": 171, "mode": [["exec"]], "port": 51719, "cli": ["show", "controller"]}||...
...||port: 51719||...
...||Mode: [[u'exec']]||...
...||Command: [u'show', u'controller']||...
...||CommandCompleter: add exec||...
...||CommandCompleter: add show||...
...||CommandCompleter: add controller||...
...||last tokens: ['show', 'controller']||...
...||modeCmd: Mode: exec, fulltree: False teminal context is : {'mode-module': 'yaci._cfgroot', 'module': 'show._controllers', 'inherited': False}||.
...||('_%CMD_TERM%_', {'mode-module': 'yaci._cfgroot', 'module': 'show._controllers', 'inherited': False}, {'prompt': '# ', 'mode': [[u'exec']]}, None)||...
...||terminal command module: {"mode-module": "yaci._cfgroot", "module": "show._controllers", "inherited": false}|...

The CLI sends this as a dictionary, we can see it from the highlighted line. The formatted version would look like this:

{
    "option": "server",
    "loglevel": [
        "disable" <-- Can be modified to debug the interaction further.
    ],
    "cols": 171,
    "mode": [     <-- Command ran by admin
        [
            "exec"
        ]
    ],
    "port": 51719, <-- Random TCP port from session.
    "cli": [      <-- Actual command
        "show",
        "controller"
    ]
}

The line with ‘_%CMD_TERM%_’ shows the command executed, along with the module used, in this example:

('_%CMD_TERM%_', {'mode-module': 'yaci._cfgroot', 'module': 'show._controllers', 'inherited': False}, {'prompt': '# ', 'mode': [[u'exec']]}, None)

The CLI module must then translate it to REST API calls.

Check access.log for API Calls

The access.log file displays the API Calls received after decoy processes the CMD:

APIC-1# tail /var/log/dme/log/access.log  
...
127.0.0.1 - - [24/May/2021:18:43:12 +0000] "POST /decoy/exec/cmd.cli HTTP/1.1" 200 0 "-" "python-requests/2.7.0..."
127.0.0.1 - - [24/May/2021:18:43:19 +0000] "GET /api/mo/topology/pod-1/node-1/sys.xml HTTP/1.1" 200 1273 "-" "python-requests/2.7.0..."
127.0.0.1 - - [24/May/2021:18:43:19 +0000] "GET /api/mo/uni/fabsslcomm/ifmcertnode-1.xml HTTP/1.1" 200 2391 "-" "python-requests/2.7.0..."
127.0.0.1 - - [24/May/2021:18:43:19 +0000] "GET /api/mo/uni/fabsslcomm/ifmcertnode-2.xml HTTP/1.1" 200 2508 "-" "python-requests/2.7.0..."
127.0.0.1 - - [24/May/2021:18:43:19 +0000] "GET /api/mo/topology/pod-1/node-2/sys.xml HTTP/1.1" 200 1265 "-" "python-requests/2.7.0..."

Check nginx.bin.log

The nginx.bin.log file includes additional details on all API Requests handled and must include information about the payload received from various DMEs before it is sent back to the requester.

In the same file, after decoy/exec/cmd.cli is called, there are several API calls logged:

admin@APIC-1:log> cat nginx.bin.log| grep "ifmcertnode|sys.xml"
17567||2021-05-24T18:43:19.817094383+00:00||nginx||DBG4||||Request received /api/mo/topology/pod-1/node-1/sys.xml||../common/src/rest/./Rest.cc||67   bico 11.322
17567||2021-05-24T18:43:19.817184335+00:00||nginx||DBG4||||httpmethod=1; from 127.0.0.1; url=/api/mo/topology/pod-1/node-1/sys.xml; url options=||../common/src/rest/./Request.cc||133

17567||2021-05-24T18:43:19.817409193+00:00||nginx||DBG4||||Request received /api/mo/topology/pod-1/node-2/sys.xml||../common/src/rest/./Rest.cc||67
17567||2021-05-24T18:43:19.817466216+00:00||nginx||DBG4||||httpmethod=1; from 127.0.0.1; url=/api/mo/topology/pod-1/node-2/sys.xml; url options=||../common/src/rest/./Request.cc||133

17567||2021-05-24T18:43:19.817589102+00:00||nginx||DBG4||||Request received /api/mo/uni/fabsslcomm/ifmcertnode-1.xml||../common/src/rest/./Rest.cc||67
17567||2021-05-24T18:43:19.817641070+00:00||nginx||DBG4||||httpmethod=1; from 127.0.0.1; url=/api/mo/uni/fabsslcomm/ifmcertnode-1.xml; url options=||../common/src/rest/./Request.cc||133

17567||2021-05-24T18:43:19.819268449+00:00||nginx||DBG4||||Request received /api/mo/uni/fabsslcomm/ifmcertnode-2.xml||../common/src/rest/./Rest.cc||67
17567||2021-05-24T18:43:19.819340589+00:00||nginx||DBG4||||httpmethod=1; from 127.0.0.1; url=/api/mo/uni/fabsslcomm/ifmcertnode-2.xml; url options=||../common/src/rest/./Request.cc||133
...

Re-run the Individual API Calls

The access.logs contain a list of exactly which API calls were sent for processing.

The goal of this step is to re-run each API call to determine:

Is there an issue with the API call itself?
Is there something different about a successful API call vs a failed one?

For example, one of the API calls generated from the "show controllers" command is:

127.0.0.1 - - [24/May/2021:18:43:19 +0000] "GET /api/mo/topology/pod-1/node-1/sys.xml HTTP/1.1" 200 1273 "-" "python-requests/2.7.0..."

Via icurl

This API Request can be re-run with the use icurl on an APIC:

icurl 'http://localhost:7777/<API-CALL-from-ACCESS.log>'

Note: Port 7777 is specifically used to allow the APIC to query itself.

With the specific call as an example:

APIC-1# bash
admin@APIC-1:~> icurl 'http://localhost:7777/api/mo/topology/pod-1/node-1/sys.xml'

<?xml version="1.0" encoding="UTF-8"?>
<imdata totalCount="1">
   <topSystem address="10.0.0.1" bootstrapState="none" childAction="" dn="topology/pod-1/node-1/sys" ... />
</imdata>

The amount and complexity of API calls required for each NXOS CLI command can differ greatly. s. In all cases, the queries can be re-played via icurl for request and response validation.

Via moquery

From the previous API calls shown, the module parses the output for the ‘show controller’ command with the information from the MOs requested.

admin@APIC-1:~> moquery -d topology/pod-1/node-1/sys -o xml
...
  <topSystem address="10.0.0.1" dn="topology/pod-1/node-1/sys" fabricDomain="ACI-POD1"
  id="1" inbMgmtAddr="0.0.0.0" inbMgmtAddr6=""
  oobMgmtAddr="192.168.1.1" oobMgmtAddr6="" podId="1" serial="..."
  state="in-service" version="4.2(6h)"/>


admin@APIC-1:~> moquery -d topology/pod-1/node-2/sys -o xml
...
  <topSystem address="10.0.0.2" dn="topology/pod-1/node-2/sys" fabricDomain="ACI-POD1"
  id="2" inbMgmtAddr="0.0.0.0" inbMgmtAddr6=""
  oobMgmtAddr="192.168.1.2" oobMgmtAddr6="" podId="1" serial="..."
  state="in-service" version="4.2(6h)"/>


admin@APIC-1:~> moquery -d uni/fabsslcomm/ifmcertnode-1  -o xml
...
  <pkiFabricNodeSSLCertificate nodeId="1" serialNumber="..." subject="/serialNumber=PID:APIC-SERVER-M2 SN:..." .../>


admin@APIC-1:~> moquery -d uni/fabsslcomm/ifmcertnode-2  -o xml
...
  <pkiFabricNodeSSLCertificate nodeId="2" serialNumber="..." subject="/serialNumber=PID:APIC-SERVER-M2 SN:..." .../>

Re-run the CLI Command via Python

As mentioned, all the "show" commands are actually a REST API call to a series of python scripts.

Due to this, a user can technically manually invoke the python script which runs the cmd. The goal here is to see if Python gives more information on why a specific CMD has issues.

For any "show" command that has an issue, invoke the command via Python to compare a successful vs a failed APIC:

apic1# ${PYTHON} -m pyclient.remote exec terminal ${COLUMNS} <some show command>

Example runs:

apic1# ${PYTHON} -m pyclient.remote exec terminal ${COLUMNS} show switch 
apic1# ${PYTHON} -m pyclient.remote exec terminal ${COLUMNS} show controller

Example where the direct Python script gives additional debug information on a failure:

a-apic1# ${PYTHON} -m pyclient.remote exec terminal ${COLUMNS} show switch

Process Process-2:
Traceback (most recent call last):
File "/usr/lib64/python2.7/multiprocessing/process.py", line 267, in _bootstrap
self.run()
File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/controller/yaci/execmode/show/_switch_nodes.py", line 75, in _systemQuery
mo = ctx.moDir.lookupByDn(dn)
File "/controller/ishell/cobra/mit/access.py", line 80, in lookupByDn
mos = self.query(dnQuery)
File "/controller/yaci/pyclient/local.py", line 36, in query
raise e
QueryError: Unable to deliver the message, Resolve timeout from (type/num/svc/shard) = switch:205:4:0

ID   Pod      Address       In-Band IPv4      In-Band IPv6            OOB IPv4             OOB IPv6                Version       Flags  Serial Number         Name 
---- ---- --------------- --------------- ------------------------- --------------- ------------------------- ------------------ ----- ---------------- ------------------ 
101    1     10.0.40.65    192.168.2.231             ::              192.168.1.101             ::               n9000-14.2(6o)   aliv      XXXXXXXXXXX       leaf101 
...
205                              ...<empty>...                                                                                   al-v      XXXXXXXXXXX       leaf205 
206    2     10.0.156.65   192.168.2.236             ::              192.168.1.106             ::               n9000-14.2(6l)   aliv      XXXXXXXXXXX       leaf206 
...

Flags - a:Active | l/s:Leaf/Spine | v:Valid Certificate | i:In-Service

Trace Object Modification via APIC CLI

Object Creation

The APIC utilizes configure terminal to modify the MIT in an NXOS-Like fashion.

The results from these operations can can be seen in the decoy.log file.

VRF Creation Example:

APIC-1# configure terminal
APIC-1(config)# tenant TestTn
APIC-1(config-tenant)# vrf context Test-CTX-1
APIC-1(config-tenant-vrf)#

Output from decoy.log:

'18279||2021-05-25 22:52:11,877||decoy||INFO||AUTH COOKIE="eyJhbGciOiJSUzI1NiIsImtpZCI6ImgycDE1NmdjYXIxMG50emx3YmR3MDFjNnV5dGEycXQzIiwidHlwIjoiand0In0.eyJyYmFjIjpbeyJkb21haW4iOiJhbGwiLCJyb2xlc1IiOjEsInJvbGVzVyI6MX1dLCJpc3MiOiJBQ0kgQVBJQyIsInVzZXJuYW1lIjoiYWRtaW4iLCJ1c2VyaWQiOjE1Mzc0LCJ1c2VyZmxhZ3MiOjQsImlhdCI6MTYyMTI4MTY2NiwiZXhwIjoxNjIxMjgyMjY2LCJzZXNzaW9uaWQiOiJUWW9pZGgyMFRIeUc5YldMUllhb0FnPT0ifQ.ksnCeOxnrNQeuNaQnmpauUG_eja70nVtaCbamxFaB1LLkMIqzJ_wk_GMN1h4eM1WLS41VraukWw8Fztd281eaSQPPWiT-ieCjWxim8Sw4spYS8XBrBBx62tot201TIEJ8mUFHUjvXpPctDsBYi9YM5lUmFxhZgYI2Lx8gg0P6sLoUydcShKKcUNgRmGWwO64LH7rMEpzyCTapJBXdkzUhJ-zm98fmOy1oGGTesBteSWP_ksH14Xq411k1ebJ83sV4tL6-FJLhcPNIKwqYJ87fqUWwZFZb5tY4JUJxrSnahKfwyidNXt5m8LCIc8pt-xbBtVihAFkAYBoXKI-OYBrwg"||/mgmt/opt/controller/decoy/decoy/_app.py||32'
'18279||2021-05-25 22:52:11,878||decoy||INFO||CLI: {"option": "server", "loglevel": ["disable"], "cols": 100, "mode": [["exec"], ["configure", "terminal"]], "port": 60003, "cli": ["tenant", "TestTn"]}||/mgmt/opt/controller/decoy/apps/execserver/execapp.py||88'
'18279||2021-05-25 22:52:11,878||decoy||INFO||port: 60003||/mgmt/opt/controller/decoy/apps/execserver/execapp.py||107'
'18279||2021-05-25 22:52:11,879||decoy||INFO||Mode: [[u'exec'], [u'configure', u'terminal']]||/mgmt/opt/controller/decoy/apps/execserver/execapp.py||120'
'18279||2021-05-25 22:52:11,879||decoy||INFO||Command: [u'tenant', u'TestTn']||/mgmt/opt/controller/decoy/apps/execserver/execapp.py||121'
'18279||2021-05-25 22:52:11,879||decoy||DEBUG||CommandCompleter: add exec||/mgmt/opt/controller/yaci/yaci/_completer.py||255'
'18279||2021-05-25 22:52:11,880||decoy||DEBUG||CommandCompleter: add configure||/mgmt/opt/controller/yaci/yaci/_completer.py||255'
'18279||2021-05-25 22:52:11,880||decoy||DEBUG||OptionKeyword: add terminal||/mgmt/opt/controller/yaci/yaci/_completer.py||1036'
'11818||2021-05-25 22:52:11,910||decoy||DEBUG||CommandCompleter: add tenant||/mgmt/opt/controller/yaci/yaci/_completer.py||255'
'11818||2021-05-25 22:52:11,911||decoy||DEBUG||ArgCompleter: add TestTn||/mgmt/opt/controller/yaci/yaci/_completer.py||531'
'11818||2021-05-25 22:52:11,912||decoy||INFO||last tokens: ['tenant', u'TestTn']||/mgmt/opt/controller/yaci/yaci/_ctx.py||617'
'11818||2021-05-25 22:52:11,912||decoy||INFO||TID: 98||CLI Command: 'tenant TestTn'||/mgmt/opt/controller/yaci/yaci/_transaction.py||67'

'11818||2021-05-25 22:52:14,331||decoy||DEBUG||[commit]: <?xml version="1.0" encoding="UTF-8"?>
        <fvTenant name='TestTn' status='created,modified'></fvTenant>||/mgmt/opt/controller/yaci/yaci/_ctx.py||1024'
'11818||2021-05-25 22:52:14,349||decoy||INFO||CLIENT-HEADERS: {'APIC-Client': u'tenant TestTn', 'Cookie': 'APIC-cookie=eyJhbGciOiJSUzI1NiIsImtpZCI6ImgycDE1NmdjYXIxMG50emx3YmR3MDFjNnV5dGEycXQzIiwidHlwIjoiand0In0.eyJyYmFjIjpbeyJkb21haW4iOiJhbGwiLCJyb2xlc1IiOjEsInJvbGVzVyI6MX1dLCJpc3MiOiJBQ0kgQVBJQyIsInVzZXJuYW1lIjoiYWRtaW4iLCJ1c2VyaWQiOjE1Mzc0LCJ1c2VyZmxhZ3MiOjQsImlhdCI6MTYyMTI4MTY2NiwiZXhwIjoxNjIxMjgyMjY2LCJzZXNzaW9uaWQiOiJUWW9pZGgyMFRIeUc5YldMUllhb0FnPT0ifQ.ksnCeOxnrNQeuNaQnmpauUG_eja70nVtaCbamxFaB1LLkMIqzJ_wk_GMN1h4eM1WLS41VraukWw8Fztd281eaSQPPWiT-ieCjWxim8Sw4spYS8XBrBBx62tot201TIEJ8mUFHUjvXpPctDsBYi9YM5lUmFxhZgYI2Lx8gg0P6sLoUydcShKKcUNgRmGWwO64LH7rMEpzyCTapJBXdkzUhJ-zm98fmOy1oGGTesBteSWP_ksH14Xq411k1ebJ83sV4tL6-FJLhcPNIKwqYJ87fqUWwZFZb5tY4JUJxrSnahKfwyidNXt5m8LCIc8pt-xbBtVihAFkAYBoXKI-OYBrwg', 'Client_Name': 'APIC-CLI', 'Request-Tag': 'tag0'}||/mgmt/opt/controller/decoy/apps/execserver/execapp.py||31'
'11818||2021-05-25 22:52:14,350||decoy||INFO||Starting new HTTP connection (1): 127.0.0.1||/mgmt/opt/controller/decoy/decoy-env/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py||203'
'11818||2021-05-25 22:52:14,381||decoy||DEBUG||"POST /api/mo/uni/tn-TestTn.xml HTTP/1.1" 200 70||/mgmt/opt/controller/decoy/decoy-env/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py||383'
'18279||2021-05-25 22:52:14,383||decoy||DEBUG||(None, {}, {'prompt': '(config-tenant)# ', 'mode': [[u'exec'], [u'configure', u'terminal'], ['tenant', u'TestTn']]}, None)||/mgmt/opt/controller/decoy/apps/execserver/execapp.py||144'

'18280||2021-05-25 22:52:48,569||decoy||INFO||AUTH COOKIE="eyJhbGciOiJSUzI1NiIsImtpZCI6ImgycDE1NmdjYXIxMG50emx3YmR3MDFjNnV5dGEycXQzIiwidHlwIjoiand0In0.eyJyYmFjIjpbeyJkb21haW4iOiJhbGwiLCJyb2xlc1IiOjEsInJvbGVzVyI6MX1dLCJpc3MiOiJBQ0kgQVBJQyIsInVzZXJuYW1lIjoiYWRtaW4iLCJ1c2VyaWQiOjE1Mzc0LCJ1c2VyZmxhZ3MiOjQsImlhdCI6MTYyMTI4MTY2NiwiZXhwIjoxNjIxMjgyMjY2LCJzZXNzaW9uaWQiOiJUWW9pZGgyMFRIeUc5YldMUllhb0FnPT0ifQ.ksnCeOxnrNQeuNaQnmpauUG_eja70nVtaCbamxFaB1LLkMIqzJ_wk_GMN1h4eM1WLS41VraukWw8Fztd281eaSQPPWiT-ieCjWxim8Sw4spYS8XBrBBx62tot201TIEJ8mUFHUjvXpPctDsBYi9YM5lUmFxhZgYI2Lx8gg0P6sLoUydcShKKcUNgRmGWwO64LH7rMEpzyCTapJBXdkzUhJ-zm98fmOy1oGGTesBteSWP_ksH14Xq411k1ebJ83sV4tL6-FJLhcPNIKwqYJ87fqUWwZFZb5tY4JUJxrSnahKfwyidNXt5m8LCIc8pt-xbBtVihAFkAYBoXKI-OYBrwg"||/mgmt/opt/controller/decoy/decoy/_app.py||32'
'18280||2021-05-25 22:52:48,571||decoy||INFO||CLI: {"option": "server", "loglevel": ["disable"], "cols": 100, "mode": [["exec"], ["configure", "terminal"], ["tenant", "TestTn"]], "port": 60821, "cli": ["vrf", "context", "Test-CTX-1"]}||/mgmt/opt/controller/decoy/apps/execserver/execapp.py||88'
'18280||2021-05-25 22:52:48,571||decoy||INFO||port: 60821||/mgmt/opt/controller/decoy/apps/execserver/execapp.py||107'
'18280||2021-05-25 22:52:48,572||decoy||INFO||Mode: [[u'exec'], [u'configure', u'terminal'], [u'tenant', u'TestTn']]||/mgmt/opt/controller/decoy/apps/execserver/execapp.py||120'
'18280||2021-05-25 22:52:48,572||decoy||INFO||Command: [u'vrf', u'context', u'Test-CTX-1']||/mgmt/opt/controller/decoy/apps/execserver/execapp.py||121'
'18280||2021-05-25 22:52:48,572||decoy||DEBUG||CommandCompleter: add exec||/mgmt/opt/controller/yaci/yaci/_completer.py||255'
'18280||2021-05-25 22:52:48,572||decoy||DEBUG||CommandCompleter: add configure||/mgmt/opt/controller/yaci/yaci/_completer.py||255'
'18280||2021-05-25 22:52:48,573||decoy||DEBUG||OptionKeyword: add terminal||/mgmt/opt/controller/yaci/yaci/_completer.py||1036'
'18280||2021-05-25 22:52:48,574||decoy||DEBUG||CommandCompleter: add tenant||/mgmt/opt/controller/yaci/yaci/_completer.py||255'
'18280||2021-05-25 22:52:48,574||decoy||DEBUG||ArgCompleter: add TestTn||/mgmt/opt/controller/yaci/yaci/_completer.py||531'
'13482||2021-05-25 22:52:48,599||decoy||DEBUG||CommandCompleter: add vrf||/mgmt/opt/controller/yaci/yaci/_completer.py||255'
'13482||2021-05-25 22:52:48,600||decoy||DEBUG||Keyword: add context||/mgmt/opt/controller/yaci/yaci/_completer.py||983'
'13482||2021-05-25 22:52:48,601||decoy||DEBUG||ArgCompleter: add Test-CTX-1||/mgmt/opt/controller/yaci/yaci/_completer.py||531'
'13482||2021-05-25 22:52:48,602||decoy||INFO||last tokens: ['vrf', 'context', u'Test-CTX-1']||/mgmt/opt/controller/yaci/yaci/_ctx.py||617'
'13482||2021-05-25 22:52:48,602||decoy||INFO||TID: 105||CLI Command: 'vrf context Test-CTX-1'||/mgmt/opt/controller/yaci/yaci/_transaction.py||67'

'13482||2021-05-25 22:52:51,393||decoy||DEBUG||[commit]: <?xml version="1.0" encoding="UTF-8"?>
        <fvCtx status='created,modified' name='Test-CTX-1'></fvCtx>||/mgmt/opt/controller/yaci/yaci/_ctx.py||1024'
'13482||2021-05-25 22:52:51,405||decoy||INFO||CLIENT-HEADERS: {'APIC-Client': u'tenant TestTn; vrf context Test-CTX-1', 'Cookie': 'APIC-cookie=eyJhbGciOiJSUzI1NiIsImtpZCI6ImgycDE1NmdjYXIxMG50emx3YmR3MDFjNnV5dGEycXQzIiwidHlwIjoiand0In0.eyJyYmFjIjpbeyJkb21haW4iOiJhbGwiLCJyb2xlc1IiOjEsInJvbGVzVyI6MX1dLCJpc3MiOiJBQ0kgQVBJQyIsInVzZXJuYW1lIjoiYWRtaW4iLCJ1c2VyaWQiOjE1Mzc0LCJ1c2VyZmxhZ3MiOjQsImlhdCI6MTYyMTI4MTY2NiwiZXhwIjoxNjIxMjgyMjY2LCJzZXNzaW9uaWQiOiJUWW9pZGgyMFRIeUc5YldMUllhb0FnPT0ifQ.ksnCeOxnrNQeuNaQnmpauUG_eja70nVtaCbamxFaB1LLkMIqzJ_wk_GMN1h4eM1WLS41VraukWw8Fztd281eaSQPPWiT-ieCjWxim8Sw4spYS8XBrBBx62tot201TIEJ8mUFHUjvXpPctDsBYi9YM5lUmFxhZgYI2Lx8gg0P6sLoUydcShKKcUNgRmGWwO64LH7rMEpzyCTapJBXdkzUhJ-zm98fmOy1oGGTesBteSWP_ksH14Xq411k1ebJ83sV4tL6-FJLhcPNIKwqYJ87fqUWwZFZb5tY4JUJxrSnahKfwyidNXt5m8LCIc8pt-xbBtVihAFkAYBoXKI-OYBrwg', 'Client_Name': 'APIC-CLI', 'Request-Tag': 'tag0'}||/mgmt/opt/controller/decoy/apps/execserver/execapp.py||31'
'13482||2021-05-25 22:52:51,407||decoy||INFO||Starting new HTTP connection (1): 127.0.0.1||/mgmt/opt/controller/decoy/decoy-env/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py||203'
'13482||2021-05-25 22:52:51,452||decoy||DEBUG||"POST /api/mo/uni/tn-TestTn/ctx-Test-CTX-1.xml HTTP/1.1" 200 70||/mgmt/opt/controller/decoy/decoy-env/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py||383'
'18280||2021-05-25 22:52:51,455||decoy||DEBUG||(None, {}, {'prompt': '(config-tenant-vrf)# ', 'mode': [[u'exec'], [u'configure', u'terminal'], [u'tenant', u'TestTn'], ['vrf', 'context', u'Test-CTX-1']]}, None)||/mgmt/opt/controller/decoy/apps/execserver/execapp.py||144'

Observe the fvTenant and fvCtx objects both set with status="created,modified".

Object Deletion

We can review the REST operations involved when any MO gets deleted.

VRF Deletion example:

APIC-1# configure terminal
APIC-1(config)# tenant TestTn
APIC-1(config-tenant)# no vrf context Test-CTX-1
APIC-1(config-tenant-vrf)#