Telemetry has the following configuration guidelines and limitations:

• For information about supported platforms, see the Nexus Switch Platform Matrix.

• Cisco NX-OS releases that support the data management engine (DME) Native Model support Telemetry.

• Support is in place for the following:

  • DME data collection

  • NX-API data sources

  • Google protocol buffer (GPB) encoding over Google Remote Procedure Call (gRPC) transport

  • JSON encoding over HTTP

• The smallest sending interval (cadence) supported is five seconds for a depth of 0. The minimum cadence values for depth values greater than 0 depends on the size of the data being streamed out. Configuring any cadences below the minimum value may result in undesirable system behavior.

• Telemetry supports up to five remote management receivers (destinations). Configuring more than five remote receivers may result in undesirable system behavior.

• Telemetry can consume up to 20% of the CPU resource.

• Beginning with Cisco NX-OS Release 10.2(1q)F, Telemetry is supported on the N9K-C9332D-GX2B platform switches.

• Beginning with Cisco NX-OS Release 10.4(1)F, telemetry is supported on the Cisco Nexus 9332D-H2R platform switches.

Configuration Commands After Downgrading to an Older Release

After a downgrade to an older release, some configuration commands or command options can fail because the older release may not support them. When downgrading to an older release, unconfigure and reconfigure the telemetry feature after the new image comes up. This sequence avoids the failure of unsupported commands or command options.

The following example shows this procedure:

• Copy the telemetry configuration to a file:

  
  switch# show running-config | section telemetry
  feature telemetry
  telemetry
    destination-group 100
      ip address 1.2.3.4 port 50004 protocol gRPC encoding GPB 
      use-chunking size 4096
    sensor-group 100
      path sys/bgp/inst/dom-default depth 0
    subscription 600
      dst-grp 100
      snsr-grp 100 sample-interval 7000
  switch# show running-config | section telemetry > telemetry_running_config
  switch# show file bootflash:telemetry_running_config
  feature telemetry
  telemetry
    destination-group 100
      ip address 1.2.3.4 port 50004 protocol gRPC encoding GPB 
      use-chunking size 4096
    sensor-group 100
      path sys/bgp/inst/dom-default depth 0
    subscription 600
      dst-grp 100
      snsr-grp 100 sample-interval 7000
  switch# 
  

• Execute the downgrade operation. When the image comes up and the switch is ready, copy the telemetry configurations back to the switch.

  
  switch# copy telemetry_running_config running-config echo-commands 
  `switch# config terminal`
  `switch(config)# feature telemetry`
  `switch(config)# telemetry`
  `switch(config-telemetry)# destination-group 100`
  `switch(conf-tm-dest)# ip address 1.2.3.4 port 50004 protocol gRPC encoding GPB `
  `switch(conf-tm-dest)# sensor-group 100`
  `switch(conf-tm-sensor)# path sys/bgp/inst/dom-default depth 0`
  `switch(conf-tm-sensor)# subscription 600`
  `switch(conf-tm-sub)# dst-grp 100`
  `switch(conf-tm-sub)# snsr-grp 100 sample-interval 7000`
  `switch(conf-tm-sub)# end`
  Copy complete, now saving to disk (please wait)...
  Copy complete.
  switch#
  

gRPC Error Behavior

The switch client disables the connection to the gRPC receiver if the gRPC receiver sends 20 errors. Unconfigure then reconfigure the receiver's IP address under the destination group to enable the gRPC receiver. Errors include:

• The gRPC client sends the wrong certificate for secure connections.

• The gRPC receiver takes too long to handle client messages and incurs a timeout. Avoid timeouts by processing messages using a separate message processing thread.

Support for gRPC Chunking

Starting with Release 9.2(1), support for gRPC chunking has been added. For streaming to occur successfully, you must enable chunking if gRPC has to send an amount of data greater than 12 MB to the receiver.

The gRPC user must do the gRPC chunking. The gRPC client side does the fragmentation, and the gRPC server side does the reassembly. Telemetry is still bound to memory and data can be dropped if the memory size is more than the allowed limit of 12 MB for telemetry. In order to support chunking, use the telemetry .proto file that is available at Cisco's GibLab, which has been updated for gRPC chunking, as described in Telemetry Components and Process.

The chunking size is from 64 through 4096 bytes.

Following shows a configuration example through the NX-API CLI:

feature telemetry
!
telemetry
  destination-group 1
    ip address 171.68.197.40 port 50051 protocol gRPC encoding GPB 
    use-chunking size 4096
  destination-group 2
    ip address 10.155.0.15 port 50001 protocol gRPC encoding GPB 
    use-chunking size 64
  sensor-group 1
    path sys/intf depth unbounded
  sensor-group 2
    path sys/intf depth unbounded	
  subscription 1
    dst-grp 1
    snsr-grp 1 sample-interval 10000
  subscription 2
    dst-grp 2
    snsr-grp 2 sample-interval 15000

Following shows a configuration example through the NX-API REST:

{
    "telemetryDestGrpOptChunking": {
        "attributes": {
            "chunkSize": "2048",
            "dn": "sys/tm/dest-1/chunking"
        }
    }
}

The following error message appears on systems that do not support gRPC chunking, such as the Cisco MDS series switches:

MDS-9706-86(conf-tm-dest)# use-chunking size 200
ERROR: Operation failed: [chunking support not available]

NX-API Sensor Path Limitations

NX-API can collect and stream switch information not yet in the DME using show commands. However, using the NX-API instead of streaming data from the DME has inherent scale limitations as outlined:

• The switch backend dynamically processes NX-API calls such as show commands,

• NX-API spawns several processes that can consume up to a maximum of 20% of the CPU.

• NX-API data translates from the CLI to XML to JSON.

The following is a suggested user flow to help limit excessive NX-API sensor path bandwidth consumption:

1. Check whether the show command has NX-API support. You can confirm whether NX-API supports the command from the VSH with the pipe option: show <command> | json or show <command> | json pretty.

   Note	Avoid commands that take the switch more than 30 seconds to return JSON output.

2. Refine the show command to include any filters or options.

   • Avoid enumerating the same command for individual outputs; for example, show vlan id 100 , show vlan id 101 , and so on. Instead, use the CLI range options; for example, show vlan id 100-110,204 , whenever possible to improve performance.

     If only the summary or counter is needed, then avoid dumping a whole show command output to limit the bandwidth and data storage that is required for data collection.

3. Configure telemetry with sensor groups that use NX-API as their data sources. Add the show commands as sensor paths

4. Configure telemetry with a cadence of five times the processing time of the respective show command to limit CPI usage.

5. Receive and process the streamed NX-API output as part of the existing DME collection.

Telemetry VRF Support

Telemetry VRF support allows you to specify a transport VRF, which means that the telemetry data stream can egress through front-panel ports and avoid possible competition between SSH or NGINX control sessions.

You can use the use-vrf vrf-name command to specify the transport VRF.

The following example specifies the transport VRF:

{
              "telemetryDestProfile": {
                "attributes": {
                  "adminSt": "enabled"
                },
                "children": [
                  {
                    "telemetryDestOptVrf": {
                      "attributes": {
                        "name": "default"
                      }
                    }
                  }
                ]
              }
            }

Certificate Trustpoint Support

Beginning in NX-OS release 10.1(1), the trustpoint keyword is added in the existing global level command.

switch(config-telemetry)# certificate ?​
trustpoint    specify trustpoint label                  
WORD          .pem certificate filename (Max Size 256)  
switch(config-telemetry)# certificate trustpoint ​
WORD          trustpoint label name (Max Size 256)​
switch(config-telemetry)# certificate trustpoint trustpoint1 ?​
WORD  Hostname associated with certificate (Max Size 256)​
switch(config-telemetry)#certificate trustpoint trustpoint1 foo.test.google.fr​

Destination Hostname Support

Beginning in NX-OS release 10.1(1), the host keyword is added in destination-group command.

switch(config-telemetry)# destination-group 1​
switch(conf-tm-dest)# ?​
certificate Specify certificate
host Specify destination host
ip Set destination IPv4 address
ipv6 Set destination IPv6 address
...​
switch(conf-tm-dest)# host ?​
A.B.C.D|A:B::C:D|WORD  IPv4 or IPv6 address or DNS name of destination​
switch(conf-tm-dest)#​

switch(conf-tm-dest)# host abc port 11111 ?​
protocol  Set transport protocol​
switch(conf-tm-dest)# host abc port 11111 protocol ?​
HTTP  ​
UDP   ​
gRPC  ​
switch(conf-tm-dest)# host abc port 11111 protocol gRPC ?​
encoding  Set encoding format​
switch(conf-tm-dest)# host abc port 11111 protocol gRPC encoding ? ​
Form-data    Set encoding to Form-data only​
GPB          Set encoding to GPB only​
GPB-compact  Set encoding to Compact-GPB only​
JSON         Set encoding to JSON​
XML          Set encoding to XML​
switch(conf-tm-dest)# host ip address 1.1.1.1 port 2222 protocol HTTP encoding JSON
<CR>​
​

Support for Node ID

Beginning in NX-OS release 10.1(1), you can configure a custom Node ID string for a telemetry receiver through the use-nodeid command. By default, the host name is used, but support for a node ID enables you to set or change the identifier for the node_id_str of the telemetry receiver data.

You can assign the node ID through the telemetry destination profile, by using the usenode-id command. This command is optional.

switch(config)# telemetry
switch(config-telemetry)# destination-profile
switch(conf-tm-dest-profile)# use-nodeid test-srvr-10
switch(conf-tm-dest-profile)#

Telemetry receiver:
==================================
node_id_str: "test-srvr-10"
subscription_id_str: "1"
encoding_path: "sys/ch/psuslot-1/psu"
collection_id: 3896
msg_timestamp: 1559669946501

Use the use-nodeid sub-command under the host command. The destination level use-nodeid configuration preceeds the global level configuration.

The following example shows the command syntax:

switch(config-telemetry)# destination-group 1​
switch(conf-tm-dest)# host 172.19.216.78 port 18112 protocol http enc json​
switch(conf-tm-dest-host)# use-nodeid ?​
WORD  Node ID (Max Size 128)​
switch(conf-tm-dest-host)# use-nodeid session_1:18112​

The following example shows the output from the Telemetry receiver:

>> Message size 923​
Telemetry msg received @  23:41:38 UTC​
    Msg Size: 11​
    node_id_str   : session_1:18112​
    collection_id : 3118​
    data_source   : DME​    
    encoding_path : sys/ch/psuslot-1/psu​
    collection_start_time   : 1598485314721​
    collection_end_time   : 1598485314721​
    data          : ​

Support for Streaming of YANG Models

Beginning in NX-OS release 9.2(1), telemetry supports the YANG ("Yet Another Next Generation") data modeling language. Telemetry supports data streaming for both device YANG and OpenConfig YANG.

Support for Proxy

Beginning in NX-OS release 10.1(1), the proxy command is included in the host command. The following is the command syntax:

switch(config-telemetry)# destination-group 1​
switch(conf-tm-dest)# host 172.19.216.78 port 18112 protocol http enc json​
switch(conf-tm-dest-host)# proxy ?​
    A.B.C.D|A:B::C:D|WORD  IPv4 or IPv6 address or DNS name of proxy server​
    <1-65535>  Proxy port number, Default value is 8080​
username  Set proxy authentication username​
password  Set proxy authentication password​

gRPC Asynchronous Mode

The gRPC asynchronous mode is available only under the host command. In normal stream condition, this mode allows the receivers to stream data in mdtDialout call without exiting or receiving WriteDone() call.

The following is the command syntax:

nxosv-1(config-telemetry)# destination-group 1
nxosv-1(conf-tm-dest)# host 172.22.244.130 port 50007 ?
nxosv-1(conf-tm-dest-host)# grpc-async ?

The following steps describe how to configure a single telemetry DME stream with a ten second cadence with GPB encoding.

switch# configure terminal
switch(config)# feature telemetry
switch(config)# telemetry
switch(config-telemetry)# destination-group 1
switch(config-tm-dest)# ip address 171.70.59.62 port 50051 protocol gRPC encoding GPB
switch(config-tm-dest)# exit
switch(config-telemetry)# sensor group sg1
switch(config-tm-sensor)# data-source DME
switch(config-tm-dest)# path interface depth unbounded query-condition keep-data-type
switch(config-tm-dest)# subscription 1
switch(config-tm-dest)# dst-grp 1 
switch(config-tm-dest)# snsr grp 1 sample interval 10000 

This example creates a subscription that streams data for the sys/bgp root MO every 5 seconds to the destination IP 1.2.3.4 port 50003.

switch(config)# telemetry
switch(config-telemetry)# sensor-group 100
switch(conf-tm-sensor)# path sys/bgp depth 0
switch(conf-tm-sensor)# destination-group 100
switch(conf-tm-dest)# ip address 1.2.3.4 port 50003
switch(conf-tm-dest)# subscription 100
switch(conf-tm-sub)# snsr-grp 100 sample-interval 5000
switch(conf-tm-sub)# dst-grp 100

This example creates a subscription that streams data for sys/intf every 5 seconds to destination IP 1.2.3.4 port 50003, and encrypts the stream using GPB encoding verified using the test.pem.

switch(config)# telemetry
switch(config-telemetry)# certificate /bootflash/test.pem foo.test.google.fr
switch(conf-tm-telemetry)# destination-group 100
switch(conf-tm-dest)# ip address 1.2.3.4 port 50003 protocol gRPC encoding GPB
switch(config-dest)# sensor-group 100
switch(conf-tm-sensor)# path sys/bgp depth 0
switch(conf-tm-sensor)# subscription 100
switch(conf-tm-sub)# snsr-grp 100 sample-interval 5000
switch(conf-tm-sub)# dst-grp 100

This example creates a subscription that streams data for sys/cdp every 15 seconds to destination IP 1.2.3.4 port 50004.

switch(config)# telemetry
switch(config-telemetry)# sensor-group 100
switch(conf-tm-sensor)# path sys/cdp depth 0
switch(conf-tm-sensor)# destination-group 100
switch(conf-tm-dest)# ip address 1.2.3.4 port 50004
switch(conf-tm-dest)# subscription 100
switch(conf-tm-sub)# snsr-grp 100 sample-interval 15000
switch(conf-tm-sub)# dst-grp 100

This example creates a cadence-based collection of show command data every 750 seconds.

switch(config)# telemetry
switch(config-telemetry)# destination-group 1
switch(conf-tm-dest)# ip address 172.27.247.72 port 60001 protocol gRPC encoding GPB 
switch(conf-tm-dest)# sensor-group 1
switch(conf-tm-sensor# data-source NX-API
switch(conf-tm-sensor)# path "show system resources" depth 0
switch(conf-tm-sensor)# path "show version" depth 0 
switch(conf-tm-sensor)# path "show environment power" depth 0 
switch(conf-tm-sensor)# path "show environment fan" depth 0 
switch(conf-tm-sensor)# path "show environment temperature" depth 0 
switch(conf-tm-sensor)# path "show process cpu" depth 0 
switch(conf-tm-sensor)# path "show nve peers" depth 0 
switch(conf-tm-sensor)# path "show nve vni" depth 0 
switch(conf-tm-sensor)# path "show nve vni 4002 counters" depth 0 
switch(conf-tm-sensor)# path "show int nve 1 counters" depth 0 
switch(conf-tm-sensor)# path "show policy-map vlan" depth 0 
switch(conf-tm-sensor)# path "show ip access-list test" depth 0 
switch(conf-tm-sensor)# path "show system internal access-list resource utilization" depth 0 
switch(conf-tm-sensor)# subscription 1
switch(conf-tm-sub)# dst-grp 1
switch(conf-tm-dest)# snsr-grp 1 sample-interval 750000

This example creates an event-based subscription for sys/fm. Data is streamed to the destination only if there is a change under the sys/fm MO.

switch(config)# telemetry
switch(config-telemetry)# sensor-group 100
switch(conf-tm-sensor)# path sys/fm depth 0
switch(conf-tm-sensor)# destination-group 100
switch(conf-tm-dest)# ip address 1.2.3.4 port 50005
switch(conf-tm-dest)# subscription 100
switch(conf-tm-sub)# snsr-grp 100 sample-interval 0
switch(conf-tm-sub)# dst-grp 100

During operation, you can change a sensor group from frequency-based to event-based, and change event-based to frequency-based by changing the sample-interval. This example changes the sensor-group from the previous example to frequency-based. After the following commands, the telemetry application will begin streaming the sys/fm data to the destination every 7 seconds.

switch(config)# telemetry
switch(config-telemetry)# subscription 100
switch(conf-tm-sub)# snsr-grp 100 sample-interval 7000

Multiple sensor groups and destinations can be linked to a single subscription. The subscription in this example streams the data for Ethernet port 1/1 to four different destinations every 10 seconds.

switch(config)# telemetry
switch(config-telemetry)# sensor-group 100
switch(conf-tm-sensor)# path sys/intf/phys-[eth1/1] depth 0
switch(conf-tm-sensor)# destination-group 100
switch(conf-tm-dest)# ip address 1.2.3.4 port 50004
switch(conf-tm-dest)# ip address 1.2.3.4 port 50005
switch(conf-tm-sensor)# destination-group 200
switch(conf-tm-dest)# ip address 5.6.7.8 port 50001 protocol HTTP encoding JSON
switch(conf-tm-dest)# ip address 1.4.8.2 port 60003
switch(conf-tm-dest)# subscription 100
switch(conf-tm-sub)# snsr-grp 100 sample-interval 10000
switch(conf-tm-sub)# dst-grp 100
switch(conf-tm-sub)# dst-grp 200

A sensor group can contain multiple paths, a destination group can contain multiple destination profiles, and a subscription can be linked to multiple sensor groups and destination groups, as shown in this example.

switch(config)# telemetry
switch(config-telemetry)# sensor-group 100
switch(conf-tm-sensor)# path sys/intf/phys-[eth1/1] depth 0
switch(conf-tm-sensor)# path sys/epId-1 depth 0
switch(conf-tm-sensor)# path sys/bgp/inst/dom-default depth 0

switch(config-telemetry)# sensor-group 200
switch(conf-tm-sensor)# path sys/cdp depth 0
switch(conf-tm-sensor)# path sys/ipv4 depth 0

switch(config-telemetry)# sensor-group 300
switch(conf-tm-sensor)# path sys/fm depth 0
switch(conf-tm-sensor)# path sys/bgp depth 0

switch(conf-tm-sensor)# destination-group 100
switch(conf-tm-dest)# ip address 1.2.3.4 port 50004
switch(conf-tm-dest)# ip address 4.3.2.5 port 50005

switch(conf-tm-dest)# destination-group 200
switch(conf-tm-dest)# ip address 5.6.7.8 port 50001

switch(conf-tm-dest)# destination-group 300
switch(conf-tm-dest)# ip address 1.2.3.4 port 60003

switch(conf-tm-dest)# subscription 600
switch(conf-tm-sub)# snsr-grp 100 sample-interval 7000
switch(conf-tm-sub)# snsr-grp 200 sample-interval 20000
switch(conf-tm-sub)# dst-grp 100
switch(conf-tm-sub)# dst-grp 200

switch(conf-tm-dest)# subscription 900
switch(conf-tm-sub)# snsr-grp 200 sample-interval 7000
switch(conf-tm-sub)# snsr-grp 300 sample-interval 0
switch(conf-tm-sub)# dst-grp 100
switch(conf-tm-sub)# dst-grp 300

You can verify the telemetry configuration using the show running-config telemetry command, as shown in this example.

switch(config)# telemetry 
switch(config-telemetry)# destination-group 100
switch(conf-tm-dest)# ip address 1.2.3.4 port 50003
switch(conf-tm-dest)# ip address 1.2.3.4 port 50004
switch(conf-tm-dest)# end
switch# show run telemetry 

!Command: show running-config telemetry
!Time: Thu Oct 13 21:10:12 2016

version 7.0(3)I5(1)
feature telemetry

telemetry
destination-group 100
ip address 1.2.3.4 port 50003 protocol gRPC encoding GPB 
ip address 1.2.3.4 port 50004 protocol gRPC encoding GPB 

You can specify transport VRF and telemetry data compression for gRPC using the use-vrf and use-compression gzip commands, as shown in this example.

switch(config)# telemetry
switch(config-telemetry)# destination-profile
switch(conf-tm-dest-profile)# use-vrf default
switch(conf-tm-dest-profile)# use-compression gzip
switch(conf-tm-dest-profile)# sensor-group 1
switch(conf-tm-sensor)# path sys/bgp depth unbounded
switch(conf-tm-sensor)# destination-group 1
switch(conf-tm-dest)# ip address 1.2.3.4 port 50004
switch(conf-tm-dest)# subscription 1
switch(conf-tm-sub)# dst-grp 1
switch(conf-tm-sub)# snsr-grp 1 sample-interval 10000

Use the following NX-OS CLI show commands to display telemetry configuration, statistics, errors, and session information.

show telemetry yang direct-path cisco-nxos-device

This command displays YANG paths that are directly encoded to perform better than other paths.

switch# show telemetry yang direct-path cisco-nxos-device
) Cisco-NX-OS-device:System/lldp-items
2) Cisco-NX-OS-device:System/acl-items
3) Cisco-NX-OS-device:System/mac-items
4) Cisco-NX-OS-device:System/intf-items
5) Cisco-NX-OS-device:System/procsys-items/sysload-items
6) Cisco-NX-OS-device:System/ospf-items
7) Cisco-NX-OS-device:System/procsys-items
8) Cisco-NX-OS-device:System/ipqos-items/queuing-items/policy-items/out-items
9) Cisco-NX-OS-device:System/mac-items/static-items
10) Cisco-NX-OS-device:System/ch-items
11) Cisco-NX-OS-device:System/cdp-items
12) Cisco-NX-OS-device:System/bd-items
13) Cisco-NX-OS-device:System/eps-items
14) Cisco-NX-OS-device:System/ipv6-items

show telemetry control database

This command displays the internal databases that reflect the configuration of telemetry.

switch# show telemetry control database ?
  <CR>                
  >                   Redirect it to a file
  >>                  Redirect it to a file in append mode
  destination-groups  Show destination-groups
  destinations        Show destinations
  sensor-groups       Show sensor-groups
  sensor-paths        Show sensor-paths
  subscriptions       Show subscriptions
  |                   Pipe command output to filter

switch# show telemetry control database 

Subscription Database size = 1

--------------------------------------------------------------------------------
Subscription ID      Data Collector Type 
--------------------------------------------------------------------------------
100                  DME NX-API                

Sensor Group Database size = 1

--------------------------------------------------------------------------------
Sensor Group ID  Sensor Group type  Sampling interval(ms)  Linked subscriptions 
--------------------------------------------------------------------------------
100              Timer              10000(Running)         1                    

Sensor Path Database size = 1

--------------------------------------------------------------------------------
Subscribed Query Filter  Linked Groups  Sec Groups  Retrieve level  Sensor Path 
--------------------------------------------------------------------------------
No                       1              0           Full            sys/fm      

Destination group Database size = 2

--------------------------------------------------------------------------------
Destination Group ID  Refcount  
--------------------------------------------------------------------------------
100                   1         

Destination Database size = 2

--------------------------------------------------------------------------------
Dst IP Addr     Dst Port   Encoding   Transport  Count     
--------------------------------------------------------------------------------
192.168.20.111       12345      JSON       HTTP       1
192.168.20.123	50001      GPB        gRPC       1    

show telemetry control database sensor-paths

This command displays sensor path details for telemetry configuration, including counters for encoding, collection, transport, and streaming.

switch(conf-tm-sub)# show telemetry control database sensor-paths
Sensor Path Database size = 4
----------------------------------------------------------------------------------------------------
Row ID     Subscribed Linked Groups  Sec Groups  Retrieve level  Path(GroupId) : Query : Filter
----------------------------------------------------------------------------------------------------
1          No         1              0           Full            sys/cdp(1) : NA : NA

GPB Encoded Data size in bytes (Cur/Min/Max): 0/0/0
JSON Encoded Data size in bytes (Cur/Min/Max): 65785/65785/65785
Collection Time in ms (Cur/Min/Max): 10/10/55
Encoding Time in ms (Cur/Min/Max): 8/8/9
Transport Time in ms (Cur/Min/Max): 0/0/0
Streaming Time in ms (Cur/Min/Max): 18/18/65

2          No         1              0           Self            show module(2) : NA : NA
GPB Encoded Data size in bytes (Cur/Min/Max): 0/0/0
JSON Encoded Data size in bytes (Cur/Min/Max): 1107/1106/1107
Collection Time in ms (Cur/Min/Max): 603/603/802
Encoding Time in ms (Cur/Min/Max): 0/0/0
Transport Time in ms (Cur/Min/Max): 0/0/1
Streaming Time in ms (Cur/Min/Max): 605/605/803

3          No         1              0           Full            sys/bgp(1) : NA : NA
GPB Encoded Data size in bytes (Cur/Min/Max): 0/0/0
JSON Encoded Data size in bytes (Cur/Min/Max): 0/0/0
Collection Time in ms (Cur/Min/Max): 0/0/44
Encoding Time in ms (Cur/Min/Max): 0/0/0
Transport Time in ms (Cur/Min/Max): 0/0/0
Streaming Time in ms (Cur/Min/Max): 1/1/44

4          No         1              0           Self            show version(2) : NA : NA
GPB Encoded Data size in bytes (Cur/Min/Max): 0/0/0
JSON Encoded Data size in bytes (Cur/Min/Max): 2442/2441/2442
Collection Time in ms (Cur/Min/Max): 1703/1703/1903
Encoding Time in ms (Cur/Min/Max): 0/0/0
Transport Time in ms (Cur/Min/Max): 0/0/0
Streaming Time in ms (Cur/Min/Max): 1703/1703/1904

switch(conf-tm-sub)#

show telemetry control stats

This command displays the statistics about the internal databases about configuration of telemetry.

switch# show telemetry control stats 
show telemetry control stats entered

--------------------------------------------------------------------------------
Error Description                                            Error Count 
--------------------------------------------------------------------------------
Chunk allocation failures                                    0
Sensor path Database chunk creation failures                 0
Sensor Group Database chunk creation failures                0
Destination Database chunk creation failures                 0
Destination Group Database chunk creation failures           0
Subscription Database chunk creation failures                0
Sensor path Database creation failures                       0
Sensor Group Database creation failures                      0
Destination Database creation failures                       0
Destination Group Database creation failures                 0
Subscription Database creation failures                      0
Sensor path Database insert failures                         0
Sensor Group Database insert failures                        0
Destination Database insert failures                         0
Destination Group Database insert failures                   0
Subscription insert to Subscription Database failures        0
Sensor path Database delete failures                         0
Sensor Group Database delete failures                        0
Destination Database delete failures                         0
Destination Group Database delete failures                   0
Delete Subscription from Subscription Database failures      0
Sensor path delete in use                                    0
Sensor Group delete in use                                   0
Destination delete in use                                    0
Destination Group delete in use                              0
Delete destination(in use) failure count                     0
Failed to get encode callback                                0
Sensor path Sensor Group list creation failures              0
Sensor path prop list creation failures                      0
Sensor path sec Sensor path list creation failures           0
Sensor path sec Sensor Group list creation failures          0
Sensor Group Sensor path list creation failures              0
Sensor Group Sensor subs list creation failures              0
Destination Group subs list creation failures                0
Destination Group Destinations list creation failures        0
Destination Destination Groups list creation failures        0
Subscription Sensor Group list creation failures             0
Subscription Destination Groups list creation failures       0
Sensor Group Sensor path list delete failures                0
Sensor Group Subscriptions list delete failures              0
Destination Group Subscriptions list delete failures         0
Destination Group Destinations list delete failures          0
Subscription Sensor Groups list delete failures              0
Subscription Destination Groups list delete failures         0
Destination Destination Groups list delete failures          0
Failed to delete Destination from Destination Group          0
Failed to delete Destination Group from Subscription         0
Failed to delete Sensor Group from Subscription              0
Failed to delete Sensor path from Sensor Group               0
Failed to get encode callback                                0
Failed to get transport callback                             0
switch#  Destination Database size = 1

--------------------------------------------------------------------------------
Dst IP Addr     Dst Port   Encoding   Transport  Count     
--------------------------------------------------------------------------------
192.168.20.123	50001      GPB        gRPC       1    

show telemetry data collector brief

This command displays the brief statistics about the data collection.

switch# show telemetry data collector brief 

----------------------------------------------------------------------
Collector Type       Successful Collections     Failed Collections        
----------------------------------------------------------------------
DME                   143                        0                        

show telemetry data collector details

This command displays detailed statistics about the data collection which includes breakdown of all sensor paths.

switch# show telemetry data collector details 

--------------------------------------------------------------------------------
Succ Collections      Failed Collections     Sensor Path
--------------------------------------------------------------------------------
150                   0                      sys/fm

show telemetry event collector errors

This command displays the errors statistic about the event collection.

switch# show telemetry event collector errors 

--------------------------------------------------------------------------------
Error Description                                  Error Count 
--------------------------------------------------------------------------------
APIC-Cookie Generation Failures                    - 0
Authentication Failures                            - 0
Authentication Refresh Failures                    - 0
Authentication Refresh Timer Start Failures        - 0
Connection Timer Start Failures                    - 0
Connection Attempts                                - 3
Dme Event Subscription Init Failures               - 0
Event Data Enqueue Failures                        - 0
Event Subscription Failures                        - 0
Event Subscription Refresh Failures                - 0
Pending Subscription List Create Failures          - 0
Subscription Hash Table Create Failures            - 0
Subscription Hash Table Destroy Failures           - 0
Subscription Hash Table Insert Failures            - 0
Subscription Hash Table Remove Failures            - 0
Subscription Refresh Timer Start Failures          - 0
Websocket Connect Failures                         - 0

show telemetry event collector stats

This command displays the statistics about the event collection which includes breakdown of all sensor paths.

switch# show telemetry event collector stats 

--------------------------------------------------------------------------------
Collection Count  Latest Collection Time    Sensor Path
--------------------------------------------------------------------------------

show telemetry control pipeline stats

This command displays the statistics for the telemetry pipeline.

switch# show telemetry pipeline stats 
Main Statistics:
    Timers:
        Errors:
            Start Fail        =     0

    Data Collector:
        Errors:
            Node Create Fail  =     0

    Event Collector:
        Errors:
            Node Create Fail  =     0    Node Add Fail     =     0
            Invalid Data      =     0

    Memory:
            Allowed Memory Limit                = 1181116006 bytes
            Occupied Memory                     = 93265920 bytes 

Queue Statistics:
    Request Queue:
        High Priority Queue:
            Info:
                Actual Size       =    50    Current Size      =     0
                Max Size          =     0    Full Count        =     0

            Errors:
                Enqueue Error     =     0    Dequeue Error     =     0

        Low Priority Queue:
            Info:
                Actual Size       =    50    Current Size      =     0
                Max Size          =     0    Full Count        =     0

            Errors:
                Enqueue Error     =     0    Dequeue Error     =     0

    Data Queue:
        High Priority Queue:
            Info:
                Actual Size       =    50    Current Size      =     0
                Max Size          =     0    Full Count        =     0

            Errors:
                Enqueue Error     =     0    Dequeue Error     =     0

        Low Priority Queue:
            Info:
                Actual Size       =    50    Current Size      =     0
                Max Size          =     0    Full Count        =     0

            Errors:
                Enqueue Error     =     0    Dequeue Error     =     0

show telemetry transport

This command displays all configured transport sessions.

switch# show telemetry transport 

Session Id      IP Address      Port       Encoding   Transport  Status    
-----------------------------------------------------------------------------------
0               192.168.20.123   50001      GPB        gRPC       Connected

show telemetry transport <session-id>

This command displays detailed session information for a specific transport session.

switch# show telemetry transport 0 

Session Id:          0                   
IP Address:Port      192.168.20.123:50001 
Encoding:            GPB                  
Transport:           gRPC                
Status:              Disconnected        
Last Connected:      Fri Sep 02 11:45:57.505 UTC
Last Disconnected:   Never               
Tx Error Count:      224                 
Last Tx Error:       Fri Sep 02 12:23:49.555 UTC

switch# show telemetry transport 1

Session Id:          1                   
IP Address:Port      10.30.218.56:51235 
Transport:           HTTP                
Status:              Disconnected        
Last Connected:      Never               
Last Disconnected:   Never                
Tx Error Count:      3                   
Last Tx Error:       Wed Apr 19 15:56:51.617 PDT

The following example shows output from an IPv6 entry.

switch# show telemetry transport 0 
Session Id: 0
IP Address:Port [10:10::1]:8000
Transport: GRPC
Status: Idle
Last Connected: Never
Last Disconnected: Never
Tx Error Count: 0
Last Tx Error: None
Event Retry Queue Bytes: 0
Event Retry Queue Size: 0
Timer Retry Queue Bytes: 0
Timer Retry Queue Size: 0
Sent Retry Messages: 0
Dropped Retry Messages: 0

show telemetry transport <session-id> stats

This command displays details of a specific transport session.

switch# show telemetry transport 0 stats 

Session Id:          0                   
IP Address:Port      192.168.20.123:50001 
Encoding:            GPB                 
Transport:           GRPC                
Status:              Connected           
Last Connected:      Mon May 01 11:29:46.912 PST
Last Disconnected:   Never               
Tx Error Count:      0                   
Last Tx Error:       None                

show telemetry transport <session-id> stats

This command displays details of a specific transport session.

Session Id:                    0                   
Transmission Stats                                 
   Compression:                disabled            
   Source Interface:           not set()
   Transmit Count:             319297              
   Last TX time:               Fri Aug 02 03:51:15.287 UTC
   Min Tx Time:                1                   ms
   Max Tx Time:                3117                ms
   Avg Tx Time:                3                   ms
   Cur Tx Time:                1                   ms

show telemetry transport <session-id> errors

This command displays detailed error statistics for a specific transport session.

switch# show telemetry transport 0 errors 
Session Id:                 0
Connection Errors
Connection Error Count:     0
Transmission Errors
Tx Error Count:             30
Last Tx Error:              Thu Aug 01 04:39:47.083 UTC
Last Tx Return Code:        No error   

show telemetry control databases sensor-paths

These following configuration steps result in the show telemetry control databases sensor-paths command output below.

feature telemetry
 
telemetry
  destination-group 1
    ip address 172.25.238.13 port 50600 protocol gRPC encoding GPB
  sensor-group 1
    path sys/cdp depth unbounded
    path sys/intf depth unbounded
    path sys/mac depth 0
  subscription 1
    dst-grp 1
    snsr-grp 1 sample-interval 1000

Command output.

switch# show telemetry control databases sensor-paths

Sensor Path Database size = 3
--------------------------------------------------------------------------------
--------------------
Row ID     Subscribed Linked Groups  Sec Groups  Retrieve level  Path(GroupId) :
Query : Filter
--------------------------------------------------------------------------------
--------------------
1          No         1              0           Full            sys/cdp(1) : NA
: NA
GPB Encoded Data size in bytes (Cur/Min/Max): 30489/30489/30489
JSON Encoded Data size in bytes (Cur/Min/Max): 0/0/0
CGPB Encoded Data size in bytes (Cur/Min/Max): 0/0/0
Collection Time in ms (Cur/Min/Max): 6/5/54
Encoding Time in ms (Cur/Min/Max): 5/5/6
Transport Time in ms (Cur/Min/Max): 1027/55/1045
Streaming Time in ms (Cur/Min/Max): 48402/5/48402
 
2          No         1              0           Full            sys/intf(1) : N
A : NA
GPB Encoded Data size in bytes (Cur/Min/Max): 539466/539466/539466
JSON Encoded Data size in bytes (Cur/Min/Max): 0/0/0
CGPB Encoded Data size in bytes (Cur/Min/Max): 0/0/0
Collection Time in ms (Cur/Min/Max): 66/64/114
Encoding Time in ms (Cur/Min/Max): 91/90/92
Transport Time in ms (Cur/Min/Max): 4065/4014/5334
Streaming Time in ms (Cur/Min/Max): 48365/64/48365
 
3          No         1              0           Self            sys/mac(1) : NA
: NA
GPB Encoded Data size in bytes (Cur/Min/Max): 247/247/247
JSON Encoded Data size in bytes (Cur/Min/Max): 0/0/0
CGPB Encoded Data size in bytes (Cur/Min/Max): 0/0/0
Collection Time in ms (Cur/Min/Max): 1/1/47
Encoding Time in ms (Cur/Min/Max): 1/1/1
Transport Time in ms (Cur/Min/Max): 4/1/6
Streaming Time in ms (Cur/Min/Max): 47369/1/47369

show telemetry transport sessions

switch# show telemetry transport sessions​
switch# show telemetry transport stats​
switch# show telemetry transport errors​
switch# show telemetry transport all​

switch# show telemetry transport sessions​
Session Id:          0                   ​
IP Address:Port      172.27.254.13:50004 ​
Transport:           GRPC                ​
Status:              Transmit Error      ​
SSL Certificate:     trustpoint1                   ​
Last Connected:      Never               ​
Last Disconnected:   Never                ​
Tx Error Count:      2                   ​
Last Tx Error:       Wed Aug 19 23:32:21.749 UTC​
…​
Session Id:          4                 ​
IP Address:Port      172.27.254.13:50006 ​
Transport:           UDP              

Telemetry Ephemeral Event

sensor-group 1​
path sys/accounting/log query-condition query-target=subtree&complete-mo=yes&notify-interval=1

sys/pim/inst/routedb-route, sys/pim/pimifdb-adj, sys/pim/pimifdb-prop​
sys/igmp/igmpifdb-prop, sys/igmp/inst/routedb, sys/igmpsnoop/inst/dom/db-exptrack, ​
sys/igmpsnoop/inst/dom/db-group, sys/igmpsnoop/inst/dom/db-mrouter​
sys/igmpsnoop/inst/dom/db-querier, sys/igmpsnoop/inst/dom/db-snoop​

Use the following NX-OS CLI commands to display the log and trace information.

show tech-support telemetry

This NX-OS CLI command collects the telemetry log contents from the tech-support log. In this example, the command output is redirected into a file in bootflash.

switch# show tech-support telemetry > bootflash:tmst.log

tmtrace.bin

This BASH shell command collects telemetry traces and prints them out.

switch# configure terminal
switch(config)# feature bash
switch(config)# run bash
bash-4.2$ tmtrace.bin -d tm-errors
bash-4.2$ tmtrace.bin -d tm-logs
bash-4.2$ tmtrace.bin -d tm-events

For example:

bash-4.2$ tmtrace.bin -d tm-logs
[01/25/17 22:52:24.563 UTC 1 29130] [3944724224][tm_ec_dme_auth.c:59] TM_EC: Authentication refresh url http://127.0.0.1/api/aaaRefresh.json
[01/25/17 22:52:24.565 UTC 2 29130] [3944724224][tm_ec_dme_rest_util.c:382] TM_EC: Performed POST request on http://127.0.0.1/api/aaaRefresh.json
[01/25/17 22:52:24.566 UTC 3 29130] [3944724224][tm_mgd_timers.c:114] TM_MGD_TIMER: Starting leaf timer for leaf:0x11e17ea4 time_in_ms:540000
[01/25/17 22:52:45.317 UTC 4 29130] [3944724224][tm_ec_dme_event_subsc.c:790] TM_EC: Event subscription database size 0 
[01/25/17 22:52:45.317 UTC 5 29130] [3944724224][tm_mgd_timers.c:114] TM_MGD_TIMER: Starting leaf timer for leaf:0x11e17e3c time_in_ms:50000
bash-4.2# 

Note	The tm-logs option is not enabled by default because it is verbose. Enable tm-logs with the tmtrace.bin -L D tm-logs command. Disable tm-logs with the tmtrace.bin -L W tm-logs command.

show system internal telemetry trace

The show system internal telemetry trace [tm-events | tm-errors |tm-logs | all] command displays system internal telemetry trace information.

switch# show system internal telemetry trace all
Telemetry All Traces:
Telemetry Error Traces:
[07/26/17 15:22:29.156 UTC 1 28577] [3960399872][tm_cfg_api.c:367] Not able to destroy dest profile list for config node rc:-1610612714 reason:Invalid argument
[07/26/17 15:22:44.972 UTC 2 28577] [3960399872][tm_stream.c:248] No subscriptions for destination group 1
[07/26/17 15:22:49.463 UTC 3 28577] [3960399872][tm_stream.c:576] TM_STREAM: Subscriptoin 1 does not have any sensor groups

3 entries printed
Telemetry Event Traces:
[07/26/17 15:19:40.610 UTC 1 28577] [3960399872][tm_debug.c:41] Telemetry xostrace buffers initialized successfully!
[07/26/17 15:19:40.610 UTC 2 28577] [3960399872][tm.c:744] Telemetry statistics created successfully!
[07/26/17 15:19:40.610 UTC 3 28577] [3960399872][tm_init_n9k.c:97] Platform intf: grpc_traces:compression,channel
switch# 

switch# show system internal telemetry trace tm-logs
Telemetry Log Traces:
0 entries printed
switch#
switch# show system internal telemetry trace tm-events
Telemetry Event Traces:
[07/26/17 15:19:40.610 UTC 1 28577] [3960399872][tm_debug.c:41] Telemetry xostrace buffers initialized successfully!
[07/26/17 15:19:40.610 UTC 2 28577] [3960399872][tm.c:744] Telemetry statistics created successfully!
[07/26/17 15:19:40.610 UTC 3 28577] [3960399872][tm_init_n9k.c:97] Platform intf: grpc_traces:compression,channel
[07/26/17 15:19:40.610 UTC 4 28577] [3960399872][tm_init_n9k.c:207] Adding telemetry to cgroup
[07/26/17 15:19:40.670 UTC 5 28577] [3960399872][tm_init_n9k.c:215] Added telemetry to cgroup successfully!

switch# show system internal telemetry trace tm-errors
Telemetry Error Traces:
0 entries printed
switch#

Streaming Paths to a Destination

This example creates a subscription that streams paths sys/cdp and sys/ipv4 to a destination 1.2.3.4 port 50001 every five seconds.

POST https://192.168.20.123/api/node/mo/sys/tm.json

Payload:
{
    "telemetryEntity": {
        "attributes": {
            "dn": "sys/tm"
        },
        "children": [{
            "telemetrySensorGroup": {
                "attributes": {
                    "id": "10",
                    "rn": "sensor-10"
                },  "children": [{
                    "telemetryRtSensorGroupRel": {
                        "attributes": {
                            "rn": "rtsensorGroupRel-[sys/tm/subs-30]",
                            "tCl": "telemetrySubscription",
                            "tDn": "sys/tm/subs-30"
                        }
                    }
                }, {
                    "telemetrySensorPath": {
                        "attributes": {
                            "path": "sys/cdp",
                            "rn": "path-[sys/cdp]",
                            "excludeFilter": "",
                            "filterCondition": "",
                            "secondaryGroup": "0",
                            "secondaryPath": "",
                            "depth": "0"
                        }
                    }
                }, {
                    "telemetrySensorPath": {
                        "attributes": {
                            "path": "sys/ipv4",
                            "rn": "path-[sys/ipv4]",
                            "excludeFilter": "",
                            "filterCondition": "",
                            "secondaryGroup": "0",
                            "secondaryPath": "",
                            "depth": "0"
                        }
                    }
                }]
            }
        }, {
            "telemetryDestGroup": {
                "attributes": {
                    "id": "20",
                    "rn": "dest-20"
                },
                "children": [{
                    "telemetryRtDestGroupRel": {
                        "attributes": {
                            "rn": "rtdestGroupRel-[sys/tm/subs-30]",
                            "tCl": "telemetrySubscription",
                            "tDn": "sys/tm/subs-30"
                        }
                    }
                }, {
                    "telemetryDest": {
                        "attributes": {
                            "addr": "1.2.3.4",
                            "enc": "GPB",
                            "port": "50001",
                            "proto": "gRPC",
                            "rn": "addr-[1.2.3.4]-port-50001"
                        }
                    }
                }]
            }
        }, {
            "telemetrySubscription": {
                "attributes": {
                    "id": "30",
                    "rn": "subs-30"
                },
                "children": [{
                    "telemetryRsDestGroupRel": {
                        "attributes": {
                            "rType": "mo",
                            "rn": "rsdestGroupRel-[sys/tm/dest-20]",
                            "tCl": "telemetryDestGroup",
                            "tDn": "sys/tm/dest-20",
                            "tType": "mo"
                        }
                    }
                }, {
                    "telemetryRsSensorGroupRel": {
                        "attributes": {
                            "rType": "mo",
                            "rn": "rssensorGroupRel-[sys/tm/sensor-10]",
                            "sampleIntvl": "5000",
                            "tCl": "telemetrySensorGroup",
                            "tDn": "sys/tm/sensor-10",
                            "tType": "mo"
                        }
                    }
                }]
            }
        }]
    }
}

Filter Conditions on BGP Notifications

The following example payload enables notifications that trigger when the BFP feature is disabled as per the filterCondition attribute in the telemetrySensorPath MO. The data is streamed to10.30.217.80 port 50055.

POST  https://192.168.20.123/api/node/mo/sys/tm.json

Payload:
{
  "telemetryEntity": {
    "children": [{
      "telemetrySensorGroup": {
        "attributes": {
          "id": "10"
        }
        "children": [{
          "telemetrySensorPath": {
            "attributes": {
              "excludeFilter": "",
              "filterCondition": "eq(fmBgp.operSt,\"disabled\")",
              "path": "sys/fm/bgp",
              "secondaryGroup": "0",
              "secondaryPath": "",
              "depth": "0"
            }
          }
        }
        ]
      }
    },
    {
      "telemetryDestGroup": {
        "attributes": {
          "id": "20"
        }
        "children": [{
          "telemetryDest": {
            "attributes": {
              "addr": "10.30.217.80",
              "port": "50055",
              "enc": "GPB",
              "proto": "gRPC"
            }
          }
        }
        ]
      }
    },
    {
      "telemetrySubscription": {
        "attributes": {
          "id": "30"
        }
        "children": [{
          "telemetryRsSensorGroupRel": {
            "attributes": {
              "sampleIntvl": "0",
              "tDn": "sys/tm/sensor-10"
            }
          }
        },
        {
          "telemetryRsDestGroupRel": {
            "attributes": {
              "tDn": "sys/tm/dest-20"
            }
          }
        }
        ]
      }
    }
    ]
  }
}

Using Postman Collection for Telemetry Configuration

An example Postman collection is an easy way to start configuring the telemetry feature, and can run all telemetry CLI equivalents in a single payload. Modify the file in the preceding link using your preferred text editor to update the payload to your needs, then open the collection in Postman and run the collection.

The telemetry application is modeled in the DME with the following structure:

model
|----package [name:telemetry]
    |  @name:telemetry
    |----objects
         |----mo [name:Entity]
              |    @name:Entity
              |      @label:Telemetry System
              |--property 
              |    @name:adminSt
              |      @type:AdminState
              |
              |----mo [name:SensorGroup]
              |    |    @name:SensorGroup
              |    |      @label:Sensor Group
              |    |--property
              |    |    @name:id [key]
              |    |      @type:string:Basic
              |    |
              |    |----mo [name:SensorPath]
              |         |   @name:SensorPath
              |         |     @label:Sensor Path
              |         |--property
              |         |    @name:path [key]
              |         |      @type:string:Basic
              |         |    @name:filterCondition
              |         |      @type:string:Basic
              |         |    @name:excludeFilter
              |         |      @type:string:Basic
              |         |    @name:depth
              |         |      @type:RetrieveDepth
              |
              |----mo [name:DestGroup]
              |    |    @name:DestGroup
              |    |      @label:Destination Group
              |    |--property
              |    |    @name:id
              |    |      @type:string:Basic
              |    |        
              |    |----mo [name:Dest]
              |         |   @name:Dest
              |         |     @label:Destination
              |         |--property
              |         |    @name:addr [key]
              |         |      @type:address:Ip
              |         |    @name:port [key]
              |         |      @type:scalar:Uint16
              |         |    @name:proto
              |         |      @type:Protocol
              |         |    @name:enc
              |         |      @type:Encoding
              |
              |----mo [name:Subscription]
                   |    @name:Subscription
                   |      @label:Subscription
                   |--property
                   |    @name:id
                   |      @type:scalar:Uint64
                   |----reldef
                   |    |  @name:SensorGroupRel
                   |    |    @to:SensorGroup
                   |    |    @cardinality:ntom
                   |    |    @label:Link to sensorGroup entry
                   |    |--property
                   |         @name:sampleIntvl
                   |           @type:scalar:Uint64
                   |
                   |----reldef
                        |  @name:DestGroupRel
                        |    @to:DestGroup
                        |    @cardinality:ntom
                        |    @label:Link to destGroup entry

The following are the guidelines and limitations for the Cloud Scale software telemetry feature:

• For information about supported platforms for Cisco NX-OS prior to release 9.3(x), see the section for Platform Support for Programmability Features in that guide. Starting with Cisco NX-OS release 9.3(x) for information about supported platforms, see the Nexus Switch Platform Matrix.

• For custom client telemetry programs, one message format is supported. Your client programs must comply with this format.

• Beginning with Cisco NX-OS Release 10.3(1)F, software telemetry is supported on the Cisco Nexus 9808 platform switches.

• Beginning with Cisco NX-OS Release 10.4(1)F, software telemetry is supported on the Cisco Nexus 9804 platform switches.

• Beginning with Cisco NX-OS Release 10.4(1)F, software telemetry is supported on N9KX98900CD-A and N9KX9836DM-A line cards with Cisco Nexus 9808 and 9804 switches.

The sample interval for a sensor group determines how and when telemetry data is transmitted to a path label. The sample interval can be configured either to periodically poll for telemetry data or gather telemetry data when events occur.

• When the sample interval for telemetry is configured as a non-zero value, telemetry periodically sends the data for the environment, interfaces, resources, and vxlan labels during each sample interval.

• When the sample interval is set to zero, telemetry sends event notifications when the environment, interfaces, resources, and vxlan labels experience operational state updates, as well as creation and deletion of MOs.

Polling for data or receiving events are mutually exclusive. You can configure polling or event-driven telemetry for each path label.

The telemetry path labels feature has the following guidelines and limitations:

• The feature supports only Cisco DME data source only.

• You cannot mix and match usability paths with regular DME paths in the same sensor group. For example, you cannot configure sys/intf and interface in the same sensor group. Also, you cannot configure the same sensor group with sys/intf and interface. If this situation occurs, NX-OS rejects the configuration.

• User filter keywords, such as oper-speed and counters=[detailed], are supported only for the interface path.

• The feature does not support other sensor path options, such as depth or filter-condition.

• The telemetry path labels has the following restrictions in using path labels:

  • Must start with prefix show in lowercase, as it is case sensitive.

    For example: show version is allowed. However, SHOW version or version is not allowed.

  • Cannot include following characters:

    • ;

    • |

    • " " or ' '

  • Cannot include following words:

    • telemetry

    • conf t

    • configure

Path Label Show Commands

Through the show telemetry usability commands, you can display the individual paths that the path label walks when you issue a query.

Command Examples

Note	The show telemetry usability all command is a concatenation of all the individual commands that are shown in this section.

switch# show telemetry usability environment
  1) label_name          : environment

     path_name           : sys/ch
     query_type          : poll
     query_condition     : rsp-subtree=full&query-target=subtree&target-subtree-class=eqptPsuSlot,eqptFtSlot,eqptSupCSlot,eqptPsu,eqptFt,eqptSensor,eqptLCSlot

  2) label_name          : environment

     path_name           : sys/ch
     query_type          : event
     query_condition     : rsp-subtree=full&query-target=subtree&query-target-filter=or(or(deleted(),created()),or(and(updated(eqptFan.operSt),ne(eqptFan.operSt,"ok")),and(updated(eqptDimm.operSt),ne(eqptDimm.operSt,"ok")),and(updated(eqptFlash.operSt),ne(eqptFlash.operSt,"ok")),and(updated(eqptSpromSup.operSt),ne(eqptSpromSup.operSt,"ok")),and(updated(eqptSpromLc.operSt),ne(eqptSpromLc.operSt,"ok"))))
switch# 

switch# show telemetry usability interface 
  1) label_name          : interface

     path_name           : sys/intf
     query_type          : poll
     query_condition     : query-target=children&query-target-filter=eq(l1PhysIf.adminSt,"up")&rsp-subtree=children&rsp-subtree-class=rmonEtherStats,rmonIfIn,rmonIfOut,rmonIfHCIn,rmonIfHCOut

  2) label_name          : interface

     path_name           : sys/mgmt-[mgmt0]
     query_type          : poll
     query_condition     : query-target=subtree&query-target-filter=eq(mgmtMgmtIf.adminSt,"up")&rsp-subtree=full&rsp-subtree-class=rmonEtherStats,rmonIfIn,rmonIfOut,rmonIfHCIn,rmonIfHCOut

  3) label_name          : interface

     path_name           : sys/intf
     query_type          : event
     query_condition     : query-target=subtree&query-target-filter=or(or(deleted(),created()),or(and(updated(ethpmPhysIf.operSt),eq(ethpmPhysIf.operSt,"down")),and(updated(ethpmPhysIf.operSt),eq(ethpmPhysIf.operSt,"up")),and(updated(ethpmLbRtdIf.operSt),eq(ethpmLbRtdIf.operSt,"down")),and(updated(ethpmLbRtdIf.operSt),eq(ethpmLbRtdIf.operSt,"up")),and(updated(ethpmAggrIf.operSt),eq(ethpmAggrIf.operSt,"down")),and(updated(ethpmAggrIf.operSt),eq(ethpmAggrIf.operSt,"up")),and(updated(ethpmEncRtdIf.operSt),eq(
ethpmEncRtdIf.operSt,"down")),and(updated(ethpmEncRtdIf.operSt),eq(ethpmEncRtdIf.operSt,"up"))))

  4) label_name          : interface

     path_name           : sys/mgmt-[mgmt0]
     query_type          : event
     query_condition     : query-target=subtree&query-target-filter=or(or(deleted(),created()),or(and(updated(imMgmtIf.operSt),eq(imMgmtIf.operSt,"down")),and(updated(imMgmtIf.operSt),eq(imMgmtIf.operSt,"up"))))
switch#

switch# show telemetry usability resources 
  1) label_name          : resources

     path_name           : sys/proc
     query_type          : poll
     query_condition     : rsp-subtree=full&rsp-foreign-subtree=ephemeral

  2) label_name          : resources

     path_name           : sys/procsys
     query_type          : poll
     query_condition     : query-target=subtree&target-subtree-class=procSystem,procSysCore,procSysCpuSummary,procSysCpu,procIdle,procIrq,procKernel,procNice,procSoftirq,procTotal,procUser,procWait,procSysCpuHistory,procSysLoad,procSysMem,procSysMemFree,procSysMemUsage,procSysMemUsed

  3) label_name          : resources

     path_name           : sys/procsys/sysmem
     query_type          : event
     query_condition     : query-target-filter=and(updated(procSysMem.memstatus),ne(procSysMem.memstatus,"OK"))

switch# 

switch# show telemetry usability vxlan 
  1) label_name          : vxlan

     path_name           : sys/bd
     query_type          : poll
     query_condition     : query-target=subtree&target-subtree-class=l2VlanStats

  2) label_name          : vxlan

     path_name           : sys/eps
     query_type          : poll
     query_condition     : rsp-subtree=full&rsp-foreign-subtree=ephemeral

  3) label_name          : vxlan

     path_name           : sys/eps
     query_type          : event
     query_condition     : query-target=subtree&target-subtree-class=nvoDyPeer

  4) label_name          : vxlan

     path_name           : sys/bgp
     query_type          : event
     query_condition     : query-target=subtree&query-target-filter=or(deleted(),created())

  5) label_name          : vxlan

     path_name           : sys/bgp
     query_type          : event
     query_condition     : query-target=subtree&target-subtree-class=bgpDom,bgpPeer,bgpPeerAf,bgpDomAf,bgpPeerAfEntry,bgpOperRtctrlL3,bgpOperRttP,bgpOperRttEntry,bgpOperAfCtrl

switch#

For each source path, the following table shows the information that is streamed when the subscription is first created (the baseline) and when event notifications occur.

For additional information, refer to Github https://github.com/CiscoDevNet/nx-telemetry-proto.

The native data source path feature has the following guidelines and limitations:

• For streaming from the RIB, MAC, and Adjacency native data source paths, sensor-path property updates do not support custom criteria like depth , query-condition , or filter-condition .

Use the NX-OS show telemetry event collector commands to display statistics and counters, or errors for the native data source path.

Displaying Statistics

You can issue show telemetry event collector stats command to display the statistics and counters for each native data source path.

Displaying Error Counters

switch# show telemetry event collector errors

-------------------------------------------------------------------------------
-
Error Description                                  Error Count 
-------------------------------------------------------------------------------
-
Dme Event Subscription Init Failures               - 0
Event Data Enqueue Failures                        - 0
Event Subscription Failures                        - 0
Pending Subscription List Create Failures          - 0
Subscription Hash Table Create Failures            - 0
Subscription Hash Table Destroy Failures           - 0
Subscription Hash Table Insert Failures            - 0
Subscription Hash Table Remove Failures            - 0
switch# 

For each source path, the following table shows the information that is streamed when the subscription is first created "the baseline" and when event notifications occur.

Displaying Syslog Path Information

Use the Cisco NX-OS show telemetry event collector commands to display statistics and counters, or errors for the syslog path.

Displaying Statistics

You can enter the show telemetry event collector stats command to display the statistics and counters for each syslog path.

The following is an example of statistics for the syslog path:

switch# show telemetry event collector stats 

-------------------------------------------------------------------------------- 
Row ID         Collection Count  Latest Collection Time     Sensor Path(GroupId) 
-------------------------------------------------------------------------------- 
1              138               Tue Dec 03 11:20:08.200 PST Cisco-NX-OS-Syslog-oper:syslog(1) 
2              138               Tue Dec 03 11:20:08.200 PST Cisco-NX-OS-Syslog-oper:syslog/messages(1)

Displaying Error Counters

You can use the show telemetry event collector errors command to display the error totals for all the syslog paths.

switch(config-if)# show telemetry event collector errors 

-------------------------------------------------------------------------------- 
Error Description                                  Error Count 
-------------------------------------------------------------------------------- 
Dme Event Subscription Init Failures                       - 0 
Event Data Enqueue Failures                                - 0 
Event Subscription Failures                                - 0 
Pending Subscription List Create Failures                  - 0 
Subscription Hash Table Create Failures                    - 0 
Subscription Hash Table Destroy Failures                   - 0 
Subscription Hash Table Insert Failures                    - 0 
Subscription Hash Table Remove Failures                    - 0 

The following is a sample of JSON output:

172.19.216.13 - - [03/Dec/2019 19:38:50] "POST /network/Cisco-NX-OS-Syslog-oper%3Asyslog%2Fmessages HTTP/1.0" 200 - 
172.19.216.13 - - [03/Dec/2019 19:38:50] "POST /network/Cisco-NX-OS-Syslog-oper%3Asyslog%2Fmessages HTTP/1.0" 200 - 
>>> URL            : /network/Cisco-NX-OS-Syslog-oper%3Asyslog%2Fmessages 
>>> TM-HTTP-VER    : 1.0.0 
>>> TM-HTTP-CNT    : 1 
>>> Content-Type   : application/json 
>>> Content-Length : 578 
    Path => Cisco-NX-OS-Syslog-oper:syslog/messages 
            node_id_str   : task-n9k-1 
            collection_id : 40 
            data_source   : YANG 
            data          : 
[ 
  [ 
    { 
      "message-id": 420 
    }, 
    { 
      "category": "ETHPORT", 
      "group": "ETHPORT", 
      "message-name": "IF_UP", 
      "node-name": "task-n9k-1", 
      "severity": 5, 
      "text": "Interface loopback10 is up ", 
      "time-of-day": "Dec 3 2019 11:38:51", 
      "time-stamp": "1575401931000", 
      "time-zone": "" 
    } 
  ] 
] 

The following is a sample KVGPB output.

KVGPB Output:
---Telemetry msg received @ 18:22:04 UTC
 
Read frag:1 size:339 continue to block on read..
 
All the fragments:1 read successfully total size read:339
 
node_id_str: "task-n9k-1"
 
subscription_id_str: "1"
 
collection_id: 374
 
data_gpbkv {
 
  fields {
 
    name: "keys"
 
    fields {
 
      name: "message-id"
 
      uint32_value: 374
 
    }
 
  }
 
  fields {
 
    name: "content"
 
    fields {
 
      fields {
 
        name: "node-name"
 
        string_value: "task-n9k-1"
 
      }
 
      fields {
 
        name: "time-of-day"
 
        string_value: "Jun 26 2019 18:20:21"
 
      }
 
      fields {
 
        name: "time-stamp"
 
        uint64_value: 1574293838000
 
      }
 
      fields {
 
        name: "time-zone"
 
        string_value: "UTC"
 
      }
 
      fields {
 
        name: "process-name"
 
        string_value: ""
 
      }
 
      fields {
 
        name: "category"
 
        string_value: "VSHD"
 
      }
 
      fields {
 
        name: "group"
 
        string_value: "VSHD"
 
      }
 
      fields {
 
        name: "message-name"
 
        string_value: "VSHD_SYSLOG_CONFIG_I"
 
      }
 
      fields {
 
        name: "severity"
 
        uint32_value: 5
 
      }
 
      fields {
 
        name: "text"
 
        string_value: "Configured from vty by admin on console0"
 
      }
 
    }
 
  }
 
}