The following options with their default value are added to existing monitor subscriber command.

UPF Monitor Subscriber CLI

Following are the options:

• W - UP PCAP Trace (ON ): This parameter is used to create PCAP trace for slowpath and fastpath.

• U - Mon Display (ON): The non-protocol events (such as statistics and charging information from ECS and so on) are also captured in slowpath PCAP files and are displayed on UPF monitor console.

• V - PCAP Hexdump (ON): This flag must be set to ON to capture the protocol packets in a text file in hexdump format on UPF.

Note	Currently, UP PCAP Trace flag must be set to ON to capture fastpath and slowpath PCAP files.

• F - Packet Capture (Full Pkt): Captures all packets from fastpath.

  Using this option, operators can choose between full and partial packet captures. By entering F, the packet capture type can be changed to either full or partial. With partial packet capture, users can enter packet sizes from 1 to 16384 bytes. For example, if input is given as 20, only the first 20 bytes of fastpath packets will be captured and the remaining packets will be dropped.

  Note	When opening the PCAP file, the summary view will display full length of the packet, but the detailed view will show only the truncated packet.

• / - Priority (0): The value is in the range from "0 – Best Effort" to "7 – Guaranteed"

  • 0 - Best Effort

  • 1 - Low

  • 2 - Med-Low

  • 3 - Medium

  • 4 - Med-High

  • 5 - High

  • 6 - Critical

  • 7 - Guaranteed

  Caution

  It is strongly recommended to not change the default value. It can adversely affect the system performance.

• N - MEH Header (OFF) : The MEH header is stripped from the IP packet if this option is configured

Show Monitor Subscriber Sessions

Following is the new CLI to show the ongoing MonSub session.

You can trigger the show monitor subscriber fastpath session all CLI command from both SMF and UPF. You can trigger the show monitor subscriber fastpath session up-ip-address CLI command from the SMF

• SessId: This is the local session id for MonSub session on UPF Sessmgr.

• CallID: Call ID on UPF.

• PSN: This is panopticon sequence number. There is a maximum of four MonSub fastpath tracing sessions on one UPF with PSN ranging from 0-3.

• Start time: Time at which MonSub tracing session starts.

• Interface Type: This is to identify the call type for which MonSub fastpath tracing session was started, whether it is Sxa, Sxb or Sxab.

Disconnect Monitor Subscriber Sessions

Following is the new CLI to disconnect the ongoing MonSub session. You can trigger the CLI from both CP and UP.

monitor subscriber fastpath disconnect sessmgr-instance upf_sessmgr_instance_id session-id local_monsub_sessid_sessmgr_level

If the MonSub session disconnect is successful, the following message dispalys on console.

Session Disconnected Successfully

If the MonSub session disconnect fails, the following message dispalys on console.

Monitor Subscriber session does not exist

Note	Only security administrator can execute the monitor disconnect CLI.

Hexdump module must be configured to provide operators the provision to configure Files names and Poll timers. The Hexdump module is one of the modules such as—EDR, UDR and so on, that are part of the CDRMOD functionality. It is recommended to configure the hexdump in a non-local context such as the ECS context. Hexdump modules are not supported in local context.

For more information on Hexdump module and its configuration, refer to the Packet Capture (PCAP) Trace chapter in the ASR5500 System Administration Guide

The naming conventions for PCAP files are discussed in the following sections

Note	Only monitor-subscriber-file-name and rotation options are used in naming PCAP files.

Slowpath File Name Convention

The slow path file names appear in the following format:

curr_slowpath_{SMGR Mon Sub Session Id}_{monsub_file_name_option_val}_{Timestamp}_{RotationCount}.pcap

or

slowpath_{SMGR Mon Sub Session Id}_{monsub_file_name_option_val}_{Timestamp}_{RotationCount}.pcap

File with ‘curr_’ prefix is the file, that is currently being written to, that is still not closed. When files are to be rotated (depending on the file rotation parameters), file without the ‘curr_’ prefix are copied to hard disk.

The SMGR MonSub Session Id – This is the session Id for MonSub session created on Uplane SMGR instance ID, which created this PCAP. This Id is local to SMGR instance, so there could be two SLOWPATH pcap captured with same ID.

When files are to be copied to hard disk, The monsub_file_name_option_val is replaced by:

• IMSI value if monitor-subscriber-file-name is set to "imsi".

• Call ID value if monitor-subscriber-file-name is set to "call-id"

• Username value if monitor-subscriber-file-name is set to ‘username’

Timestamp is in the following format "MMDDYYYYHHMMSS", where:

• MM - Month, DD - Date and YYYY - Year.

• HH -Hour, MM - Minutes and SS - Seconds.

RotationCount is a 9-digit value that is incremented every time an old file is rotated, and a new file is generated.

Rotation of slowpath files is determined by following option in hexdump-module file configuration:

rotation { num-records number | time seconds| volume bytes }

• num-records: num-records specifies the number of packets after which a new file is generated and ‘RotationCount’ in the filename is incremented. The range of number is between 100 to 10240, and the default value is 1024.

• time: time specifies the time to wait in seconds before a new file is generated and ‘RotationCount’ in the filename is incremented. seconds must be an integer from 30 through 86400. The default value is 3600.

• volume: volume specifies the number of bytes after which a new file is generated and ‘RotationCount’ in the filename is incremented. bytes must be an integer from 51200 through 62914560. The default value is 102400.

Note	The tarriff-time parameter under rotation is ignored as it is not suitable for PCAP file capture.

The following are examples of the file naming conventions for slowpath PCAP files.

• For the ‘imsi’ option where IMSI is ‘112233445566778’, slowpath files are named as:

  slowpath_S0_112233445566778_07152019050907_000000000.pcap

• For ‘call_id’ option where Call Id is ‘01317b22’, slowpath files are named as:

  slowpath_S0_01317b22_07152019050907_000000000.pcap

Note	The parameter tarrif-time is not applicable for PCAP file capture.

Fastpath File Name Convention

The fastpath file names appear in the following format:

vpp_{S}_{B}_{encap}_{monsub_file_name_option}_{Timestamp}_{FileCount}.pcap

• S is replaced by either ‘S1’, ‘S2’,'S3’, or 'S4’.

• B is replaced by either ‘B0’,‘B1’,'B2', or 'B3' depending on the bundle generated by Panopticon.

• monsub_file_name_option is replaced by:

  • IMSI value if monitor-subscriber-file-name is set to "imsi".

  • Call ID value if monitor-subscriber-file-name is set to "call-id"

  • Username value if monitor-subscriber-file-name is set to ‘username’

Timestamp is in the following format "MMDDYYYYHHMMSS", where:

• MM - Month, DD - Date and YYYY - Year.

• HH -Hour, MM - Minutes and SS - Seconds.

RotationCount is a 9-digit value that is incremented every time an old file is rotated, and a new file is generated.

Fast path "FileCount" is not the same as the slowpath "RotationCount" parameters and hence ‘hexdump-module file rotation’ parameters are ignored while naming fastpath files.

In Phases 1 of the feature, fastpath generated file names are like ‘vpp_S1_B0_ip.pcap’ or ‘vpp_S1_B1_ip.pcap’, they are renamed to following when being copied over to non-volatile storage:

• vpp_S1_B0_ip_01317b22_07152019050907_000000000.pcap

• vpp_S1_B1_ip_01317b22_07152019050908_000000001.pcap

• vpp_S1_B0_ip_01317b22_07152019050908_000000002.pcap

In MonSub phase 3, a PCAP “bundle” is replaced with a single PCAP file that uses Ethernet encapsulation.

In Phase 3, each fastpath session file is captured in the Ethernet PCAP file that is ‘vpp_S0_B0_eth.pcap’ and they are renamed to following when being copied to a non-volatile storage:

vpp_S0_B0_eth_01317b22_07152019050907_000000000.pcap

For ‘callid’ option where Call Id is ‘12345678ef’:

• slowpath_S0_12345678ef _07152019050907_000000000.pcap

• vpp_S1_B0_eth_12345678ef _07152019050907_000000000.pcap

For ‘username’ option where username is ‘9890098900’:

• slowpath_S0_07152019050907_000000000_9890098900.pcap

• vpp_S1_B0_eth_07152019050907_000000000_9890098900.pcap

Fastpath PCAP files are written to the /records/pcap directory in same card and CPU complex where the SMGR owns the subscriber session resides.

/records directory is mapped to the "tmpfs" filesystem that is mapped to RAM. In this state, the files are suffixed with a ".pending" extension. For example:

Note	The files size at this stage is not the actual file size when it is written to a persistent storage.

Once the fastpath tracing mechanism has written the files, they are converted to ‘.pcap’ files and renamed as given below. Additionally, there is a file that ends with a ".done" extension:

-rw-rw-r--    1 root     root       8689188 Oct 16 22:06 vpp_S0_B0_eth.pcap

After the PCAP files are written by fastpath tracing mechanism, the Co-Proc functionality instantiates and copies the files to a hard disk or a persistent storage.

The above file location process for Fastpath is also applicable to Slowpath.

The target file location in all cases is: /hd_raid/records/hexdump, except for the case in the hexdump module configuration where use-harddisk is enabled and the directory option under the hexdump file is to a custom value. For example, if the directory option is set to a value "abc" then the target location for the PCAP file will be: /hd_raid/records/hexdump/abc/.

In this feature implementation, a predefined location is set for PCAP files.

• To make sure that /records/pcap directory is not populated when issues are encountered with the use of use-harddisk and hexdump module configurations.

• For regular cleanup from /hd_raid/records/hexdump directory.

File Transfer to External Location

Once the files have been copied to the hard disk, they can be copied over to an external server using the command: transfer-mode option under the hexdump command in the hexdump-module configuration.

Apart from transfer-mode , other relevant options under hexdump can be used for external file transfer. Operators can use these commands to avoid excessive storage during fastpath processing.

Following are the Limitations:

• Restarting trace immediately after quitting may result in fastpath files in /records/pcap directory to be overwritten. It is recommended to restart the session after a brief moment (a few seconds).

• When MonSub trace is stopped, the tear down process can take a few seconds, so it is recommended to wait for few seconds. A maximum of five seconds (hexdump poll timer value in seconds) before toggling the MonSub trace to start, else, operators may observe MAX TRACING SESSIONS REACHED momentarily.

• Show monitor subscriber fastpath sessions CLI does not display the MonSub sessions that are being stopped. Hence there is a transient period where new MonSub sessions can be rejected due to max sessions reached, whereas show CLI shows less sessions. It is recommended that operators wait for some time before starting a new MonSub trace session.

• Changing fastpath configuration options is only possible when UP Pcap Trace is set to OFF.

• When MT=ON in the Multi-PDN, then once MT=OFF, new PDN tracing is not started due to MAX TRACING REACHED, and then all other tracing is STOPPED. This is because the first new PDN tracing is started and then all previous PDN's were STOPPED for MT=OFF case.

• It is recommended to not to launch the same UE MonSub sessions from different CLIs.

• In slowpath PCAP, the egress DL packets does not show the GTPU-U header because the functionality to add GTP-U is with fastpath. So, ingress and egress DL packets shows up the duplicates, unless there is some packet modification like HTTP X-headers applied over the ingress packets.

• Toggling C and D options does not impact the PCAP capture.

• For Multi-PDN, the fastpath filenames does not use the Call Id, because, by definition the multi-PDN case has more than one call id and hence a higher-level configuration such as IMSI is more suitable for naming the files.

• Only the named options explicitly mentioned in this document are supported from hexdump-module file configuration.

• Number of streams that can be traced in fastpath is limited to 5000. Stream is defined as a TCP or UDP flow which is made up of (source IP address, destination IP address, source port, and destination port, transport protocol (TCP or UDP)).

• Fastpath packets cannot be streamed to an external server. They are stored on the hard-disk and transferred (either manually) or by using transfer-mode options.

• The UP PCAP trace must be set to ON to capture fastpath and slowpath PCAP files.

• MonSub CLI option ‘<SPACE> Pause’ is only to pause console events. There is no impact on other tracing events (slowpath PCAP, fastpath PCAP and protocol packets tracing in a text file in hexdump format) with this option.

• The UP trace PCAP file does not contain the initial PFCP Sx Request/Response, due to race condition.

• The ICMP Packets and a first packets of TCP and UDP streams flow through both slowpath and fastpath. Default values of GTPU (option 26) and User L3 (Option 19) are set to OFF. As a result, these packets are not captured in slowpath captures. If Option 26 is set to ON then these packets are captured in slowpath PCAP captures. As mentioned in previous point, option 19 has no effect on slowpath PCAP capture.

• Data Events flag must be set to ON to capture fastpath and slowpath PCAP files.

• Only firstPDN tracing is supported forPure-Scall. This limitation will be fixed with multiple subscriber tracing support.

• The Mon sub tracing is not supported for option Next-SAEGW Call on UP.

• The Mon sub tracing is not supported for option Next call by APN for Pure-S call type.

• On ASR 5500 setup with the default value of poll-timer, all the packets may not be captured due to a known issue. To avoid large number of packets to be rejected, it is recommended to change the poll-timer value to the lowest possible (10ms).

• If context replacement occurs (if the same subscriber reattaches without a detach) then the slowpath captures for the new call continues to be in the old slowpath files.

Use this configuration to set the frequency of PCAP file capture check.

configure 
   context context_name 
      hexdump-module 
         hexdump monitor-subscriber-poll-timeout poll_timer_value  
         end 

NOTES:

• hexdump monitor-subscriber-poll-timeout : This option specifies how frequently the check for newly captured PCAP files in the volatile storage must be done before they are copied to persistent storage.

• poll_timer_value : Specifies the poll timer value in milliseconds. It must be an integer in the range of 10 ms to 60 seconds. Default: 30 seconds.

  Note	It is strongly recommended to not configure the timer with a value less than 5 seconds.

• This option is only applicable when MonSub is enabled for the products that have fastpath functionality - PGW, SAEGW on ASR-5500 and VPC-SI.

Use the following configuration to specify the file name of the PCAP file which contains IMSI, Call ID or Username.

configure 
   context context_name 
      hexdump-module 
         file rotation { num-records number | tariff-time minute minutes hour hours | time seconds| volume bytes | monitor-subscriber-file-name { imsi | username | call-id }  
         end 

NOTES:

• monitor-subscriber-file-name { imsi | username | call-id } : This option specifies if the name of the captured PCAP files will contain IMSI, Call Id or Username. This option is only applicable on products that have fastpath functionality (PGW, SAEGW on ASR 5500 and VPC-SI) AND only when Monitor Subscriber functionality is enabled. Default: IMSI.

• rotation { num-records number | tariff-time minute minutes hour hours | time seconds | volume bytes } : Specifies when to close a hexdump file and create a new one.

  • num-records number : Specifies the maximum number of records that should be added to a hexdump file. When the number of records in the file reaches this value, the file is complete.

    number must be an integer from 100 through 10240. Default: 1024

  • tariff-time minute minutes hour hours : Specifies to close the current hexdump file and create a new one based on the tariff time (in minutes and hours).

    minutes must be an integer from 0 through 59.

    hours must be an integer from 0 through 23.

  • time seconds : Specifies the period of time to wait (in seconds) before closing the current hexdump file and creating a new one.

    seconds must be an integer from 30 through 86400. Default: 3600

    Important

    It is recommended to set the rotation time to 30 seconds.

  • volume bytes : Specifies the maximum size of the hexdump file (in bytes) before closing it and creating a new one.

    bytes must be an integer from 51200 through 62914560. Note that a higher setting may improve the compression ratio when the compression keyword is set to gzip. Default: 102400

The following SNMP trap(s) are added in support of the Monitor Subscriber feature:

• MonSubProcessInitFailure: This trap is triggered when MonSub handler process has failed for a particular process and service.