Packet I/O on IOS XR
This section illustrates how, with the Packet I/O functionality, you can use Linux applications to manage communication with the IOS XR interfaces. It describes how the OS environment must be set up to establish packet I/O communication with hosted applications.
Exposed IOS-XR Interfaces in Linux
Feature Name |
Release Information |
Description |
---|---|---|
Automatic Synchronization of Secondary IPv4 addresses from XR to Linux OS |
Release 7.9.1 |
Now the configured interface secondary IPv4 addresses on the Cisco IOS XR software are automatically synchronized to Linux operating system. The third-party applications on Cisco IOS XR can use the secondary IPv4 addresses without any manual intervention. Earlier, you had to configure the secondary IPv4 addresses on the Linux operating system manually. |
The secondary IPv4 addresses that are configured for an XR interface are now synchronized into the Linux operating system automatically. With this secondary IPv4 address synchronization, the third party applications that are deployed on Cisco IOS XR can now use the secondary IPv4 addresses. Prior to this release, only primary IPv4 addresses were supported and the secondary IPv4 addresses had to be configured manually in the Linux operating system.
Exposed XR interfaces (EXIs) and address-only interfaces support secondary IPv4 address synchronization:
-
EXIs have secondary IP addresses added to their corresponding Linux interface
-
Address-only interfaces have secondary IP addresses added to the Linux loopback device. For additional information on address-only interfaces, see show linux networking interfaces address-only.
The restrictions of secondary IPv4 addresses synchronization are:
-
Secondary IPv4 addresses are not synchronized from Linux to XR for Linux-managed interfaces.
-
The ifconfig Linux command only displays the first configured IPv4 address. To view the complete list of IPv4 addresses, use the ip addr show Linux command.
For additional information on secondary IPv4 addresses, see ipv4 address (network).
You can run bash commands at the IOS XR router prompt to view the interfaces and IP addresses stored in global VRF. When you access the Cisco IOS XR Linux shell, you directly enter the global VRF.
SUMMARY STEPS
- From your Linux box, access the IOS XR console through SSH, and log in.
- View the ethernet interfaces on IOS XR.
- Check the IP and MAC addresses of the interface that is in
Up
state. Here, interfacesHundredGigE0/0/0/24
andMgmtEth0/RP0/CPU0/0
are in theUp
state. - Verify that the bash command runs in global VRF to view the network interfaces.
- Access the Linux shell.
- (Optional) View the IP routes used by the
to_xr
interfaces.
DETAILED STEPS
Step 1 |
From your Linux box, access the IOS XR console through SSH, and log in. Example:
|
||
Step 2 |
View the ethernet interfaces on IOS XR. Example:
|
||
Step 3 |
Check the IP and MAC addresses of the interface that is in Example:
|
||
Step 4 |
Verify that the bash command runs in global VRF to view the network interfaces. Example:
The |
||
Step 5 |
Access the Linux shell. Example:
|
||
Step 6 |
(Optional) View the IP routes used by the Example:
|
Setting up Virtual IP Addresses
Feature Name |
Release Information |
Description |
---|---|---|
Virtual IP address in the Linux networking stack |
Release 7.5.2 |
Virtual IP addresses allow a single IP address to connect to the current active RP after an RP switchover event. In addition, this functionality enables your network stack to support virtual IP addresses for third-party applications and IOS XR applications that use the Linux networking stack. The following commands are modified: |
Interfaces configured on IOS XR are programmed into the Linux kernel. These interfaces allow Linux applications to run as if they were running on a regular Linux system. This packet I/O capability ensures that off-the-shelf Linux applications can be run alongside IOS XR, allowing operators to use their existing tools and automate deployments with IOS XR.
The IP address on the Linux interfaces, MTU settings, MAC address are inherited from the corresponding settings of the IOS XR interface. Accessing the global VRF network namespace ensures that when you issue the bash command, the default or the global VRF in IOS XR is reflected in the kernel. This ensures default reachability based on the routing capabilities of IOS XR and the packet I/O infrastructure.
Virtual addresses can be configured to access a router from the management network such as gRPC using a single virtual IP address. On a device with two or more RPs, the virtual address refers to the management interface that is currently active. This functionality can be used across RP failover without the information of which RP is currently active. This is applicable to the Linux packet path.
Procedure
Command or Action | Purpose |
---|---|
You can use the following commands to verify the IP Address in the Linux networking stack: |
|
Third-Party Application Networking in Named VRFs
Feature Name |
Release Information |
Description |
---|---|---|
Virtual Routing and Forwarding for Linux Third-Party Applications using Data Port |
Release 7.9.1 |
This feature empowers you to run your native Linux applications on Cisco IOS XR as-is, without any modifications. You can now configure a host of utilities that allows for easy integration of Linux devices and applications. These utilities allow applications hosted in containers to interact with native Cisco IOS XR applications (hosted in the XR control plane). The following commands are modified:show linux networking vrfs. |
Cisco IOS XR now supports the use of standard Linux APIs to send and receive packets, update routes, interface state, interface IP addresses, and so on.
The supported utilities are:
-
Default Route Source Address
-
East-West Communication
-
Hardware LPTS Support for Traffic Protection
-
Management Route Export
-
Automatic Mapping of Deprecated TPA Configuration
-
Software Forwarding
-
Statistics Synchronization
-
VRF Disable
Default Route Source Address
The Default Route Source Address utility allows you to specify an interface in which the address should be used as the source hint on Linux's default route.
This source hint is used for traffic where:
-
The application is not bound to a specific address.
-
The traffic is destined over a nonconnected route. This is commonly seen as Rx-inject traffic and represents most of the traffic that is sent by Linux.
Ensure that the interface is synchronized to Linux, to qualify as a valid source hint interface.
-
Its VRF must not be disabled.
-
On XR platforms, it must not be the East-West interface.
-
It is a supported interface type.
-
If explicitly configured, it must be in the specified VRF.
The following configuration parameters are used to select the interface to be used:
-
If an interface is specified explicitly and valid, it is used.
-
If active-management is specified, the lowest-numbered valid management interface on the active RP is used. The identity of this interface will change after RP switchover.
-
If no configuration is specified, the lowest-numbered valid loopback interface in the VRF is used.
The address that is chosen from the selected source hint interface depends on the address family:
-
IPv4: The primary address is used, when present. Secondary addresses are not considered.
-
IPv6: The IP address that is numerically the lowest is used.
vrf blue
!
linux networking
vrf blue
east-west Loopback3
address-family ipv4
source-hint default-route interface Loopback2
!
address-family ipv6
source-hint default-route interface Loopback2
!
!
!
interface Loopback2
vrf blue
ipv4 address 192.0.2.1 255.255.255.255
ipv6 address 2001:db8::1/128
!
interface Loopback3
vrf blue
ipv6 address 2001:db8::ea57/128
!
Use the following show command to verify whether the default source hint interface address is configured:
RP/0/RP0/CPU0:ios#show linux networking vrfs vrf blue
VRF blue (Linux network namespace vrf-blue):
Status: active
IPv4 default route source hint: 192.0.2.1
IPv6 default route source hint: 2001:db8::1
IPv4 XR East-West: none
IPv6 XR East-West: 2001:db8::ea57
tpa
vrf < vrf-name >
address-family { ipv4 | ipv6 }
update-source dataports { < interface > | active-management }
East-West Communication
The East-West Communication utility allows you to specify a Cisco IOS XR interface that should be used for communication between Linux and Cisco IOS XR applications.
Configuring an interface as East-West for a virtual routing and forwarding (VRF) ensures that all listed addresses are reserved for East-West communication, with the following behaviour:
-
Traffic cannot be routed from Linux to other devices using this IP address.
-
Traffic destined to the listed addresses cannot be received by Linux applications.
-
The IP addresses will not appear in Linux.
-
For Linux applications: Traffic might be sourced from any local IP address present in Linux. Traffic must be sent to one of the reserved East-West IP addresses.
-
For Cisco IOS XR applications: Traffic must be sourced from one of the reserved East-West IP addresses. Traffic might be sent to any local IP address present in Linux.
Ensure the following, for the interface to be qualified as a valid East-West interface:
-
Be in a VRF that is not disabled.
-
Have one or more IP addresses.
-
The following configuration is used to select the interface to be used:
-
If an interface is specified explicitly and valid, it is used.
-
If no configuration is specified, Loopback1 is used.
-
-
All IP addresses on the interface are reserved for East-West.
vrf blue
!
linux networking
vrf blue
east-west Loopback3
address-family ipv4
source-hint default-route interface Loopback2
!
address-family ipv6
source-hint default-route interface Loopback2
!
!
!
interface Loopback2
vrf blue
ipv4 address 192.0.2.1 255.255.255.255
ipv6 address 2001:db8::1/128
!
interface Loopback3
vrf blue
ipv6 address 2001:db8::ea57/128
!
Use the following show command to verify whether the east-west communication is configured:
RP/0/RP0/CPU0:ios#show linux networking vrfs vrf blue
VRF blue (Linux network namespace vrf-blue):
Status: active
IPv4 default route source hint: 192.0.2.1
IPv6 default route source hint: 2001:db8::1
IPv4 XR East-West: none
IPv6 XR East-West: 2001:db8::ea57
tpa
vrf < vrf-name >
east-west < interface >
Hardware LPTS Support For Traffic Protection
The Hardware Local Packet Transport Services (LPTS) Support for Traffic Protection utility allows you to specify traffic protection rules to be factored into as an LPTS programming that is done by the Linux Packet I/O. This is in addition to the existing method where the rules were implemented using the Linux kernel's software-based nftables firewall. The nftables firewall is a subsystem of the Linux kernel, and provides filtering and classification of network packets. The nftables firewall is retained as a fallback, but augmented by higher performance LPTS rules.
Linux Packet I/O programs the LPTS in response to Linux socket operations, to ensure that Linux clients can receive traffic from other devices. When traffic protection rules are configured, this feature applies filtering to the programmed LPTS rules to allow a restricted subset that matches the traffic protection rules.
tpa
vrf < vrf-name >
address-family { ipv4 | ipv6 }
protection
allow protocol { tcp | udp } local-port < local-port >
{ remote-address < remote-address >/< prefix-len >
| local-address < local-address >/< prefix-len >
| interface < interface-name > }
Management Route Export
The Management Route Export utility allows for a subset of Cisco IOS XR static routes that resolve over the active management interfaces to be replicated to Linux. This avoids the need for a line card NPU inject and FIB lookup for routing Linux traffic matching these management routes.
In order for the routes to be exported from Cisco IOS XR to Linux, you must ensure that the routes:
-
Resolve over the management interface.
-
Are static.
-
Not recursive.
-
Not the default XR route.
A specified source hint interface is qualified only if:
-
Its VRF is not disabled.
-
The interface is in the same VRF as the management interface.
-
On Cisco IOS XR platforms, it is not the East-West interface.
-
It is a supported interface type.
If the specified interface is valid, then the address that is chosen from it depends on the address family:
-
IPv4: The primary address is used, when present. Secondary addresses are not considered.
-
IPv6: The IP address that is numerically the lowest is used.
linux networking
vrf default
address-family ipv4
source-hint management-route interface Loopback2
!
!
!
interface Loopback0
ipv4 address 192.0.2.128 255.255.255.255
!
interface Loopback2
ipv4 address 192.0.2.200 255.255.255.255
!
interface MgmtEth0/RP0/CPU0/0
ipv4 address 192.0.2.1 255.255.255.240
!
router static
address-family ipv4 unicast
192.0.2.16/28 192.0.2.2
192.0.2.32/28 192.0.2.2
!
!
Use the following show command to verify whether the east-west communication is configured:
Note |
The management ethernet is directly connected to a device with the unicast route IP address. |
RP/0/RP0/CPU0:ios#bash vrf default ip route
default dev to_xr scope link src 192.0.2.128 metric 2048 mtu 1500 advmss 1460
192.0.2.1/30 dev Mg0_RP0_CPU0_0 proto static scope link src 192.0.2.200
192.0.16.0/24 via 192.0.2.2 dev Mg0_RP0_CPU0_0 proto static src 192.0.2.200 metric 2048
192.0.17.0/24 via 192.0.2.2 dev Mg0_RP0_CPU0_0 proto static src 192.0.2.200 metric 2048
The verification for source hint config is to check that all Linux routes resolving via the management ethernet interface are using the source address from the configured device. The verification for management route export is to check that all static routes resolving via the management ethernet interfaceare exported to Linux.
tpa
vrf < vrf-name >
address-family { ipv4 | ipv6 }
update-source destination < management-interface > source < interface >
Mapping of Deprecated TPA Configuration
The Automatic Mapping of Deprecated TPA Configuration utility supports seamless migrations from a Cisco IOS XR environment to Linux, with the Packet I/O functionality. The configuration
is translated from the deprecated TPA configuration (under tpa
) to Linux, with the Packet I/O configuration (under linux networking
).
The configuration will be automatically translated to the equivalent Linux Packet I/O configuration, after installation.
The following scenarios are relevant for this utility:
-
Applying deprecated TPA configuration on a Cisco IOS XR device that supports Linux Packet I/O.
-
Upgrading a Cisco IOS XR device from a version that does not support Linux Packet I/O, to a version that supports Linux Packet I/O.
-
Downgrading a Cisco IOS XR device from a version that supports Linux Packet I/O, to a version that does not support Linux Packet I/O.
-
The deprecated configuration is available until all Cisco XR platforms are migrated to support Linux Packet I/O.
Note |
Downgrading to an unsupported version of Linux Packet I/O cannot be done automatically. The definitions required to support Linux Packet I/O configuration does not exist on releases earlier to Cisco IOS XR Release 7.9.1. |
Software Forwarding
The Software Forwarding utility allows you to choose software forwarding over hardware forwarding. Software forwarding is provided primarily for compatibility with Cisco IOS XR networking stack, where hardware forwarding could not route packets over the management interface.
When software forwarding is configured, the Net I/O will be used for forwarding packets. The packet path might be slow, although no change to Linux reachability is noticeable. You can use software forwarding to avoid injecting traffic toward line card NPUs in scenarios where the Linux traffic in a VRF will be sent over management interfaces.
linux networking
vrf default
address-family ipv6
default-route software-forwarding
!
!
!
tpa
vrf < vrf-name >
address-family { ipv4 | ipv6 }
default-route mgmt
Statistics Synchronization
The Statistics Synchronization utility allows you to specify the intervals when interface statistics for all interfaces are synchronized to Linux, when using the Linux ethtool interface, to gather interface statistics.
For supported configurations, Cisco IOS XR's statsd infra is polled at specified intervals to retrieve cached interface statistics for all interfaces that are exposed to Linux, as an exposed Cisco IOS XR interface (those visible to the Linux ip link command).
However, statistics are not gathered for interfaces in disabled VRFs, or for those interfaces which are not synchronized to Linux as an exposed interface.
This example shows how the bundle-ether interface packet statistics are synchronized between Cisco IOS XR and Linux. The packet and byte counters that are maintained by Linux for Cisco IOS XR interfaces display only the traffic that is sourced in Linux. You can configure to periodically synchronize these counters with the Cisco IOS XR statistics for the interfaces.
-
Following is the configure for statistics synchronization, including the direction and synchronization interval. linux networking statistics-synchronization from-xr every { 30s | 60s | 2m | 3m | 4m | 5m | 6m | 7m | 8m | 9m | 10m }
The following example shows statistics synchronization in global configuration: Router(config)#linux networking statistics-synchronization from-xr every 30s
The following example shows statistics synchronization in exposed-interface configuration: Router(config)#linux networking exposed-interfaces interface bundle-ether 1 statistics-synchronization from-xr every 10s
where—
-
from-xr: The direction indicating that the interface packet statistics will be pushed from Cisco IOS XR to the Linux kernel.
-
every: Shows the frequency at which to synchronize statistics. The intervals that are supported for global configuration are 30s and 60s. The intervals that are supported for exposed interfaces are 5s, 10s, 30s, or 60s. The interval
s
is in seconds.
-
-
Verify that the statistics synchronization is applied successfully on Cisco IOS XR. Router#show run linux networking linux networking vrf default address-family ipv4 protection protocol tcp local-port all default-action deny permit interface bundle-ether 1 ! ! ! ! exposed-interfaces interface bundle-ether 1 linux-managed statistics-synchronization from-xr every 10s ! ! !
You can use the show tech-support linux networking command to display debugging information, with regard to statistics synchronisation.
tpa
statistics update-frequency < 1 - 99999999 >
Note |
The integer values here are mapped to the nearest matching value in supported configuration:
|
VRF Disable
The VRF Disable utility enables you to specify the virtual routing and forwarding (VRF) that should not be synchronized to Linux, and will not be used by applications using the Linux packet path. This configuration improves performance. Communication using Linux Packet I/O (including East-West communication) will not be functional in the VRF or network namespace which was disabled.
The usage of the VRF Disable utility depends on whether you are using the Cisco IOS XR default VRF or the nondefault VRF:
-
For the default VRF, no interfaces, routes, or addresses are synchronized to Linux, but a network namespace called "vrf-default" still exists.
-
For nondefault VRFs, the corresponding network namespace is deleted.
vrf green
!
linux networking
vrf green
disable
!
!
Use the following show command to verify whether the VRF is disabled:
RP/0/RP0/CPU0:ios#show linux networking vrfs vrf green
VRF green (Linux network namespace not created):
Status: VRF disabled
tpa
vrf < vrf-name >
disable
Program Routes in Linux
The basic routes required to allow applications to send or receive traffic can be programmed into the kernel. The Linux network stack that is part of the kernel is used by normal Linux applications to send/receive packets. In an IOS XR stack, IOS XR acts as the network stack for the system. Therefore to allow the Linux network stack to connect into and use the IOS XR network stack, basic routes must be programmed into the Linux Kernel.
Procedure
Step 1 |
View the routes from the bash shell. Example:
|
Step 2 |
Programme the routes in the kernel. Two types of routes can be programmed in the kernel:
|
Configure VRFs in Linux
VRFs configured in IOS XR are automatically synchronized to the kernel. In the kernel, the VRFs appear as network namespaces (netns). For every globally-configured VRF, a Linux network namespace is created. With this capability it is possible to isolate Linux applications or processes into specific VRFs like an out-of-band management VRF and open-up sockets or send or receive traffic only on interfaces in that VRF.
Every VRF, when synchronized with the Linux kernel, is programmed as a network namespace with the same name as a VRF but with
the string vrf
prefixed to it. The default VRF in IOS XR has the name default
. This name gets programmed as vrf-default
in the Linux kernel.
The following example shows how to configure a custom VRF blue
:
Procedure
Step 1 |
Identify the current network namespace or VRF. Example:
|
Step 2 |
Configure a custom VRF Example:
|
Step 3 |
Verify that the VRF Example:
|
Step 4 |
Verify that the VRF Example:
|
Step 5 |
Access VRF Example:
to-xr interface because there is no IOS XR interface in this VRF.
|
Step 6 |
Configure an interface in the VRF Example:vrf-blue from IOS XR:
|
Step 7 |
Verify that the HundredGigE 0/0/0/24 interface is configured in the VRF Example:
|
Step 8 |
Verify that the interface is configured in the VRF Example:
|
Open Linux Sockets
The socket entries are programmed into the Local Packet Transport Services (LPTS) infrastructure that distributes the information through the line cards. Any packet received on a line card interface triggers an LPTS lookup to send the packet to the application opening the socket. Because the required interfaces and routes already appear in the kernel, the applications can open the sockets — TCP or UDP.
Procedure
Step 1 |
Verify that applications open up sockets. Example:
|
Step 2 |
Verify that the socket is open. Example:
Netcat starts listening on port 5000, which appears as an IPv4 TCP socket in the netstat output like a typical Linux kernel. This socket gets programmed to LPTS, creating a corresponding entry in the hardware to the lookup tcp port 5000. The incoming traffic is redirected to the kernel of the active RP where the netcat runs. |
Send and Receive Traffic
Connect to the nc socket from an external server. For example, the nc socket was started in the vrf-default
network namespace. So, connect over an interface that is in the same VRF.
[root@localhost ~]#nc -vz 192.168.122.22 5000
Ncat: Version 7.50 ( https://nmap.org/ncat )
Ncat: Connected to 192.168.122.22:5000.
Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds.
Manage IOS XR Interfaces through Linux
The Linux system contains a number of individual network namespaces. Each namespace contains a set of interfaces that map to a single interface in the XR control plane. These interfaces represent the exposed XR interfaces (eXI). By default, all interfaces in IOS XR are managed through the IOS XR configuration (CLI or YANG models), and the attributes of the interface (IP address, MTU, and state) are inherited from the corresponding configuration and the state of the interface in XR.
With the new Packet I/O functionality, it is possible to have an IOS XR interface completely managed by Linux. This also means that one or more of the interfaces can be configured to be managed by Linux, and standard automation tools can be used on Linux servers can be used to manage interfaces in IOS XR.
Note |
Secondary IPv4 addresses cannot be managed by Linux. |
Configure an Interface to be Linux-Managed
This section shows how to configure an interface to be Linux-managed.
Procedure
Step 1 |
Check the available exposed-interfaces in the system. Example:
|
Step 2 |
Configure the interface to be managed by Linux. Example:
|
Step 3 |
View the interface details and the VRF. Example:
|
Step 4 |
Verify the configuration in XR. Example:
|
Step 5 |
Verify the configuration from Linux. Example:
|
Configure New IP address on the Interface in Linux
This section shows how to configure a new IP address on the Linux-managed interface.
Procedure
Step 1 |
Configure the IP address on the interface. Example:
|
Step 2 |
Verify that the new IP address is configured. Example:
|
Configure Custom MTU Setting
This section shows how to bring up the interface and configure a custom MTU in a Linux-managed interface.
Procedure
Step 1 |
Configure the MTU setting. Example:
|
Step 2 |
Verify that the MTU setting has been updated in Linux. Example:
|
Step 3 |
Check the effect on the IOS XR configuration with the change in MTU setting on this interface. Example:
The output indicates that the interface acts as a regular Linux interface, and IOS XR configuration receives inputs from Linux. |
Configure Traffic Protection for Linux Networking
Traffic protection provides a mechanism to configure Linux firewalls using IOS XR configuration. These rules can be used to restrict traffic to Linux applications. You can restrict traffic to Linux applications using native Linux firewalls or configuring IOS XR Linux traffic protection. It is not recommended to use both mechanisms at the same time. Any combination of remote address, local address and ingress interface can be specified as rules to either allow or deny traffic. However, at least one parameter must be specified for the traffic protection rule to be valid.
Note |
If traffic is received on a protocol or port combination that has no traffic protection rules configured, then all traffic is allowed by default. |
This example explains how to configure a traffic protection rule on IOS XR to deny all traffic on port 999 except for traffic arriving on interface HundredGigE0/0/0/25.
Procedure
Step 1 |
Configure traffic protection rules. Example:
where —
|
Step 2 |
Verify that the traffic protection rule is applied successfully. Example:
|