The following restrictions apply to MLP over Serial Interfaces:

• The MLP over Serial Interfaces feature supports a maximum of ten member links per bundle. The member links can be any combination of T1/E1 or fractional T1s/E1s (for example, NxDS0). Member-link interface speed above T1/E1 is not supported in the MLP over Serial Interfaces feature. For better MLP performance, all the member links in a bundle must be of the same bandwidth.
• Member links in a bundle cannot be of different encapsulation types.
• You cannot manually configure the bandwidth of an MLP bundle by using the bandwidth command on the multilink interface. The bandwidth of an MLP bundle is managed based on the aggregate bandwidth of all the active member links on the bundle. As the links are dynamically added or removed from an MLP bundle, the bandwidth is updated to reflect the aggregate of the active links. The bandwidth can be rate limited by applying an hierarchical QoS policy on the multilink interface and applying a shaper to the parent class-default class.
• MLP over Frame Relay is not supported; only MLP over Serial PPP link is supported. Customers who require multilink support in a frame relay environment can use the Multilink Frame Relay (MLFR-FRF.16) feature.
• The legacy IOS compression feature compress [mppc | stac | predictor] is not supported.
• LFI is supported on MLP bundles with any number of links in the bundle. However, when using a bundle with more than one member link, the order of the priority packets (PPP encapsulated) is not guaranteed. Priority-packet distribution is handled in a manner similar to IP per-packet load sharing. MLP guarantees nonpriority packet ordering that manages reordering at the peer device, based on the MLP packet sequence number.
• Order issues with the LFI multiple-member link in case of priority traffic can be addressed in some platforms using Multiclass Multilink Protocol (MCMP-RFC 2686), which is an extension of the MLP. The Cisco ASR 1000 Series Aggregation Services Routers do not support MCMP.
• Only the MLP long-sequence number format is supported for the packet header format option.

The following restrictions apply to MLP over Ethernet at PTA and LAC:

• MLPoE using EtherChannel is not supported.
• For MLP virtual access bundles, the default Layer 3 (that is IP and IPv6) maximum transmission unit (MTU) value is 1500. For more information about MTU, see the MTU section.
• For MLPoE PTA variations (MLPoE, MLPoVLAN, and MLPoQinQ), the default bandwidth of the member-link session is 1 Gbps instead of the data rate communicated by the DSLAM to the PTA router. If a bandwidth statement is added to the virtual template, the bandwidth is applied to the bundle instead of the member link. This is not the desired behavior. (To define the data rate of an MLPoE PTA-type bundle, apply a QoS policy on the bundle session that includes a parent shaper on the class-default class with an explicit data rate defined. Do not use the shape percent command in this parent shaper because the shape percent command uses the default data rate of 1 Gbps as the base rate for percent calculation. However, the percent-based rates can be defined in the child (nested) policy, if an hierarchical policy is being defined.
• If the DSLAM between the CPE and PTA communicates the link rate through the PPPoE dsl-sync-rate tags (Actual Data-Rate Downstream [0x82/130d] tag), the PTA device passes this data to the RADIUS server, but the Cisco ASR 1000 Series Aggregation Services Routers do not act upon it. The data rate of the session remains as described in the previous list item.

The following restrictions apply to MLP over ATM at PTA and LAC:

• ATM Autosense is supported to allow the dynamic selection of MLPoA or MLPoEoA.
• For ATM, the link-level bandwidth is a part of the ATM Permanent Virtual Circuits (PVC) configuration based on the unspecified bit rate (UBR) or variable bit rate (VBR) configurations. The bundle bandwidth is the aggregate of the member-link session bandwidth.

Note	The MLP over Ethernet over ATM at PTA and LAC has the same restrictions as the MLP over ATM at PTA and LAC.

In case of MLP over LNS (Ethernet) LAC switching, the MLP member-link session and the packet payload is transparent at the LAC device because it does not terminate the MLP session or the bundle interface. Hence, the LAC device does not bind the number of member-link sessions associated with a bundle. Similarly, the LFI functionality is transparent at the LAC device because the traffic is switched or passed through traffic.

The following restrictions apply to MLP over LNS:

• MLPoLNS bundles are supported with only Ethernet as the trunk between the LAC and LNS.
• Layer 2 Tunnel Protocol (L2TP) over IPsec is not supported.
• QoS (other than downstream Model-F shaping) on interfaces and tunnels towards the customer premise equipment (CPE) is not supported.
• When the CPE client initiates the PPP LCP connection, the multilink negotiation included as part of the LCP negotiation may fail if the LAC has not yet established connection with the LNS (which is typically the case). The LNS renegotiates the Multilink LCP options with the CPE client when the LAC initiates the connection to the LNS. (To allow this renegotiation of LCP options, the lcp renegotiation always command must be configured in the VPDN group at the LNS).
• Although per-packet load balancing is not supported, the configuration is not blocked and the functionality is operational (but not tested). Per-packet load balancing cannot be used with MLPoLNS because MLPoLNS requires a single-path per-destination IP address.
• Unlike the MLP over Serial mode or the MLP PTA mode, packets may traverse several network hops between the CPE and LNS devices in an MLPoLNS network. As a result of this multihop topology, even on a single-link bundle, MLP encapsulated packets may arrive at the receiver in an out-of-order state. Hence, the MLPoLNS receiver operates in a loose, lost-fragment detection mode. In this mode, if an MLP fragment is lost, the received MLP waits for a short time to receive the lost fragment. In addition, the MLP receiver limits the amount of out-of-order MLP data received before the fragment is declared lost. In Cisco IOS XE software, the default timeout value is 1 second. This may create problems in an environment with high packet loss and scaled MLP configurations because it requires the receiver to potentially buffer large amounts of data for each MLP bundle. Since the buffer space that is available is a finite resource, worst-case depletion of buffers can bleed over and begin affecting packet buffering on other MLP bundles. (The MLP lost-fragment timeout can be configured on the multilink virtual template interface using the ppp timeout multilink lost-fragment (seconds) (milliseconds) configuration command).

By default, in MLPoLNS, the Cisco IOS XE software informs the MLP that packets may arrive out of order. This works well for upstream traffic, but does not address the order issue at the peer CPE device. The peer CPE device should also be configured to allow for receipt of out-of-order packets. In Cisco devices, this can be managed by configuring the ppp link reorders command at the bundle interface.

• When the Cisco ASR 1000 Series Aggregation Services Routers function as both a PTA device and an LNS device simultaneously, locally terminated member links (PTA) and member links that are forwarded from the LAC are not supported within the same bundle.

The following restrictions apply to all variations of broadband MLP at PTA and LNS modes:

• When defining an MLP bundle with multiple member-link sessions, we recommend that all the member-link sessions utilize the same physical interface or subinterface. If other broadband sessions are sharing the same interface, ensure that all the member-link sessions utilize the same physical interface or subinterface.
• The following issues might occur because of splitting links across separate physical interfaces or subinterfaces:
  • MLP is a sequenced protocol and all the packets and fragments must be reordered and reassembled at the receiver, based on the MLP sequence number before the receiver forwards them. In such a scenario, packets traversing separate physical interfaces may cause additional packet latency disparity between links due to transmission delays and other issues associated with using multiple physical paths. The reordering and reassembly processing may require additional MLP buffering at the receiver.
  • MLP on the Cisco ASR 1000 Series Aggregation Services Routers performs congestion management of the MLP bundle based on the congestion state of the member-link sessions that make up the bundle. If member-links are distributed across multiple interfaces and sufficient congestion is detected in one or more member links, the bundle may be back pressured due to the congestion even if all the links in the bundle are not congested. By keeping all the links on the same physical interface or subinterface, the chance of back pressure due to one link being congested is reduced.

QoS refers to the ability of a network to provide improved service to selected network traffic over various underlying technologies, including Frame Relay, ATM, Ethernet and 802.1 networks, SONET, and IP-routed networks. In particular, QoS features provide improved and more predictable network service.

For serial deployments, QoS is applied to an MLP bundle using the multilink configuration command. For broadband deployments, QoS is applied to an MLP bundle using the virtual-template command. When a router dynamically creates the virtual access interface from the virtual template, the QoS policy is applied to the corresponding bundle.

QoS is characterized by the following features and restrictions:

• To rate limit a broadband MLP bundle session, use a hierarchical QoS (HQoS) policy with a parent shaper in the class-default class.

• The Cisco ASR 1000 Series Aggregation Services Routers support HQoS queuing only in the egress (output) direction, and not in the ingress direction.

• The Cisco IOS XE software supports Model-F QoS with MLP. Model-F QoS on the L2TP tunnel is not supported on the Cisco ASR 1002-X Router and the FP100 line card.

  • In Cisco IOS XE Release 3.7.1S, support was added for Model-F QoS on the L2TP tunnel when the device acts as an LNS. A parent shaper policy can be applied to the physical subinterface that connects the LNS to the LAC device. This enables the shaping of the aggregate traffic going downstream to the LAC device.

  • If a Model-F shaper is attached to the LAC-facing interface after the sessions are established through that interface, the sessions must be bounced to handle the priority traffic appropriately.

• In Cisco IOS XE Release 3.4S, the shape average shape-rate account user-defined offset atm command supports only the broadband MLP interface and not the MLP over serial interface. The range for the offset argument is from -48 to 48 bytes. In Cisco IOS XE Release 3.6S, the shape average shape-rate account user-defined offset atm command supports MLP over serial interface.

• ATM cell loss priority (CLP) Match (classification) and Set (marking) are not supported with broadband MLP.

• When packets transit the MLP transmit path, they are subject to two separate stages of queuing. The first stage is at the MLP bundle interface, where QoS may be applied, and the second one is at the MLP member-link interface. At the MLP bundle interface, the packets are processed according to the applied QoS policy. Packets classified as priority are given preferential treatment over nonpriority traffic. For the priority classification to be honored at the MLP member-link interface, the bundle must have ppp multilink interleave enabled. Interleaving allows a packet to be queued to a separate priority queue at the member-link. If interleaving is not enabled on the bundle, the priority packet is placed in the member link session default queue and the knowledge that it is a priority packet will be lost. This is especially important if there are other PPP or MLP sessions sharing the same physical interface or subinterface. Without interleaving, priority traffic on the other sessions are given preferential treatment over the MLP priority packets that were reclassified as nonpriority packets at the MLP member-link queuing stage. For additional information on interleaving, see the Downstream LFI section.

The interface-level bandwidth command must not be used to define the bandwidth at the bundle level on the virtual template interface or the multilink interface. By default, the bundle bandwidth is the aggregate of the bandwidth of the individual member links that make up the bundle.

For ATM, the link-level bandwidth is part of the ATM Permanent Virtual Circuits (PVC) configuration based on the unspecified bit rate (UBR) or variable bit rate (VBR) configurations. The member-link bandwidth cannot be set for an MLPoE session on a PTA device. To define the bandwidth for an MLPoE-type bundle on a PTA device, a QoS policy must be applied to the bundle interface that shapes the bundle bandwidth at the class-default class with a parent shaper.

In PPPoE and MLPoE broadband networks, the DSL access multiplexer (DSLAM) placed between the customer premises equipment (CPE) and LAC or PTA, inserts a PPPoE vendor tag. This tag includes information such as, media rate, characteristics, and identification pertaining to the circuit or session.

For more information about Ethernet-based networks, see DSL Forum TR-101 Migration to Ethernet-Based DSL Aggregation April 2006 at:

http:/​/​www.broadband-forum.org/​technical/​download/​TR-101.pdf http://www.broadband-forum.org/technical/download/TR-101.pdf

The PTA passes media-rate information to the RADIUS server for selecting an appropriate QoS policy to the bundle session based on the reported bandwidth. In the context of MLP over LNS, the LAC passes media-rate information to both the RADIUS server and the LNS router. The LNS router uses the media-rate information to define the bandwidth of the corresponding member-link session. If the upstream connection at the LAC is MLPoE, MLPoEoVLAN, or MLPoEoQinQ, the DSLAM may provide the media rate information to the LAC. If the DSLAM does not provide the media rate, the member-link session bandwidth can be configured using the l2tp tx-speed rate and l2tp rx-speed rate commands within the vpdn-group configuration command or downloaded from the RADIUS server using the l2tp-tx-speed and l2tp-rx-speed attributes.

For MLP Virtual Access bundles (IP and IPv6), the default Layer 3 MTU value is 1500. When the MLP bundle's member links are Ethernet, as in MLPoE, MLPoEoVLAN, and MLPoEoQinQ, the default MTU value of 1500 may cause an issue when sending IP packets that are close to this size.

For example, when a router sends a 1500-byte IP packet over MLPoE, the actual packet size transmitted is 1528: 14 (Ethernet header) + 8 (PPPoE header) + 6 (MLP header) + 1500 (IP) = 1528. The peer router drops the incoming packet as a giant because it exceeds the default expected maximum packet size.

The 1500-byte MTU size does not take into account any PPPoE or MLP header overhead, and hence, causes packets greater than 1493 bytes to be dropped by the peer. To address this issue, perform one of the following tasks:

• Lower the MTU on the MLP bundle to 1492.

• Increase the MTU on the Ethernet interface to 9216, the maximum MTU size supported on Cisco ASR 1000 Series Aggregation Services Routers.

Note	In Cisco ASR 1000 Series Route Processor 1 (RP1), 2RU, and 2RU-Fixed chassis, the MTU size for the Management Ethernet interface (interface gigabitethernet 0) is up to 2370 bytes.

Although LFI is thought of as a single feature, it is actually two independent features within MLP. MLP link fragmentation allows larger packets to be Layer 2 fragmented by MLP, and the fragments to be distributed across the various member links in the MLP bundle. These fragments are MLP encapsulated and sequenced. These fragments are then collected, reordered, and reassembled at the peer termination point for the MLP bundle interface.

Note	For more information about interleaving with QoS, see the Quality of Service section..

Interleaving enables you to reduce transmission delay on delay-sensitive voice, video, and interactive application data by interleaving it with the MLP fragments. When interleaving is configured, the packets on the bundle interface that QoS classifies as priority packets are interleaved. These priority packets are PPP encapsulated and interleaved with the MLP-encapsulated fragments or packets. When the peer router receives the PPP packets, they can be immediately forwarded, whereas, the received MLP-encapsulated packets have to be reordered and reassembled before being forwarded. While link fragmentation and interleaving can be configured on any multilink bundle, this LFI functionality is beneficial only on bundles of 1 Mbps or less. Packet transmission delays of higher bandwidth bundles are such that QoS prioritization of priority traffic should be sufficient to guarantee preferential treatment of the priority traffic without the need for LFI.

One downside of interleaving is that when there are two or more links in an MLP bundle, the order of the PPP-encapsulated packets cannot be guaranteed. In most applications sending data, such as, voice, video, and Telnet, this is not an issue because the gap between the packets on a given flow is large enough that the packets must not pass each other on the multiple links in the bundle. Since the order cannot be guaranteed for the priority PPP-encapsulated packets that are interleaved, IP Header Compression (IPHC) is skipped on any packet that is classified as priority-interleaved packet. IPHC continues to occur for nonpriority packets that are sent as MLP encapsulated because MLP guarantees reordering before the packets are forwarded to IPHC.

The Multi-Class Multilink Protocol (MCMP) (RFC-2686) addresses the issues related to ordering of priority-interleaved packets. Currently, the MCMP is not supported on the Cisco ASR 1000 Series Aggregation Services Routers.

MLP LFI must be configured on the Cisco ASR 1000 Series Aggregation Services Routers to enable LFI.

In the context of interface multilink or interface virtual template, use any of the following commands to enable link fragmentation:

• ppp multilink fragment delay (delay in milliseconds)

• ppp multilink fragment delay (maximum fragment size, in bytes)

• ppp multilink interleave

For MLP using serial links, link fragmentation can also be enabled by configuring the ppp multilink fragment size (maximum fragment size, in bytes) command on the member-link serial interface.

If the MLP bundle has only one active member link and interleaving is not enabled, MLP fragmentation is disabled. In addition, all the packets are sent PPP encapsulated instead of MLP encapsulated. When a second link in the bundle becomes active or interleaving is enabled, MLP and fragmentation is enabled.

If the ppp multilink interleave command is not configured, only MLP link fragmentation is enabled. To enable interleaving, you must also configure the ppp multilink interleave command at the interface multilink level or the interface virtual template level. In addition to configuring interleaving as indicated here, you must also define a QoS policy with one or more priority classes, and attach the QoS to this interface using the service-policy output policy-map-name command. This command classifies the priority traffic, that is interleaved by the MLP.

See the QoS and LFI configuration examples in the “Configuring Multilink PPP Connections” chapter in the Wide-Area Networking Configuration Guide: Multilink PPP.

When configuring MLP fragmentation on the various Cisco platforms, the functionality of MLP fragmentation and interleaving support on the various platforms may differ. This section explains the configuration options and their interpretation in the context of the Cisco ASR 1000 Series Aggregation Services Routers.

Based on the values of the MLP fragmentation configuration commands, the MLP feature calculates two values that are used during MLP fragmentation: link weight and maximum fragment size. These parameters are calculated for each member link in the bundle.

First, a link weight must be determined for each member link. The link weight indicates the number of bytes, and the MLP uses this value to balance the data amongst the links in the bundle. This parameter is especially important when the links in a bundle are of unequal bandwidth. The link weight is based on a combination of the bandwidth of the member link and the PPP multilink fragment delay value. If you do not configure the fragment delay value, a default delay value of 30 milliseconds is used:

Link Weight = (Member Link Interface Bandwidth in bps/8) * Fragment Delay

Caution

Configuring the fragment delay to a smaller value results in smaller fragment size because the fragment delay value determines the default fragment size on the member link. This, in turn, implies loss of bandwidth due to the added Layer 2 header overhead. This is important for broadband MLP, which can have Layer 2 headers of 4 to 58 bytes in length.

The default maximum fragment size must be calculated per member link. The default maximum fragment size used will be the lesser value obtained from either of the following calculations:

• Link Weight – Multilink PPP + PPP Header Overhead (8)

• Interface MTU – Multilink PPP Header Overhead (4)

After the default maximum fragment size is calculated, if you have configured the ppp multilink fragment size (maximum) command at the multilink, virtual template, or serial interface level, the default maximum fragment size is compared against the configured maximum value and is capped accordingly. If the fragment size is configured at the serial interface level and the multilink interface level, the serial interface configuration takes precedence.

Effective from Cisco IOS XE Release 3.7.(0)S, support for the IP Type of Service (ToS) Reflect feature was added on the VPDN group or VPDN template for the L2TP tunnel when the Cisco ASR 1000 Series Aggregation Services Routers act as LNS devices for broadband MLP sessions. Later, this feature was also added to the following maintenance releases: Cisco IOS XE 3.4.2, 3.5.1, and 3.6.2.

The IP Type of Service (ToS) Reflect feature allows the IP header ToS value from the inner IP header to be reflected in the ToS of the outer L2TP IP header.

Caution

To prevent MLP packet reordering and fragment or packet holes, the ToS data should not be used to reclassify and requeue or drop packets at the LAC. Any drops or reordering of MLP packets may cause MLP reordering or reassembly delays and additional packet loss in the receiving CPE device.

The following example shows how to configure IP ToS reflect:

vpdn-group vpdn-1
accept-dialin
protocol l2tp
virtual-template 1
session-limit 100
terminate-from hostname VPDN-1
lcp renegotiation always
no l2tp tunnel authentication
ip tos reflect

Effective from Cisco IOS XE Release 3.7.1, support was added for setting the ToS value in the outer L2TP IP header using the QoS set tunnel action or the policer set tunnel action.

The following configuration options of the set actions are supported when applied to the output QoS policy of the multilink virtual template interface. This functionality is not supported in the Model-F QoS policy attached to the member-link parent subinterface.

• set ip dscp tunnel xx
• set ip prec tunnel xx
• set dscp tunnel xx
• set prec tunnel xx
• police set-dscp-tunnel-transmit xx
• police set-prec-tunnel-transmit xx

The following example shows how to set the ToS value using the police set-prec-tunnel-transmit option:

policy-map ppp
class class-default
police cir 4000000 conform-action set-prec-tunnel-transmit 3
Set action example:
policy-map ppp
class gold
set ip prec tunnel 1