About Direct Server Return
The direct server return feature enables a server to respond directly to clients without having to go through the load balancer, which eliminates a bottleneck in the server-to-client path. In traditional load balancer deployments, the load balancer is in the path of the client-to-server communication: both the client-to-server request path and the server-to-client response path. While the amount of data in the requests from the client-to-server direction are relatively small, the server-to-client response traffic is much higher: approximately 10 times that of client-to-server request data. The load balancer in the path of this high volume response traffic becomes a bottleneck and adversely affects the communication.
For direct server return deployments, a virtual IP address is shared by the load balancer and server. Clients always address their request to the virtual IP address that is intended to reach the load balancer, and the direct response from the server-to-client use this virtual IP address as the source address. Cisco Application Centric Infrastructure (ACI) enabled with data-path learning of the IP source address poses problems when it learns the virtual IP address from the server-to-client traffic, leading to the disruption of Client-to-load balancer request traffic. To allow for the proper operation of a direct server return deployment, the ACI fabric must ensure that the request-response traffic between the communicating endpoints are delivered to their intended destination correctly. This requires that the data-path IP address learning on the leaf switches must be controlled in such a way that there is no interruption to client-to-load balancer, load balancer-to-server, and server-to-client traffic.
The following figure illustrates the data path in a direct server return deployment:
-
The load balancer and all of the back-end servers are configured with the virtual IP address. The load balancer alone responds to Address Resolution Protocol (ARP) requests for this virtual IP address. After load balancing the client request, the load balancer re-writes the destination MAC address in the packet and forwards the MAC address to one of the back-end servers.
-
The virtual IP address is configured on the back-end server, but ARP is disabled to prevent back-end servers from responding to ARP requests for this virtual IP address.
-
The server sends the return traffic directly to the client, by-passing the load-balancer.
Layer 2 Direct Server Return
Layer 2 direct server return is the common or traditional deployment, also known as direct routing, SwitchBack, or nPath. In this deployment, the virtual IP address is shared by the load balancer and server. The load balancers and servers must be layer 2 adjacent. A layer 2 direct server return deployment has the following limitations:
-
You lose flexibility in server placement
-
You need an extra server configuration to suppress Address Resolution Protocol (ARP) responses to client virtual IP address requests
-
Port selection is layer 3 and protocol dependent; port selection cannot happen at layer 2 (load balancer to server communication)
A layer 2 direct server return deployment has the following traffic flow:
-
Client to load balancer
Source IP Address
1.1.1.1
Destination IP Address
2.2.2.2
Destination MAC Address
A.A.A
-
Load balancer to server
Source IP Address
1.1.1.1
Destination IP Address
2.2.2.2
Destination MAC Address
B.B.B
-
Server to client
Source IP Address
2.2.2.2
Destination IP Address
1.1.1.1
Destination MAC Address
MAC address of the default gateway
About Deploying Layer 2 Direct Server Return with Cisco Application Centric Infrastructure
The following information applies to deploying layer 2 direct server return with Cisco Application Centric Infrastructure (ACI):
-
The virtual IP address (2.2.2.2) moves within the ACI fabric
-
The load balancer-to-server and server-to-client traffic with the same source virtual IP address (2.2.2.2)
-
The server-to-client traffic is routed; the traffic is addressed to the gateway MAC address in the fabric
-
The data-path learning of the source IP address from the server moves to the virtual IP address within the fabric
-
-
There are no issues for the client IP address (1.1.1.1) appearing from difference sources
-
The client IP address appears as the source IP address from both the client and the load balancer in the fabric
-
The load balancer and server are layer 2 adjacent and the load balancer-to-server traffic is layer 2 forwarded
-
There is no data-path IP address learning from layer 2 forwarded traffic in the fabric
-
Even if the client IP address appears as the source IP address from the load balancer in the fabric, the client IP address is not learned
-
Guidelines and Limitations for Configuring Direct Server Return
Follow these guidelines and limitations when deploying Direct Server Return:
-
The VRF where the VIP is deployed must be set to "enforced" mode.
-
The VRF must be set to "ingress" enforcement.
-
Shared services are not supported for this configuration.
-
EP Move Detection Mode: GARP-based Detection must be enabled for the bridge domain.
-
Unicast routing must be enabled for the bridge domain.
-
The EPG where the VIP is located must have a contract associated with it (the contract drives the configuration in hardware).
-
The Layer 4 to Layer 7 VIP option is configurable only under EPG but not under the EPG collection for VRF (also known as vzAny).
-
Client to VIP traffic must always go through the proxy spine.
-
The load balancer must be in one-armed mode.
-
The server and load balancer EPG must be the same device or the load balancer EPG must be deployed on all server EPG ToRs.
-
The server EPG and load balancer EPG must be in the same bridge domain.
-
Configuring a Layer 4 to Layer 7 virtual IP (VIP) address under microsegmented EPGs or their corresponding Base EPGs is not supported.
Supported Direct Server Return Configuration
The following figure illustrates the supported direct server return configuration:
The following information applies to the supported configuration:
- The server load balancer and servers are in the same subnet and bridge domain
- The server load balancer should operate in 1 ARM mode; the inside and outside legs of server load balancer should point to the same bridge domain
- The consumer and provider endpoint groups should be under the same private network; no shared service configuration is supported