Cisco SFS 7000 Series InfiniBand Server Switches

Since the disaggregation of the mainframe began in the 1980s, servers and storage continue to commoditize, lowering the costs of individual components but increasing the number of managed parts. This increase in volume, combined with the ever-increasing need to keep pace with new business applications, results in more management expenses in an increasingly complex datacenter. To reduce these management costs and improve time to market, the promise of utility or on-demand computing has emerged. But this promise has been largely unrealized as old proposals focus on individual components instead of the whole solution, neglecting the application or network infrastructure that connects servers together. There has to be a better way.

Introducing VFrame, a datacenter provisioning and orchestration product from Cisco Systems. VFrame enables the delivery of utility computing in the datacenter, or the ability to rapidly commission and decommission a shared pool of server and I/O resources on demand. VFrame takes a fresh networking approach to provisioning datacenter resources by integrating orchestration and control intelligence into the infrastructure that interconnects resources together.

VFrame is a systems management software product that creates virtual "compute services" by programming and coordinating a fabric of server switches. A server switch is a new class of data center infrastructure that provides a platform to interconnect discreet server resources together into a high performance fabric, to connect that server fabric with shared pools of I/O and storage resources. When those resources are mapped together, it creates a virtual "compute service". Similar to how a VLAN operates in the Ethernet world today, a compute service groups physical servers, storage resources, and IP resources into resource pools. VFrame provisions these compute services based on any number of criteria, including business application, time-of-day, required compute power, or standby servers for higher availability.

VFrame enables administrators to:

VFrame enables rapid service deployment by transforming servers into pools of reconfigurable, diskless servers. At the core of VFrame is the programmable Server Switch, a programmable InfiniBand-based switching platform. The Server Switch connects every server on a single high-speed unified fabric, and then maps the physical server (now a simplified CPU and memory resource only) to a remote, virtual I/O subsystem and server image stored in SAN storage. VFrame can program the server switch to change this server mapping in real time, allowing a physical device to take on another identity quickly. VFrame can program the server switch to change this server mapping in real time, allowing a physical device to take on another identity quickly. VFrame can deploy and repurpose physical resources based on business policies such as time of day, failover, or different types of load.

The Server Switch creates a unified, wire-once fabric that dramatically simplifies the datacenter architecture by aggregating I/O and server resources. Multiple types of I/O cards, including HBAs, NICs, and dedicated cluster cards can be combined into a single high-speed, low-latency 10Gbps unified InfiniBand fabric. And by creating virtual HBAs and IP interfaces in each server, administrators maintain seamless access to SANs and LANs, while reducing the expensive overhead of Fibre Channel HBAs. This dramatic consolidation reduces the number of expansion slots and enables the ability to architect based on average load across multiple servers, not peak demand. By aggregating multiple adapters and eliminating local storage, the size of the server is driven by CPU and memory requirements only. This often results in a reduction in the size and cost of the server, as well as associated space, power and cooling.

Virtualizing I/O on the server also enables multiple servers to be shared by different applications by enabling the ability to rapidly change server identities. By simply changing the server to storage mappings stored in the Server Switch, physical machines can switch between different operating systems and applications. The server's identity is stored in the fabric, and the physical server box is simply another resource to be used regardless of physical location. Administrators can create business policies in VFrame that repurpose servers based on time of day, CPU or application load, or other metrics. For example, in a clustered database or web server environment, this allows a group of physical server devices to be added during peak hours, and released during less busy times. Alternatively, a group of servers can be shared across multiple applications for N+1 failover, eliminating the need to dedicate unused standby servers for every application.

VFrame can improve TCO by more than 30%. In addition to capital savings by reducing the size and number of required servers through just-in-time provisioning, VFrame reduces operational costs by automating regular tasks. VFrame saves management time by increasing the number of servers an individual administrator can manage. VFrame shortens time to bring new resources online, and eliminates the need to physically touch or rewire servers to change how resources are assigned. At the server level, the server architecture is dramatically simplified, with fewer interfaces per server and less complexity, as well as a lower failure and replacement rate due to fewer moving parts with the removal of local storage.

This flexibility and centralized management model significantly reduces server downtime. By booting remotely over the network to a centralized SAN or LAN, administrators can improve server MTBF by eliminating the single most common point of failure: local storage. With a unified fabric, administrators also eliminate extra adapters, further removing moving parts and simplifying the server. Administrators can then manage storage and I/O upgrades centrally, enabling the ability to migrate and change without bringing down applications for significant periods of time. For example, administrators can change which physical hardware a server is running on with one click in the VFrame user interface, which changes the mapping in the Server Switch and brings up the same "virtual server" stored in the fabric with different physical hardware.

Physical virtualization enables both service migration and failover. If the old hardware exhibits issues, administrators can proactively migrate to new hardware, while maintaining existing server image and I/O settings. Conversely, administrators can build an updated image offline and migrate the physical hardware to a new image. Or if a server fails, VFrame can detect the physical hardware fault and automatically migrate that server identity to a new location, choosing the replacement from a shared failover pool. Similarly, I/O virtualization enables the administrator to centrally expand or service centralized I/O without affecting server uptime.

Virtual I/O also eliminates I/O deficiencies or bottlenecks in dense server packaging. I/O is particularly an issue in blade servers, where multiple blades in a chassis create more I/O than the outbound pipes can sustain. In the previous model, blades share a few common Fibre Channel and Ethernet ports at the blade chassis level. In the virtual I/O model, all blades are connected via a 10Gbps network "backplane", and use common external Fibre Channel and Ethernet ports, which can be scaled linearly by adding additional expansion modules. This kind of expansion is simply not possible in the previous model with limited space for multiple adapters, resulting in I/O bottlenecks that can prevent enterprise-class applications from being deployed. With today's leading blade solutions, switching to an architecture based on Virtual I/O significantly improve the amount of aggregate I/O coming out of the chassis. Similarly, this also frees other dense server packages from their architectural constraints resulting from the lack of I/O expansion slots.