Lawrence Livermore National Laboratory deployed Nexus switches and fabric extenders.
Challenge
Located in Livermore, California, Lawrence Livermore National Laboratory is a national security laboratory, with a mission to help ensure national security and apply science and technology to the important issues of our time. The Laboratory is managed by Lawrence Livermore National Security, LLC for the U.S. Department of Energy's National Nuclear Security Administration.
For many years, LLNL housed its financial and human resources databases in numerous small data centers throughout the facility. In the near term, consolidating all servers and storage into a single data center would simplify management and reduce costs. In the longer term, it would enable the laboratory to manage more information, more efficiently, by virtualizing the network infrastructure, servers, and storage area network (SAN).
Requirements for the new data center solution included high availability, support for 10 Gigabit Ethernet, and ease of management. "We wanted to build a foundation to adopt new network technologies as they become available, to keep the organization efficient," says Steve Devine, network engineer, LLNL.
Solution
LLNL renovated an existing building to create a state-of-the-art data center to meet the lab's business computing needs for years to come. The IT department selected Cisco® solutions for the data network and the SAN. "A single-vendor solution for the data center simplifies troubleshooting and support and eliminates interoperability issues that can affect availability," says Dave Sipes, network engineer, LLNL. "All of this together increases customer satisfaction." The laboratory has implemented two of the four data center quadrants.
Quadrant One: Cisco Catalyst 6500 Switches
In the first quadrant, each row of rack-optimized servers connects over Gigabit Ethernet to a pair of Cisco Catalyst® 6500 Series Switches. These switches, in turn, connect over 10 Gigabit Ethernet to the core, which consists of two more Cisco Catalyst 6500 Series Switches. A Cisco Virtual Switching System (VSS) joins the pair of core switches, enabling the IT department to manage them as one.
One row consists of blade servers, which connect to dual Cisco 4948 10GE switches. These switches connect to the core over 10 Gigabit Ethernet, creating a 20 Gbps port channel.
Quadrant Two: Cisco Nexus 5000 Switches
Before proceeding with the second quadrant, the LLNL team asked Cisco about its data center roadmap. "You get few opportunities to build a brand-new data center, and we wanted a design that would help us remain a world-class data center for the next 15 to 20 years," Sipes says.
The design of the second quadrant is similar to the first, except that each server row connects to a pair of Cisco Nexus 5010 Series Switches at the end of the row. Each switch is configured with four Cisco Nexus 2148 Fabric Extenders, which mix Gigabit Ethernet and 10-Gigabit Ethernet connectivity in one compact device. LLNL selected the Cisco Nexus 5010 Switch because of its high 10-Gbps port density, low cost per port, and interoperability with the Cisco Catalyst 6500 core switches. "We'll need high 10-Gpbs port density as we begin replacing our servers, and getting that port density a small footprint conserves valuable space," says Devine.
Results
Simplified Management
The LLNL IT staff configures the Cisco Nexus 5010 Switch and all connected Cisco 2148 Fabric Extenders as one device. A single point of management simplifies provisioning and management, and significantly reduces the time required to add network capacity. "The Cisco Nexus 2148 Fabric Extenders also take less time to change out than switch modules," Sipes says. The close similarity between the Cisco NX-OS and the Cisco IOS® Software interface enabled LLNL network engineers to manage the Cisco Nexus switches without specialized training.
Support for Virtualization
LLNL plans to virtualize all of its data center resources: servers, switching, and storage access. "The Cisco Nexus family gives us the scalable bandwidth and simplified management we need to adopt virtualization," Sipes says. "If you're building a single-story house now but plan to add a second story later, it's smartest to build the foundation for the second story now. The Cisco Nexus family provides the strong foundation we need to implement virtualization."
Cost Savings
The Cisco Nexus family contributes to lower data center costs:
• Lower capital costs: "Virtualizing the access row switches reduces the number of switches we need to support," says Devine. "We can quickly add just the right number of network ports with Cisco Nexus 2148 Fabric Extenders instead of buying a whole new switch."
• Unified fabric: LLNL purchased the Cisco Nexus 5010 Switch Fibre Channel over Ethernet (FCoE) license so that it can later consolidate to a unified fabric for the data network and SAN. "Support for FCoE in Cisco Nexus switches will save us from laying two sets of fiber and implementing two sets of switches, one for the network and one for the SAN," says Sipes. "We'll save management costs on an ongoing basis."
Next Steps
As part of its virtualization strategy, LLNL is considering using the Cisco Unified Computing System, which integrates compute, network, storage access, and virtualization in a single cohesive system. The Cisco UCS will access the SAN through the Cisco Nexus 5000 over FCoE, reducing fiber costs as well as management overhead. "If all 12 servers in a given row need SAN access, we can connect them all to the Cisco Nexus 5000 Switch instead of running 12 fiber connections all the way to the SAN Switch," Sipes says. "We'll be able to manage one connection point from all hosts." LLNL is also evaluating the Cisco Nexus 1000V Switch to facilitate server virtualization.
Technical Implementation
Each server row connects to the core through redundant Cisco Nexus stacks. Each stack is built with a Cisco Nexus 5010 Switch on top, three Cisco Nexus 2148 Fabric Extenders, and one Cisco Catalyst 2960 Switch. The data center has one row of blade servers, which uplink to a Cisco Catalyst 4948 Switch over Gigabit Ethernet.
For More Information
To find out more about Cisco Nexus 5000 Switches and Nexus 2000 Series Fabric Extenders, visit: http://www.cisco.com/go/nexus