AI PODs for Inferencing Data Sheet

           

Cisco AI-ready PODs are purpose-built infrastructure solutions designed for any organization to harness the power of artificial intelligence. Whether you’re starting out with AI or scaling complex, high-performance workloads beyond legacy architecture, Cisco AI PODs deliver the performance, efficiency, and control you need to drive innovation. With ready-to-deploy configurations, centralized management, and seamless scalability, Cisco AI PODs help you unlock AI potential at every stage of your journey.

Product overview

Now, not every firm is at the same stage when rolling out an AI solution. To address this, Cisco is introducing our first product line, Cisco AI-ready Infrastructure Stacks. These stacks are meant to empower businesses to quickly harness the potential of artificial intelligence with confidence and ease. By providing streamlined, ready-to-use infrastructure with best-in-class management tools, we’re eliminating the complexities of AI deployment, making it accessible even for firms new to this technology. With rapid, validated setups and consistent refreshes, Cisco AI-ready Infrastructure Stacks offer a scalable path from initial deployment to advanced applications, ensuring your organization stays competitive in an AI-driven world.

Cisco AI-ready Infrastructure Stacks provide a comprehensive suite of solutions tailored for various AI and ML use cases. These validated designs simplify and automate AI infrastructure, enabling organizations to deploy AI models efficiently across different environments—from edge inferencing to large-scale data centers. Cisco^® infrastructure stacks cater to a range of requirements:

●      MLOps with Red Hat OpenShift AI to streamline machine learning operations and model deployment.

●      Gen AI Modeling utilizing Intel^® Gaudi 2 processors and Cisco Nexus^® 9000 Series Switches, optimized for high-performance AI tasks.

●      Data Modeling platforms powered by Cisco’s robust networking and compute capabilities.

●      Digital Twin frameworks, supported by NVIDIA AI Enterprise and integrated data systems.

Each solution is built on Cisco Validated Designs, ensuring seamless integration and reliable performance for AI applications. Organizations can choose from converged infrastructure such as FlashStack and FlexPod.

Customers across different segments, from small businesses to large enterprises, will find immense value in Cisco AI Infrastructure. Small businesses can utilize cost-effective PODs for development and basic workloads, while larger enterprises can harness custom PODs for intensive AI and data processing tasks. The primary business benefits include improved productivity, reduced costs, and enhanced scalability, while the primary technical value lies in the PODs' ability to deliver high performance and reliability, ensuring seamless integration and operation within existing network infrastructures.

Features and benefits

Table 1.        Features and benefits

Prominent feature

Transforming AI Architecture Design with Cisco’s Full Stack Solution

This solution combines the best of hardware and software technologies to create a robust, scalable, and efficient AI-ready infrastructure tailored to diverse needs.

1.     NVIDIA AI Enterprise: These components provide a powerful foundation for managing AI workloads. They offer optimized performance, streamlined operations, and advanced management capabilities, enabling users to focus on developing and deploying AI models without worrying about underlying infrastructure complexities.

2.     OpenShift by Red Hat: This Kubernetes-based container platform simplifies the orchestration and deployment of containerized AI applications. It provides flexibility and scalability, allowing users to easily manage and scale their AI projects as they grow.

3.     Cisco Unified Computing System^™ (Cisco UCS): Serves as the hardware backbone, delivering high performance and reliability with its multiple server configurations. This ensures that the compute power required for demanding AI workloads is readily available and scalable.

4.     Converged Infrastructure Solutions:

◦    FlashStack (Cisco UCS servers with Pure Storage) and FlexPod (Cisco UCS servers with NetApp storage) both provide high-performance storage and compute capabilities essential for handling large-scale AI and machine learning tasks. These solutions offer integrated, pre-validated configurations that simplify deployment and management, enhancing operational efficiency and reducing time to market.

By leveraging Cisco's full stack solution, customers benefit from:

●      Easy to Buy:

◦    Cisco AI PODs are available as pre-designed, validated bundles that are ready to order, ensuring that customers can quickly select the right setup for their specific AI needs. This eliminates the complexity of choosing individual components and reduces decision-making time.

◦    By offering orderable bundles this quarter, Cisco enables faster time-to-value for customers, allowing them to accelerate their AI initiatives without unnecessary delays.

●      Easy to Deploy and Integrate:

◦    Cisco AI PODs are designed for plug-and-play deployment, ensuring that organizations can rapidly integrate them into existing data center or cloud environments with minimal effort. This seamless deployment capability is supported by Cisco’s Intersight and Nexus Dashboard, providing centralized management, real-time visibility, and automation to reduce operational complexity.

◦    The pre-designed bundles signify that they are pre-tested and validated, ensuring that all components work harmoniously together right out of the box. This reduces the risk of deployment issues and speeds up time-to-production for AI workloads.

◦    Whether deployed in a converged infrastructure setup like FlashStack and FlexPod, Cisco AI PODs offer flexibility and scalability. This ensures easy integration into a wide range of existing infrastructure setups, providing a future-ready solution that can grow with organizational needs.

●      Easy to Manage:

◦    The pre-integrated and pre-tested nature of the solution reduces the complexity of designing AI infrastructure. It provides a clear, replicable formula for optimal performance, eliminating guesswork.

◦    The solution supports a range of deployment sizes from small edge inferencing setups to large, high-performance environments. This scalability ensures that the infrastructure can grow with the organization’s needs.

◦    Integrated management tools from NVIDIA and Red Hat simplify the day-to-day operations of AI infrastructure, making it easier for IT teams to support AI projects.

For individuals and organizations at the ideation stage, Cisco's full stack solution provides a clear, structured path to developing a robust AI architecture. It combines the latest in AI management, container orchestration, and high-performance hardware, making it an ideal choice for anyone looking to build and scale their AI capabilities.

Scalability on Demand

Requirements change often, and you need a system that doesn’t lock you into one set of resources when you find that you need another. Cisco AI-ready Infrastructure Stacks are by nature growth ready and seamlessly scales with your Generative AI inferencing needs because of the modular nature of the Cisco UCS X-Series. It’s as simple as adding or removing servers, adjusting memory capacities, and configuring resources in an automated manner as your models evolve and workloads grow using Cisco Intersight. Additionally, you even have the flexibility to vary the CPU to GPU ratio or choose between Intel Xeon^® Scalable or AMD EPYC processors within a single chassis, depending on your specific use case. Lastly, we will continue to drive further expansion of the PODs through support for nodes beyond UCS X-Series.

●      Dynamic Resource Allocation: Easily add or reduce compute and storage resources.

●      Flexible CPU/GPU Ratios: Customize the balance of compute power.

●      Automated Scaling: Adjust resources automatically as workloads grow.

Future Ready

The Cisco AI-ready Infrastructure Stacks are built to be future ready, evolving with new technologies and adapting to emerging business objectives. The UCS X-Series modular system is designed to support future generations of processors, storage, nonvolatile memory, accelerators, and interconnects. It can be selectively upgraded piece by piece without the need to purchase, configure, maintain, power, and cool discrete management modules and servers. Cloud-based management is kept up to date automatically with a constant stream of new capabilities delivered by the Intersight software-as-a-service model. The resultant extended lifecycles for hardware will lower ongoing costs over time.

●      Modular Design: Easily upgrade components without major overhauls.

●      Longevity: Support for upcoming hardware generations.

●      Investment Protection: Safeguard your technology investments with forward-compatible infrastructure.

Customization

Cisco’s customizable configurations allow you to host models of your choice, edit configurations, and connect to diverse data sources and services. This flexibility ensures optimal performance and control over your AI workloads, tailored to your specific needs and deployment sizes.

●      Model Flexibility: Host any AI model to meet your unique requirements.

●      Configurable Setups: Modify hardware setups as needed.

●      Comprehensive Control: Manage and control your deployment environment fully.

Comprehensive Platform

Operationalizing AI is a daunting task for any Enterprise. One of the biggest reasons AI/ML efforts fail to move from proof-of-concept to production is due to the complexity associated with streamlining and managing data and machine learning that deliver production-ready models. Instead of ad-hoc ML efforts that add technical debt with each AI/ML effort, it is important to adopt processes, tools, and best-practices that can continually deliver and maintain models with speed and accuracy. Cisco AI-ready Infrastructure Stacks provide a purpose-built, full stack solution that accelerates AI/ML efforts by reducing complexity. The design uses the Cisco UCS X-Series modular platform with the latest Cisco UCS M7 servers, Cisco UCS X440p PCIe Nodes with NVIDIA GPUs, all centrally managed from the cloud using Cisco Intersight. Building on this accelerated and high-performance infrastructure, Red Hat OpenShift AI enhances the solution by integrating essential tools and technologies to accelerate and operationalize AI consistently and efficiently. NVIDIA AI Enterprise software further complements the solution, offering key capabilities such as virtual GPUs, a GPU operator, and a comprehensive library of tools and frameworks optimized for AI. These features simplify the adoption and scaling of AI solutions, making it easier for enterprises to operationalize AI technologies.

●      End-to-End Solution: From infrastructure to AI frameworks.

●      Generative AI Support: Optimized for AI model deployment at scale.

●      Enterprise-Grade: Robust enough to meet large-scale enterprise demands.

Simplified Management

Optimize your infrastructure management with Cisco Intersight. Our platform offers centralized provisioning, real-time visibility, and automation, simplifying the management of UCS servers and GPUs. This streamlines operations and reduces manual intervention, enhancing efficiency and reducing complexity.

●      Centralized Management: Manage all resources from a single interface.

●      Real-Time Visibility: Monitor performance and status in real-time.

●      Automation: Reduce manual intervention with automated management processes.

Product specifications

The Cisco AI Data Center and Edge Inference Pod is expertly engineered for edge inferencing applications, facilitating computation directly near the user at the network's edge, close to the data source. This strategic design minimizes latency and maximizes efficiency by processing data locally, rather than relying on a centralized cloud or data center. Supporting advanced models like Llama 2-7B, GPT-2B, and other Small Language Models (SLMs), the Data Center and Edge Inference Pod is highly versatile and capable. The inclusion of the integrated X-Series Direct fabric interconnect within the chassis eliminates the need for additional hardware, thereby reducing complexity and streamlining operations.

Table 2.        Specifications for Cisco AI Data Center and Edge Inference Pod

The Cisco AI RAG Augmented Inference Pod is purpose-built for demanding AI workloads, supporting larger models such as Llama 2-13B and OPT 13B. Equipped with enhanced GPU and node capacity, it efficiently manages complex tasks, delivering increased computational power and efficiency. This advanced POD can accommodate optimizations like Retrieval-Augmented Generation (RAG), which leverages knowledge sources to provide contextual relevance during query service, significantly reducing hallucinations and improving response accuracy. This makes the Cisco RAG Augmented Inference Pod an ideal choice for advanced applications requiring robust AI capabilities and exceptional performance.

Table 3.        Specifications for Cisco AI RAG Augmented Inference Pod

The Cisco AI Scale Up Inference Pod for High Performance is meticulously optimized to support large-scale models like CodeLlama 34B and Falcon 40B. With its substantial GPU and node capacity, this pod is engineered to deliver exceptional performance for the most complex AI tasks. It significantly enhances Retrieval-Augmented Generation (RAG) capabilities by supporting larger vector databases and improving vector search accuracy through advanced algorithms.

Table 4.        Specifications for Cisco AI Scale Up Inference Pod for High Performance

The Cisco AI Scale Out Inference Pod for Large Deployment is designed to offer unparalleled flexibility, allowing multiple models to run concurrently within a single chassis. It is ideal for organizations that require robust lifecycle management or high availability of models. This pod enhances accuracy by allowing the optimal model to be selected for each specific task and effortlessly scales out to support multiple users. This makes the pod an exceptional choice for organizations seeking scalable, efficient, and reliable AI infrastructure to handle diverse and demanding AI workloads.

Table 5.        Specifications for Cisco AI Scale Out Inference Pod for Large Deployment

System requirements

Table 6.        System requirements

Ordering information

Table 7.        Ordering guide for Cisco AI Data Center and Edge Inference Pod

Overall MLB PID: UCSX-AI-EDGE

Table 8.        Ordering guide for Cisco AI RAG Augmented Inference Pod

Overall MLB PID: UCSX-AI-RAG

Table 9.        Ordering guide for Cisco AI Scale Up Inference Pod for High Performance

Overall MLB PID: UCSX-AI-LARGERAG

Table 10.     Ordering Guide for Cisco AI Scale Out Inference PODs for Large Deployment

Overall MLB PID: UCSX-AI-LARGEINF

 

Warranty information

The Cisco UCS X210c M7 Compute Node has a three-year Next-Business-Day (NBD) hardware warranty and a 90-day software warranty.

Augmenting the Cisco Unified Computing System (Cisco UCS) warranty, Cisco Smart Net Total Care^® and Cisco Solution Support services are part of Cisco's technical services portfolio. Cisco Smart Net Total Care combines Cisco's industry-leading and award-winning foundational technical services with an extra level of actionable business intelligence that is delivered to you through the smart capabilities in the Cisco Smart Net Total Care portal. For more information, please refer to https://www.cisco.com/c/en/us/support/services/smart-net-total-care/index.html.

Cisco Solution Support includes both Cisco^® product support and solution-level support, resolving complex issues in multivendor environments on average 43 percent more quickly than with product support alone. Cisco Solution Support is a critical element in data center administration, helping rapidly resolve issues encountered while maintaining performance, reliability, and return on investment.

This service centralizes support across your multivendor Cisco environment for both our products and solution partner products that you have deployed in your ecosystem. Whether there is an issue with a Cisco product or with a solution partner product, just call us. Our experts are the primary point of contact and own the case from first call to resolution. For more information, please refer to https://www.cisco.com/c/en/us/services/technical/solution-support.html.

Product sustainability

Information about Cisco’s Environmental, Social and Governance (ESG) initiatives and performance is provided in Cisco’s CSR and sustainability reporting.

Table 11.     Cisco environmental sustainability information

Cisco and partner services

Cisco and partner services for AI-ready Infrastructure

How can you quickly adopt and maximize the value of your AI infrastructure technology investments to accelerate business value? From onboarding and adopting to optimizing your next technology transition, Cisco and our certified partners provide comprehensive services to support your AI Infrastructure, ensuring seamless integration and optimal performance.

Cisco Solution Attached Services for AI PODs for Inferencing provide subscription-led advisory and consulting services to fully leverage your AI infrastructure and models for inferencing tasks within enterprise environments. Choose the service level that best fits your needs—strategic guidance from Cisco experts, collaboration with your team, or letting Cisco handle everything—to help deploy, adopt, and optimize your solutions. Our services help you minimize disruption by delivering:

●      Planning and deployment to help with the implementation and onboarding of Cisco AI PODs

●      Advisory services to assist you with adopting your AI infrastructure and applications

●      Ongoing optimization during the subscription period to help you configure, add, and enhance AI-specific features and architecture expansion

For more details, please review the description of Cisco Solution Attached Services.

Cisco Capital

Flexible payment solutions to help you achieve your objectives

Cisco Capital makes it easier to get the right technology to achieve your objectives, enable business transformation and help you stay competitive. We can help you reduce the total cost of ownership, conserve capital, and accelerate growth. In more than 100 countries, our flexible payment solutions can help you acquire hardware, software, services and complementary third-party equipment in easy, predictable payments. Learn more.

For more information

Unlock the Future of AI with Cisco AI Pods

Discover how Cisco AI-ready Infrastructure can revolutionize your business. Request a demo, sign up for a free assessment, or contact our virtual sales rep to explore our tailored solutions. For more insights, visit our Cisco AI Solutions Overview page and access white papers, case studies, and more. Ensure your business stays ahead with cutting-edge technology designed to optimize productivity and growth.