Migrating Hadoop to ACI

What

Cisco IT offers Hadoop as a service to developers who write business intelligence and analytics applications. In July 2016, we moved our Hadoop environment to Application-Centric Infrastructure (ACI). ACI is a software-defined networking (SDN) architecture. It manages endpoint connectivity for each application by applying the policy that the developer defines. To write a policy, developers first define endpoint groups (EPGs) for each application component—for example, the database, application, and web tier. Then they define contracts that control how the EPGs can communicate with each other, strengthening security. The contract includes QoS, firewall access control lists (ACLs), and load balancing. The Cisco Application Policy Infrastructure Controller (APIC) automatically enforces the right policy for each application using Cisco Nexus 9000 switches.

“Our developers have written more than 30 Hadoop applications, all of which now run on ACI,” says Virendra Singh, Cisco IT architect. For example, engineers use Hadoop for quality analysis. Sales teams developed an application to recommend sales approaches in different situations. Marketing uses a Hadoop application to automatically tag content so that it appears when customers search by keywords. Our IT team uses Hadoop to measure employee sentiment about services. Cisco Services has one Hadoop application that identifies service-renewal opportunities and another that provides analytics services to customers.

Why

We migrated Hadoop to ACI to strengthen application security, speed up application provisioning, and increase application performance.

Improve Hadoop Security

Hadoop does not inherently have strong security. Before upgrading to ACI, we used ACLs to specify the applications and systems that could connect to a Hadoop cluster. But ACL configuration is a manual process—time-consuming and error-prone. Mistakes such as mistyping a port name can result in unauthorized access. “ACI makes Hadoop more secure and simpler to configure by using automated application security policies,” says Jag Kahlon, Cisco IT architect. “We avoid the risk that a developer might miss writing an ACL for a particular situation.”

Security is not only simpler but also stronger. “In a traditional network, any system can talk to any other system on the same subnet,” Kahlon says. “With ACI you can easily limit the servers that can connect to a particular application.”

Speed Up Application Deployment from Weeks to Minutes

Before, adding a new Hadoop application to the enterprise Hadoop environment took 3-4 weeks. First the developer had to submit a request. Our IT team needed to approve the request and assign people to work on it. Then we had to validate that the ACL worked. Meanwhile, the business customer couldn’t take advantage of the new Hadoop application. Now, connecting a new Hadoop application is as simple as clicking to apply the policy. Currently the networking team applies the policy. In October 2017, application teams will be able to manage ACLs themselves, and our networking team will no longer be involved.

Speed Up Hadoop Application Performance by 21-34 Percent

Moving Hadoop to ACI decreased latency. “We’ve seen 21-34 percent faster application performance because decreased latency increases throughput between application nodes,” says Singh. With faster access to information, Cisco employees can take action sooner, helping to run the business better.

For More Information

Cisco IT White Paper: Cisco IT ACI Design

Cisco IT Insights: Improving Security with Application-Centric Infrastructure

Cisco Nexus 9000 Switches

To read additional Cisco IT case studies about a variety of business solutions, visit Cisco on Cisco: Inside Cisco IT.

To view Cisco IT webinars and events about related topics, visit Cisco on Cisco Webinars & Events.

Note

This publication describes how Cisco has benefited from the deployment of its own products. Many factors may have contributed to the results and benefits described. Cisco does not guarantee comparable results elsewhere.

CISCO PROVIDES THIS PUBLICATION AS IS WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING THE IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Some jurisdictions do not allow disclaimer of express or implied warranties; therefore, this disclaimer may not apply to you.