Cisco
Cisco Secures the Cloud with AppDynamics
Key Benefits
Improved resource through dramatic reduction of false positives
Increased performance through better visibility and employee collaboration
Facilitated a move towards DevOps model
Challenges
Cisco Security Cloud Operations blocks over 250 million threats per day. The team’s main challenge was actioning the 88,000 alerts generated annually by their existing monitoring tools. Valuable time and resources were being wasted, as 96% of these were false positives.
The sheer volume of data processed across CWS’s 23 global data centers raised obvious management challenges for the team. With 2,000 physical servers and 5,000 VMs serving more than 200 applications in total, Cisco required a scalable APM solution able to support its complex environment.
At the same time, the organization has witnessed a 12% increase in users over the past year, with average daily transactions growing from 5.5 billion to 6.37 billion. Constantly upscaling compute power in order to meet this demand was not sustainable. Cisco needed a way to scale effectively and manage traffic, while ensuring flawless execution.
“AppDynamics was rolled out globally across 23 data centers — deploying a total of 15,000 agents in just one week — all to a single AppDynamics report server.”
Nitin ThakurTechnical Operations Manager @Cisco Security Technology Group
About Cisco
Cisco Systems is the worldwide leader in designing, manufacturing, and selling Internet Protocol-based networking and associated services. It provides a broad line of products for transporting data, voice, and video within buildings and across campuses.
Cisco’s Security Cloud Operations spans multiple product offerings and sits within its Security Business Group. Cisco Cloud Web Security (CWS) provides industry-leading security and control for the distributed enterprise. Users are protected everywhere, all the time, when using CWS through Cisco worldwide threat intelligence, advanced threat defense capabilities, and roaming user protection.
The team is responsible for managing the site’s performance and capacity, quickly finding the root cause of issues and fixing problems fast to ensure seamless service.
Solution
Cisco Security Cloud Operations required a robust, next-generation Application Performance Management solution able to cope with the scale of its infrastructure and high volumes of daily traffic.
The organization first engaged AppDynamics in May 2013, deciding to run a proof-of-concept across its main security scanners, using AppDynamics in a performance environment to spot issues.
Nitin Thakur, technical operations manager, Cisco Security Technology Group, said, “From our initial evaluation, it was clear AppDynamics was able to provide the functionality we required to cope with the sheer volume of transactions we handle on a daily basis. We evaluated other products on the market, but for us, AppDynamics provided true end-to-end visibility, and we were impressed with the ease of deployment. Crucially, its future roadmap meant the company was best placed to help us scale for future growth.”
After a successful proof-of-concept, AppDynamics was rolled out globally across the organization’s 23 data centers — deploying a total of 15,000 agents — in just one week, and all reporting into a single AppDynamics-hosted SaaS controller.
“AppDynamics has enabled us to move towards data-driven troubleshooting rather than ‘gut-feels.’ The solution gives us the application intelligence to know when things aren’t functioning optimally.”
Nitin ThakurTechnical Operations Manager @Cisco Security Technology Group
“AppDynamics gives us great visibility into what is happening on the estate, helping us to dramatically reduce the number of escalations.”
Nitin ThakurTechnical Operations Manager @Cisco Security Technology Group
Previously, Cisco received 88,000 alerts a year from standard monitoring tools, of which only 3,000 were genuine alerts requiring remedial action. Through implementing AppDynamics, Cisco has been able to dramatically reduce false positives over the past twelve months.
“Prior to AppDynamics, our network operations center team was working overtime looking in the wrong places, at the wrong problems. Even if the NOC team only spent an average of five minutes looking at each erroneous alert, that amounts to over 7,000 man hours per year of effort that could be better spent either working on critical issues, or helping to drive innovation forward. Since introducing the AppDynamics platform, we have been able to identify problems we weren’t even looking for. AppDynamics has enabled us to move towards data-driven troubleshooting rather than ‘gut-feels.’ The solution gives us visibility when we need it and the application intelligence to know when things aren’t functioning optimally,” said Thakur.
Since introducing AppDynamics, Cisco has identified unique slow business transactions affecting services, as well as addressing multiple configuration errors. For example, Thakur explained, “We spotted a configuration issue which meant 17 million requests were being sent back to our central hub unnecessarily. With AppDynamics, it took just five minutes to find and fix this issue. We were not even looking for this problem and only identified it when it was presented to us.” Among other benefits, this has resulted in a four-to-five percent increase in memory utilization across the entire platform.
AppDynamics provides a common language between operations, development, and test. Introducing the platform has helped enable us to move towards a DevOps model, which in turn had a positive impact on employee collaboration,” said Thakur. “The visibility gleaned from AppDynamics has made employees feel more empowered to reach problem resolution and able to influence business outcomes.
“Because we make application releases so frequently, for us, it was critical to have APM in production. AppDynamics gives us great visibility into what is happening on the estate, helping us to dramatically reduce the number of escalations and continue delivering a seamless service to our customers.”
Future plans
Cisco sees Application Runbook Automation (RBA) as a critical part of the monitoring solution, and intends to roll this out extensively in the near future. “The goal is have no alerts that humans interact with directly, ensuring issues are automatically remediated and routed to development teams where necessary,” said Thakur.
“My goal is to be able to let Chaos Monkey loose and sit back to watch AppDynamics monitor and automate the required remediation,” added Thakur.
Cisco also plans to further integrate AppDynamics with more Security Cloud Operations products.
“AppDynamics provides the functionality required to cope with the sheer volume of transactions we handle on a daily basis.”
Nitin ThakurTechnical Operations Manager @Cisco Security Technology Group
