When we launched AppDynamics with Cisco Secure Application in early 2021, it was the industry’s first integrated application performance management (APM) and runtime application security offering. We made a bold bet that consolidated monitoring would become increasingly important and provide significant benefits such as improved security capabilities and reduced costs.
It was the right bet.
The speed and effectiveness with which our AppDynamics team was able to help our customers rapidly and successfully defend their business-critical applications against the recent Log4j vulnerabilities is a testament to the unique value that can be achieved by integrating security into your full-stack observability strategy.
Let’s briefly take a high-level look at what we did and then dive deeper into the technological, operational and organizational benefits of our approach.
What happened? The Log4j security vulnerabilities explained
The initial Log4j vulnerability (CVE-2021-44228) — aka Log4Shell or LogJam — has been widely discussed, but in a nutshell, a zero-day vulnerability was disclosed in the popular Apache Log4j Java logging library. (Log4j is used directly or as a dependency by millions of servers for both enterprise applications and cloud-based services.) It received the maximum Common Vulnerability Scoring System (CVSS) severity score of 10.0 and is very accessible and exploitable, allowing attackers to gain full control of affected servers and applications.
But the bad news related to the Log4j library didn’t end there. Three additional vulnerabilities (CVE-2021-45046, CVE-2021-45105 and CVE-2021-44832) were disclosed in short order, further raising already widespread concerns among and posing serious challenges for IT teams around the world.
Rising to the challenge with Cisco Secure Application
After disclosure of the first Log4j vulnerability, our security experts took immediate action to help our customers defend their business-critical applications against the threat. We released a detailed security advisory providing the results of our internal product-related investigation and recommended AppDynamics software updates within 24 hours and followed it up with a series of communications and additional useful resources. (Head to our blog to learn more about our customer response.)
Even more importantly, we were able to directly facilitate our customers’ efforts to identify Log4j vulnerabilities at runtime and block exploits — without friction or overhead — thanks to the unique capabilities of our leading-edge Runtime Application Self-Protection (RASP) product. It really felt good to have our customers tell us things like “I can finally go to bed after two days of uninterrupted work” after seeing Cisco Secure Application in action in their application environment.
Cisco Secure Application identifies where Log4j vulnerabilities exist in production. (Click/tap image to enlarge it.)
What makes our security solution stand out from the pack? Let’s peek under the hood.
Code-level protection that’s essential to a defense-in-depth (DiD) strategy
Defense-in-depth is a security strategy that’s a widely accepted practice and continues to be advocated for within the security community, especially in the face of threats like the Log4j vulnerabilities. Simply put, you need multiple layers of security in case one of them fails or is incapable of effectively protecting your applications.
As far as runtime application security goes, it’s important to prevent malicious payloads from exploiting vulnerabilities — without subjecting the underlying infrastructure to unnecessary load. Each defensive layer plays its role based on what it can see to help decrease the overall security and performance risk. And the expectation is that the number of threats will lessen as traffic moves through them, which reduces the risk to and the load on components the closer they are to the application itself.
While perimeter defenses such as a firewall can block network-layer and transport-layer traffic based on allow or deny policies, protocol violations and anomalies related to things like packet size and source geolocation, however, they don’t provide much if any protection against application-layer exploits. For example, a web application firewall (WAF) can block HTTP requests based on application-layer data matching — often referred to as signatures — as well as protocol violations. But it often requires manual adjusting (aka tuning) to reduce alert noise and blocking of legitimate requests. And if the malicious actor is an insider or can gain a foothold through another vulnerability behind your perimeter defenses, you need something more in place.
Here’s where a RASP solution comes in. It can block exploits based on code-level runtime behavior using method parameters or application logic that can be triggered from any event and is the last line of defense before code is exercised.
Defense-in-depth reduces the risk and the load the application must accept. (Click/tap image to enlarge it.)
Cisco Secure Application sits alongside your code and runs wherever it does. This means that it can detect and block exploits when your application is executing code based on a request or a response no matter where traffic originates from.
In the case of the Log4j vulnerabilities, Cisco Secure Application can enforce a single policy across all applications and block any specified vulnerable class (JndiManager.lookup) from making any type of network connection (LDAP, DNS, HTTP, etc.), no matter what the trigger was for that behavior (HTTP, gRPC, file read, db dip, etc.).
Cisco Secure Application uses a single policy for all apps that block vulnerable Java classes triggered by any event from establishing any network connection. (Click/tap image to enlarge it.)
Application-layer protection that’s impervious to obfuscation tactics
One of the bad actors’ favorite pastimes is the cat-and-mouse game of finding workarounds to signatures used to match against request data. They intentionally hide malicious payloads with encoding, string replacement, concatenation or other approaches that make it hard to match until an application pieces all the data together for handling in its logic. This method, referred to in the security world as obfuscation, is top of mind as IT teams try to protect against Log4j exploits. In a recent Log4j-related Dark Reading piece, Mike Benjamin, vice president of security research at Fastly, says, “The nested templates used in Log4j attacks allow for attackers to both obfuscate the strings included as well as try to steal information.” He also states that ”the obfuscation makes it more difficult for defenders to block or alert without false positives.”
A quick perusal of Twitter provides evidence that he’s correct.
Some quick #log4j exploit #log4shell updates.
We’re seing more and more encoded payloads to avoid detection through WAF, the bypass possibilities are probably endless, there’s too many ways to escape detection.
It will be rly hard to sort out the false positives from real ones
— Emy | eq 🌈 (@entropyqueen_) December 12, 2021
Cisco Secure Application, however, is ready and waiting for this type of behavior, and it doesn’t care if an attacker attempts to use a known (or unknown) obfuscation tactic to get around signature-based blocking from a WAF or an intrusion prevention system (IPS). It focuses on the resulting action that the application takes using a specific class in the Log4j library rather than the user-generated input that may have triggered it.
Automatic, continuous protection without friction or overhead
Although extremely valuable for rapid detection of and response to runtime security issues such as the Log4j incident, the traditional RASP approach has not seen widespread adoption by application teams. These solutions have traditionally required a separate security monitoring agent to reside on an application server in addition to any APM agents, which can cause potential resource conflicts such as instability and performance degradation.
Because Cisco Secure Application was built by the same engineering team as our APM agent, it provides code-level runtime security via a single, unified APM/security agent. The (minimal) additional overhead it creates comes at an average cost of less than 1% CPU usage, less than 6 ms of latency and 4-6 MB of memory. This means you can safely use it in production without impacting the end-user experience or application performance.
Cisco Secure Application protects applications against attempted exploits of the Log4j vulnerabilities without the overhead. (Click/tap image to enlarge it.)
Correlated application and business insights to prioritize response efforts and bring teams together
When companies began putting together their Log4j response plan, questions immediately arose as to how to prioritize and coordinate between siloed application and security teams. Many didn’t have a complete, readily available software bill of materials (SBOM) for their applications and services, which made it difficult for them to identify what was vulnerable and connect all the need-to-know stakeholders through shared data.
Cisco Secure Application eliminates this gap by leveraging the metadata that AppDynamics already uses to monitor the health of your applications to maintain continuous context across different stakeholder groups. It correlates security details with application insights for unified visibility into the impacted service area and breaks down the barriers between application and security teams to allow for rapid response.
But those aren’t the only benefits that consolidated performance and security monitoring provide. According to Gartner analysts Cameron Haight and Neil MacDonald, there are also several other advantages, including:
- Reduced technology risks, as APM technology has already been proven in large, production-oriented environments.
- Consolidation of the database supporting both operational and security analytics against the application’s monitored data.
- Lower software and support costs due to agent consolidation.
Some final thoughts
Developing totally secure code is impossible. The Log4j incident is just one more in a long line of zero-day vulnerabilities, and the nature of today’s globally interconnected software supply chain means vulnerable application components will remain an ongoing threat for the foreseeable future — with potentially significant consequences for enterprises without adequate protection in place.
As you think about where to go from here, taking a combined approach to security and observability by implementing a solution such as Cisco Secure Application can provide significant benefits such as ease of use, precision and performance. But ultimately the largest benefit is that end users don’t care if their application problems are performance- or security-related. In other words, both high CPU utilization and a Log4J remote code execution (RCE) attack can result in service degradation. A combined observability/security approach could support more rapid mean time to detect (MTTD) and mean time to repair (MTTR), which are critical metrics for both operations and security. And that can help all of us get a good night’s sleep.
Want to learn more?
Check out our other Log4j-related resources, including a step-by-step guide on how to use Cisco Secure Application to block Log4j exploits.
AppDynamics product investigation and software updates
Security advisory: Apache Log4j vulnerability
Security advisory: CVE-2021-45105 in Apache Log4j
AppDynamics Community
How do I use AppDynamics with Cisco Secure Application to find vulnerabilities and block exploits?
Cisco Talos threat intelligence and detection
Threat advisory: Critical Apache Log4j vulnerability being exploited in the wild
Other
Defend Log4j vulnerabilities with Cisco Secure Application (video)
As a senior director of product management, Randy Birdsall specializes in AppSec and Observability bringing over two decades of developer and technical leadership to the AppDynamics team. During his time at AppDynamics and Cisco, Randy has driven strategy, portfolio growth and marketing initiatives leading product managers, technical marketing engineers, solution architects and field engineers to support GTM strategy and roadmap development across many focuses including security, IoT, collaboration, cloud computing, system administration and network architecture.