Five tips for developing an observability mindset

August 17 2022
 

In the race to create more sophisticated apps, microservices and scalable cloud environments, monitoring is not enough. As environments gain complexity, an observability mindset is paramount in transforming data into mission-critical insights.


When it comes to application performance, monitoring is the measurement of “known” unknowns — for example, measuring load, CPU and disk space. But what monitoring doesn’t account for is the “unknown” unknowns, like new metrics created by deploying, shifting and updating Kubernetes clusters in the cloud, a growing list of APIs or serverless services where nothing is constant. Just like in business, where a fixed mindset doesn’t fully support resilience, a monitoring mindset is fixed and can’t provide the level of awareness needed to optimize a tech stack. For that, you need an observability mindset.

Gain insights into the hidden parts of your tech stack

A core challenge hindering organizations is that IT teams can’t analyze, accelerate or innovate what they can’t see. Yes, monitoring helps and yes, monitoring gets lumped in with observability — often used interchangeably — but really, they’re not the same. To get insights for optimal business outcomes and to remain competitive and differentiated in the market, you need both.

The state of application performance monitoring today

The app economy continues to grow, fueled by user demand for fast, always-on, flawless and secure application experiences. Furthermore, if apps or their microservices are not already hosted in the cloud and deployed through Kubernetes or containers, at some point, they most likely will be. Containers ease innovation, scalability and high availability but the constant deployment, redeployment and use of scalable, serverless infrastructure creates a world of “unknown” unknowns. In that environment, status quo monitoring tools rely on assumptions but as complexity rises, so does a need to see the whole stack — especially when app performance degrades. And moving beyond monitoring to embrace rapid changes requires a new mindset built on observability.

Observability mindset vs. monitoring mindset

Monitoring is a solution that enables IT teams to have visibility into the tech stack based on a set of predefined, fixed and known actions and it is assumptive. These tools have a valued place in defining fixed sets of service-level objectives (SLOs) and determining which service-level indicators (SLIs) show that SLOs have been upheld.

Example assumptions include:

  • Apps made up of a set number of microservices.
  • Known latency when writing data to a disk and measuring latency.
  • Locating issues with memory, CPU and throughput.
  • Preset views into apps running on containers, virtual machines (VMs) or bare metal.
  • When system and instrumentation metrics are the primary source of information for debugging code.
  • Dashboards and telemetry that serve ops engineers.
  • When the focus of monitoring is uptime and failure prevention.
  • Examination of correlations across a limited (or small) number of dimensions.

 

These fixed measurements are based on understanding what fails, how a system already works and how it should work. Which makes monitoring critical to ensuring optimal uptime and a first line of defense to safeguard user experience while protecting apps from zero day and other security threats. To do that well, IT leaders should foster an observability mindset that begins with no preconceived notions, where tech stack visibility is leveraged to explore, explain and define patterns and anomalies — and findings are used to augment processes and procedures that accelerate innovation. Together, monitoring solves application performance and security glitches in real time and observability can enable issue resolution before a glitch ever happens. Both are important for driving business forward and gaining resiliency needed to rise as a market leader.

Building an observability mindset

Monitoring opens an opportunity to see things that would typically be masked. Observability is more of an omniscient way of looking at your IT ecosystem. Rather than seeing things you already know exist, observability enables discovery of the unknowns; the things you suspect are there but remain undiscovered because monitoring hasn’t exposed them. For example, you understand there’s data to be seen somewhere that’s accessible as metrics, logs, telemetry and events but you haven’t found them yet — like a needle in a haystack.

Liz Fong-Jones, principal developer and member of the OpenTelemetryTM governance committee explains, “Viable observability tools must proactively measure and analyze systems, and involve many use cases beyond just break/fix.” Her examples include instrumenting code as it’s being written — the same way unit tests are written, or being able to visualize what’s happening inside a CI/CD pipeline. And she views those sorts of capabilities as a true definition of comprehensive observability.

AppDynamics: a monitoring tool, an observability tool or both?

AppDynamics has effectively always been both a monitoring and an observability tool. With it, you gain a map of applications including all the tiers, nodes, etc. The platform correlates these components and updates them upon implementation and out-of-the-box, it delivers dynamic baselining that provides a full view into app health. It alerts and ranks anomalies based on your input and it’s been used, from the beginning, as an observability tool for IT leaders who leverage its full capabilities to move beyond monitoring to comprehensive observation. Organizations are gaining recognition for the abundance of unknowns and the importance of augmenting app monitoring with OpenTelementry to incorporate metrics, logs and traces pulled from those environments too. Observability helps with this.

 

“Together, AppDynamics and ThousandEyes provide us with full-stack observability to handle demand spikes with ease and deliver next-level experiences to our customers throughout the year, while also underpinning a new wave of innovative cloud-based services”

Madhu Peddaboina
Director of Information Technology eCommerce, The Children’s Place

Five tips for developing and growing an observability mindset

Similar to a growth mindset, which is considered a leadership characteristic, an observability mindset can also be learned. For those naturally curious, the leap to searching for things in the tech stack that can’t yet be seen (observability) will likely feel more natural. But it does not prevent those who are more pragmatic and focused on what they can see from developing that level of curiosity. The first step is understanding the business growth potential when a full view of the IT ecosystem is seen and understood.

To get started:

  1. Lead with curiosity, dissect output from monitoring and begin connecting dots between the “what-ifs” and let your observability practice grow with no clearly definable end.
  2. Remember the reason an application exists and use that as the north star for decisions.
  3. Review metrics previously used to manage app performance and availability. Understand why they were chosen and determine if they are still relevant.
  4. Understand which business-critical KPIs align with the app. These could be direct metrics i.e. items sold per second, network latency, etc. or when direct metrics are not available, assign proxy metrics and let these be additional decision north stars.
  5. Review incidents over the past 6-12 months and assess gaps in metric collection. Discover which could have been eliminated, identified and remediated if it had been monitored.

 

For more information on how AppDynamics enables organizations to gain full visibility into their unique technology ecosystems and supports both monitoring and observability mindsets, check out what our customers are saying!

Sujata Singhal is a channel sales engineer at AppDynamics, providing world-class sales technical support to channel partners, sales teams and customers. A subject matter expert on application performance management, Sujata contributes specific expertise and knowledge around Cisco Unified Contact Center Enterprise (UCCE). In addition, she loves learning about customer and partner needs and pairing them with solutions that drive the best outcomes.