The inevitable rise of full-stack observability

As most enterprises today leverage the power of technologies like artificial intelligence (AI), machine learning (ML), 5G, and internet of things, it is no longer enough for IT teams to be able to monitor and respond to network irregularities. From quick response protocols, the security paradigm is now tilting towards predictive analysis, where the unknown unknowns can be unearthed.

This is where full-stack observability comes in— an approach to IT security gaining popularity among technologists due to its panoramic scope capabilities. The days of IT teams having to be called to troubleshoot end-user problems may soon be in the rearview mirror, as full-stack observability provides for the detection and correction of anomalies even before they reach end users.

Frontier Enterprise asked Joe Byrne, Executive CTO of Cisco AppDynamics, about the growing appetite for full-stack observability, roadblocks to implementing the solution, and his company’s projects in the pipeline. Here’s what he said.

AppDynamics’ IT professionals survey found that there is an appetite for full-stack observability among technologists. How can such solutions help organisations differentiate themselves from the competition?

Technologists know that they simply can’t afford any slip-ups when it comes to delivering exceptional digital experiences to customers and employees, and that means that they need to ensure they’re able to monitor and optimise IT performance at all times. In order to do this, technologists are urgently looking to build on their current application monitoring tools and techniques to get a unified view on availability and performance up and down the IT stack.

This is why full-stack observability has now become a major priority for businesses all over the world. Full-stack observability enables IT operations, development, and networking teams to quickly and easily identify anomalies, understand root causes through dependency analysis and resolve issues before they impact customers and employees. 

In the Cisco AppDynamics report – The Journey to Observability – 92% of Singaporeans technologists said that the appetite for full-stack observability within their organisation has increased over the last 12 months. And in addition, 90% say that they believe the shift to full-stack observability will be transformational for their business. Talk turned to action during 2021 and the transition to full-stack observability is very much underway. 

The report showed that where organisations have progressed with their full-stack observability plans, business leaders have been able to see immediate, tangible results and differentiate themselves from their competition. These benefits include improved productivity within the IT department, reduced operational costs due to teams having to spend less time reacting to performance issues, and improved collaboration across the business. 

Perhaps most significantly, business leaders are starting to feel the impact of having their technologists focused on key strategic priorities, such as digital transformation and enhanced customer experience, rather than the constant firefighting of the past two years. They know what a huge difference it can make to their organisation if they can get their most innovative brains concentrating on those innovation-driven projects that can really shape the future of a business and a market.

What are some of the factors today that keep some companies from adopting full-stack observability? What are their most common implementation challenges?

The transition towards full-stack observability is very much underway in 2022 and while implementation is still in the early stages, it’s positive to note that 56% of all Singaporean organisations have already started executing their plans, and a further 38% are likely to do so in the next 12 months. That means that 94% of organisations are on track and will be somewhere along their journey to full-stack observability in the coming year.

But before one can fully enjoy the benefits of full-stack observability, there are some challenges to overcome. The report highlights that 92% of technologists recognise that there is more work to be done to deploy full-stack observability within their organisation. It includes concerns around integration issues (46%), increasing complexity (44%) and implementation (40%).

In addition to these, there are also understandable concerns over a lack of skills to truly deliver on observability. Implementing a full-stack observability solution is not a simple task and tech teams will need new skills, processes, and thinking. This is where technology partners have a vital role to play in easing the pressure on technologists during this most challenging period and providing them with the support and information they need to build a business case and execute against a strategic implementation program. 

What are your top few technology priorities at AppDynamics? What are some of the most exciting things you’re working on?

A recent development is AppDynamics’ adoption of OpenTelemetry. With OpenTelemetry, companies are empowered with standardised telemetry data from all parts of the technology stack – with control of how it flows between observability and monitoring solutions. This is an important step in realising the full benefits of full-stack observability, and just one area where AppDynamics is innovating right now.

You’ve held different tech positions in various companies throughout your career. What specific lessons have you learned so far that you’re applying today?

One of the most significant things I have learned is that people are the most critical part of building quality software. Having a technology team that takes pride in their work and wants to see and understand the impact the software they are creating is having on their business and the end user is essential to building a successful product.

This is also essential to implementing a thorough full-stack observability strategy. It’s not just about the visibility, insights, and actions the platform can provide, but understanding that in order to know the overall health of the application, the team building and supporting the product must focus on the application itself. So, everything matters, from the network it communicates through, to the infrastructure hosting the application and the data accessed in the database. Teams that operate in silos and are only concerned with their specific area of expertise make it difficult to grok the enormous amount of Telemetry created and waste a ton of time trying to correlate it.

Another key learning is that not enough companies focus on the application itself and the experience the user is having. Monitoring the back-end servers, CPU, memory, network— those are all important. But why does all of that exist, you might ask? It exists to provide the application running in that infrastructure an optimised environment to allow for a positive user experience. If you cannot validate that the end user’s experience is optimal, then it’s like going to practice all week and not showing up for game day. 

What developments do you foresee in full-stack observability for the next three to five years? How will emerging technologies like 5G, AI, and machine learning affect its evolution?

I think we will see this trend in the rapid adoption of full-stack observability continue to gain momentum. Technologists find themselves with a unique opportunity to have a game-changing impact on their organisations, and they’re rightly feeling excited and confident about doing just that.

You also mention some interesting technologies which will impact this momentum. The potential of 5G is almost limitless for APAC – 14% of connections will be running on 5G networks by 2025, according to GSMA. 5G promises greater speeds, more immersive experiences, and even new services still beyond our imagination. The biggest impact that 5G will have on application owners is the extent to which it will further increase customer expectations of digital services and reliance upon a greater number of services. This is a great opportunity for brands to have deeper relationships with their customers and drive greater loyalty.

AI and ML are already important components of observability. And as IT environments continue to expand and become more complex, and the volume of data explodes – AI and ML will become critical. This data deluge will make it increasingly difficult for human teams to manage application environments alone. That’s why observability solutions of the future will embrace AI and ML and feature even more automation than they do today. With AI-powered alerting and remediation tools, IT teams will be able to proactively optimise and more accurately resolve issues before they become full-blown problems.