As AppDynamics’ Technical Evangelist for Analytics and Business Intelligence, Marco Coulter is passionate about the interaction between humans and technology. A former startup CTO, Marco progressed from operator to leadership roles at CSC, CA Technologies and 451 Research. He earned the nickname “the tech whisperer” for his ability to translate business drivers for a technical audience and technical concepts for business leaders. We sat down with Marco to get his insights on the topic of DevOps—specifically the concerns that organizations have when considering a DevOps initiative.
AppDynamics Team: We’ve heard countless times about the benefits of DevOps, such as its ability to make your company’s tech more stable and lower operational costs. And yet DevOps marks a change—especially for operations teams—and often triggers one of the most powerful human emotions: fear. Why is that?
Marco Coulter: This is a combination of two primal fears: the unknown and change. If you move down the DevOps path, will it actually be better for both the enterprise and individuals? That’s fear of the unknown. For the individual who has been in development or operations for a few years, they may be concerned that their expertise will be devalued by moving to DevOps. Of the enterprise experts I chat with, they are generally positive after moving to DevOps. They have taken away a lot of the monotonous tasks through automation, and learned some new marketable skills. Whether looking for people to build applications or run applications or both, we’re looking for some exposure to DevOps when we hire. Even the IBM mainframe folks are implementing DevOps.
AppDynamics Team: Companies often underestimate the amount of work required in a DevOps transformation. According to a Gartner study, 75% of DevOps initiatives through 2022 will fail to meet their goals due to issues around organizational learning and change. So it seems there are many challenges in a DevOps initiative. Can you tell us about some of them?
Marco: Interesting that Gartner ties this back to organization, learning and change. Remember the goals of DevOps are around velocity, quality, and performance. These have to be made better and for that you need to measure before you start, along the path and after you think you are complete. DevOps is an efficiency booster and only measurement will prove success. Come up with a statement around “we expect DevOps to improve [existing metrics] by [change level] for [application] without impacting [existing customer success metrics].” That helps everyone understand what is about to change and why, and helps address some of the fear we talked about earlier.
AppDynamics Team: It’s been said that people-related factors tend to be the greatest challenges, not technology, in a DevOps effort. True?
Marco: Absolutely, and this is not just among the technology folks. The business side of the house also has to commit to the change. The common questions I get asked concern team structure for DevOps. The basic skills and processes for DevOps have books written on them, but how to manage the human side is THE most important predictor of success.
AppDynamics Team: An effective DevOps effort needs metrics that drive smart automation decisions, and yet organizations often struggle with DevOps metrics. How can companies overcome this hurdle?
Marco: The DevOps ribbon (below) is very clear about monitoring as a critical success factor. Yet it gets overlooked. The weakness of this ribbon visualization is that it makes it look like each stage has equal importance and equal timeframe. The truth is, you will run a piece of code for much longer than it takes to create it. I once wrote an interest function for a bank that ran in production for ten years. Monitoring that code was much more important than the creation phase.
Some measures around DevOps will be process-related: deployment frequency, change volume, deployment time, lead time, and defect escape rates for testing. These measure the effectiveness of your automated build and test processes, but little of your human processes. Monitoring the application in production, you need to measure availability, performance, transaction error rate, change failure rate whether from outages or degraded service, time to detection, and mean time to recovery. It’s always important to understand the before and after measures of these, which is why you need baselines. Scale is an aspect here as well, and response time and performance should remain stable, even after an increase in user volume or a new deployment. On top of all these systemic aspects sits the customer experience within business transactions and any service-level metrics you need to track and report on. I know that is a lot of different metrics to monitor, but these are what will support justifications that the DevOps approach is better and support your requests to write off technical debt in the future. So it is worth doing.
AppDynamics Team: DevOps efforts can fail for many reasons: unrealistic expectations, tracking metrics that don’t align with business goals, keeping ITOps and engineering/development teams in traditional silos, and so on. Which of these issues are most common and how can organizations overcome them?
Marco: The commonest issue is starting without a plan—”Let’s go DevOps and see what happens!” I recommend start small and share skills. Select the application that is important to the business and will benefit from improved velocity, quality, and performance. Build a small team and be clear that the team is temporary and will return to their prior career to share the skills they learn while on the temporary team. Also—and this is critical—get a business sponsor for the application, and get an executive sponsor who sits above any silos that may exist—people who can give clear guidance to the customer voice and settle demarcation disputes.
AppDynamics Team: You’ve spoken a lot on the topic of “digital transformation.” Tell us what it is and how it can benefit a DevOps transition.
Marco: Actually, I would phrase that second half the other way around: DevOps can benefit a digital transformation. DT is often confused with digitization. Digitization takes an existing non-digital process and recreates it in the digital world. Maybe you were taking orders over the phone on paper, and walking them over to the warehouse to see if stock is available and to arrange delivery. After digitization, you enter them on the computer, which directs the details to the folks in the warehouse. DT is much more than that. Instead, you would empower the customer to order online and see current stock so they adjust their order to different products before they even checkout. Now, you are transforming your business.
As you begin asking your enterprise, “How can we change this process to better empower the customer?” the answers start coming fast. Now you have to introduce all these changes quickly and be able to adjust, retreat, alter and update as fast as the ideas pour in. That is when you suddenly realize you cannot wait months for the next release, only to find out you did not get it quite right. Customers will simply abandon your application. This is where DevOps will be critical in supporting speed of change and speed of reaction. But only if the IT metrics are correlated to the business metrics.
AppDynamics Team: Any comforting advice to pass along to companies fearful of taking on a DevOps initiative?
Marco: Nope … OK, just kidding. DevOps is not an all-or-nothing proposition. It does not mean a Java coder has to learn how to lay CAT5 cable, or an operator needs to learn the subtleties of TensorFlow. Work out how to break it into smaller bites. One of the key pieces of advice is to avoid creating a permanent DevOps team. Consider calling it a DevOps Center of Excellence or DevOps Resource Team, and make it a temporary one from existing folks just to ramp up the processes and tools. Then let them flow back to their prior teams, along with the fresh DevOps skills. Who to pick for the team? Find those who already are generous in sharing their knowledge.
AppDynamics Team: What role can APM play in a DevOps initiative?
Marco: Measurement makes DevOps complete. Not only on a day-to-day basis of watching your key applications perform as customers use them, but measurements also will help support your arguments around technical debt, justifying resource increase and adjusting to time-based variation in load. Measurements will quickly expose whether you are getting the reset of DevOps right. Measure the pipe, equip the team, skill up, monitor out to the customer usage and business transactions.
AppDynamics Team: Great insights, Marco. Thanks for sharing.
Read more on how you can integrate AppDynamics in your DevOps processes.