Design For Success – Design, Implement, Refine, Succeed

When an organization gets to the point where they realize they need an Application Performance Management (APM) solution, things are usually on fire and getting more unmanageable by the day. For any organization, the purchase of an APM toolset is a large budget item and the pressure is getting these new tools into production and delivering value today, not tomorrow. In these circumstances, it is often difficult to make the time to really think through the process and do the implementation correctly. I know all too well how this feels, because I have been in this position with APM solutions on several occasions. I have deployed almost every major APM solution in this exact situation – executives breathing down my neck to both fix problems and make sure that the money spent delivers value quickly. An alternate title for this post may be “Don’t Make the Same Mistakes I Did” or even “My Pain, Your Gain”.

Tactical Deployment vs. Strategic Enablement

When it comes to enabling agent-based APM solutions in your environment, it’s typically a relatively easy task. Put agent binaries on the machines you want to monitor, modify your application startup parameters to use the agent binaries, restart the application, and presto, you have an application monitored by APM – or so you can tell your bosses. However, the reality is, if you don’t think through how the agents are organized within the solution, and you don’t plan for how you will automate the deployment and enablement of those agents, you will get nowhere near the value you could get from the money spent on your solution. It is very hard to put the brakes on and wait to do things right, but even with the smallest level of effort ahead of time, you can reap 10 times that effort in lost productivity and greater visibility down the line. This small amount of effort is what enables your APM solution rather than merely deploying a product.

A Tale of Two Deployments

I currently spend my time in the field helping AppDynamics enterprise customers get the most value from their APM investment. However, these stories come from my past experience deploying APM solutions at large Fortune 500 companies, pre-AppDynamics.

At one of these companies, we had just made a significant investment in an APM solution, and the pressure was on to deploy the solution overnight. I thought it was really as simple as I described earlier, so I went about making plans to get this deployment done. The mindset was purely tactical (get agents out, turn agents on, PROFIT). When it came to questions like how the application should organized within the solution (what is a tier, what is a node, how should tiers be grouped together) it seemed as simple as just grouping tiers owned by one group together into a single bucket, one “Application” in AppDynamics terms. So this is what I did. In a matter of a few days, after some herculean efforts to get change requests in, planning for application restarts, getting change management exceptions, etc, all of the agents were up and reporting in to the APM solution. However, when I started looking at the visibility I gained from this solution I was perplexed because calls that should be flowing from one tier to another tier simply weren’t showing up as I expected. As it turns out, data that flows from one tier to another tier that was part of another “Application” were showing as a call to an external system rather than a call flowing through the other tier as I expected. Consequently, a significant reconfiguration effort had to made in order to get this data flow corrected. Even a little time spent initially understanding how to design the hierarchy in the APM tool would have avoided this significant headache, nevermind the credibility lost by deploying something that didn’t deliver the value promised.

At another company, I took the time to make a bit of a sketch of how I thought the data should flow from tier to tier. One meeting with the application team who owned the monitored application was sufficient to whiteboard this high level picture, and from here determine what the proper agent configuration should be. With this knowledge, I then developed scripts that did the work of pushing the agent binaries out to the servers and configuring the relevant properties to enable them. In a few days longer than was required to simply push the agents manually and ask questions later, I had the agents deployed, and a fully enabled APM solution.

Lessons Learned and Applying Lessons to AppDynamics Deployments

The most important thing to consider when deploying any APM solution is how the tool enables you to understand true application performance. One key element of this is the application hierarchy inside of your APM tool. A good way to think of this in AppDynamics terms is as follows:

  1. Application – This is the top level of the hierarchical configuration. All services that talk to each other should be in the same AppDynamics Application. For most customers, from very small to the biggest customers using AppDynamics, the starting point should be a single Application. Only when it is absolutely known that a monitored process communicates with no other monitored processes should they be broken out into a separate application.
  2. Tier – A tier should combine all runtime elements that implement similar services. If there are several runtimes that are load balanced, this is a good indication that they belong in the same tier. If two runtimes implement the same services, they should be grouped together.
  3. Node – This is the lowest level element in the AppDynamics monitoring hierarchy and the easiest to understand. This is a single monitored run time, whether it be a JVM, a .NET CLR, or a PHP server.

Take the time to briefly sketch out a high-level flow map, and this will get you thinking in the right way about how to lay these things out. If you need assistance, reach out to AppDynamics field enablement teams to help you understand how this should be laid out. Additional information can be found in the AppDynamics Center of Excellence Best Practices Guides.

Lastly, take the time to automate your agent rollout and enablement. Building an APM solution should be an iterative process, where you can easily deploy and remove agents, and reconfigure them as needed. Doing this small amount of pre work will save you much of the pain I experienced in my earliest APM deployments.

Take five minutes to get complete visibility into the performance of your production applications with AppDynamics Pro today.

Deploying APM in the Enterprise Part 7: Dashboards and Reports – Get the Business on Your Side

Welcome back to my series on Deploying APM in the Enterprise. In Part 6: Spread the APM Love we talked about ways to increase user adoption of your APM tool and thereby make your organization more successful.

This week we focus on Dashboards and Reports. I’m a huge believer in unlocking the information and intelligence contained within your monitoring tools, and turning that data into actionable information. Over time your tooling ecosystem will collect a wealth of data that can be used for capacity planning, business intelligence, development planning, and many other business and IT activities that require information about usage patterns and operational statistics. By unlocking this value, and surfacing it to the business and IT organizations in a meaningful way, you take another step up the maturity ladder and solidify your rockstar status within your organization. A huge benefit of providing useful information to the business is that they will fight for your tools and projects if they are ever in question!


Dashboards should be used to provide real-time insight into critical business and IT indicators. A failed server doesn’t always mean that there is impact to the business.

Several of our customers are using AppDynamics dashboards to better understand business activity. For example, we have multiple e-commerce customers that are tracking revenue and order volumes in real time.

Let’s take a look at some different perspectives within the organization and explore what type of dashboard each role requires.


Most managers don’t need to know the sordid details about the infrastructure that supports the business applications. For each application within the managers realm of responsibility they need to understand key business indicators and have an overall indication of their application performance and health. Take a look at the dashboard below…


This is the type of dashboard that managers want to keep an eye on throughout the workday. Any impact to the business will be easily identified on this dashboard so that the manager can make sure the impact is being addressed. There should be alerts associated with these metrics so that the operations center is notified of business impact but it’s not always an IT problem when business metrics deviate from normal behavior. It’s possible that sales volume is impacted by a competitor offering lower prices on the same product. This will show up as business impact and there is nothing that the IT staff can do to fix it. This is a business problem that needs to be addressed by the appropriate department.

Of course any IT infrastructure problems that impact the business will also show up on the managers dashboard but only by way of business metrics that have deviated from normal behavior. This fosters communication between IT and the business which should lead to improved cooperation and coordination over time.

Application Support

Application support rides the fence between the business and IT. If something goes wrong with their application, the business makes sure that app support hears about it and gets the problem resolved in a timely manner. Application support teams need a view of key business indicators (similar to what the managers are looking at) as well as key IT indicators so they know when there are problems with the application or the infrastructure they depend upon.


The operations teams need to know when infrastructure components are nearing capacity, throwing errors, or failing completely. It is their responsibility to ensure the proper functionality of the infrastructure and as such need an infrastructure centric view of the components they are responsible for. This is a very classic enterprise monitoring view that has been long established and still has its place in the modern monitoring world. With that said, its beneficial to show the operations teams a few key business indicators so they know how urgent it is to replace that failed piece of hardware.


Given that dashboards are best utilized to provide real-time status information, reports are the go-to solution for information that drives action in the future. Reports can be about the application, infrastructure, business, or anything else that makes sense given the data you ave to work with. What I want to focus on in this blog are the insights I have picked up over the years when developing and utilizing reports at my past companies.

  • Reports should contain information that is actionable. There is little value in receiving a report that you cannot use to decide if action is required.
  • Reports need to be concise. Very few people are interested in reading a 50 page report. Try to keep them to a few pages or at least summarized in the first couple of pages with supporting details in the rest of the report.
  • Don’t send an avalanche of reports. People usually don’t have the time or desire to read multiple reports per day. Ideally you should send reports only when there is something in the report that needs attention.
  • If you can, include the source of the report and a description of what they report is used for. I’ve received reports in the past where I didn’t know what system sent them and why I needed to see them.
  • Know your audience. Make sure the business gets business related information and IT gets technology related information.
  • Don’t blast reports to email groups (usually). Most reports need to be seen by only a few people in your organization. Email distribution groups tend to contain way more people than are truly interested in your report. No need to clog up the email system with a 50 page PDF sent to 2000 addresses.

Reports are important and can help your organization avoid issues down the road but you need to implement them carefully or they become just another piece of internal spam. The best report that I can ever receive is the one that only shows up when there is a problem brewing within the next few weeks, clearly identifies the problem, and I am the right person to make sure it gets addressed. That would be reporting done right!

Dashboards and reports need to be implemented properly to get the most out of your monitoring investments. You can amplify the value of all of your monitoring investments by combining, analyzing, and displaying the data contained within each disparate repository. The most mature organizations will build out their monitoring ecosystem and then invest in analytics to derive business and technology value and to create the best dashboards and reports possible.

Thanks for taking the time to read this series. The final post will be next week and will focus on how to keep up the momentum and stay relevant when it comes to monitoring within your organization.

Click here to read Deploying APM in the Enterprise Part 8: Stay Thirsty (and Relevant) My Friends