Percentiles Made Easy

September 05 2013
 

Explanation and definition of what percentiles are and how they should be used to understand your application performance better.


Performance monitoring comes in all different shapes and sizes but the end goal is the same. Performance monitoring tools exist to help ensure a fast and stable quality of service. Statistically speaking there are many methods to determine just how good of an overall experience your application is providing but today we are going to focus on Percentiles. I’ll explain what percentiles are and how they should be used to understand your application performance better.

When you want to know how your application is performing from a macro perspective you need to understand the concept of percentiles. Let’s take a look at an example. Figure 1 below is a histogram which represents transaction response times (you can click on the chart to see a larger version in a new tab). Each bar represents a grouping of transactions that fall within 400ms of each other with the fastest group on the left.

histogram

Figure 1: Histogram of response times showing average response time, and 95th percentile. (Click chart for larger version in new tab)

μ(a) from our histogram is the arithmetic mean or average response time. You’ll also notice a vertical dotted line labeled 95. This line represents the associated percentile. For example the 95th percentile (shown) is roughly 15 seconds; this means that 95% of the transactions completed in 15 seconds or less. Ideally we want as many of our transactions as possible to have the fastest response time possible; so in an ideal world the 95th percentile line would shift left to approach μ(a). Percentiles, when compared with averages, tell us how consistent our application response times are. Let’s take a look at percentiles and averages plotted on the same chart.

PercentileChart

Figure 2: Average response time and 95th percentile plotted on same chart. (Click chart for a larger version in a new tab)

Figure 2 above shows the average response time (ART) in green and the 95th percentile plotted in blue. You can see that the ART peaks out at about 1400 ms while the 95th percentile tops out at about 2600 ms. What does this tell us and what should our response be? Well, it depends on how fast you want your application to run.

  • If having a lot of transactions that take between 1.4 and 2.6 seconds is okay for your particular application then you are probably in good shape and don’t have much to do except for tracking down the outliers (the 5% of transactions that take longer than 2600 ms) and figuring out if you can make them faster.
  • If you need the majority of your transactions completing in less than 1400 ms then know you have a lot of work to do with optimizing your platform since so many transactions take longer than 1400 ms.

The whole point of this discussion is to provide an introduction to percentiles and to let you know how to interpret and react to the data once you have it plotted out. Percentiles are just another statistical tool that is available to help manage application performance and like any tool you need to know how to use it for it to be valuable. If you’re looking to make your application faster and more stable you can try AppDynamics Pro for free today by clicking here.

Jim Hirschauer
Jim Hirschauer is a Technology Evangelist for AppDynamics. He has an extensive background working in highly available, business critical, large enterprise IT operations environments. Jim has been interested in application performance testing and monitoring since he was a Systems Administrator working in a retail bank. His passion for performance analysis led him down a path where he would design, implement and manage the cloud computing monitoring architecture for a top 10 investment bank. During his tenure at the investment bank, Jim created new processes and procedures that increased overall code release quality and dramatically improved end user experience.

Thank you! Your submission has been received!

Oops! Something went wrong while submitting the form