Finding the right metric to report on end-user performance can be a challenge given the menu of metrics collected by the many synthetic monitoring products in the market.  In the past, two metrics dominated synthetic benchmarking. The first is the time to download an entire page, excluding browser pauses to execute JavaScript or other browser components. The second is the time to download an entire page inclusive of the time the browser stopped to load a plug-in or execute JavaScript. These two metrics were the most accepted standards that legacy synthetic providers like Keynote and Dynatrace advocated.  The problem with these metrics is that they really don’t convey much context on what the user actually experienced on the page. Furthermore, user-configured, arbitrary page completion like counting the number of ready states from IE, or waiting for two or three seconds of network inactivity, counted the page as complete, even though other components still had to load.  When measuring just a single page, it can give you a false sense of speed when the wrong page completion method is selected.

With the modern browsers came the inclusion of the navigation timing specification, allowing synthetic and real-user measurements to track more than the full completion by extracting this information via an API. The new information gave insight into each stage of a web page load by tracking Document Object Model (DOM) events, including the loadEventEnd, when a page theoretically becomes interactive to the user.  

The problem with these metrics is that they are not indicative of the experience the user has.  Many modern sites have asynchronous calls that occur after the loadEventEnd that are critical to the rendering and functionality of the page; or the loadEventEnd has to account for all the content that is still downloading and executing below the fold of the page.  The question remains, when did the user perceive to have a completed page?

WebPageTest has introduced a new metric designed to address how quickly a page appears complete for an end-user, giving us a performance number for the time to render a page above the fold. User perception of speed is impacted by how quickly they see the content they are looking for. Based on an algorithm that looks at how the screen renders as measurements are taken, an AppDynamics Browser Synthetic monitor can report back the visually complete time for each page visited throughout a journey. Let’s look at an example comparing two retailers:

In this example, Retailer Two does a much better job of optimizing and presenting content to the end-user, gaining a performance edge over the competition. However, if we used the traditional synthetic metrics like loadEventEnd and Fully Loaded, we would erroneously think that Retailer Two was delivering a worse performance to their end-users and lagging behind the competition.

The browser synthetic tests afford us a deeper understanding of performance metrics, given we can directly observe the behavior of each page load as it happens, including when a user sees that their page has fully loaded. When using synthetic measurements as competitive benchmarks, it’s important to focus on the user-impacting metrics and not legacy numbers that may mislead about the experience of the end-user. The conversation needs to be moved from “fully loaded” and “time to loadEventEnd” to “What did the user actually experience above the fold?” To compare one site versus another, visually complete is the metric that truly reflects the end-user experience.