It’s time for another update to our series on Top Application Performance Challenges. Last time I looked at Java’s synchronization mechanism as a source for performance problems. This time around I take on what is likely the Performance Engineer’s bread and butter … slow database access!

Behind this small statement lies a tricky and multifaceted discussion. For now, I’m going to focus on just one particular aspect – the Object Relational Mapper (ORM).

The ORM has become a method of choice for bringing together the two foundational technologies that we base business applications on today – object-oriented applications (Java, .NET) and relational databases (Oracle, mySQL, PostgreSQL, etc.). For many developers, this technology can seem like a godsend, eliminating the need for them to drill-down into the intricacies of how these two technologies interact. But at the same time, ORMs can place an additional burden on applications, significantly impacting performance while everything looks fine on the surface.

Here’s my two cents on ORMs and why developers should take a longer look under the hood:

In the majority of cases, the time and resources taken to retrieve data are orders of magnitude greater than what’s required to process it. It is no surprise that performance considerations should always include the means and ways of accessing and storing data.

I already mentioned the two major technology foundations on which we build business applications today, object oriented applications, used to model and execute the business logic, and relational databases, used to manage and store the data. Object oriented programs unite data and logic into object instances, relational databases on the other hand isolate data into columns and tables, joined by keys and indexes.

This leaves a pretty big gap to bridge, and it falls upon the application to do the legwork. Since bridging this gap is something many applications must do, enter the ORM as a convenient, reusable framework. Hibernate, for instance, is quite likely the most popular ORM out there.

While intuitive for an application developer to use (ORMs do hide the translation complexities) an ORM can also be a significant weight on an application’s performance.

Let me explain.

Take a Call Graph from AppDynamics and follow the execution path of a transaction, method by method, from the moment a user request hits the application until calls to the database are issued to retrieve the requested information. Then size up the layers of code this path has to go through to get to the data. If your application has implemented an ORM like Hibernate, I assure you’ll be surprised how much stuff is actually going on in there.

No finger-pointing intended. A developer will emphasize that the benefits of just using the ORM component (without having to understand how its working) greatly increases his productivity. Point taken. It does pay, however, to review an ORM’s data access strategy.

I recently worked with a company and saw transactions (single user requests) that each bombard the database with 3000+ distinct calls. Seems a little excessive? You would be right. More over, nobody knew that this was happening and certainly nobody intended for it to be so.

In many cases simple configuration settings or using a different ‘fetch’ method offered by the ORM itself can affect performance significantly. Whether for instance the ORM accesses each row of the customer table individually to fill an array of customers in your code or is actually constructing a query that encompasses all of the expected result-set and gets them in one fell swoop does make a big difference.

Seems obvious, right? But do you actually know what your ORM really does when retrieving the information?

You might be surprised.

application development, application monitoring, database calls, developers, slow SQL

In my last post, I wrote about how using Business Transactions as a management unit is critical for managing modern-day applications efficiently. Sticking to this train of thought, I will focus on how this applies to various aspects of application management. The first area I want to cover is monitoring and troubleshooting.

In a highly distributed system, there are 100s of CPUs, 100s of JVMs/CLRs, and millions of lines of code running. Now, if you want to attach service levels to every component of that system, you would either be eyeballing dashboards most of the time or trying to maintaining the configuration associated with alerting.

As I mentioned in the last post, if the business grows, it will require more capacity and more infrastructure—which means newer pieces (and ones that are moving around rapidly). Add the configuration attached with lines of code and you can pick either “daunting” or “impossible” to describe the task at hand. Long testing and staging cycles have become a thing of the past.

At the same time, the DevOps tribe is adapting to and embracing the new application landscape rapidly. Their biggest need is the need for speed, which translates into efficiency driven by intelligence in every aspect of application management in production. This is where using Business Transactions can make monitoring more efficient and easier to accomplish.

But wait a minute – am I really suggesting that you watch Business Transaction service levels (which are your business’ bottom line) instead of getting overwhelmed with alerts based on 1000s of key metrics? Doesn’t that mean you are ignoring things that can be going wrong by not giving them enough attention?

Au contraire! In fact, you can actually focus on more with this approach versus traditional monitoring techniques. Let me explain.

Traditional monitoring has all been about averages. You look at some service or some method and its average over time, then set up alerts associated with it. Doing so is great for catching slow performance degradation or systemic outages. But it won’t help you catch outliers where there is no pattern associated with errors or slow requests.

Let’s look at a couple of examples to understand this better. For brevity I will talk only about slow requests (but the same argument can also be applied to errors).

1) A frequent cause of slow requests, resulting in a poor user experience, pertains to a transaction associated with user input.  For example, in a shopping cart application, the user might add a particular item to his or her cart—which results in the application slowing down.

2) Here’s another one. Sometimes one particular node, which is part of a big cluster of say 50 nodes, has an issue in servicing requests but might be doing ok on CPU and memory usage.

In both of these cases, averages would hide the problem. Here’s an example.

Scenario
Over a period of time, an online checkout application experienced 5,000 Total Checkouts. Of those checkouts, 4,800 were Normal Checkouts and 200 were Slow Checkouts.

In cases like these, it is very likely that the average response time of the normal transaction—along with that of the bad transactions—averages out to a very normal rate. But the bottom line is that 200 users had a bad checkout, and that needs to be fixed. By focusing on business transactions only and reducing the points of monitoring, you are actually able to identify and address more performance concerns than you might have otherwise.

So what exactly are we watching here?

a) Response time for a transaction – this is averaged over all requests for this transaction. We don’t need to watch the key methods being executed in the request since the overall response time for the request is the single indicator. A method-level drill down is needed only when the response time is slow.

b) Number of slow requests over time – If we set up thresholds over the transaction and watch every request, we are able to identify exactly how many requests were outliers. Of course, for this to be useful, the system needs to be able to collect diagnostic information as slow requests happen and not afterwards. This also ensures that when the request is being serviced by a problematic node, it is represented appropriately.

We’re not the only ones who believe that monitoring Business Transactions is critical to better performance in production. The customers and prospects we speak to are serious about having a system in place that can watch all requests instead of just watching averages in their system. It’s becoming a real-world demand with real-world benefits.

The last thing I want to mention before signing off, is that AppDynamics does all of the above. We follow all requests to monitor SLAs—not just averages, but actual numbers—and we do so at an extremely low overhead, enabling us to jump in and get diagnostic data as bad requests happen!

That wasn’t too much of a plug, right? Until next time…

APM Thought Leadership, application monitoring, business transaction, DevOps

Taking a break from the rush of activity at VMWorld, today we released the results of our first Application Virtualization Outlook survey.  With virtualization a top priority for many CIOs in the next year, we wanted to hear from IT professionals about their plans to virtualize mission-critical apps. Most importantly, we wanted to learn how quickly they’re making the transition, or whether there are obstacles getting in their way.

We found the survey results pretty interesting – overall, it seems like many application owners recognize the potential benefits of virtualization, but lack the confidence that their Tier-1 apps will continue to perform as needed when moved to a virtual environment. Let’s take a closer look at a few of these results and how we came to this conclusion:

More than 80 percent of professionals polled had virtualized their non-critical systems, but on the flip side, only 14 percent had virtualized more than three quarters or more of their Tier 1 apps (see charts below).  That’s a big divide in the pace of adoption.

The number one obstacle to virtualizing apps? “People Issues,” i.e., application owners who impede the virtualization process.  Respondents also cited worries about performance degradation and application design.

And one of our favorite questions- almost one-third of respondents reported that there are people in their organization who would say, “My Tier 1 application will be virtualized over my dead body!!”  This definitely proves that application virtualization is can elicit an emotional response in concerned application owners.

But, let’s not get discouraged. Just about everyone surveyed agreed that there are some great benefits to be had with virtualization. In particular, respondents noted that server consolidation, power and cost savings were likely payoffs, along with disaster recovery and agility improvements.

So how can virtualization teams assuage the concerns of their colleagues and reap these benefits? By providing hard evidence that the owners’ application will perform well in a virtual environment.

Application owners are responsible for meeting SLAs and for maintaining 100% up-time – and they’re not going to hand over their application without proof that those business objectives won’t be compromised.

The best way to get these facts is to conduct tests to establish a baseline performance in a non-virtual environment, and then provide a comparison of application performance, pre-and post-virtualization.  This “apples-to-apples” strategy presents app owners with evidence that their application will continue to perform to their high standards in a virtual environment.

For more information about how we help companies manage application performance in virtual environments, check this out.

APM Thought Leadership, application monitoring, cloud computing, virtualization