There’s been a generational shift in how Java enterprise applications are created: they have been broken down from a monolithic architecture into multiple services, and they’re highly interconnected and distributed. How can Java developers and Operations teams adapt to these changes?

This keynote will discuss the 4 Big Things that Java professionals need to design for now:

• Cloud: Most applications built will have some part of its service in the cloud
• Big Data: With the advent of NoSQL, Hadoop, and distributed caches, how should we now approach the data layer?
• Agile Development & Operations: Developers won’t just be responsible for the code, but how it’s deployed. How does that affect the DevOps relationships?
• Failure is an option: Distributed systems won’t just invite but demand failure, so how can failure become part of the initial design?

This talk will present recommended strategies and approaches for these new design imperatives.

You can watch the keynote here

Agile Development, appdynamics, application monitoring, Big Data, cloud, DevOps, Stephen Burton, WJAX, WJAX 2011

I’ve always been a skeptic of JMX monitoring, largely because I felt it was often wrongly positioned as the pillar stone of application monitoring software. An application for me is a collection of business services or transactions that users perform, that causes application logic (code) to request and process data. Without visibility into the performance of business transactions and code execution, JMX monitoring can be seen as just another infrastructure monitoring tool for the JVM. However, when application and JMX monitoring are combined into a single tool, they can offer powerful capabilities for managing application performance.

What is JMX Monitoring?

JMX Monitoring is done by querying data from “Managed Beans” (MBeans) that are exposed via a JVM port (JMX console). An MBean represents a resource running inside a JVM and provides data on the configuration and usage of that resource. MBeans are typically grouped into “domains” to denote where resources belong to. Typically in a JVM you’ll see multiple domains. For example, for an application running on tomcat you’d see “Catalina” and “Java.lang” domains. “Catalina” represents resources and MBeans relating to the Apache Tomcat container, and “Java.lang” represents the same for the JVM run-time (e.g. Hotspot). You might also see custom domains that belong to an application, given how trivial it is to write custom MBeans via the JMX interface.

Read the Full Post…

apm, Application Performance Management, Introscope JMX, JMX, JMX Metrics, JMX Monitoring, Tomcat JMX, Weblogic JMX

What happens when mission critical Java applications slow down or keep crashing in production? The vast majority of IT Operations (Ops) today bury their heads in log files. Why? because thats what they’ve been doing since IBM invented the mainframe. Diving into the weeds feels good, everyone feels productive looking at log entries, hoping that one will eventually explain the unexplainable. IT Ops may also look at system and network metrics which tell them how server resource and network bandwidth is being consumed. Again, looking at lots of metrics feels good but what is causing those server and network metrics to change in the first place? Answer: the application.

IT Ops monitor the infrastructure that applications run on, but they lack visibility of how applications actually work and utilize the infrastructure. To get this visibility, Ops must monitor the application run-time. A quick way to get started is to use the free tools that come with the application run-time. In the case of Java applications, both JConsole and VisualVM ship with the standard SDK and have proved popular choices for monitoring Java applications. When we built AppDynamics Lite we felt their was a void of free application monitoring solutions for IT Ops, the market had plenty of tools aimed at developers but many were just too verbose and intrusive for IT Ops to use in production. If we take a look at how JConsole, VisualVM and AppDynamics Lite compare, we’ll see just how different free application monitoring solutions can be.

Read the Full Post…

apm, appdynamics lite, application monitoring, Application Performance Management, Free application monitoring, Java Monitoring, JConsole, JVM, JVM Monitoring, Oracle, Oracle Java Monitoring, VisualVM

I made my first public appearance last week at JavaOne and had a blast mixing it with the dev community and the various exhibitors. Prior to being bitten by radioactive byte code, I’d attended JavaOne as a developer and had fond memories of vast crowds, packed session rooms, and nonstop partying. While JavaOne ran in parallel to Oracle Open World again this year, the event actually felt independent despite San Francisco city being littered with Oracle posters. Walking into the venue every morning felt like it was still the biggest Java conference in the world, especially when you had corridors of developers checking email on bean bags. Java is very much alive, despite skeptics claiming its dead or has no future. If you had to write a new mission critical application for your business today, I’d expect the majority of organizations would still opt for Java despite the hype around other languages like Ruby, Python and the return of PHP.

I was attending JavaOne with AppDynamics as an exhibitor, and I’m pleased to report things went very well for us. What was surprising is that many attendees already knew who we were and what we did, and that wasn’t just from U.S. attendees. I spoke to lots of developers from Europe who were already using AppDynamics Lite and were keen to see a demo of our latest Pro edition.

We also met several attendees who were in the process of evaluating APM toolsets for their organizations. which was great. APM is definitely becoming a priority now for many application teams, with most struggling to get decent performance and visibility in production. I had one alarming conversation with an architect while he was briefing me on his team’s success criteria for selecting an APM solution. I heard the words “All the monitoring vendors tell me they run in production with a few percent overhead so I’ll take their word for it.” For me that’s like agreeing to a mortgage without asking each bank what their actual interest rates and terms & conditions are. My advice to this chap was along the lines of “trust no vendor and prove all overhead claims in production.” The reality today is very few APM vendors can run and scale in production–even though they all sound the same!

Speaking of APM vendors, both OpNet and Quest invited me over to their booth and asked if I’d mind having my photo taken with them. Being an APM superhero I was more than happy to accept their offer; it’s actually good to banter with competitors who have a sense of humor. I did try my best with the other APM vendor, but all the booth staff declined while staring at the floor–something about getting into trouble with their boss if they were seen with App Man. Maybe they were afraid of my X-Ray vision…

Here’s a brief photo diary of what I got up to at JavaOne:

Video Summary:

AppDynamics Customer TiVo stopped by to say Hello:

Grabbing a Coffee at Starbucks:

Meeting Dubu Panda from BMC:

Saying hello OpNet:

Saying hello to Quest:

Enjoying a Beer after a long day:

apm, App Man, Application Performance, Java, JavaOne, JavaOne11, OpNet, Oracle Open World, Quest

Back in April this year, we announced new monitoring support for NoSQL tecnhnologies like Cassandra and Memcached which was received really well by our customers. Due to further demand we’ve seen over the past few months from prospects and customers we’ve officially added MongoDB to our platform support, thus extending our coverage of NoSQL technologies.

For those not familiar with MongoDB, here’s how it differs from the traditional relational database just so you know what data and context AppDynamics provides for monitoring MongoDB applications:

Read the Full Post…

apm, Application Performance Management, BSON, BSON Query Performance, Business Transactions, MongoDB, MongoDB Monitoring, MongoDB Performance

Remember the good old days where developers and DBAs would argue over who and what was killing the relational database? “It’s your crap SQL,” “You forgot to create an index,” “You don’t know what an index is”…and so on. Do you remember when the DBA occasionally spoke and served out humble pie on how to make SQL statements go faster? Well my friends, those biblical days could soon be over with the adoption of NoSQL technologies. Or is that not true? Will the DBA perform a mighty rollback?

If you look back at the last decade, there is no doubt nearly all business today is connected or conducted online. In fact I can’t physically remember the last time Mrs. App Man and I walked into the bank, wrote a letter, or returned a movie rental. Half our life is done online and it’s not stopping anytime soon now that Mrs. AppMan has a new shiny MacBook to shop on. IDC and EMC recently published a report that stated 70% of the digital universe this year will be generated by users at home, work, and on the go. In total this amounts to 880 billion gigabytes of data being created this year. All that data has got to go somewhere, and it’s left to applications to query and make this data available to users whereever they are in the world. Data availability and performance has never been so important.

Read the Full Post…

apm, Application Performance Management, Big Data, Cassandra, Coherernce, DBA, Distributed Caches, Hadoop, NoSQL

Many root causes ago I was working with a customer who suspected they had a memory leak in production. Their JVM console event logs were showing the famous OutOfMemory exception and these were being thrown periodically every three to four days causing production outages. To stop these exceptions, the operations team would restart all JVMs at midnight every night in order to prevent system wide impact to customers during business hours. And if Ops forgot to restart the JVMs (which they did on several occasions), production went bang.

It’s worth pointing out at this stage that “OutOfMemory” exceptions in log files doesn’t automatically mean your application has a memory leak. It simply means your application is using or needs more memory than you’ve allocated to it at run-time. A leak is just one candidate of several potential candidates that cause memory to grow over time until all resource is exhausted.

Read the Full Post…

apm, Application Performance Management, Heap Dump, Java Memory Leak, Memory Profiler, Out of Memory, OutOfMemory Exception

San Francisco’s QCon was expecting a smaller crowd, but ended up bursting at the seams: the event sold out weeks ahead of time and in many sessions it was standing room only.

Targeted at architects and operations folks as much as developers, QCon was heavy on the hot topics of the day: SOA, agile, and DevOps. But if there was a consistent trend throughout the three days, it was “No more theory. Show us the practice.”

At Jesper Boeg’s talk for example—“Raising the Bar: Using Kanban and Lean to Super Optimize Your Agile Implementation”—the talk was peppered with some good sound bites (“If it hurts, do it more often and bring the pain forward”). But it also stressed the meat: Boeg demonstrated a “deployment pipeline” that represented an automated implementation of the build, deploy, test, and release process—a way to find and eliminate bottlenecks in agile delivery.

Similarly, John Allspaw started high in his talk—sharing his ideas on the areas of ownership and specialization between Ops and Dev, a typical DevOps presentation—but backs up the theory with code-level discussions of how logging, metrics, and monitoring works at Etsy.  (His blog entry on the subject and complete Qcon slides can be found on his blog, Kitchen Soap.)

Adrian Cockroft, who is leading a trailblazing public cloud deployment of production-level applications at Netflix, also wrapped theory around juicy substance. He “showed the code” and the screenshots of his company’s app scaling and monitoring tools (you can find his complete slide presentation here).

Not everyone took the time to drill down, though. Tweets from QCon attendees showed that the natives got restless in talks that stayed too high level:

“OK, just because you can draw a block diagram out of something doesn’t mean it makes sense.”

“Ok, we get it. Your company is very interesting, now get to the nerd stuff.”

“These sessions are high-level narratives. Show me the code, guys! Devil’s in the details.”

At the same time, they would shower plaudits and congratulations on speakers who gave them what they wanted: something new to learn.

When the Twitter stream started to compare QCon’s activities with an event happening concurrently in the city, Cloud Expo, the nature of the attendees was draw into sharp relief:

“At #cloudexpo people used laptops during sessions to check email… At #qconsf they are writing code.”

When it comes to agile, SOA, DevOps, and other problems of the day, people are ready for answers.

APM Thought Leadership, cloud computing, developers, DevOps, IT operations

Here’s a guest blog from one of AppDynamics’ international partners, Stefan Zoltai from sysPerform. Stefan wanted to write about how he used AppDynamics to solve a performance problem for a major telecom company in Switzerland—and we said, sure!  Take it away, Stefan…

I’d like to talk about how we used AppDynamics for a major production troubleshooting exercise—and how AppDynamics passed with flying colors.

Swisscom is the leading telecommunications company in Switzerland with about 5.7 million mobile customers and 1.8 million broadband connections. Swisscom is present on the Swiss market with a full portfolio of wireless, wire- and IP-based data and voice-based communication services.

Swisscom’s (Internet) Messaging had engaged sysPerform to assist with the analysis of their Tomcat 6 / Java 1.6 based WebMail application. WebMail has been under scrutiny for about a year now—ever since it manifested both performance and stability problems. Prior analysis efforts, conducted with a number of available tools, did not lead to the determination of the actual root cause(s) since the aforementioned problems only occurred in production under load and could not be reproduced in other environments. WebMail is rated at a throughput of 300 tx/sec.

We realized immediately that without a deep, detailed view into the application’s runtime, in production and under load, we would not be able to determine the actual root cause.

To analyze the application, we selected AppDynamics’ application performance management solution.  Since this solution has been developed specifically for high throughput, distributed production environments, we were able to obtain a high-level overview of the application as well as conduct a deep root cause analysis down to code-level execution without generating measurable overhead. Again, we did all of this at 300 transactions per second of throughput.

Thanks to AppDynamics’ ability to create a dynamic baseline of application performance, we were able to isolate the major bottlenecks on the first day and discuss a solution with the developers at Swisscom.  We were able to quickly learn the application’s performance and stability characteristics — and after only 5 days of development, we deployed a specific, major fix to address the main issue and massively improve performance.  At the moment, we are continuing our analysis efforts since stability and performance are the focus of an ongoing quality process.

[UPDATE: For Swisscom's perspective on the use of AppDynamics, check out Mika Borner's blog]

This example clearly demonstrates that operating a modern, distributed application without an adequate monitoring solution is effectively the same as “flying blind.” 60%-80% of all performance problems are caused by the application itself, and need to be analyzed from the inside out. We can confirm these numbers from many of other engagements with similar customers. External causes like hardware or network issues have become increasingly rare; it’s the problems deep inside the application that truly matter.

Intelligent application performance management however is not a means to itself, but must be evaluated in terms of economical considerations as well. Our experience indicates that an APM solution shows an ROI within just a few months. Among the reasons for such a quick ROI is the aforementioned extremely fast root cause analysis.

If you’re reading this in Switzerland, feel free to contact me with questions!

– Stefan Zoltai, Founder, SysPerform GmbH

Email: sz@sysperform.ch

Twitter: http://twitter.com/SysPerform

APM Thought Leadership, appdynamics, application monitoring, business transaction, dynamic baselines

Memory, Memory, Memory…

Memory is a critical part of Java, in particular, the management of memory. As a developer, memory management is not something you want to be doing on a regular basis, nor is it something you want to do manually. One of the great benefits of Java is its ability to take care of the memory model for you. When objects aren’t in use, Java helps you out by doing the clean up.

Read the Full Post…

APM Thought Leadership, application development, application monitoring, Java, memory leaks, memory thrash