The AppD Approach: Increasing Mobile Engagement with Business iQ

With 2.3 billion smartphone users worldwide, enterprises can’t afford to ignore the mobile channel. Last year, $2 billion of Black Friday’s total sales of $5.03 billion were spent on mobile devices. And mobile isn’t just about shopping. People are now spending equal amounts of time on mobile devices and desktop computers. Flurry Insights pegs the average time that people devote to mobile devices at 5 hours a day.

In an attempt to capture a fraction of their attention, nearly three-quarters of enterprises surveyed by Gartner last year deployed an average of 8 mobile applications. But their efforts were only modestly rewarded. That is because total mobile engagement is depressingly low. According to Localytics, 63% of users visit a mobile app less than 10 times, while 24% use an app only once.

Product managers are trying to address the engagement crisis by creating more compelling user experiences. But they find it hard to measure their success. They often have to guess: Is the new feature I rolled out last week pleasing users or annoying them? It turns out that producing great mobile experiences requires the same deep visibility as more traditional applications do. Product managers and their operations teams need to understand user journeys and to quickly diagnose performance issues. This is simple to do with AppDynamics Business iQ, of course, but I suspect that few product managers understand how easy it is to set up and use our solution.

I was reminded of this during a recent customer engagement at a large bank. The product manager was updating one of the bank’s mobile apps, and she asked me if AppDynamics could help her determine if users liked the new features she was adding or not.

Given that there will be around 200 billion mobile applications downloaded this year, I suspect her question is a common one. In the rest of this blog post, I’ll walk through the steps I showed the product manager so that others can get started just as quickly.

The first step is to go to the “Analytics” tab and “Add a new Search.”

Since feature registration is one of the Business Transactions automatically discovered by the APM agent, we can use AppDynamics Query Language (ADQL) to select all the features that have been registered by all users.

The screenshot below (a mockup) shows an ADQL statement that will count all the features that have been registered during a selected time period. You can see that the top features registered are “Quick Balance,” “Push Notification,” and “Quick Transactions.”

As developers continue to release new features for the application, the Analytics Dashboard will automatically display the registration occurrences of new features without any reconfiguration (i.e. ADQL statement).

Let’s say that “Quick Balance” is the new feature that the product manager has just released. The fact that there are 350 registrations indicates that the users are responding favorably to the pop-up screen promoting the feature.

However, as the promotion continues, the product manager notices a growing number of deregistrations from the Analytics Dashboard. By the end of the week, Quick Balance is showing the highest number of deregistrations.

Let’s assume the high number of deregistrations also coincides with negative feedback left on a handful of mobile app rating sites, providing quantitative confirmation that users do not like it.

By drilling into the “Quick Balance” transactions, the product manager can see at a glance that the performance of the feature is being affected by errors.

At this point, she can share the dashboard with IT Ops and ask them to follow up on the problems associated with Quick Balance. Once the performance issues are corrected, the Analytics Dashboard should register a decrease in deregistrations and the rating sites will also likely begin to show positive feedback.

Successful product management in the digital world requires rapid insight into user experience. Learn more about AppDynamics Business iQ and how you can gain real-time awareness of application performance, user experience, and their impact on your business performance.

How to Get a Seat at the Table with Business iQ Visualizations

In an era of software-driven business, there is no doubt that IT deserves a seat at the table with top business decision-makers. But how do you quickly convince the Business that IT is about more than servers and software? One way to demonstrate the critical role IT metrics play in business decisions is to simply click over to the Analytics tab of your AppDynamics screen. If you are new to Analytics you may be surprised by the powerful visualizations that are available out of the box.

For example, if you are an eCommerce firm, you can quickly learn the overall Conversion Rate as well as the percent of “Abandoners” for each step of your customers’ Check Out Journey, simply by utilizing the Funnel Widget:

If you are an insurance company, you could learn the “health” of every step of your on-line Customer Journeys by product:

If you are a travel agency, you will be able to visualize your revenue stream by customer segment and see the impact of defective transactions by location:

Some have gotten the impression, based in part on their experience with other business intelligence tools, that getting real, actionable business information from their IT systems will require a major investment of time. They worry about the steps it takes to extract business data from within their technical systems. What they don’t realize is that a lot of this work is done by AppDynamics out of the box. This is especially true when you leverage Business iQ on top of our core APM and EUM offerings.

I’ll show you how easy this is. Let’s walk through examples of some correlated business and performance information you can learn just by playing around with the visualization widgets.In these examples we’ll focus on e-commerce, but you can easily imagine relevant business metrics for your own use case.

If you place APM metrics, like Average Response Time and/or Number of Errors, next to the sequential order of customer events, you can better understand what is the business impact of your performance problems.

Or you can also try placing a funnel with Conversion Rates with the Abandoners numbers for all major steps in your Customer Journeys just below the “overall health” of every step of their respective business journeys (represented by the rows of circles):

A little more advanced “play” can involve ADQL (AppDynamics Query Language) queries, creating metrics for the ADQL queries’ search results and Health Rules for those metrics.

Let us start with the ADQL queries. In the Controller UI, go to Analytics > Searches and create the required ADQL queries, for example:

-What is the number of Unique Visitors?

SELECT distinctcount(segments.userData.SessionID) FROM transactions WHERE application = "eCommerce"

-What is the number successful Orders Placed?

SELECT count(*) FROM transactions WHERE application = "eCommerce" AND transactionName = "CheckOut" AND userExperience != "ERROR"

-What is the total amount of Revenue?

SELECT sum(segments.userData.ProductPrice) FROM transactions WHERE application = "eCommerce" AND transactionName = "CheckOut"

-What is the total amount of Revenues at risk?

SELECT sum(segments.userData.ProductPrice) FROM transactions WHERE application = "eCommerce" AND transactionName = "CheckOut" AND userExperience = "ERROR"

Save the most interesting ADQL queries search results so that you can create metrics that are updated in near real time. In the Controller UI, click over to Analytics > Metrics and then simply create from your saved queries. E.g.:

-> # of Orders Placed Metric
-> # of Unique Visitors Metric
-> Check Out Total Processed Metric
-> Check Out Total Revenues at Risk Metric

Use a time series graph chart widget and “Check Out Total Processed Metric” data to visualize the total amount of Revenue trend over time. Notice that you can include a dynamic baseline so that your thresholds are based on average behaviour, calculated hour by hour for a given time period

(i.g. Daily Trend, Weekly Trend, Monthly Trend etc.). Using the baseline helps you to find an answer to the following question: Is the total amount of Revenue normal at the given hour?

Now you may wish to create Health Rules for the remaining business metrics so you can visualize them and receive alerts in the same way you would for your technical performance metrics. In the Controller UI, go to Analytics > Alert & Respond > Health Rules and create your Health Rules for the new business metrics. We’ll use static thresholds here for simplicity, but again dynamic baselines are likely the better way to go for your use-case.

-eComm – Orders lower than expected:

Critical condition: # of Orders Placed < 700
Warning condition: # of Orders Placed < 800

-eComm – Unique Visitors lower than expected

Critical condition: # of Unique Visitors < 10
Warning condition: # of Unique Visitors < 60

-eComm – Check Out Processed lower than expected

Critical condition: Check Out Total Processed < 200
Warning condition: Check Out Total Processed < 100

-eComm – Revenues at risk higher than expected

Critical condition: ({RevenuesAtRisk}/{CheckOutTotalProcessed}*100) > 10 (%)
Warning condition: ({RevenuesAtRisk}/{CheckOutTotalProcessed}*100) > 2 (%)

Now you can create and link some Health Status widgets with the relative Health Rules:

And finally you can put all of your interesting components together in Dashboards, for example:

Refer to the AppDynamics documentation for more ideas on ways to explore your data.

It doesn’t take long to create these kinds of visualizations, but the insight you can offer to your business leaders is incredibly powerful. As you get comfortable using these basic building blocks you will see opportunities to assemble them into more complex analyses such as business health, the steps of the business journey, performance by segment, and more.

Learn more about Business iQ and how it can give you a seat at the table.

Krzysztof Gawronski is part of AppDynamics Global Services team, which is dedicated to helping enterprises realize the value of business and application performance monitoring. AppDynamics’ Global Services’ consultants, architects, and project managers are experts in unlocking the cross-stack intelligence needed to improve business outcomes and increase organizational efficiency.

AI’s Arrival in the Enterprise Will Have Profound Implications for IT

At the end of Twentieth Century in the wake of Deep Blue’s triumph over Garry Kasparov, it was popular for computer scientists to speculate about when human begins would begin to interact with artificial intelligence. It was generally believed that machines with true reasoning capabilities were decades away. In an interview with author Michio Kaku, published in Visions: How Science Will Revolutionize the 21st Century, AI expert and Carnegie Mellon professor Hans Moravec predicted that robots would be able to model the world and anticipate the consequences of different actions sometime between 2020 and 2030.

Twenty years later, we are no longer wondering about how artificial intelligence (AI) will first appear in our lives. It has arrived in the form of virtual assistants like Alexa and self-driving cars. But this can give a misleading impression of what we can expect from AI in the next few years. AI software is not going to evolve human-like reasoning capabilities anytime soon.

Indeed, most of what is described as AI is really machine-learning algorithms that act largely as detectors. These algorithms analyze massive amounts of data and learn to discriminate between normal and anomalous behavior. AI, where it exists, is similar to a decision-support system for reacting to behavior as it changes. But even in these early stages, machine learning and AI are changing the game for IT operations. In the next few years, the impact of machine learning and AI will be profound.

The problem enterprises are facing is that computing environments have simply grown too large and too complex for human beings to monitor alone. To effectively monitor enterprise systems, IT must track millions of metrics per second. This is not a challenge that can be met by putting another screen on the wall of the network operations center. There are already too many screens, and just contemplating the number of screens that would be required is overwhelming. Even more daunting is figuring out the five or ten metrics that matter the most out of five or ten million as every new millisecond brings the system to a new dynamic state.

The company I founded, Perspica, which was was acquired last year by AppDynamics/Cisco, solved this problem for our customers by applying machine learning and AI to massive amounts of streaming telemetry data generated by applications and IT infrastructure. What Perspica did was surface all the relevant metrics and then use those metrics to accelerate root cause analysis and reduce the mean time to repair. But Perspica’s ability to grow beyond that was limited by the data that we had access to. In fact, everyone involved in machine learning and AI at that time faced the same limitation. We lacked a source of truth on which to train our algorithms to go beyond what they had already achieved.

But this limitation is rapidly being overcome. Increasingly, data scientists are gaining access to new sets of what we call labelled data—sets of numbers or strings of text that a computer can understand as a true representation of something else. Data scientists who work with IT data, in particular, are finding that enough labelled data exists that we can realistically begin talking about automating large parts of IT in the next two or three years. And that is only the beginning.

In the future, every enterprise is going to have some combination of machine learning and AI to monitor its computing environments, and, equally important to understand how changes to those environments affect business goals. As these systems are deployed, they will become smarter and more sophisticated. Every application, every server, and every port on that server will have its own unique AI model, which means if you have 50 applications running on 10,000 servers you will need to train 500,000 models. This is not something that is going to be created overnight. But once these models are put in place, self-healing systems will become standard.

We’ll see AI playing a role in everyday IT and business events. For example, imagine a large airline that is planning on holding a worldwide promotion. The airline’s IT department rolls out new application code as a canary deployment. But the monitoring system soon reveals the new release is performing worse than the old code. While the business owner and IT staff are realizing that the code push has failed, the airline’s AI system is determining the root cause is a disk space issue and taking steps to address the problem.

For many years, Perspica and others were doing detection. Today, as we broaden the libraries, increase the sets of problems that have solutions and bundle those solutions together, we’ll be able to start doing remediation. Moravec, it seemed, had the timeline correct.

What will happen in the next twenty years? The media sometimes promotes “fear of AI.” But I see AI making business more profitable and people more productive. It will improve service quality, reliability, and availability not only in IT but across all industries. And in this way, AI will not only have profound implications for IT. It is also bound to improve the human condition.

Customer Spotlight: Carhartt Wins 2018 CIO 100 Award

Today, our customer Carhartt was honored as a 2018 CIO 100 Award Winner by and to say that we are thrilled is simply an understatement. It’s an incredible accolade and a nod to the tireless work that CIO, John Hill, and his team have done to transform Carhartt into a premiere fashion and workwear brand.

The CIO 100 Award program recognizes organizations around the world that exemplify the highest level of operational and strategic excellence in information technology – and Carhartt’s digital transformation was not overlooked. Founded in 1889 with just two sewing machines and five employees, Carhartt has since grown into a global brand, overcoming the Great Depression, and embracing the digital age to build a vibrant online sales platform and loyal customer base.

Progressive CIOs like John are leading their companies through exciting change in their industries and we’re thrilled Carhartt chose AppDynamics Business iQ to aid in their digital transformation.

“Retailers live and die by the consumer experience, whether that experience comes in person, through mobile or on the web,” John explains in a press release announcing the win. “It was imperative to understand how to position our business for the evolving consumer needs of today and in the future. With Business iQ, we now have a direct lens into the health of business transactions across our entire digital shopping journey in real time, so we can continue to deliver the highest quality user experience to our consumers.”

I got a chance to steal a few minutes of John’s time to congratulate him on this win and to find out what else Carhartt has been up to. Check out our brief Q&A:

Prathap Dendi: The CIO 100 award is a recognition of the innovation that you and your team have been able to implement. What does this acknowledgement mean for you personally and for Carhartt as a whole?

John Hill: The CIO 100 award is a great honor for Carhartt and represents our commitment as an IT organization to further drive Carhartt into the digital era through innovation, enablement, thought leadership, and collaboration.

Prathap Dendi: What has the past year looked like in terms of collaboration with the IT and Business teams at Carhartt? What do you attribute that to?

John Hill: Increasing collaboration between our business product owners and our IT teams has been instrumental in driving our ability to quickly deliver better solutions that are more stable, efficient, and effective.  The increased transparency in real time access to application/business metrics and data has made these collaboration efforts more effective than ever before.

Prathap Dendi: If you could describe the growth you’ve witnessed over the past year in one word, what would it be?

John Hill: Awesome.

Prathap Dendi: How do your teams feel about having access to real-time data and being able to collaborate more with the business teams?

John Hill: The ability to see in real-time what is happening in the application is key to supporting a 24/7 platform.  The ability to see issues as they happen and trace them back to their root causes has enabled our teams to become more proactive and confident in the solutions they are delivering. The business teams have also embraced the ability to see real time data regarding new features and releases. It has increased the trust and confidence that both teams have in each other.

Prathap Dendi: How have the other executives at Carhartt responded?

John Hill: We have received great feedback from senior leaders as we continue to expose key real-time business data to key stakeholders throughout the organization. This data is empowering leaders to make better, faster decisions that directly impact the business as it happens.

Prathap Dendi: What’s next for Carhartt? What are you hoping to delve into in the next six months, year?

John Hill: Deeper user experience metrics and analysis, complete business process monitoring across additional systems of record.

Prathap Dendi: What has been your experience working with AppD?

John Hill: AppD has been a true partner. They have engaged at every level to deliver a complete solution. AppD brought their extensive APM experience to our implementation and enabled us to hit the ground running. They immediately identified a number of key processes that could be monitored and analyzed through the tool.

I’m so excited for everything that John and the Carhartt team have accomplished, and I’m looking forward to seeing what else is in store for them. Here’s to many more wins like this one!

Take a guided tour today for an in-depth look at how Business iQ puts application teams in a position to drive the business.  

The Virtual Assistant is the Future of Business

What makes a good user experience (UX)? Many factors come into play, including the user’s work habits, environment and goals. An exceptional UX must meet the customer’s precise needs, applying elegance and simplicity to deliver seamless interaction. As noted by Nielsen Norman Group, a leading UX consultancy, one first must draw a distinction between UX and the user interface (UI). Using the example of movie review website, firm co-founders Don Norman and Jacob Nielsen write: “Even if the UI for finding a film is perfect, the UX will be poor for a user who wants information about a small independent release if the underlying database only contains movies from the major studios.”

This analogy applies as well to the world of application performance management (APM) software, where the conventional UI—the aforementioned dashboard—is optimized to meet the needs of the IT professional, who can drill down to explore technical issues in greater depth. Often, however, this presentation is visual overkill for the mobile-toting business user seeking a far narrower range of insights.

For business customers, simpler is almost always better, and that’s why conversational UX is the superior choice for extracting real-time business insights from vast and varied sources of data. According to a recent study by CapTech, an IT management consulting firm, businesses are rapidly adopting conversational UX-based virtual assistants, such as chatbots, Amazon Alexa, Apple Siri and Google Assistant. While conversational UX is beginning to appear in some APM solutions too, it has targeted the IT user and not the business. At AppDynamics we are beginning to experiment with this focus on the business user, beginning with some prototypes of an “intelligent bot,” a smart virtual assistant that employs proactive reporting, on-demand interaction, and automated task execution—all combined with machine learning—to extend the capabilities of APM far beyond its traditional IT roots.

Needed: Simpler Data Analysis

Business users today favor popular, mobile-based social platforms—think Slack, Skype, Facebook Messenger and Cisco WebEx—for workplace interaction. At the same time, self-service analytics is a growing trend in the enterprise, one requiring intuitive tools that enable the user to glean insights from company data without assistance from IT or a data scientist. This self-serve approach can bring analytics “to the masses” via a familiar, intuitive UI, often on a mobile device.

The dashboard isn’t the answer here, particularly when business users want to be alerted only when key metrics are impacted. A conversational UX, voice or text, can revolutionize business by providing intelligent automation that enables users to see in real time how technology is impacting their operations and customers. By empowering companies to measure and baseline everything, conversational UX becomes a central nervous system for the entire organization. The key benefit here is proactive visibility, such as real-time alerts on sudden changes in company sales, revenue or customer churn.

A Smarter Virtual Assistant

The conversational assistant can deliver value to a much broader user base—not just IT—by providing deeper, more automated and proactive business monitoring. Think of it as a concierge service for monitoring and optimization. By sending alerts for key metrics, the virtual assistant frees users from having to log into a dashboard to view their KPIs, since their assistant does it for them.

What kinds of business services could this assistant deliver? Some examples:

    • Real-time proactive reporting: users are alerted to KPIs they care about, such as a sudden drop-off in sales or revenue.
    • On-demand interaction: The ability to make natural language queries such as “What were total sales in the past hour?” Or, “How did this customer segment perform over the past two hours?”
    • Context-driven recommendation and task execution: The assistant might ask the user, “Total sales have declined 25% in the past hour, what would you like to do?” Suggested actions could include “Provide more information,” “Notify IT,” “Snooze alert for 30 minutes,” or other options.
    • Customer segment monitoring: When a user requests the customer journey for a particular service or application, the assistant could provide a graphical representation revealing bottlenecks or critical metrics (such as a drop in the number of conversions) that must be addressed immediately.

The true benefit of the business-focused virtual assistant is its ability to provide proactive reporting, on-demand interaction and automated task execution—all without the need for pesky logins or dashboards. It could also deliver key metrics, such as high-level KPIs, via an easy-to-use UI available on a variety of platforms, including mobile.

By incorporating machine learning capabilities, the virtual assistant could also deliver prescriptive insights that help businesses detect and prevent fraud in real-time, improve retail forecasts, create more accurate pricing models, and much more. More than simply a conversational UX, this cognitive helper would become a powerful recommendation engine enabling a natural conversation channel for business users to engage with their APM software.

The intelligent virtual assistant is smart automation that will change the way business operates. Rather than analyzing massive data sets long after they’ve gone cold, business users will soon be able to access real-time information—uncovering insights that enable them to see trends and changes as they happen. Here at AppDynamics, we are seeing promising results with virtual assistant prototypes, and we’re starting to work with customers on early use cases.

Gartner Report: The ‘3B’s’ of Engagement: Business Architecture, Business Process, and Business Outcomes

Today, many IT leaders are missing opportunities to help their business grow, add value, and transform operations because they aren’t aligned with business leadership.

In Gartner’s recent report, The ‘3B’s’ of Engagement: Business Architecture, Business Process, and Business Outcomes, Gartner shares how CIOs can leverage existing competencies to successfully engage with business leaders, enabling strategic planning, innovation, and growth initiatives.

The report gives solutions for fixing communication issues between IT and business teams, and offers insight into how IT leaders can demonstrate relevance and vested interest in delivering desired enterprise performance results.

The report also reveals another stumbling block rarely considered: CIOs are often victims of their own success.

“A solid reputation as a great utility service provider has become an Achilles heel for many CIOs,” notes the Garter report. “Their success in infrastructure, operations, and back-office automation precludes them from being invited to participate in discussions about front-office opportunities for growth, new value and transforming business.”

Another challenge that CIOs face is an inability to clearly communicate and demonstrate IT’s impact on the business.

“Even when [IT and business] teams work well together and have strong competencies in business architecture, business process and business outcomes, CIOs struggle to link their teams’ contributions to the aspects of business operations (aka the actual business artifacts) they represent.”

Gartner identifies these as key challenges for companies, as CIOs already have immense competency in three areas – business architecture, business processes, and business outcomes – which are crucial for spotting opportunities for growth.

So, how can CIOs communicate with business leaders to ensure they’re at the table for growth and innovation initiatives?

The solution, according to Gartner, is to use those three Bs – business architecture, business processes, and business outcomes – as a hub for communication, or a translator device, if you will.

Gartner suggests that using business-oriented terms will help CIOs “demonstrate relevance and vested interest in delivering desired enterprise performance results by explicitly linking competencies in business architecture, business process, and business outcomes to the actual business designs, processes, and outcomes they enable.”

The report also lays out multiple tactics for clear communication with business owners, including:

– Establishing a lingua franca

– Examples of visualization

– Creating a Level 0 diagram

– And more

For additional details and insights, download Gartner’s full report now: The ‘3B’s’ of Engagement: Business Architecture, Business Process, and Business Outcomes now.

Gartner The ‘3B’s’ of Engagement: Business Architecture, Business Process and Business Outcomes, Janelle B. Hill, Patrick Meehan, 16 March 2016.

Gartner Report: Business Outcomes are the Milestones on an Application Strategy Roadmap

As the enterprise world continues to move from multiple store locations to multiple clouds, the call for IT leadership and application owners to work more closely with business owners has become critical. After all, at a digital-first enterprise, those responsible for building and maintaining the digital part of the company need to be involved in strategies for driving business success. And in turn, business leaders need to be involved when application strategies are created.

Naturally, the realization of these two powerhouses meshing perfectly like the cogs of a machine has been slow to come. One of the major stumbling blocks is a general language barrier.

The Gartner report, Business Outcomes Are the Milestones on an Application Strategy Roadmap, now available for download, describes it this way: “Application leaders struggle to communicate with business leaders about application strategies because they talk about them in terms of arcane application acronyms and technology implementations.”

Gartner goes on to explain that the disconnect, unfortunately, is even more complicated than just the words we use. There’s a gap in understanding by application leaders on how their implementation impacts the business.

“Application leaders don’t understand how the various IT requests relate to the business strategy.” They also “can’t define in a business sense what capabilities need to be delivered.”

Success is hard to find when there’s not even a mutually-understood definition of success.

And the issues continue even if an IT initiative does get implemented. As Gartner puts it, “Application leaders don’t have measurable performance goals of the capabilities being delivered to determine if the effort was a success.”

And when IT can’t show success, it puts a lot on the line, including funding for future initiatives.

It’s On Us

According to Gartner, to solve the issue, application leaders need to adopt and fully understand new, business-centric metrics. We believe that will get business leaders nodding with familiarity and excitement. Gartner calls these essential metrics business outcomes —a specific and measurable target action that is taken in response to a business direction or disruption. Simply put, the business benefits that will result from a particular IT initiative. These business-centric leading indicators can come in the form of cost-benefits, increased revenue, or improvements in customer experience. The bottom line is pretty much the bottom line. But it can’t stop there. Application leaders need to then be able to prove that the desired outcome was (or was not) attained using these same business outcomes.

For more insights on how IT and business leaders can improve collaboration and communication, download the Gartner report now.

Business Outcomes Are the Milestones on an Application Strategy Roadmap is available for download now, and gives clear steps, strategies, and examples to get you on your way to a more productive collaboration with business leaders.

Gartner Business Outcomes Are the Milestones on an Application Strategy Roadmap, Bill Swanton, 7 February 2017.

AppD on AppD: Scaling Our Custom Dashboards Platform

This blog post is a developer’s perspective on how using our own AppDynamics software has helped us find and fix performance-related issues – and how other developers can do the same. 

One of the most challenging aspects of developing cloud-based platforms is scalability. As we innovate and build new features, it is essential for developers to ensure these new platform features are scalable and do not impact the performance of our applications, especially as any impact on performance is likely to impact all tenants on the cluster.

This is a challenge that we experienced firsthand. My team here at Appdynamics is responsible for building custom dashboards, one of the most powerful features of AppDynamics. It allows you to group relevant metrics into one central dashboard as well as build sophisticated dashboards with drill down capabilities.

As we added more capabilities and support for the different types of metrics, we realized that scaling our dashboards was becoming a challenge. Many of our customers were exploring different dashboard capabilities and features, which was truly stressing our endpoints. This is when our practice of using AppDynamics internally – “AppD on AppD” – helped give us insight and visibility into the issues we faced while developing new features on the platform.

Our AppD on AppD Approach

Recently, we made some changes on the backend of a REST endpoint that returns metrics for different types of widgets on custom dashboards. Our team’s internal process is to ensure every feature we implement is stress-tested under a real type of load profile, so we made sure to test this as well. Below is what we discovered.

On local dev environments, the feature worked fine and the data returned by the API took a few milliseconds. Next, we deployed our changes to our performance environment, which is similar to an actual production instance with large amounts of data. We immediately noticed that under stress, the UI which uses the endpoint was very slow and the average response time of the endpoint was considerably high. Luckily, since AppDynamics was monitoring the performance environment, it was easy for us to dig into the issue.

We configured a business transaction to monitor the REST endpoint with the slow response time and within a few minutes, we collected transaction snapshots which gave us valuable behind-the-scenes information of the REST API calls. There were two things that caught our attention:

Resolved Issue #1

A particular method which was being called repeatedly was taking more time to execute than before. The method itself took around 100-150 milliseconds, but if it was called 100 times in a single transaction, it would take around 15,000 milliseconds to execute, which is roughly 15 seconds.

The image below shows the total time it took for all the calls of this method in this single transaction.

Here is a code snippet of that method:

After a few minutes of looking at the full implementation of this method, we found that String.replace could be a potential problem here, which happens to be slower than StringUtils.replace. As a result, we made a minor change and modified the code to use StringUtil implementation. Here is some information on StringUtils.replace vs String.replace.

Resolved Issue #2

Another issue we noticed was that there were too many database calls for a single request.

We made a few optimizations here, including caching the data and bulking-up the queries. After applying these fixes, we measured the performance again and saw the average response time for the endpoint improve drastically.

Lastly, we created our own custom dashboard to measure performance of our specific endpoint, showing us different metrics like average response time, errors, and calls per minute along with thresholds and baselines. We also created a scheduled report that sends our team a dashboard snapshot everyday, allowing us to easily spot outliers and proactively address issues. We also could have created alerts, which notify you when certain conditions are met or exceeded based on your configuration. Scheduled reports and alerts make it easy to spot outliers and proactively address issues.

Without AppDynamics, it would have been difficult for a developer to quickly pinpoint these issues within a large codebase. But with AppDynamics, not only does it become easier to find issues in production, but developers can proactively ensure that features are robust and scalable before deploying them to production. This reduces the time spent on performance-related issues, and instead, gives developers more time to innovate and write code.

Learn more about our business-centric dashboards or get a demo from our sales team today.

How to Rev Up Your Business Performance During the Holiday Season

Both buyers and businesses love Black Friday and Cyber Monday. The online retail demand however is not limited to just those two days. It often stretches from days prior to Black Friday and after Christmas.

This year, Forrester predicts that online holiday sales in the United States will reach $129 billion in 2017 – a 12% growth in holiday sales over 2016. And this shouldn’t come as a surprise, as consumers continue to spend more online due to the convenience, selection, and value.

As a result, online retailers understand the importance of delivering a seamless digital experience to their customers. But unfortunately, technical issues have plagued retailers during this peak season, likely costing them millions of dollars in sales.

For example, last year, Macy’s website went down the morning of Black Friday and was experiencing outages through the afternoon. As a result, most users were greeted with a “temporary shopping jam” message due to overflow shopping traffic.

The year before, Neiman Marcus and Target experienced similar issues.

With some of the biggest retailers struggling to maintain a seamless digital experience for their customers during the holiday season, many businesses are now asking themselves the following questions:

– How can you offer an improved digital experience while ensuring your website meets the sustained and peak demand the holiday season is bound to usher in?

– How do you know if those coupons you are offering will ultimately result in increased sales?

– How can you track user journeys to analyze in real-time what the impact on the business is and where are the drop-offs?

– How do you take all of the above and tie it back to a problem which may be related to your application or infrastructure to fix them in real-time?

The answers to these questions can be found with AppDynamics Business iQ. An earlier post explored how you can leverage AppDynamics Business iQ to analyze how application performance is impacting customer experience and business outcomes. Business iQ allows developers, ops leaders, and marketers and executives to share a common interface for reviewing the health and performance of the business in real-time.

For example, Nasdaq, uses Business iQ to understand usage trends of their applications. Dhisco, an online payment company, segments its largest hotel chain customers to measure critical metrics such as total bookings and room availability across its major customers. And apparel company, Carhartt, uses Business iQ to monitor its purchase funnels to quickly identify application performance issues and how they may be impacting conversion rates.

How Carhartt Leverages Business iQ on Cyber Monday

Below is an excerpt from a Carhartt Case Study.

Established more than 128 years ago, Carhartt created the first purpose-built bib overalls for railroad workers. John Hill, Carhartt’s chief information officer, says that as Carhartt expanded its global footprint, it was important for IT to deliver a core set of capabilities that could scale as needed to different regions, rather than having different infrastructures and different application capabilities for every market.

To help with their digital transformation, Carhartt enlisted AppDynamics for real-time performance monitoring. Carhartt’s IT team had been relying on an alerting system to identify issues in four on-premises environments. But the system didn’t meet the team’s need to react in real time to changes in applications or the market. And the information it provided didn’t overlap with business needs.

Hill knew it would take time to achieve end-to-end visibility across Carhartt’s entire ecosystem, but he also believed he could realize immediate value from AppDynamics by focusing on specific processes like ordering, inventory, and checkout.

Leading up to Cyber Monday 2016, the team built a dashboard that tracked sales velocity and response times. “We wanted to see what was happening in real time as orders were coming in and how the infrastructure was supporting those orders,” Hill says.

On Cyber Monday, the engineering team was settled into their war room to monitor the action, where they were later joined by members of the senior management team. It turned out Hill had sent out a link to the new dashboard to the leadership. For the first time, Carhartt’s executives could watch orders being placed and processed. “The president of the company joked that the Business iQ dashboard was addictive,” Hill says.

As business leaders looked on, the engineering team spotted an issue in the order processing system that could have had a direct impact on the day’s revenue. The issue was fixed in ten minutes. “Before deploying AppDynamics and Business iQ, we wouldn’t have known about the problem until customers started complaining to the help desk,” Hill says. “We were able to quickly readjust and resolve the problem without our customers knowing that a problem was starting to occur.”

In addition, Hill notes that the real-time capabilities of Business iQ changed the relationship IT had with their business colleagues. The biggest change for his team was the ability to be nimble and respond quickly to the business needs in a high pressure situation like Cyber Monday and create real impact for the business.

Read the full Carhartt Case Study to see AppDynamics BiQ enabled Carhartt to have a seamless Cyber Monday that registered more than a record million dollars in sales.

To learn more about AppDynamics, check out The AppPerformers, a compilation of 50 short case studies of how enterprises have leveraged AppDynamics to improve their application and business performance.

Why Metrics Must Guide Your DevOps Initiative

Metrics-oriented thinking is key to continuous improvement – and a core tenant of any agile or DevOps philosophy. Metrics are factual and once agreed upon, these facts are used to drive discussions and methods. They also allow for a collaborative effort to execute decisions that contribute towards business outcomes.

DevOps, although becoming a commonly used job title, is not a role or person and there is no playbook or rule set to follow. Instead, DevOps is a philosophy which spans people, process, and technology. The goal is releasing better software more rapidly, and keeping said software up and running by joining development and operational responsibilities together. Additionally, DevOps aims to improve business outcomes, but there are challenges in selecting the right metrics and collecting the metric data. Continuous improvement requires continuous change, measurement, and iteration. What’s more, the agreed-upon metrics drive this cycle, but also create insights for the broader organization.


Data-Driven DevOps

A successful DevOps transformation focuses on a couple areas. To start, a culture change is needed between development and operations teams. Another core tenant of DevOps is measurement. In order to accomplish a true DevOps transformation, it’s important to measure the current situation and regularly review metrics which indicate improvement or degradation. One of the core tenants of DevOps is measurement, and using said measurements as facts when driving decision making. These metrics should span several areas which may have been considered disjointed in the past.

To help DevOps teams think of possible metrics and how these metrics relate to key initiatives, Gartner recently released this useful metrics pyramid for DevOps:

Many of these metrics span development, operations, and most importantly – the business. They measure efficiency, quality, and velocity. However, Gartner points out that the hardest part is often defining what we can collect, take action upon, audit, and use to drive a lifecycle. The second challenge (which Gartner does not discuss), is how these metrics should be linked together to offer meaningful insights. If the metrics do not allow linkage between a release and business performance, attribution gaps remain. And unfortunately, many enterprises today analyze metrics that have a lack of linkage or relationship between them.

To help with these relationships, context is critical. Without context, metrics can be open to interpretation, especially as you move up the Gartner pyramid. And this is precisely where AppDynamics can step in and help.

Based on our measurement technology, AppDynamics can link metrics together to allow you to attribute earnings or cash flow with a release or change that represents an incremental improvement in the application. We represent these within Business iQ, our event-based analytics system for the highly valuable data we extract.

Additionally, AppDynamics provides metrics to drive visibility inside the application without creating an additional burden for developers. In fact, with automated instrumentation as part of AppDynamics APM, metric data is produced consistently and comprehensively across all teams. This is extremely beneficial as many teams have different ways of collecting data, which can traditionally lead to inconsistencies. But with AppDynamics, consistent measurements are always obtained from the application components and desired business outcomes of the application.

Learn more about how Business iQ can accelerate your data-driven DevOps initiative.