CAT | APM Thought Leadership
In the third part of this series I discussed preparing to launch a mobile application with load testing and beta testing and highlighted the differences between the iOS and Android ecosystems. In this post I will dive into monitoring your production mobile application.
Production consideration: Crash and Error Reporting
Crash and error reporting is a requirement not only for the development of your application, but also for testing and production. There are quite a few crash-reporting tools available including AppDynamics, Crashlytics, Crittercism, NewRelic, BugSense, HockeyApp, InstaBug, and TestFlight. All of these tools have are capable of reporting fatal errors in your application to help developers track down the root cause of bugs. The problem with crash and error reporting is that it only tracks issues after they have affected users. Both the Apple App Store and the Google Play Store provide basic crash reporting metrics.
The harsh reality is that mobile applications have a fickle audience that is heavily reliant on curated app stores and reviews. Reviews can make or break a mobile application, as can being featured in an app store. A common best practice is to allow in-app feedback to preempt negative reviews by engaging the user early on. There are a variety of services that make it easy to provide in-app feedback like Apptentive, Appboy, and Helpshift.
This is why being proactive with quality assurance and production monitoring has a significant impact on the success of an application. Not only must the application work as designed, but also the experience must be polished. The expectation in the mobile community is significantly higher than on the web.
Production consideration: Analytics & Instrumentation
Smart executives are data driven and mobile applications can be a plethora of business intelligence. When it comes to instrumentation the earlier you instrument and more metrics you track, the better informed you will be. Analytics and instrumentation are crucial for making informed and smart decisions about your business.
Who is your audience? What platforms and devices do they use? What user flows are the most common? Why do users abandon? Where are users located? What is the performance of your application?
Tracking important demographics of your audience like operating systems, devices, carriers, application versions, and geography of users is key. These metrics allow you to target your limited resources to where they are needed most. There are quite a few analytics platforms built for mobile including Google Analytics, Flurry, Amazon Analytics, FlightPath, MixPanel, KissMetrics, Localytics, and Kontagent.
All of these tools will give you better insights into your audience and enable you to make smarter decisions. There are important metrics to track like total # of installations, average session lifetime, engagement and growth, and geography of your users. Once you have basic user demographics you can use MixPanel or KissMetrics to track user activity with custom event tracking. The more instrumentation you add to your application the more metrics and customer intelligence you will have to work with.
Production consideration: Application Performance Monitoring
Application performance management tools enable you to discover the performance of your mobile and server-side applications in production. APM allows you to understand your application topology, third party dependencies, and the performance of your production application on both the client-side and the server-side. Modern application performance management solutions like AppDynamics track crashes and errors, the performance of the mobile application, and correlates performance to your backend platform, while providing rich user demographics and metrics on the end user experience. With modern business reporting tools you can evaluate the health of your application and be proactive when performance starts to deteriorate.
End User Monitoring allows you to understand the application experience from the eyes of your real users. There are quite a few solutions to monitoring the end user experience in the market place. AppDynamics, Crittercism, New Relic, and Compuware allow you in instrument your application and gain visibility into production performance problems.
Business consideration: Real-time business metrics
Once you have launched a successful mobile experience you need to understand how that experience affects your business. If you have a business critical application like the Apple Store Checkout application or the Fedex Package Management application your business is dependent on performance and use of your application. You can gain valuable insight into your business if you track and correlate the right metrics. For example, how does performance affect revenue, or what is the average price for a checkout transaction. Understanding your core business metrics and correlate them to your mobile experience for maximum business impact.
Business consideration: Monetization
Your plan is to retire off this application, so you need to have a monetization strategy. The most common ways to make money from applications are Pay to play (charge a fee for your application), freemium (offer a free and pro upgrade), offer in-app purchases (like levels, tokens, and credit), and traditional advertising. There are many services to enable mobile advertising like Apple’s iAds, Google’s Admob, Amazon’s Mobile Ads, Flurry, inMobi, Millennial Media, and moPub. All of these strategies require precision execution, but some strategies work better for specific types of apps. Experiment with multiple strategies and do what works best for your business.
It is no longer enough just to have a presence on the web. In fact more and more companies are going mobile first. The mobile landscape is constantly evolving and the mobile market is seeing continued growth year over year.
Want to start monitoring your iOS or Android application today? Sign up for our beta program to get the full power of AppDynamics for your mobile apps. Take five minutes to get complete visibility into the performance of your production applications with AppDynamics Pro today.
As always, please feel free to comment if you think I have missed something or if you have a request for content in an upcoming post.Link to this post:
In the second part of this series I discussed developing a mobile application and choosing a backend platform and building for various network conditions. In this post I will dive into some considerations when launching a mobile application.
Mobile app audiences are a notoriously fickle bunch and a poor first impression often results in a very harsh app store review that will negatively impact your apps growth. When an app store rating can make or break your application you have to be diligent in making sure every user has a stellar experience. The best way to do this is thoroughly test your mobile experience and load testing your backend to ensure you can handle peak traffic.
The key to a successful launch is great planning and testing. Launching mobile applications are significantly more difficult than the common web application. Not only is the audience more fickle, but you also have to adhere to third-party processes and procedures. Thorough quality assurance, crash and error reporting, load testing, and proactive production monitoring are essential to launching a successful mobile application.
Launch consideration: Testing native applications across mobile devices
Testing mobile applications is notoriously difficult due to the vast number of devices. There are a few services that make this easier for engineers. I have seen a few strategies for testing mobile devices – usually you go to Amazon and buy the top twenty devices for Android and iOS and manually test your application across every device manually. Mobile device labs of this sort are quite expensive to setup and maintain and often require some level of automation to be productive. Alternatives to setting up your own mobile lab is to use a mobile app testing platform like TheBetaFamily. They offer an easy way to test your native application across many different devices and audiences with ease.
Launch consideration: Capacity planning and load testing
Capacity planning is key to the successful launch of any web application (or mobile backend). If you want to understand what can go wrong look no further than the failure to launch of healthcare.gov. Understanding your limits and potential capacity is a requirement for planning how much traffic you can handle. Making an educated assumption about potential growth and you can come up with a plan for how many concurrent users you might need to support.
Once you understand your maximum concurrent users you can test your backend infrastructure to be sure your mobile experience doesn’t suffer. There are quite a few tools available to help you load test and evaluate the scalability of your backend platform. Apica, Soasta, and Blazemeter offer services that allow you to simulate your mobile application being used at high levels of concurrency.
Launch consideration: Beta testing
Beta testing is the last quality assurance step before you can make your app generally available. Testflight, HockeyApp, and Ubertesters allow you to distribute your application for testing to a select group of users. When it comes to beta testing the more users you can convince to give feedback and the larger distribution of devices the better. These beta testing and distribution tools enable you to easily gather feedback early on about what isn’t working in your application and save you from the embarrassment of negative app store reviews due to obvious problems. A/B testing is also a great way to find out which flows work best as part of your beta testing experience. This is an essential step to a successful launch – the more beta testers you can find the better.
Launch consideration: Hard launch or Soft launch?
Once you have beta tested and decided you have a great application that is battle tested for production you need to decide how to launch. The real question is hard launch or soft launch. The traditional hard launch is straightforward. Your app is approved in the app store and you go live. There are a few different strategies for soft launches of major applications. The most common is to soft launch outside of your primary market. If you are planning to release in the USA you can easily pick another region with similar characteristics like Canada, Australia, or the United Kingdom. The benefit of soft launching in a secondary market means you can validate assumptions earlier and beta testing your key audience. Soft launching can validate product/market fit, app experience, usability, and app/game/social mechanics. The result is your first experience with your key demographic will be based on the data you learned from your sample audience. The end result will be a much more polished and proven app experience.
Launch consideration: App store submission process
The application submission process varies greatly depending on the app store. This is where you get to sell your application with a marketing description, search keywords, and screenshots of your app in action. You can specify pricing and what regions/markets you want your app to be available in.
With Apple it is customary to wait up to two weeks for Apple to review your application and approve it for production. Apple routinely rejects applications for being low quality, using unsupported APIs, and for not following design guidelines. Google on the other hand offers a streamlined release process that takes less than one hour, but doesn’t offer the first line of protection that Apple provides by not allowing apps with obvious flaws.
Mobile insights with AppDynamics
With AppDynamics for Mobile, you get complete visibility into the end user experience across mobile and web with the end user experience dashboard. Get a better understanding of your audience and where to focus development efforts with analytics on device, carriers, OS, and application versions:
Want to start monitoring your iOS or Android application today? Sign up for our beta program to get the full power of AppDynamics for your mobile apps.
In the next post in this series I will dive into monitoring a production mobile app and the various tools that are available. As always, please feel free to comment if you think I have missed something or if you have a request for content in an upcoming post.
Take five minutes to get complete visibility into the performance of your production applications with AppDynamics Pro today.Link to this post:
We’re happy to announce that AppDynamics Founder & CEO Jyoti Bansal was recently named one of the Top 101 Trust Leaders in Cloud & Hosting for 2014 by Total Product Marketing, a digital marketing and design agency. The list features prominent members of the cloud industry including Marc Benioff who have contributed to the cloud’s growth in recent years by developing technologies and providing services that their customers can trust with their data and their business.
Jyoti was selected as one of 2014’s top Trust Leaders for his work at AppDynamics, an application performance management (APM) provider that allows the owners of cloud or hybrid applications to easily diagnose and resolve performance issues in a production environment.
Learn more about why Jyoti was selected and see the full list of Trust Leaders on the Total Product Marketing site.
Link to this post:
If you’ve used a GPS product before you’re probably familiar with Garmin. They are the leading worldwide provider of navigation devices and technologies. In addition to their popular consumer GPS products, Garmin also offers devices for planes, boats, outdoor activities, fitness, and cars. Because end user experience is so important to Garmin and its customers, ensuring the speed of its web and mobile applications is one of its highest priorities.
This blog post summarizes the content of a case study that can be found by clicking here.
The reality in today’s IT driven world is that there are many different issues that need to be solved and therefore many different use cases for the products designed to help with these issues. Garmin has shown it’s industry leadership by taking advantage of AppDynamics to address many use cases with a single product. This shows a high level of maturity that most organizations strive to achieve.
“We never had anything that would give us a good deep dive into what was going on in the transactions inside our application – we had no historical data, and we had no insight into database calls, threads, etc.,” – Doug Strick
Real-Time and Historic Visibility in Production
You just can’t test everything before your application goes to production. Real users will find ways to break your application and/or make it slow to a crawl. When this happens you need historical data to compare against as well as real-time information to quickly remediate problems.
Garmin found that AppDynamics, an application performance management solution designed for high-volume production environments, best suited its needs. Garmin’s operations team feels comfortable leaving AppDynamics on in the production environment, allowing them to respond more quickly and collect historical data.
The following use cases are examples of how Garmin is attacking IT and business problems to better serve their company and their customers.
Use Case 1: Memory Monitoring
Before they started using AppDynamics, Garmin was unable to track application memory usage over time. Using AppDynamics they were able to identify a major memory issue impacting customers and ensure a fast and stable user experience. Click here to read the details in the case study.
Use Case 2: Automated Remediation Using Application Run Book Automation
AppDynamics’ Application Run Book Automation feature allows organizations to trigger certain tasks, such as restarting a JVM or running a script, with thresholds in AppDynamics. Garmin was able to use this feature very effectively one weekend while waiting for a fix from a third party. Click here to read the details in the case study.
Use Case 3: Code-level Diagnostics
“We knew we needed a tool that could constantly monitor our production environment, allowing us to collect historical data and trend performance over time,” said Strick. “Also, we needed something that would give us a better view of what was going on inside the application at the code level.” – Doug Strick
With AppDynamics, Garmin’s application teams have been able to rapidly resolve several issues originating in the application code. Click here to read the details in the case study.
Use Case 4: Bad Configuration in Production
Garmin also uses AppDynamics to verify that new code releases perform well. Because Garmin has an agile release schedule, new code releases are very frequent (every 2 weeks) and it’s important to ensure that each release goes smoothly. One common problem that occurs when deploying new code is that configurations for test are not updated for the production environment, resulting in performance issues. Click here to read the details in the case study.
Use Case 5: Real-time Business Metrics (RTBM)
IT metrics alone don’t tell the whole story. There can be business problems due to pricing issues our issues with external service providers that impact the business and will never show up as an IT metric. That’s why AppDynamics created RTBM and why Garmin adopted it so early on in their deployment of AppDynamics. With Real-Time Business Metrics, organizations like Garmin can collect, monitor, and visualize transaction payload data from their applications. Strick used this capability to measure the revenue flow in Garmin’s eCommerce application. Click here to read the details in the case study.
“AppDynamics has given us visibility into areas we never had visibility into before… Some issues that used to take several hours now take under 30 minutes to resolve, or a matter of seconds if we’re using Run Book Automation.” - Doug Strick
Garmin has shown how to get real value out of its APM tool of choice and you can do the same. Click here to try AppDynamics for free in your environment today.Link to this post: