A Day in the Connected Life of the Internet of Things

Applications have become the lifeblood of today’s businesses. Gone are the days when you had to pay bills by mail, or walk inside a bank branch to transfer money. And hailing a rideshare requires just a few taps or swipes on your mobile phone or smart watch.

My Experience with IoT

On a recent trip to San Francisco, I experienced firsthand how the Internet of Things is impacting our daily lives in ways most people don’t imagine.

My day started with a short walk to the AppDynamics office. On the way I stopped at a famous restaurant chain to grab breakfast. I soon realised I couldn’t get my order quickly, as there was a long queue of people waiting to be served.

Not a great start to my day, I thought.

Then I noticed a timesaver—a kiosk where I could place my order. I used it to bypass the queue, and within a few minutes I had received my croissant and smoothie. Nice!

This is just a simple example of how connected devices in retail can enhance the customer experience. Of course, the global potential of IoT is far greater.

IoT is Everywhere

At home we rely on virtual assistants to interact with a multitude of connected devices, such as smart bulbs, robot vacuums, security cameras, door locks, thermostats, electric roof windows, and blinds and shutters to create a healthier indoor environment. But the smart home is just one of a vast variety of IoT use cases. Take kiosks, for instance. We use airport kiosks to check in and drop off our luggage. Movie theatre kiosks allow us to easily pick up tickets we’ve bought online.

The list of connected devices making our lives easier grows longer each day. Popular and emerging IoT use cases include:

  • Smart cities using IoT sensors to collect data, combined with advanced monitoring and analytics to enhance government services
  • Smart grids—utility companies leveraging IoT technology to improve power quality and reliability issues, as well as customer interaction
  • Industrial IoT—bringing automation to the factory floor; leveraging artificial intelligence and machine learning (AI/ML), cloud computing, and self-optimizing equipment and facilities
  • Connected cars sharing data with other devices inside and outside the vehicle—including other vehicles on the road
  • Connected health with telemedicine and remote assistance
  • Smart supply chains delivering IoT-driven operational efficiencies, including improved asset tracking, forecasting, and fleet and inventory management
  • Smart farming to enable precision agriculture
  • Smart retail—better inventory management, predictive equipment maintenance, customer interaction, and more

Almost 31 Billion Devices by Next Year

The number of connected devices worldwide has soared in recent years—from 15.4 billion in 2015 to 26.6 billion this year. In 2020, that number is expected to reach nearly 31 billion. Global spending on the Internet of Things will reach $745 billion in 2019, a 15.4% increase over $646 billion in 2018, IDC estimates, with IoT spending topping $1 trillion in 2022.

But for IT operations, the Internet of Things brings a new set of concerns. Transmitting massive volumes of data collected from an ever-growing number of connected devices—and then storing and processing this data in the cloud—can lead to high bandwidth usage and increased costs. And while edge computing, which occurs at or near the source of the data rather than back at the datacenter, can help mitigate these issues, businesses must gain visibility into applications running on IoT devices, regardless of their physical location.

A related concern, according to a recent Gartner report, is that infrastructure and operations teams may be unable to use tools designed to monitor traditional applications, networks and systems in their IoT ecosystems.

New Tools for IoT

AppDynamics IoT monitoring extends visibility to connected device applications, delivering performance diagnostics and usage analytics that enable IT teams to better understand and resolve IoT performance issues.

By providing insights into device and user behaviour, AppDynamics not only helps validate business outcomes, but also enables you to understand how application performance impacts your business. It provides end-to-end visibility—from connected devices and applications to backend services in the private, public or hybrid cloud, and even in serverless environments.

AppDynamics IoT monitoring allows business to determine which KPIs to measure in relation to device type and hardware/firmware version, and lets developers decide when, where and how to collect data from IoT devices, including those on the edge.

IoT Monitoring Use Cases

Here are ways that AppDynamics can help businesses with kiosk and point-of-sale (POS) applications:

Edge Performance Management
Gauge device health by monitoring errors, network requests, and usage—all while extending and correlating business transactions to edge devices and ensuring a smooth registration and hand-off.

Release Validation
Release newer versions of software and compare business KPIs to ensure the edge is always up to date.

Device and User Segmentation
Gain visibility into specific devices, platforms, data and users to ensure a consistent user experience across your entire ecosystem.

AppDynamics also monitors many business KPIs, including:

  • Conversion rate
  • Average transaction value
  • Device uptime
  • Sell through
  • Sales count

 

The AppDynamics Device Dashboard (below) allows you to monitor important KPIs, device status, and custom device application information. You can use filters to drill down to view metrics for a specific device.

AppDynamics also monitors other important information, such as:

  • Device ID
  • Firmware version
  • Software version
  • Event type
  • Errors by type

 

The Network Requests Dashboard lets you view outgoing network requests and details, and helps you analyse network request metrics.

The Errors Dashboard enables you to view crashes, exceptions, and custom errors, both fatal and non-fatal. You can view details for a specific error event and download the stack trace for debugging.

Today, POS and kiosks are the first line of revenue for many companies in retail, travel, financial and other industries. Learn how AppDynamics IoT monitoring can improve your operational efficiency and enhance your customers’ experience!

Internet of Things: From Buzz to Billions

Initially introduced as a solution for enterprises looking to solve issues with industrial grade switches and gateways, the IoT is rapidly becoming a vast, global system of interrelated devices designed to address economic, environmental and societal challenges.

From smart water solutions that leverage performance sensors to measure water use and quality, to devices that monitor robotic arms on manufacturing floors, the IoT is everywhere. Edge computing and IoT are dictating how enterprises, government agencies, hospitals, plant floors and even farms are doing business.

Accenture forecasts the industrial IoT market will top $123 billion by 2021, and McKinsey predicts the IoT ecosystem will eventually be worth $581 billion for information and communication technology (ICT)-based spending.

Since its introduction, one critical component of the IoT has changed: products no longer exist in isolation. New technologies enable products and services to interact and learn from one another, bringing added complexity to backend infrastructures. As a result, enterprises are looking for a tool (not a toolbox) to help monitor, manage and monetize their IoT investment.

According to a recent consumer survey by CSG, end users are embracing connected devices and IoT technology in their daily lives. The majority of respondents (60%) said the IoT’s greatest appeal is its potential to make their lives easier. And with the rapid growth of the IoT market, new applications are reaching home, office and city environments, making it more important than ever for businesses to ensure optimal application performance.

But there is a strong need for help in solving critical application issues on the edge—problems of great concern to end users. A 2018 survey of U.S. consumers by security firm F-Secure found that 66% of respondents have delayed purchasing IoT home devices due to privacy or security concerns.

Indeed, the IoT revolution is already underway. The question is, are you ready?

Preparing for IoT

The goal for successful IoT deployment is to create the ultimate customer experience for end users. The AppDynamics purpose-built IoT SDK puts enterprises in the driver’s seat. Developers decide when, where and how often data is sent, and the business side determines which KPIs to measure in relation to device type, hardware and firmware version. Sometimes an agent isn’t the answer; smart devices and sensors have CPU, memory and consumption limitations, so being able to control data behavior is a great strength of the AppDynamics IoT SDK.

Not sure if your customers or prospects have an IoT play? Remember, just like every business has become a software business, every business will become an IoT business. Traffic lights, parking meters, irrigation systems, smart buildings, connected banks, factories, railways and bathrooms…believe me, I could go on and on.

Learn how AppDynamics IoT monitoring can help you secure your share of the growing Internet of Things market!

Big Data Monitoring

The term “Big Data” is quite possibly one of the most difficult IT-related terms to pin down ever. There are so many potential types of, and applications for Big Data that it can be a bit daunting to consider all of the possibilities. Thankfully, for IT operations staff, Big Data is mostly a bunch of new technologies that are being used together to solve some sort of business problem. In this blog post I’m going to focus on what IT Operations teams need to know about big data technology and support.

Big Data Repositories

At the heart of any big data architecture is going to be some sort of NoSQL data repository. If you’re not very familiar with the various types of NoSQL databases that are out there today I recommend reading this article on the MongoDB website. These repositories are designed to run in a distributed/clustered manner so they they can process incoming queries as fast as possible on extremely large data sets.

MongoDB Request Diagram

Source: MongoDB

An important concept to understand when discussing big data repositories is the concept of sharding. Sharding is when you take a large database and break it down into smaller sets of data which are distributed across server instances. This is done to improve performance as your database can be highly distributed and the amount of data to query is less than the same database without sharding. It also allows you to keep scaling horizontally and that is usually much easier than having to scale vertically. If you want more details on sharding you can reference this Wikipedia page.

Application Performance Considerations

Monitoring the performance of big data repositories is just as important as monitoring the performance of any other type of database. Applications that want to use the data stored in these repositories will submit queries in much the same way as traditional applications querying relational databases like Oracle, SQL Server, Sybase, DB2, MySQL, PostgreSQL, etc… Let’s take a look at more information from the MongoDB website. In their documentation there is a section on monitoring MongoDB that states “Monitoring is a critical component of all database administration.” This is a simple statement that is overlooked all too often when deploying new technology in most organizations. Monitoring is usually only considered once major problems start to crop up and by that time there has already been impact to the users and the business.

RedisDashboard

Dashboard showing Redis key metrics.

One thing that we can’t forget is just how important it is to monitor not only the big data repository, but to also monitor the applications that are querying the repository. After all, those applications are the direct clients that could be responsible for creating a performance issue and that certainly rely on the repository to perform well when queried. The application viewpoint is where you will first discover if there is a problem with the data repository that is actually impacting the performance and/or functionality of the app itself.

Monitoring Examples

So now that we have built a quick foundation of big data knowledge, how do we monitor them in the real world?

End to end flow – As we already discussed, you need to understand if your big data applications are being impacted by the performance of your big data repositories. You do that by tracking all of the application transactions across all of the application tiers and analyzing their response times. Given this information it’s easy to identify exactly which components are experiencing problems at any given time.

FS Transaction View

Code level details – When you’ve identified that there is a performance problem in your big data application you need to understand what portion of the code is responsible for the problems. The only way to do this is by using a tool that provides deep code diagnostics and is capable of showing you the call stack of your problematic transactions.

Cassandra_Call_Stack

Back end processing – Tracing transactions from the end user, through the application tier, and into the backend repository is required to properly identify and isolate performance problems. Identification of poor performing backend tiers (big data repositories, relational databases, etc…) is easy if you have the proper tools in place to provide the view of your transactions.

Backend_Detection

AppDynamics detects and measures the response time of all backend calls.

Big data metrics – Each big data technology has it’s own set of relevant KPIs just like any other technology used in the enterprise. The important part is to understand what is normal behavior for each metric while performance is good and then identify when KPIs are deviating from normal. This combined with the end to end transaction tracking will tell you if there is a problem, where the problem is, and possibly the root cause. AppDynamics currently has monitoring extensions for HBase, MongoDB, Redis, Hadoop, Cassandra, CouchBase, and CouchDB. You can find all AppDynamics platform extensions by clicking here.

HadoopDashboard

Hadoop KPI Dashboard 1

HadoopDashboard2

Hadoop KPI Dashboard 2

Big data deep dive – Sometimes KPIs aren’t enough to help solve your big data performance issues. That’s when you need to pull out the big guns and use a deep dive tool to assist with troubleshooting. Deep dive tools will be very detailed and very specific to the big data repository type that you are using/monitoring. In the screen shots below you can see details of AppDynamics monitoring for MongoDB.

MongoDB Monitoring 1

 MongoDB Monitoring 2

MongoDB Monitoring 3

MongoDB Monitoring 4

If your company is using big data technology, it’s IT operations’ responsibility to deploy and support a cohesive performance monitoring strategy for the inevitable performance degradation that will cause business impact. See what AppDynamics has to offer by signing up for our free trial today.

What’s up with the Network and Storage teams?

yoda-talk-to-handA few weeks ago I was presenting at CMG Performance and Capacity 2013 and during my presentation we (myself and a few audience members) got slightly side-tracked. Our conversation somehow took a turn and became a question of why it was so hard to get performance data from the network and storage teams. Audience members were asking me why, when they requested this type of data, they were typically stonewalled by these organizations.

I didn’t have a good answer for this question and in fact I have run into the same problem. Back when I was working in the Financial Services sector I was part of a team that was building a master dashboard that collected data from a bunch of underlying tools and displayed it in a drill-down dashboard format. It was, and still is, a great example of how to provide value to your business by bringing together the most relevant bits of data from your tooling ecosystem.

This master dashboard was focused on applications and included as many of the components involved in delivering each application as possible. Web servers, application severs, middleware, databases, OS metrics, business metrics, etc… were all included and the key performance indicators (KPIs) for each component were available within the dashboard. The entire premise of this master dashboard relied upon getting access to the tools that collected the data via API or through database queries.

The only problems that our group faced in getting access to the data we needed was with the network and storage teams. Why was that? Was it because these teams did not have the data we needed? Was it because these teams did not want anyone to see when they were experiencing issues? Was it for some other reason?

I know the network team had the data we required because they eventually agreed to provide the KPIs we had asked for. This was great, but the process was very painful and took way longer than it should have. Still, we eventually got access to the data. The big problem is that we never got access to the storage data. To this day I still don’t know why we were blocked at every turn but I’m hoping that some of the readers of this blog can share their insight.

Back in the day, when my team was chasing the storage team for access to their monitoring data, there weren’t really any tools that we could find for performance monitoring of storage arrays besides the tools that came with the arrays. These days I would have been able to get the data I needed for NetApp storage by using AppDynamics for Databases (which includes NetApp performance monitoring capabilities). You can read more about it by clicking here.

Have you been stonewalled by the network or storage or some other team? Did you ever get what you were after? Based upon my experiences talking with a lot of different folks at different organizations this seems to be a significant problem today. Are you on a network or storage team? Does your team freely share the data they have? Please share your experience, insight, or questions in the comments below. Just to clarify, I hold no ill will against any of you network or storage professionals out there. I’d just like to hear some opinions and gain some perspective.