Monitoring Docker Container Performance: Stats & Metrics

Metrics are critical to monitoring performance, troubleshooting incidents, and managing resources. And while data collection is important, the true value of metrics comes from the ability to analyze that information for actionable insights. The Docker stats command displays real-time reporting of the following resource usage statistics: 

CPU

The CPU metric represents what percentage of the total host capacity is being utilized. Consequently, a container's stat command could be at max capacity while only showing a 50% CPU utilization rate if there were two containers that were allocated equal shares by Docker.

Memory 

The MEM USAGE/LIMIT MEM and MEM % metrics report the memory stats of the container, including usage, the memory limit, and the percentage of container utilization. The memory usage limit represents the memory limit of the host machine, unless it is configured for the specific container.

BLOCK I/O

The BLOCK I/O section displays the total bytes written and read to the container file system, which is particularly useful to locate containers that are writing data, which could indicate a data persistence problem.

NET I/O

The NET I/O column communicates how many bytes are received (RX) and (TX) by that Docker container, and represents network traffic.

Docker offers three different options to obtain this data, with a different level of information available for each.

Pseudo-files

Cgroup pseudo files offer detailed information that can facilitate an in-depth analysis of performance and happens to be the fastest way to read Docker metrics, making it the most efficient option if you have many containers to monitor. One of the significant benefits of cgroups pseudo files is that it's easy to write tools around them since root access is not required. 

You can find cgroups pseudo files under /sys/fs/cgroup in Linux, or under /cgroup, depending on your system. You can set the $CONTAINER_ID environment variable to the long ID of the container being monitored by replacing CONTAINER_NAME with the name of your container in the following command: 

export CONTAINER_ID=$(docker inspect --format="{{.Id}}" CONTAINER_NAME)

You can also run docker ps to set the $CONTAINER_ID to avoid copying the long ID.

Stats command 

The Docker stats command provides an efficient yet effective snapshot of resource-usage and is ideal on a smaller scale, such as a limited number of containers on a single host. The Docker stats command display shows CPU stats, memory metrics, block I/O, and network IO metrics for all running containers, but you can limit the data by specifying the container names or IDs you're interested in monitoring.

API 

The API metric returns a detailed live stream of data, which includes not only CPU usage, memory, and I/O data, but also JSON object information and metrics about the container. It does this by having the Docker daemon enter unix:///var/run/docker.sock, which allows local connections by the root user on the default socket, but can be configured to a different port or socket. 

Access API information as follows for a JSON stream:

$ curl -v --unix-socket /var/run/docker.sock \  http://localhost/containers/CONTAINER_ID/stats

Substitute CONTAINER_ID with the actual ID of the container being inspected.

Although Docker's built-in tools make it possible to manually check metrics, an integrated monitoring solution can consolidate all of your system's data, including Docker metrics, into a single view to improve visibility, alerting, and correlation. 

AppDynamics’ Docker Monitoring Extension simplifies the task of monitoring the health and performance of your system by leveraging AppDynamics functionality like dynamic baselining, health rules, policies, and actions for Docker metrics, while also correlating them with the metrics that exist in the Docker environment. Create an out-of-the-box custom dashboard using key Docker metrics for an easy and comprehensive experience with monitoring the Docker environment.