What is OpenTelemetry?

Today, telemetry data plays a vital role in providing technologists with a complete picture of app behavior and performance. However, collecting such data from many different systems presents a significant challenge. Enter OpenTelemetry — a nascent project of the Cloud Native Computing Foundation (CNCF) and the result of a merger between the OpenTracing and OpenCensus projects.

In 2019, OpenTracing (a CNCF vendor-agnostic tracing project) and OpenCensus (a Google-led vendor-agnostic tracing and metrics library) merged to form OpenTelemetry with the goal of simplifying the telemetry ecosystem by providing a unified set of instrumentation libraries and specifications for observability telemetry. The result is a complete telemetry system suitable for monitoring both modern distributed services and more traditional environments. By offering a single set of APIs and libraries, it simplifies the collection and transfer of telemetry data used by technologists. 

As a vendor-agnostic service, OpenTelemetry is compatible with a growing list of leading OSS and commercial backends, including Jaeger and Zipkin.

Observability, or the ability to monitor a system’s performance and understand its internal states by virtue of the external data it produces, is generally achieved by utilizing four fundamental data groups:

• Metrics provide time-based numerical measurements on elements of the application ecosystems. For instance, there are an average of 255 transactions per second.
• Events track individual actions, producing a time-stamped inventory of data related to operations that are defined by the user.
• Logging collects application-generated structured or unstructured text added to help troubleshoot. There is much debate about the effectiveness of log data in a microservices context, as it is difficult to scale and often considered overly expensive.
• Traces provide an understanding of how requests flow through the system. This can be used to understand interacting parts and their behaviors. 

These data items can help provide technologists with a richer picture of application health and performance. The four categories are interconnected, meaning metrics can be used to locate misbehaving distributed traces, and logs associated with trace data can be used to find out exactly what's causing an issue, etc.

Understanding the OpenTelemetry ecosystem

The OpenTelemetry API is used by app developers to instrument their code. Library authors use it to integrate instrumentation within their libraries. The API is further divided into four parts:

Tracer API

The Tracer API enables the generation of spans. A span is a named, timed operation that represents a contiguous segment of work. Multiple spans are pieced together to create a trace.

Metric API

With the Metric API, developers can access different instruments like counters and observers.

Context API

The Context API can add valuable context information like W3C Trace Context. 

Lastly, there are also a set of semantic conventions included in the OpenTelemetry API that have rules for naming spans and attributes, as well as for associating errors with spans.

Key components for OpenTelemetry integration

For the implementation of the OpenTelemetry API, there's the OpenTelemetry SDK, with three parts that are comparable to the aforementioned APIs. OpenTelemetry provides one API and SDK per language, with the latter allowing for additional configuration like transaction sampling and request filtering.

The in-process Exporter is an important component, as this is what enables developers to choose which backend they want the telemetry to be sent to once translated to standard OTLP or proprietary formats. 

The OpenTelemetry Collector gathers the data, then processes and exports that data using proprietary or vendor-agnostic implementation.

Developers can easily obtain quick-start guides and documentation for OpenTelemetry APIs and SDKs to quickly ingest data with compatible languages like Javascript, Python, Go, Ruby, C#, and more.

The primary benefit of OpenTelemetry is that it provides a unified standard for creating and ingesting telemetry data, much like container orchestration standards set years ago by Kubernetes. OpenTelemetry provides a consistent collection mechanism and format, without locking technologists to a specific vendor to store and analyze this data.

Using metrics, events, logging, and tracing (MELT) to practice observability, OpenTelemetry is able to provide developers and site reliability engineers with a more complete impression of app performance than was previously provided by basic monitoring. 

Observability helps achieve business objectives

With observability, the goal is to enable developers, security analysts, and product managers to better understand and fix problems in the app (reliability, security, etc.) and supporting infrastructure and services that could have a negative impact on the business. 

The telemetry data can be used to ascertain if the system is operating as it should be or to reveal the threat of a potential intrusion or attempted DDoS attack. 

Having this watchful eye over the system at all times means that issues can be flagged instantly and addressed before any disruption in service is caused.

OpenTelemetry simplifies observability integration

Before OpenTelemetry, there wasn't a centralized and flexible observability solution. IT professionals had to stick with specific vendors and backends.  

OpenTelemetry aims to address this challenge by providing a standardized format for the collection of telemetry data in order to improve observability, enabling technologists to effectively identify and resolve availability and performance issues.

With language-specific integrations for popular frameworks, it can automatically capture relevant traces and metrics, in addition to handling context propagation. 

OpenTelemetry's automatic instrumentation agents can also collect telemetry from some apps without requiring any code changes. All of the collected data can then be sent to a backend of the developer's choice with no specific vendor requirements.

The industry stands to benefit from the consolidation of observability ecosystems. Vendor-neutral open standards are vital for increased adoption, and OpenTelemetry is going to make integrating observability more viable. 

With many parts still in the beta phase, the OpenTelemetry project has only just begun. At the moment, spans and traces are the most thoroughly developed components. However, the project also currently covers metrics with “manual code change” for some languages, and might eventually integrate logging.

The team is working towards the goal of providing a generally available, production-quality release in the coming year. This will result in the stabilization of core components, in addition to the integration of automated instrumentation through agents, additional language compatibility and improved metrics capabilities — all of which are in active development. The project is, and will likely remain, focused on getting tech health info from systems in a standard way. 

Backward compatibility is a key feature of the project, allowing OpenTelemetry to be implemented with applications that use either the OpenTracing or OpenCensus systems, and providing developers with an easy path to make the transition. OpenTelemetry is one of CNCF’s biggest open source projects, with growing developer support for the ecosystem. 

With ample opportunity to get involved in the project through GitHub, OpenTelemetry is well poised to dominate the cloud native telemetry landscape.

While integrating observability into an IT system isn't a goal on its own, it should be viewed as a vital component in the quest to achieve business objectives. Helping to relate how infrastructure (whether compute, storage, or network) and supporting services that influence an application’s health is the first step towards positively impacting user experiences and business outcomes.