Cisco Cloud Observability on AWS: Deploying is easy with the AppDynamics add-on for Amazon EKS Blueprints with Terraform

In this blog, I’ll show you how to deploy the AppDynamics Kubernetes and App Service Monitoring solution for cloud native application observability using Helm charts and the Amazon EKS Blueprints for Terraform module. Now, you can do it in just minutes.

What is Cisco Cloud Observability?

It is Cisco’s Software-as-a-Service (SaaS) product for observing, securing and optimizing large, managed Kubernetes deployments. It provides real-time observability across your entire technology stack: applications, software-defined compute, storage, services, network and other infrastructure, through the collection and correlation of metrics, events, logs and traces (MELT).

This illustration shows a high-level overview of how Cisco Cloud Observability works:

The Kubernetes and App Service Monitoring solution provides visibility into your Kubernetes infrastructure and services for application performance monitoring (APM) and allows you to gain visibility into key Kubernetes metrics from various entities, such as clusters, namespaces, workloads, pods and ingress controllers.

Cisco AppDynamics provides Helm charts that deploy the following components:

The Kubernetes and App Service Monitoring solution consists of a set of Operators and Collectors deployed via Helm charts as defined in the following tables:

Operators

Operator	Usage
AppDynamics Operator OpenTelemetry Operator for Kubernetes	These two operators are responsible for the life-cycle management of appdynamics-collectors and allow you to inject and configure OpenTelemetry auto-instrumentation libraries. Currently, Java, NodeJS and Python are supported for auto-injection.
These two operators are responsible for the life-cycle management of appdynamics-collectors and allow you to inject and configure OpenTelemetry auto-instrumentation libraries. Currently, Java, NodeJS and Python are supported for auto-injection.	This operator is responsible for the lifecycle management of Kubernetes-related agents.

Collectors

Collector	Usage
Cluster Collector	Collects Kubernetes data.
Infrastructure Collector	Collects server and container data, known as Host Monitoring.
Log Collector	Collects logs.
Cisco AppDynamics OpenTelemetry Collector	Receives OTLP data from the Infrastructure Collector and Log Collector.

What is the Amazon EKS Blueprints for Terraform Project?

The Amazon EKS Blueprints for Terraform project is an open-source framework implemented in Terraform that enables platform administrators to configure and manage their Amazon EKS clusters with ease. It is maintained by AWS solution architects and the EKS Blueprints community.

By providing a standard framework for add-ons, AWS allows companies to consolidate tools and best practices for securing, scaling, monitoring and operating containerized infrastructure into a central platform that can then be used by developers across an enterprise.

With the EKS Blueprints add-on framework, deploying the AppDynamics Kubernetes and App Service Monitoring solution becomes a simple task that can be accomplished in a matter of minutes.

Get started

Before installing the Kubernetes and App Service Monitoring solution into your EKS cluster, ensure that you meet the following requirements:

• Your account is set up on Cisco Cloud Observability. See Account Administration.
• You are connected to the EKS cluster that you want to monitor.
• You have administrator privileges on the monitored cluster to run the Helm chart commands.

Deployment tools You’ll need

To deploy the AppDynamics add-on for Amazon EKS Blueprints, you will first need to install the following set of command-line utilities on either your local machine or a suitable VM:

1. AWS Command Line Interface (v2)
   The AWS Command Line Interface (AWS CLI) is an open source tool that enables you to interact with AWS services using commands in your command-line shell. With minimal configuration, the AWS CLI enables you to start running commands that implement functionality equivalent to that provided by the browser-based AWS Management Console from the command prompt in your terminal program. The AWS CLI version 2 is the most recent major version of the AWS CLI and supports all the latest features. Please see the AWS CLI installation guide to get started.
2. Terraform 
   Terraform is a tool for building, changing and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions. The infrastructure Terraform can manage low-level components such as compute instances, storage and networking, as well as high-level components such as DNS entries, SaaS features etc. Terraform is used to automate the deployment of the AppDynamics add-on for Amazon EKS Blueprints. Installation guides for your platform can be found here.
3. Helm 
   Helm is the package manager for Kubernetes that streamlines installing and managing Kubernetes applications. Helm is the best way to find, share and use software built for Kubernetes. The Helm platform binaries can be downloaded from GitHub.
4. Kubectl 
   Kubectl is a command-line tool used for communicating with the Kubernetes API server to deploy and manage applications. Using Kubectl, you can inspect cluster resources; create, delete and update components; look at your new cluster; and bring up example apps. Instructions for Installing or updating Kubectl can be found in the Amazon EKS User Guide.NOTE: You must use a Kubectl version that is within one minor version difference of your Amazon EKS cluster control plane. For example, a 1.27 Kubectl client works with Kubernetes 1.26, 1.27 and 1.28 clusters.
5. yq 
   yq is a lightweight and portable command-line YAML processor. yq uses jq-like syntax but works with YAML files as well as JSON, XML, properties, CSV and TSV. It doesn’t yet support everything jq does — but it does support the most common operations and functions, and more are being added continuously. The yq binaries can be downloaded from GitHub.

With these pre-requisite tools installed, you’re ready to deploy.

Get the Code

First, you’ll need to clone the AppDynamics Add-On for Amazon EKS Blueprints project from the Cisco DevNet repository on GitHub:

$ cd ~ 
$ git clone https://github.com/CiscoDevNet/appdynamics-eks-blueprints-addon.git 
$ cd appdynamics-eks-blueprints-addon

Generate and download Kubernetes Operators and Collectors Files

Cisco Cloud Observability provides Helm charts for monitoring Kubernetes and app services. Follow the steps below to install and configure them:

1. 1. From your Tenant UI, navigate to “Configure” (the gear icon) and click to expand the panel.
   2. Click on “Kubernetes and APM”.

 

From the Kubernetes and App Services page:

3. Enter your “Credential set name” in the text field.

4. Enter your “Kubernetes cluster name” in the text field.

NOTE: By convention, it is common to use the actual name of the EKS cluster to make it easy to identify your deployment. It is also typical to use the same text in both fields. For example, if an EKS cluster was named “my-eks-test-cluster,” you would copy that name to both fields.

5. Click on the “Generate file” button.

From the Kubernetes and App Services page:

6. Click “View documentation” to view the detailed documentation. [Optional]

7. Click “Download operators-values.yaml file”.

8. Click “Download collectors-values.yaml file”.

9. Click “Done”.

10. Copy the “operators-values.yaml” and “collectors-values.yaml” files to the add-on project home directory for Terraform. (Please adjust as needed if you originally downloaded these files to a different directory than ~/Downloads.)

$ cd ~/appdynamics-eks-blueprints-addon/examples/addon/ 

$ cp ~/Downloads/operators-values.yaml . 

$ cp ~/Downloads/collectors-values.yaml .

Verify your connection to AWS and the EKS cluster

Now let’s verify your connection to your AWS account and the EKS cluster.

1. Set the AWS environment variables:

$ export AWS_REGION=<your_aws_region>
$ export AWS_EKS_CLUSTER=<your_aws_eks_cluster_name>

2. Invoke the Security Token Service (STS) to verify access to your AWS account via the AWS CLI:

$ aws sts get-caller-identity

 

The output should display something like:

{
"UserId": "ABCDEFGHIJKLMNOPQRSTU",
"Account": "012345678901",
"Arn": "arn:aws:iam::012345678901:user/some.user"
}

 

3. Retrieve the AWS EKS ‘kubeconfig’ and verify access to the EKS cluster:

$ aws eks --region $AWS_REGION update-kubeconfig --name $AWS_EKS_CLUSTER
$ kubectl config current-context
$ kubectl get nodes -o wide

Here you should be able to see the updated context, the current context, the names of the EKS cluster nodes; as well as the status, Kubernetes version, internal and external IP addresses and OS image as shown in the image below:

Prepare the Terraform configuration for your environment

Before we can execute the Teraform commands and deploy AppDynamics Helm charts, you need to override some custom Terraform variables related to your Cisco Cloud Observability tenant and EKS cluster by generating a custom “terraform.tfvars” file. A “terraform.tfvars” file is used to override default Terraform variables at runtime.

To make this as easy as possible, we created a script to extract key values from the downloaded “operators-values.yaml” and “collectors-values.yaml”. These key values contain information specific to your tenant, such as operators and collectors’ endpoints, client secret, tenant ID and token URL.

Let’s perform a quick check to ensure that all the files are in the correct place.

1. From the add-on project home directory for Terraform, verify that the “operators-values.yaml” and “collectors-values.yaml” files are in the current directory:

$ cd ~/appdynamics-eks-blueprints-addon/examples/addon/ 

$ ls -alF

Verify the downloaded files are in the current directory as shown:

2. From the current directory, run the following script:

$ ../../bin/extract_cnao_config_values_for_terraform.sh

The output should display something like:

Begin processing Helm Chart files... 
Extracting CNAO configuration values... 
Substituting EKS Cluster name variable... 
Substituting Helm Chart variables... 
Removing temporary backup file... 
CNAO configuration values extraction complete.

3. Examine the “terraform.tfvars” file:

$ cat terraform.tfvars

You should see that the EKS Cluster name and Helm chart variables are now uncommented and populated with the correct data.

Deploy the AppDynamics add-on for Amazon EKS Blueprints

With the Terraform variables configured; you are now ready to deploy the add-on.

From the current directory, run the following Terraform Life Cycle commands in order:

$ terraform --version 

$ terraform init 

$ terraform validate 

$ terraform plan -out terraform-addon.tfplan 

$ terraform apply terraform-addon.tfplan

In the Terraform output from the “apply” operation, verify the following:

1: The “appdynamics-cnao-operators” were created.
2: The “appdynamics-cnao-collectors” were created.
3: The Terraform Apply operation was completed successfully.

Observe your EKS cluster

With the AppDynamics monitoring solution deployed, let’s verify that we are getting observability data from our EKS cluster by taking a quick look within the Cisco Cloud Observability Tenant UI.

1. From your Tenant UI, scroll down the page until you see the “Kubernetes” section and click on the “Clusters” link.

2. In the “Filter View” text box, enter the following string: substituting the name of your EKS cluster.

HINT: As you type the UI will offer intelligent auto-completion text that will make it easy to complete the filter statement.

EntityStatus = 'active' && attributes(k8s.cluster.name) = 'AppD-EKS-Addon-e0atm-EKS'

3. Click “Apply”.

4. With this list of EKS cluster now filtered, click on your cluster’s link.

In the “Relations” panel on the left side of the page, you can explore the contextual relationships between the various components of your environment, such as “Workloads”, “Pods”, “Containers”, “Hosts” etc.

In the center panel, you can observe “Health Violations”, “Logs”, “Pod Status Events” and even “Cost Overview” information (if configured).

In the “Health and Alerting” panel, you can view additional properties of the selected entity or view “Anomaly Detection” details and “Health Rules” associated with the entity in focus.

Congratulations, you’ve completed the exercise. We hope you’ve gained some valuable insight along with practical skills today!

Remove the AppDynamics add-on for Amazon EKS Blueprints

When the time comes, you’ll want to remove the AppDynamics add-on. Terraform simplifies this task with the “destroy” command.

From the current directory, run the following Terraform Life Cycle commands in order:

$ cd ~/appdynamics-eks-blueprints-addon/examples/addon/ 
$ terraform destroy -auto-approve

Conclusion

Cisco Cloud Observability enables correlation and enrichment of telemetry data across your entire technology landscape — mapping all dependencies, relationships and ephemeral services — tying them directly to business impact. And the AppDynamics add-on for Amazon EKS Blueprints makes deployment with Terraform a standardized and repeatable process for observing your cloud native applications and infrastructure.

To learn more, visit Cisco Cloud Observability, powered by the Cisco Observability Platform.

Ed Barberis

Ed Barberis is an Advisory Sales Engineer with Cisco AppDynamics. He is an accomplished architect, technologist and DevOps specialist focused on the design, development and delivery of cloud native infrastructure and applications. Based in Detroit, Ed enjoys cycling and playing the trumpet in his spare time.