AWS CloudWatch - Monitoring Extension

NOTE 

This monolithic extension is deprecated for performance reasons and is broken down into multiple individual extensions . Please use the individual AWS Namespace extensions.

Use Case 

The AWS CloudWatch custom monitor captures statistics from Amazon CloudWatch and displays them in the AppDynamics Metric Browser.

This extension works only with the standalone machine agent.

 

Metrics are categorized under the AWS Product Namespaces: 

AWS ProductNamespace
Auto ScalingAWS/AutoScaling
BillingAWS/Billng
Amazon DynamoDBAWS/DynamoDB
Amazon ElastiCacheAWS/ElastiCache
Amazon Elastic Block StoreAWS/EBS
Amazon Elastic Compute CloudAWS/EC2
Elastic Load BalancingAWS/ELB
Amazon Elastic MapReduceAWS/ElasticMapReduce
AWS OpsWorksAWS/OpsWorks
Amazon RedshiftAWS/Redshift
Amazon Relational Database ServiceAWS/RDS
Amazon Route 53AWS/Route53
Amazon Simple Notification ServiceAWS/SNS
Amazon Simple Queue ServiceAWS/SQS
AWS Storage GatewayAWS/StorageGateway

 

Specific metrics under each of these namespaces can be found at this link http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/CW_Support_For_AWS.html

 

Installation 

  1. Download the attached CloudWatchMonitor.zip file.
  2. Unzip the CloudWatchMonitor.zip file into the <machine-agent-home-dir>/monitors directory.
  3. In the newly created "CloudWatchMonitor" directory, edit the "AWSConfigurations.xml" file configuring the parameters specified below.
  4. Restart the machine agent.
  5. In the AppDynamics Metric Browser, look for: Application Infrastructure Performance | Amazon Cloud Watch. 

Configuration 

In the AWSConfigurations.xml, the following can be configured: 

  1. The AWS account credentials (i.e. the access key and the secret key). If you wish to encrypt the credentials, please refer to "Password Encryption Support" section.
  2. ProxyParams if a proxy is used to connect to AWS
  3. The supported AWS namespaces that you can retrieve metrics for (you can enable or disable metrics for specific namespaces). You can also add your custom namespace if required.
  4. Regions (enable regions to monitor the running AWS Products in the corresponding region)
  5. Use of instance name in Metrics. Default value is false. Note, this is only applicable for AWS/EC2 namespace.
  6. The list of disabled metrics associated with their corresponding AWS namespaces
  7. The list of metrics and associated metric type you wish to retrieve. Defaults to 'Ave' if not specified. Allowed metric types: ave, max, min, sum, samplecount
  8. The max number of retry attempts for failed retryable requests (ex: 5xx error responses from a service) or throttling errors. Allowed values: 0 for disabled, up to max value of 3

List of CloudWatch Regions can be found here http://docs.aws.amazon.com/general/latest/gr/rande.html#cw_region 

This is a sample AWSConfigurations.xml file:

<?xml version="1.0"?>
<Configurations>
    <AWSCredentials>
        <AccessKey>XXXXXXXXXXXXXXXXXXXX</AccessKey>
        <SecretKey>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</SecretKey>
     </AWSCredentials>

<ProxyParams>
<Host></Host>
<Port></Port>
<UserName></UserName>
<Password></Password>
</ProxyParams> <!--Individual namespaces can be disabled by simply commenting them out --> <SupportedNamespaces> <SupportedNamespace>AWS/EC2</SupportedNamespace>
<SupportedNamespace>AWS/AutoScaling</SupportedNamespace>
<SupportedNamespace>AWS/EBS</SupportedNamespace>
<SupportedNamespace>AWS/ELB</SupportedNamespace>
<SupportedNamespace>AWS/ElastiCache</SupportedNamespace>
<SupportedNamespace>AWS/Redshift</SupportedNamespace>
<SupportedNamespace>AWS/DynamoDB</SupportedNamespace>
<SupportedNamespace>AWS/RDS</SupportedNamespace>
<SupportedNamespace>AWS/Route53</SupportedNamespace>
<SupportedNamespace>AWS/SQS</SupportedNamespace>
<SupportedNamespace>AWS/ElasticMapReduce</SupportedNamespace
<SupportedNamespace>AWS/StorageGateway</SupportedNamespace>
<SupportedNamespace>AWS/OpsWorks</SupportedNamespace>
<SupportedNamespace>AWS/SNS</SupportedNamespace>
<SupportedNamespace>AWS/Billing</SupportedNamespace>

<!-- Custom Namespace -->
<SupportedNamespace>MyCustomNamespace</SupportedNamespace>
</SupportedNamespaces>

<EC2InstanceName>
<UseNameInMetrics>true</UseNameInMetrics>
<TagFilterName>tag-key</TagFilterName>
<TagKey>Name</TagKey>
</EC2InstanceName>
<Regions> <Region>us-east-1</Region>
<Region>us-west-2</Region> <Region>us-west-1</Region> <Region>eu-west-1</Region>
<Region>eu-central-1</Region> <Region>ap-southeast-1</Region> <Region>ap-southeast-2</Region> <Region>ap-northeast-1</Region> <Region>sa-east-1</Region> </Regions>

<DisabledMetrics>
<Metric namespace="AWS/EC2" metricName="CPUUtilization"/>
<Metric namespace="AWS/EC2" metricName="Some Metric"/>
</DisabledMetrics>

<MetricTypes>
<Metric namespace="AWS/ELB" metricName="RequestCount" metricType="sum"/>
</MetricTypes>

<MaxErrorRetrySize>3</MaxErrorRetrySize>
</Configurations>

 

The monitor.xml contains two parameters, 1.Path to the AWSConfigurations.xml file  2. Custom metric-prefix path 

Here is the monitor.xml file: 

<monitor>
    <name>AmazonMonitor</name>
    <type>managed</type>
    <description>Amazon cloud watch monitor</description>
    <monitor-configuration></monitor-configuration>
    <monitor-run-task>
       <execution-style>periodic</execution-style>
       <execution-frequency-in-seconds>60</execution-frequency-in-seconds>
       <name>Amazon Cloud Watch Monitor Run Task</name>
       <display-name>Cloud Watch Monitor Task</display-name>
       <description>Cloud Watch Monitor Task</description>
       <type>java</type>
       <execution-timeout-in-secs>60</execution-timeout-in-secs>
       <java-task>
<classpath>aws-cloudwatch-monitoring-extension.jar</classpath>
<impl-class>com.appdynamics.extensions.cloudwatch.AmazonCloudWatchMonitor</impl-class>
</java-task>
<task-arguments>
<argument name="configurations" is-required="true" default-value="monitors/CloudWatchMonitor/conf/AWSConfigurations.xml"/>
<argument name="metric-prefix" is-required="true" default-value="Custom Metrics|Amazon Cloud Watch|"/> </task-arguments> </monitor-run-task>
</monitor>

 

Note : By default, a Machine agent or a AppServer agent can send a fixed number of metrics to the controller. To change this limit, please follow the instructions mentioned here. For eg.

    java -Dappdynamics.agent.maxMetrics=2500 -jar machineagent.jar

 

Password Encryption Support

To avoid setting the raw AccessKey and SecretKey in the AWSConfigurations.xml, please follow the process to encrypt the AWSCredentials and set the EncryptedAccessKey and EncryptedSecretKey in the AWSConfigurations.xml

  1. Download the util jar to encrypt the AWS Credentials from https://github.com/Appdynamics/maven-repo/blob/master/releases/com/appdynamics/appd-exts-commons/1.1.2/appd-exts-commons-1.1.2.jar

  2. Encrypt AccessKey and SecretKey from the commandline with a common EncryptionKey.

      java -cp appd-exts-commons-1.1.2.jar com.appdynamics.extensions.crypto.Encryptor EncryptionKey AccessKey
    java -cp appd-exts-commons-1.1.2.jar com.appdynamics.extensions.crypto.Encryptor EncryptionKey SecretKey
    
  3. In the AWSConfigurations.xml, replace the existing properties in <AWSCredentials></AWSCredentials> with
      <AWSCredentials>  
<EncryptionKey></EncryptionKey> <EncryptedAccessKey></EncryptedAccessKey> <EncryptedSecretKey></EncryptedSecretKey>
</AWSCredentials>

copying the EncryptionKey, EncryptedAccessKey and EncryptedSecretKey.

 

Custom Dashboard

CloudWatchDashboard.png

 

Contributing

 

Always feel free to fork and contribute any changes directly via GitHub.

 

Support

 

For any questions or feature request, please contact Appdynamics Help.

 

Version:

1.0.7

Compatibility:

3.7+

Last Update:

7/5/2015

 

Version history

1.0.0: Corrected default path to AWSConfigurations.xml file in monitor.xml

1.0.3: Added the logic to page the responses and fetch the data in multi threads

1.0.4 Added support for custom namespace and use of ec2 instance name

1.0.5 Fixed the issue of TypeCasting ConcurrentHashMap into HashMap

1.0.5.1 Added support for metric type selection

1.0.6 Added support for error retry

1.0.7 Added support for proxy and AWS keys encryption

1.0.8 Support for eu-central-1 region