AWS S3 Monitoring Extension

AWS S3 Monitoring Extension

 

Use Case

Captures S3 statistics from Amazon CloudWatch and displays them in the AppDynamics Metric Browser.

 

This extension works only with the standalone machine agent.

 

Note: By default, the Machine agent can only send a fixed number of metrics to the controller. This extension can potentially report thousands of metrics, so to change this limit, please follow the instructions mentioned here.

 

Installation

  1. Download and unzip AWSS3Monitor-<version>.zip into <machine_agent_dir>/monitors/
  2. Edit config.yaml file in AWSS3Monitor/conf and provide the required configuration (see Configuration section)
  3. Restart the Machine Agent.

 

Configuration

 

config.yaml

Note: Please avoid using tab (\t) when editing yaml files. You may want to validate the yaml file using a yaml validator.

 

SectionFieldsDescriptionExample
accountsFields under this section can be repeated for multiple accounts config
awsAccessKeyAWS Access Key
awsSecretKeyAWS Secret Key
displayAccountNameDisplay name used in metric path"MyAWSS3"
regionsRegions where S3 is registeredAllowed values:
"ap-southeast-1",
"ap-southeast-2",
"ap-northeast-1",
"eu-central-1",
"eu-west-1",
"us-east-1",
"us-west-1",
"us-west-2",
"sa-east-1"
credentialsDecryptionConfig---------------
enableDecryptionIf set to "true", then all aws credentials provided (access key and secret key) will be decrypted - see AWS Credentials Encryption section
decryptionKeyThe key used when encypting the credentials
proxyConfig---------------
hostThe proxy host (must also specify port)
portThe proxy port (must also specify host)
usernameThe proxy username (optional)
passwordThe proxy password (optional)
metricsConfig---------------
metricTypesFields under this section can be repeated for multiple metric types override
metricNameThe metric name"CPUUtilization"
statTypeThe statistic typeAllowed values:
"ave"
"max"
"min"
"sum"
"samplecount"
---------------
excludeMetricsMetrics to exclude - supports regex"CPUUtilization",
"Swap.*"
metricsTimeRange
startTimeInMinsBeforeNowThe no of mins to deduct from current time for start time of query5
endTimeInMinsBeforeNowThe no of mins to deduct from current time for end time of query.
Note, this must be less than startTimeInMinsBeforeNow
0
---------------
maxErrorRetrySizeThe max number of retry attempts for failed retryable requests1
concurrencyConfig
noOfAccountThreadsThe no of threads to process multiple accounts concurrently3
noOfRegionThreadsPerAccountThe no of threads to process multiple regions per account concurrently3
noOfMetricThreadsPerRegionThe no of threads to process multiple metrics per region concurrently3
---------------
metricPrefixThe path prefix for viewing metrics in the metric browser."Custom Metrics|Amazon S3|"

 

Below is an example config for monitoring multiple accounts and regions:

accounts:
  - awsAccessKey: "XXXXXXXX1"
    awsSecretKey: "XXXXXXXXXX1"
    displayAccountName: "TestAccount_1"
    regions: ["us-east-1","us-west-1","us-west-2"]

  - awsAccessKey: "XXXXXXXX2"
    awsSecretKey: "XXXXXXXXXX2"
    displayAccountName: "TestAccount_2"
    regions: ["eu-central-1","eu-west-1"]

credentialsDecryptionConfig:
    enableDecryption: "false"
    decryptionKey:

proxyConfig:
    host: 
    port:
    username:
    password:    

metricsConfig:
    metricTypes:
      - metricName: "GroupInServiceInstances"
        statType: "max"

      - metricName: "GroupMinSize"
        statType: "sum"        

    excludeMetrics: ["GroupMaxSize", "GroupTotal.*"]

    metricsTimeRange:
      startTimeInMinsBeforeNow: 5
      endTimeInMinsBeforeNow: 0

    maxErrorRetrySize: 0

concurrencyConfig:
  noOfAccountThreads: 3
  noOfRegionThreadsPerAccount: 3
  noOfMetricThreadsPerRegion: 3

metricPrefix: "Custom Metrics|Amazon S3|"

 

AWS Credentials Encryption

To set an encrypted awsAccessKey and awsSecretKey in config.yaml, follow the steps below:

  1. Download the util jar to encrypt the AWS Credentials from here.
  2. Run command:

    java -cp appd-exts-commons-1.1.2.jar com.appdynamics.extensions.crypto.Encryptor EncryptionKey CredentialToEncrypt
    
    For example: 
    java -cp "appd-exts-commons-1.1.2.jar" com.appdynamics.extensions.crypto.Encryptor test myAwsAccessKey
    
    java -cp "appd-exts-commons-1.1.2.jar" com.appdynamics.extensions.crypto.Encryptor test myAwsSecretKey
    
  3. Set the decryptionKey field in config.yaml with the encryption key used, as well as the resulting encrypted awsAccessKey and awsSecretKey in their respective fields.

 

Metrics

 

Typical metric path: Application Infrastructure Performance|<Tier>|Custom Metrics|Amazon S3|<Account Name>|<Region>|Bucket Name|<Bucket Name>|Storage Type|<Storage Type> followed by the metrics defined in the link below:

 

Contributing

Always feel free to fork and contribute any changes directly via GitHub.

 

Support

For any questions or feature request, please contact AppDynamics Center of Excellence.

 

Version:1.0.0
Agent Compatiblity3.7+
Last Update15 Oct 2015