Captures S3 statistics from Amazon CloudWatch and displays them in the AppDynamics Metric Browser.
\<machine_agent_dir\>/monitors/.<br/>Please place the extension in the <b>“monitors”</b> directory of your Machine Agent installation directory. Do not place the extension in the <b>“extensions”</b> directory of your Machine Agent installation directory.
In order to use the extension, you need to update the config.yml file that is present in the extension folder. The following is a step-by-step explanation of the configurable fields that are present in the
If SIM is enabled, then use the following metricPrefix -
metricPrefix: "Custom Metrics|AWS S3|"
Else, configure the "COMPONENT_ID" under which the metrics need to be reported. This can be done by changing the value of
metricPrefix: "Server|Component:<COMPONENT_ID>|Custom Metrics|AWS S3|".
metricPrefix: "Server|Component:100|Custom Metrics|AWS S3|"
Provide accessKey(required) and secretKey(required) of your account(s), also provide displayAccountName(any name that represents your account) and
regions(required). If you are running this extension inside an EC2 instance which has IAM profile configured then you don’t have to configure accessKey and secretKey values, extension will use IAM profile to authenticate. You can provide multiple accounts and regions as below -
accounts: - awsAccessKey: "XXXXXXXX1" awsSecretKey: "XXXXXXXXXX1" displayAccountName: "TestAccount_1" regions: ["us-east-1","us-west-1","us-west-2"] - awsAccessKey: "XXXXXXXX2" awsSecretKey: "XXXXXXXXXX2" displayAccountName: "TestAccount_2" regions: ["eu-central-1","eu-west-1"]
If you want to encrypt the awsAccessKey and awsSecretKey then follow the “Credentials Encryption” section and provide the encrypted values in awsAccessKey and awsSecretKey. Configure
true and provide the encryption key in
#Encryption key for Encrypted password. credentialsDecryptionConfig: enableDecryption: "true" encryptionKey: "XXXXXXXX"
Provide all valid proxy information if you use it. If not, leave this section as is.
proxyConfig: host: port: username: password:
To report metrics only from specific dimension values, configure the
dimension section as below -
dimensions: - name: "StorageType" displayName: "Storage Type" values: [".*"] #.* will fetch AllStorageTypes - name: "FilterId" displayName: "Filter Id" values: ["EntireBucket"] - name: "BucketName" displayName: "Bucket Name" values: ["Sample"]
If these fields are left empty, the metrics which require that dimension will not be reported.
In order to monitor everything under a dimension, you can simply use “.*” to pull everything from your AWS Environment.
Configure the metrics section.
For configuring the metrics, the following properties can be used:
|Property||Default value||Possible values||Description|
|alias||metric name||Any string||The substitute name to be used in the metric browser instead of metric name.|
|statType||“ave”||“AVERAGE”, “SUM”, “MIN”, “MAX”||AWS configured values as returned by API|
|aggregationType||“AVERAGE”||“AVERAGE”, “SUM”, “OBSERVATION”||Aggregation qualifier|
|timeRollUpType||“AVERAGE”||“AVERAGE”, “SUM”, “CURRENT”||Time roll-up qualifier|
|clusterRollUpType||“INDIVIDUAL”||“INDIVIDUAL”, “COLLECTIVE”||Cluster roll-up qualifier|
|multiplier||1||Any number||Value with which the metric needs to be multiplied.|
|convert||null||Any key value map||Set of key value pairs that indicates the value to which the metrics need to be transformed. eg: UP:0, DOWN:1|
|delta||false||true, false||If enabled, gives the delta values of metrics instead of actual values.|
- name: "BucketSizeBytes" alias: "Bytes per minute (Unit - byte; StatType - sum)" statType: "ave" aggregationType: "AVERAGE" timeRollUpType: "AVERAGE" clusterRollUpType: "INDIVIDUAL" delta: false multiplier: 1
All these metric properties are optional, and the default value shown in the table is applied to the metric(if a property has not been specified) by default.
CloudWatch metrics are delivered on a best-effort basis. This means that the delivery of metrics is not guaranteed to be on-time.
There may be a case where the metric is updated in CloudWatch much later than when it was processed, with an associated delay.
For S3, the delay can be 4 - 5 minutes. There is a possibility that the extension does not capture the metric, which is the reason there is a time window. The time window allows
the metric to be updated in CloudWatch before the extension collects it.
metricsTimeRange: startTimeInMinsBeforeNow: 9 endTimeInMinsBeforeNow: 4
This field is set as per the defaults suggested by AWS. You can change this if your limit is different.
The maximum number of retry attempts for failed requests that can be re-tried.
Please avoid using tab (\t) when editing yaml files. Please copy all the contents of the config.yml file and go to Yaml Validator . On reaching the website, paste the contents and press the “Go” button on the bottom left.
If you get a valid output, that means your formatting is correct and you may move on to the next step.
The AWS S3 Extension provides two categories of metrics for AWS S3 as listed here.
AWS S3 CloudWatch Storage Metrics
The AWS S3 Extension provides storage metrics, by default.These metrics are provided daily, free of cost by CloudWatch.
However, the behavior observed in CloudWatch for the storage metrics is that at 00:00 UTC, the metrics are provided with a timestamp of the previous day.<br/>
For example, if the current timestamp is Oct 5th - 00:00 UTC, the storage metrics are reported in CloudWatch with a timestamp of
Oct 4th - 00:00 UTC. The metrics are delayed by 24 hours. This is the reason the extension also reports storage metrics with a latency of 24 hours.
AWS S3 CloudWatch Request Metrics
Request metrics are are paid metrics available in AWS CloudWatch every 1 minute with some latency(upto 4 min). Request metrics have to be enabled as described here.
The extension also supports filtering of metrics on a subset of objects in S3. To get filter-level metrics, a metrics filter has to be enabled on the buckets that are being monitored as described here.
Please visit this page to get detailed instructions on password encryption. The steps in this document will guide you through the whole process.
Workbench is an inbuilt feature provided with each extension in order to assist you to fine tune the extension setup before you actually deploy it on the controller. Please review the following document on How to use the Extensions WorkBench
Please follow the steps listed in this troubleshooting-document in order to troubleshoot your issue. These are a set of common issues that customers might have faced during the installation of the extension. If these don’t solve your issue, please follow the last step on the troubleshooting-document to contact the support team.
If after going through the Troubleshooting Document you have not been able to get your extension working, please file a ticket and add the following information.
Please provide the following in order for us to assist you better.
Stop the running machine agent.
Delete all existing logs under
Please enable debug logging by editing the file
<MachineAgent>/conf/logging/log4j.xml. Change the level value of the following
<logger> elements to debug.
<logger name="com.singularity"> <logger name="com.appdynamics">
Start the machine agent and please let it run for 10 mins. Then zip and upload all the logs in the directory
Attach the zipped
<MachineAgent>/conf/* directory here.
Attach the zipped
<MachineAgent>/monitors/ExtensionFolderYouAreHavingIssuesWith directory here.
For any support related questions, you can also contact email@example.com.
Always feel free to fork and contribute any changes directly here on GitHub.
|Controller Compatibility||4.4 or Later|
|Last Update||Oct 4, 2018|
|List of Changes||Change Log|