AppDynamics Monitoring Extension for use with NGINX and NGINX Plus

Use Case

Nginx is a web server which can also be used as a reverse proxy, load balancer, mail proxy and HTTP cache. The Nginx monitoring extension gets metrics from the nginx server and displays them in the AppDynamics Metric Browser. This extension supports both Nginx and Nginx Plus.

Prerequisites

Before the extension is installed, the prerequisites mentioned here need to be met. Please do not proceed with the extension installation if the specified prerequisites are not met.

In order to use this extension, you do need a Standalone JAVA Machine Agent or SIM Agent. For more details on downloading these products, please visit here. The extension needs to be able to connect to the Nginx in order to collect and send metrics. To do this, you will have to either establish a remote connection in between the extension and the product, or have an agent on the same machine running the product in order for the extension to collect and send the metrics.

Installation

Note: For the following steps to work, nginx should be running with "ngx_http_stub_status_module" or "ngx_http_api_module" enabled. Please make sure you have required changes in the nginx.conf.

  1. Download and unzip the nginx-monitoring-extension-2.0.0.zip to the "<MachineAgent_Dir>/monitors" directory.
  2. Edit the file config.yml as described below in Configuration Section, located in <MachineAgent_Dir>/monitors/NginxMonitor and update the server(s) details.
  3. All metrics to be reported are configured in metrics.xml. Users can remove entries from metrics.xml to stop the metric from reporting, or add new entries as well.
  4. Restart the Machine Agent.

Please place the extension in the "monitors" directory of your Machine Agent installation directory. Do not place the extension in the "extensions" directory of your Machine Agent installation directory. In the AppDynamics Metric Browser, look for Application Infrastructure Performance|<Tier>|Custom Metrics|NginX Monitor and you should be able to see all the metrics.

Directory Structure

Directory/FileDescription
src/main/resources/confContains the monitor.xml
src/main/javaContains source code of the Nginx monitoring extension
targetOnly obtained when using maven. Run 'mvn clean install' to get the distributable .zip file
pom.xmlMaven build script to package the project (required only if changing Java code)

Configuration

Config.yml

Configure the extension by editing the config.yml file in <MACHINE_AGENT_HOME>/monitors/NginxMonitor/.

  1. Edit the file config.yml located at <MachineAgent_Dir>/monitors/NginxMonitor The metricPrefix of the extension has to be configured as specified here. Please make sure that the right metricPrefix is chosen based on your machine agent deployment, otherwise this could lead to metrics not being visible in the controller.

  2. The extension supports reporting metrics from multiple Nginx instances. The monitor provides an option to add Nginx server/s for monitoring the metrics provided by the particular end-point. Have a look at config.yml for more details. For example:

      metricPrefix:  "Server|Component:<TIER_ID>|Custom Metrics|Nginx|"
    
      servers:
        - displayName: "Nginx Server" # mandatory
          uri: "http://localhost/nginx_status" # append port if needed
          username: ""
          password: ""
          encryptedPassword:
          nginx_plus: "false"  # true for nginx plus else false
    
      encryptionKey: ""
    
      connection:
        sslCertCheckEnabled: false
        socketTimeout: 10000
        connectTimeout: 10000
    
       # For each server you monitor, you will need a total of 8(by default) thread.
       # By default we want to support 5 servers, so it is 5 * 8 = 40 threads.
      numberOfThreads: 12
    
  3. If you want to monitor nginx plus then put nginx_plus as true and make sure ngx_http_api_module is configured.

         nginx_plus: "true"  # true for nginx plus else false
    
  4. Configure the numberOfThreads. For example, If number of servers that need to be monitored is 5, then number of threads required is 5 * 12 = 60

    numberOfThreads: 60
    
Metrics.xml

You can add/remove metrics of your choice by modifying the provided metrics.xml file. This file consists of all the metrics that will be monitored and sent to the controller. Please look how the metrics have been defined and follow the same convention, when adding new metrics. You do have the ability to chosoe your Rollup types as well as set an alias that you would like to be displayed on the metric browser.

  1. Stats Configuration Add the stats url which has api version(1/2/3) information as shown below.

    <stats url="/api/3">
    
  2. Metric Stat Configuration Add the metric to be monitored under the metric tag as shown below.

    <stat suburl="processes" name="Processes-Status">
        <metric attr="respawned" alias="Respawned" aggregationType = "AVERAGE" timeRollUpType = "AVERAGE" clusterRollUpType = "COLLECTIVE"/>
    </stat>
    

For configuring the metrics, the following properties can be used:

PropertyDefault valuePossible valuesDescription
aliasmetric nameAny stringThe substitute name to be used in the metric browser instead of metric name.
aggregationType"AVERAGE""AVERAGE", "SUM", "OBSERVATION"Aggregation qualifier
timeRollUpType"AVERAGE""AVERAGE", "SUM", "CURRENT"Time roll-up qualifier
clusterRollUpType"INDIVIDUAL""INDIVIDUAL", "COLLECTIVE"Cluster roll-up qualifier
multiplier1Any numberValue with which the metric needs to be multiplied.
convertnullAny key value mapSet of key value pairs that indicates the value to which the metrics need to be transformed. eg: UP:1, OPEN:1
deltafalsetrue, falseIf enabled, gives the delta values of metrics instead of actual values.

All these metric properties are optional, and the default value shown in the table is applied to the metric (if a property has not been specified) by default.

Metrics

Nginx Monitoring Extension can collect metric by hitting the available Endpoints which are configured in the metrics.xml.

Nginx Metrics
Metric NameDescription
Active ConnectionsNumber of all open active connections
Server: AcceptsNumber of accepted requests
Server: HandledNumber of handled requests
Server: RequestsTotal number of requests
ReadingNginx reads request header
WritingNginx reads request body, processes request, or writes response to a client
WaitingNginX keep-alive connections or currently active
Nginx Plus Metrics
Requests
Metric NameDescription
totalThe total number of client requests
currentThe current number of client requests
Server Zones
Metric NameDescription
processingThe number of client requests that are currently being processed
requestsThe total number of client requests received from clients
responses/totalThe total number of responses sent to clients
responses/ 1xx, 2xx, 3xx, 4xx, 5xxThe number of responses with status codes 1xx, 2xx, 3xx, 4xx, and 5xx
discardedThe total number of bytes discarded from clients
receivedThe total number of bytes received from clients
sentThe total number of bytes sent to clients
Upstreams
Metric NameDescription
activeThe current number of active connections
backupA boolean value indicating whether the server is a backup server
downtimeTotal time the server was in the “unavail” and “unhealthy” states
failsThe total number of unsuccessful attempts to communicate with the server
receivedThe total number of bytes received from this server
requestsThe total number of client requests forwarded to this server
sentThe total number of bytes sent to this server
stateurrent state, which may be one of “up”, “down”, “unavail”, or “unhealthy”.
unavailTimes the server became unavailable
weightWeight of the server
responses/totalThe total number of responses obtained from this server
responses/ 1xx, 2xx, 3xx, 4xx, 5xxThe number of responses with status codes 1xx, 2xx, 3xx, 4xx, and 5xx
health_checks/checksThe total number of health check requests made
health_checks/failsThe number of failed health checks
health_checks/unhealthyHow many times the server became unhealthy (state “unhealthy”)
health_checks/last_passedBoolean indicating if the last health check request was successful and passed tests
Connections
Metric NameDescription
acceptedThe total number of accepted client connections
droppedThe total number of dropped client connections
activeThe current number of active client connections
idleThe current number of idle client connections
Caches
Metric NameDescription
sizeThe current size of the cache
max_sizeThe limit on the maximum size of the cache specified in the configuration
coldA boolean value indicating whether the “cache loader” process is still loading data from disk into the cache
hit, stale, updating, revalidated/responsesThe total number of responses read from the cache
hit, stale, updating, revalidated/bytesThe total number of bytes read from the cache
miss, expired, bypass/responsesThe total number of responses not taken from the cache
miss, expired, bypass/bytesThe total number of bytes read from the proxied server
miss, expired, bypass/responses_writtenThe total number of responses written to the cache
miss, expired, bypass/bytes_writtenThe total number of bytes written to the cache

Credentials Encryption

Please visit this page to get detailed instructions on password encryption. The steps in this document will guide you through the whole process.

Extensions Workbench

Workbench is an inbuilt feature provided with each extension in order to assist you to fine tune the extension setup before you actually deploy it on the controller. Please review the following document on How to use the Extensions WorkBench.

Troubleshooting

Please follow the steps listed in this troubleshooting-document in order to troubleshoot your issue. These are a set of common issues that customers might have faced during the installation of the extension. If these don't solve your issue, please follow the last step on the troubleshooting-document to contact the support team.

Support Tickets

If after going through the Troubleshooting Document you have not been able to get your extension working, please file a ticket and add the following information.

Please provide the following in order for us to assist you better.

1. Stop the running machine agent.
2. Delete all existing logs under <MachineAgent>/logs.
3. Please enable debug logging by editing the file <MachineAgent>/conf/logging/log4j.xml. Change the level value of the following <logger> elements to debug.
    <logger name="com.singularity">
    <logger name="com.appdynamics">
4. Start the machine agent and please let it run for 10 mins. Then zip and upload all the logs in the directory <MachineAgent>/logs/*.
5. Attach the zipped <MachineAgent>/conf/* directory here.
6. Attach the zipped <MachineAgent>/monitors/ExtensionFolderYouAreHavingIssuesWith directory here.

For any support related questions, you can also contact help@appdynamics.com.

Contributing

Always feel free to fork and contribute any changes directly here on GitHub.

Version

NameVersion
Extension Version2.0.0
Controller Compatibility3.7 or Later
Product Tested On1.13.3 and later
Last Update12/02/2019
Changes listChangeLog