MongoDB - Monitoring Extension

Use Case

The MongoDB custom monitor captures statistics from the MongoDB server and displays them in the AppDynamics Metric Browser.

This extension works only with the standalone machine agent.

Metrics include:

  • Server up time
  • Global lock time
  • Operations currently queued, number waiting for the read-lock or write-lock
  • Total active connections, number of read and write operations
  • Memory metrics including bits, resident RAM, virtual memory, mapped memory, mapped memory with journaling
  • Current and available connections
  • Index counters including index access, hits and misses, resets
  • Background flushing metrics such as number of times, total time, average time, last time
  • Network traffic sent and received (in bytes), number of distinct requests received
  • Number of database operations including: insert, query, update, delete, get more, and total number of commands
  • Number of asserts since the server process started: regular, warnings, message, user, and number of times the rollover counter has rolled
  • Database related stats
  • Cluster related stats (status, health and uptime)
 

Installation

  1. Download and unzip MongoMonitor.zip into <MACHINE_AGENT_HOME>/monitors/
  2. Edit the config.yml in MongoMonitor directory to configure the parameters specified in the below section.
  3. Restart the machineagent
  4. In the AppDynamics Metric Browser, look for: Application Infrastructure Performance | <Tier> | Custom Metrics | Mongo Server.

Configuration

Note : Please make sure to not use tab (\t) while editing yaml files. You may want to validate the yaml file using a yaml validator

  1. Configure the Mongo instances by editing the config.yml file in <MACHINE_AGENT_HOME>/monitors/MongoMonitor/.

    For eg.

        # MongoDB host and port. If ReplicaSet is enabled, configure all OR subset of members of the cluster.
        servers:
          - host: "localhost"
            port: 27017
          - host: "localhost"
            port: 27018
    
        # Specify this key if Password Encryption Support is required else keep it empty
        # If specified, adminDBPassword and databases passwords are now the encrypted passwords.
        passwordEncryptionKey: ""
    
        # Admin DB username and password. Required if mongod is started with --auth (Authentication) else keep empty
        # The user should have clusterMonitor role as a minimum
        adminDBUsername: "admin"
        adminDBPassword: "admin"
    
        # Change ssl to true if mongod is started with ssl. Then specify the pemFilePath, if not keep empty.
        ssl: false
        pemFilePath: ""
    
        # https://docs.mongodb.com/manual/reference/command/serverStatus/
        serverStatusExcludeMetricFields: [locks, wiredTiger, tcmalloc, opcountersRepl, metrics]
    
        #prefix used to show up metrics in AppDynamics
        metricPathPrefix:  "Custom Metrics|Mongo Server|"
        #This will create it in specific Tier. Replace 
        #metricPrefix:  "Server|Component:|Custom Metrics|Mongo Server|"
    
    
  2. Configure the path to the config.yml file by editing the in the monitor.xml file in the <MACHINE_AGENT_HOME>/monitors/MongoMonitor/ directory. Below is the sample

     <task-arguments>
         <!-- config file-->
         <argument name="config-file" is-required="true" default-value="monitors/MongoMonitor/config.yml" />
          ....
     </task-arguments>
    

Note : By default, a Machine agent or a AppServer agent can send a fixed number of metrics to the controller. To change this limit, please follow the instructions mentioned here. For eg.

    java -Dappdynamics.agent.maxMetrics=2500 -jar machineagent.jar

Password Encryption Support

To avoid setting the clear text password in the config.yml, please follow the process below to encrypt the passwords

  1. Download the util jar to encrypt the password from https://github.com/Appdynamics/maven-repo/blob/master/releases/com/appdynamics/appd-exts-commons/1.1.2/appd-exts-commons-1.1.2.jar and navigate to the downloaded directory
  2. Encrypt password from the commandline java -cp appd-exts-commons-1.1.2.jar com.appdynamics.extensions.crypto.Encryptor myKey myPassword
  3. Specify the passwordEncryptionKey and encrypted adminDBPassword in config.yml
  4. For every db, use the same encryption key to encrypt the password and specify the encrypted password in config.yml

Metrics

Server Stats

Metric NameDescription
Up Time (ms)The duration of time that the server is up

Metric Category: Asserts

Metric NameDescription
Message 
Regular 
Rollover 
User 
Warning 

Metric Category: Background Flushing

Metric NameDescription
FlushesNumber of times the database has been flushed
Total (ms)Total time (ms) that the mongod process spent writing data to disk
Average (ms)Average time (ms) that the mongod process spent writing data to disk
Last (ms)Time (ms) that the mongod process last spent writing data to disk

Metric Category: Connections

Metric NameDescription
CurrentNumber of current connections to the database server from clients. This number includes the current shell connection as well as any inter-node connections to support a replica set or sharded cluster.
AvailableNumber of unused available connections that the database can provide. Consider this value in combination with the value of Current to understand the connection load on the database.

Metric Category: Global Lock

Metric NameDescription
Total TimeThe total time since the globalLock was started and created
Active Clients
Metric NameDescription
TotalNumber of active client connections to the database
ReadersNumber of readers performing read operations
WritersNumber of writers performing write operations
Current Queue
Metric NameDescription
TotalNumber of operations queued before a lock
ReadersNumber of operations waiting for the read-lock
WritersNumber of operations waiting for the write-lock

Metric Category: Index Counter

Metric NameDescription
Accesses 
Hits 
Misses 
Resets 

Metric Category: Memory

Metric NameDescription
BitsTarget Architecture
ResidentAmount of RAM (MB) currently used by the database process
VirtualMB currently used by the mongod process
MappedAmount of mapped memory (MB) used by the database
Mapped With JournalAmount of mapped memory (MB), including memory used for journaling.

Metric Category: Network

Metric NameDescription
Bytes InThe amount of network traffic (bytes) received by the database
Bytes OutThe amount of network traffic (bytes) sent from the database
Number RequestsNumber of distinct requests that the server has received

Metric Category: Operations

Metric NameDescription
InsertNumber of insert operations
QueryNumber of query operations
UpdateNumber of update operations
DeleteNumber of delete operations
GetMoreNumber of getmore operations
CommandTotal number of commands issued to database

Replica Stats

For each replica the following metrics are reported.

Metric NameDescription
Up Time (ms)The duration of time that the server is up
HealthConveys if the member is up (i.e. 1) or down (i.e. 0.)
[State](http://docs.mongodb.org/manual/reference/replica-states/)Value between 0 and 10 that represents the replica state of the member.

DB Stats

Metric NameDescription
avgObjSizeThe average size of each document in bytes. This is the dataSize divided by the number of documents
collectionsContains a count of the number of collections in that database
dataSizeThe total size in bytes of the data held in this database including the padding factor
fileSizeThe total size in bytes of the data files that hold the database
indexesContains a count of the total number of indexes across all collections in the database
indexSizeThe total size in bytes of all indexes created on this database
nsSizeMBThe total size of the namespace files (i.e. that end with .ns) for this database
numExtentsContains a count of the number of extents in the database across all collections
objectsContains a count of the number of objects (i.e. documents) in the database across all collections
storageSizeThe total amount of space in bytes allocated to collections in this database for document storage

Metric Category: Collection Stats

Metric NameDescription
countThe number of objects or documents in this collection
lastExtentSizeThe size of the last extent allocated
nindexesThe number of indexes on the collection
numExtentsThe total number of contiguously allocated data file regions
paddingFactorThe amount of space added to the end of each document at insert time
sizeThe size of the data stored in this collection
storageSizeThe total amount of storage allocated to this collection for document storage
systemFlagsReports the flags on this collection that reflect internal server options
totalIndexSizeThe total size of all indexes. The scale argument affects this value
userFlagsReports the flags on this collection set by the user

 

Contributing

Always feel free to fork and contribute any changes directly via GitHub.

 

Support

For any questions or feature request, please contact help@appdynamics.com.

 

 

Version:

1.3.1

MongoDB Tested on:

3.2

Last Update:

06/15/2016

 

 
Changes in versions:
1.0.1: empty username and password fields allow connection to the MongoDB server now (in case no username & password is set)
1.0.2: bugfixes
1.0.3: Made compatible with JDK 1.5.
1.0.4: Added support for multiple DBs
1.0.5: Added support for multiple DBs and support for collections in DB
1.0.6: Fixed the connection leak
1.0.7: Resolving the install directory and support for absolute path for properties.xml
1.0.8: Added SSL support
1.0.9 Password encryption support
1.1.0 Fixed NPE's
1.2.0 Added Replica stats and moved configuration to config.yaml
1.3.1 Added Server Stats filtering