VoltDB Monitoring Extension

Use Case

 

The VoltDB Monitoring Extension collects the stats from VoltDB and reports them to the AppDynamics Controller.

This extension works only with the standalone machine agent.

 

Installation

 

  1. Download the file VoltDBMonitor.zip into <machineagent install dir>/monitors/
  2. Unzip the downloaded file and cd into VoltDBMonitor
  3. Open the monitor.xml file and provide values for voltdb-host, voltdb-port, user-name, password and mode
  4. Restart the Machine Agent.
  5. In the AppDynamics controller, look for events in Custom Metrics|VoltDB|

Directory Structure

 

Directory/FileDescription
src/main/javaContains source code to VoltDB Monitoring Extension
src/main/resourcesContains monitor.xml
targetOnly obtained when using maven. Run 'maven clean install' to get the distributable .zip file
pom.xmlMaven script file (required only if changing Java code)

 

XML Examples

 

monitor.xml

ParamDescription
voltdb-hostVoltDB host
voltdb-portVoltDb port. For client API default port is 21212 and for REST API default port is 8080
user-nameUser who has permission to execute system procedures
passwordpassword
modeMode in which user wants to gather the stats. Possible values are clientAPI/restAPI
<monitor>
        <name>VoltDBMonitor</name>
        <type>managed</type>
        <description>Volt DB monitor</description>
        <monitor-configuration></monitor-configuration>
        <monitor-run-task>
                <execution-style>periodic</execution-style>
                <execution-frequency-in-seconds>60</execution-frequency-in-seconds>
                <name>VoltDB Monitor Run Task</name>
                <display-name>VoltDB Monitor Task</display-name>
                <description>VoltDB Monitor Task</description>
                <type>java</type>
                <execution-timeout-in-secs>60</execution-timeout-in-secs>
                <task-arguments>
                    <argument name="voltdb-host" is-required="true" default-value="localhost" />
                    <argument name="voltdb-port" is-required="true" default-value="21212" />
                    <!--User should have access to execute system procedures -->
                    <argument name="user-name" is-required="false" default-value="admin" />
                    <argument name="password" is-required="false" default-value="voltdb" />
                    <!--Mode through which you want to fetch stats. Possible values [clientAPI/restAPI] -->
                    <argument name="mode" is-required="false" default-value="clientAPI" />
               </task-arguments>
                <java-task>
                    <classpath>voltdb-monitoring-extension.jar</classpath>
                    <impl-class>com.appdynamics.monitors.voltdb.VoltDBMonitor</impl-class>
                </java-task>
        </monitor-run-task>
</monitor>

VoltDB

Steps to secure VoltDB with username and password

  1. Create Role in VoltDB

    To create a role user has to execute the below query
    CREATE ROLE admin WITH sysproc,adhoc,defaultproc;
    The above query createa role with admin with sysproc, adhoc and defaultproc permissions.

  2. In the deployment.xml add the following lines

    <security enabled="true"/> 
    <users>
      <user name="admin" password="voltdb" roles="admin"/>
    </users>
    

    It will enable security and you can only access the VoltDB with user 'admin' and password 'voltdb'.

    For more info on securing VoltDB please visit http://voltdb.com/docs/UsingVoltDB/ChapSecurity.php#SecurityHowItWorks

Enabling Json interface

If you want to fetch stats using REST API, you must enable json interface. You do this by adding the following lines to deployment.xml

 <httpd enabled="true">
   <jsonapi enabled="true" />
 </httpd>   

Metrics

Index

Metrics related to Index

NameDescription
Custom Metrics/VoltDB/Stats/Index/{SITE_ID}/{PARTITION_ID}/{TABLE_NAME}/{INDEX_NAME}/ENTRY_COUNTThe number of index entries currently in the partition
Custom Metrics/VoltDB/Stats/Index/{SITE_ID}/{PARTITION_ID}/{TABLE_NAME}/{INDEX_NAME}/IS_COUNTABLEA byte value specifying whether the index maintains a counter to optimize COUNT(*) queries
Custom Metrics/VoltDB/Stats/Index/{SITE_ID}/{PARTITION_ID}/{TABLE_NAME}/{INDEX_NAME}/IS_UNIQUEA byte value specifying whether the index is unique (1) or not (0)
Custom Metrics/VoltDB/Stats/Index/{SITE_ID}/{PARTITION_ID}/{TABLE_NAME}/{INDEX_NAME}/MEMORY_ESTIMATEThe estimated amount of memory (in kilobytes) consumed by the current index entries

 

IOSTATS

Metrics related to IO operations

NameDescription
Custom Metrics/VoltDB/Stats/IO/{CONNECTION_HOSTNAME}/{CONNECTION_ID}/BYTES_READThe number of bytes of data sent from the client to the host
Custom Metrics/VoltDB/Stats/IO/{CONNECTION_HOSTNAME}/{CONNECTION_ID}/BYTES_WRITTENThe number of bytes of data sent from the host to the client
Custom Metrics/VoltDB/Stats/IO/{CONNECTION_HOSTNAME}/{CONNECTION_ID}/CONNECTION_IDNumeric ID of the client connection invoking the procedure
Custom Metrics/VoltDB/Stats/IO/{CONNECTION_HOSTNAME}/{CONNECTION_ID}/MESSAGES_READThe number of individual messages sent from the client to the host
Custom Metrics/VoltDB/Stats/IO/{CONNECTION_HOSTNAME}/{CONNECTION_ID}/MESSAGES_WRITTENThe number of individual messages sent from the host to the client

 

LIVECLIENTS

Metrics related to client connections currently active on the cluster

NameDescription
Custom Metrics/VoltDB/Stats/Live Clients/{CLIENT_HOSTNAME}/ADMINA byte value specifying whether the connection is to the client port (0) or the admin port (1)
Custom Metrics/VoltDB/Stats/Live Clients/{CLIENT_HOSTNAME}/CONNECTION_IDNumeric ID of the client connection invoking the procedure
Custom Metrics/VoltDB/Stats/Live Clients/{CLIENT_HOSTNAME}/OUTSTANDING_REQUEST_BYTESThe number of bytes of data sent from the client currently pending on the host
Custom Metrics/VoltDB/Stats/Live Clients/{CLIENT_HOSTNAME}/OUTSTANDING_RESPONSE_MESSAGESThe number of messages on the host queue waiting to be retrieved by the client
Custom Metrics/VoltDB/Stats/Live Clients/{CLIENT_HOSTNAME}/OUTSTANDING_ TRANSACTIONSThe number of transactions (that is, stored procedures) initiated on behalf of the client that have yet to be completed

 

MEMORY

Metrics related to server in the cluster

NameDescription
Custom Metrics/VoltDB/Stats/Memory/{HOSTNAME}/INDEXMEMORYThe amount of memory (in kilobytes) currently in use for storing database indexes
Custom Metrics/VoltDB/Stats/Memory/{HOSTNAME}/JAVAUNUSEDThe amount of memory (in kilobytes) allocated by Java but unused. (In other words, free space in the Java heap.)
Custom Metrics/VoltDB/Stats/Memory/{HOSTNAME}/JAVAUSEDThe amount of memory (in kilobytes) allocated by Java and currently in use by VoltDB
Custom Metrics/VoltDB/Stats/Memory/{HOSTNAME}/POOLEDMEMORYThe total size of memory (in megabytes) allocated for tasks other than database records, indexes, and strings
Custom Metrics/VoltDB/Stats/Memory/{HOSTNAME}/RSSThe current resident set size. That is, the total amount of memory allocated to the VoltDB processes on the server
Custom Metrics/VoltDB/Stats/Memory/{HOSTNAME}/STRINGMEMORYThe amount of memory (in kilobytes) currently in use for storing string and binary data that is not stored "in-line" in the database record
Custom Metrics/VoltDB/Stats/Memory/{HOSTNAME}/TUPLEALLOCATEDThe amount of memory (in kilobytes) allocated for the storage of database records (including free space)
Custom Metrics/VoltDB/Stats/Memory/{HOSTNAME}/TUPLECOUNTThe total number of database records currently in memory
Custom Metrics/VoltDB/Stats/Memory/{HOSTNAME}/TUPLEDATAThe amount of memory (in kilobytes) currently in use for storing database records

 

PARTITIONCOUNT

Metrics related to total number of partitions and the host

NameDescription
Custom Metrics/VoltDB/Stats/Partition Count/{HOSTNAME}PARTITION_COUNT

 

PROCEDURE

Metrics related to every stored procedure that has been executed on the cluster

NameDescription
Custom Metrics/VoltDB/Stats/Procedure/{PROCEDURE}/ABORTSThe number of times the procedure was aborted
Custom Metrics/VoltDB/Stats/Procedure/{PROCEDURE}/AVG_EXECUTION_TIMEThe average length of time (in nanoseconds) it took to execute the stored procedure
Custom Metrics/VoltDB/Stats/Procedure/{PROCEDURE}/AVG_PARAMETER_SET_SIZEThe average size (in bytes) of the parameters passed as input to the procedure
Custom Metrics/VoltDB/Stats/Procedure/{PROCEDURE}/AVG_RESULT_SIZEThe average size (in bytes) of the results returned by the procedure
Custom Metrics/VoltDB/Stats/Procedure/{PROCEDURE}/FAILURESThe number of times the procedure failed unexpectedly
Custom Metrics/VoltDB/Stats/Procedure/{PROCEDURE}/INVOCATIONSThe total number of invocations of this procedure at this site
Custom Metrics/VoltDB/Stats/Procedure/{PROCEDURE}/MAX_EXECUTION_TIMEThe maximum length of time (in nanoseconds) it took to execute the stored procedure
Custom Metrics/VoltDB/Stats/Procedure/{PROCEDURE}/MAX_PARAMETER_SET_SIZEThe maximum size (in bytes) of the parameters passed as input to the procedure
Custom Metrics/VoltDB/Stats/Procedure/{PROCEDURE}/MAX_RESULT_SIZEThe maximum size (in bytes) of the results returned by the procedure
Custom Metrics/VoltDB/Stats/Procedure/{PROCEDURE}/MIN_EXECUTION_TIMEThe minimum length of time (in nanoseconds) it took to execute the stored procedure
Custom Metrics/VoltDB/Stats/Procedure/{PROCEDURE}/MIN_PARAMETER_SET_SIZEThe minimum size (in bytes) of the parameters passed as input to the procedure
Custom Metrics/VoltDB/Stats/Procedure/{PROCEDURE}/MIN_RESULT_SIZEThe minimum size (in bytes) of the results returned by the procedures
Custom Metrics/VoltDB/Stats/Procedure/{PROCEDURE}/TIMED_INVOCATIONSThe number of invocations used to measure the minimum, maximum, and average execution time

 

PROCEDUREINPUT

Metrics related to every stored procedure that has been executed on the cluster

NameDescription
Custom Metrics/VoltDB/Stats/Procedure Input/{PROCEDURE}/AVG_PARAMETER_SET_SIZEThe average parameter set size in bytes
Custom Metrics/VoltDB/Stats/Procedure Input/{PROCEDURE}/INVOCATIONSThe total number of invocations of this procedure
Custom Metrics/VoltDB/Stats/Procedure Input/{PROCEDURE}/MAX_PARAMETER_SET_SIZEThe maximum parameter set size in bytes
Custom Metrics/VoltDB/Stats/Procedure Input/{PROCEDURE}/MIN_PARAMETER_SET_SIZEThe minimum parameter set size in bytes
Custom Metrics/VoltDB/Stats/Procedure Input/{PROCEDURE}/TOTAL_PARAMETER_SET_SIZE_MBThe total input for all invocations of this stored procedure measured in megabytes

 

PROCEDUREOUTPUT

Metrics related to every stored procedure that has been executed on the cluster

NameDescription
Custom Metrics/VoltDB/Stats/Procedure Output/{PROCEDURE}/AVG_RESULT_SIZEThe average result set size in bytes
Custom Metrics/VoltDB/Stats/Procedure Output/{PROCEDURE}/INVOCATIONSThe total number of invocations of this procedure
Custom Metrics/VoltDB/Stats/Procedure Output/{PROCEDURE}/MAX_RESULT_SIZEThe maximum result set size in bytes
Custom Metrics/VoltDB/Stats/Procedure Output/{PROCEDURE}/MIN_RESULT_SIZEThe minimum result set size in bytes
Custom Metrics/VoltDB/Stats/Procedure Output/{PROCEDURE}/TOTAL_RESULT_SIZE_MBThe total output returned by all invocations of this stored procedure measured in megabytes

 

PROCEDUREPROFILE

Metrics related to every stored procedure that has been executed on the cluster

NameDescription
Custom Metrics/VoltDB/Stats/Procedure Profile/{PROCEDURE}/ABORTSThe number of times the procedure was aborted
Custom Metrics/VoltDB/Stats/Procedure Profile/{PROCEDURE}/AVGThe average length of time (in nanoseconds) it took to execute the stored procedure
Custom Metrics/VoltDB/Stats/Procedure Profile/{PROCEDURE}/FAILURESThe number of times the procedure failed unexpectedly
Custom Metrics/VoltDB/Stats/Procedure Profile/{PROCEDURE}/INVOCATIONSThe total number of invocations of this procedure
Custom Metrics/VoltDB/Stats/Procedure Profile/{PROCEDURE}/MAXThe maximum length of time (in nanoseconds) it took to execute the stored procedure
Custom Metrics/VoltDB/Stats/Procedure Profile/{PROCEDURE}/MINThe minimum length of time (in nanoseconds) it took to execute the stored procedure

 

REPLICATION(DR)

Metrics to shows the current state of replication and how much data is currently queued for the DR agent

NameDescription
Custom Metrics/VoltDB/Stats/Replication/{PARTITION_ID}/LASTACKTIMESTAMPThe timestamp of the last acknowledgement received from the DR agent
Custom Metrics/VoltDB/Stats/Replication/{PARTITION_ID}/TOTALBUFFERSThe total number of buffers in this partition currently waiting for acknowledgement from the DR agent
Custom Metrics/VoltDB/Stats/Replication/{PARTITION_ID}/TOTALBYTESThe total number of bytes currently queued for transmission to the DR agent
Custom Metrics/VoltDB/Stats/Replication/{PARTITION_ID}/TOTALBYTESINMEMORYThe total number of bytes of queued data currently held in memory

 

SNAPSHOTSTATUS

Metrics to shows every snapshot file in the recent snapshots performed on the cluster

NameDescription
Custom Metrics/VoltDB/Stats/Snapshot Status/{HOSTNAME}/{TABLE}/DURATIONThe length of time (in milliseconds) it took to complete the snapshot
Custom Metrics/VoltDB/Stats/Snapshot Status/{HOSTNAME}/{TABLE}/END_TIMEThe timestamp when the snapshot was completed (in milliseconds)
Custom Metrics/VoltDB/Stats/Snapshot Status/{HOSTNAME}/{TABLE}/SIZEThe total size, in bytes, of the file
Custom Metrics/VoltDB/Stats/Snapshot Status/{HOSTNAME}/{TABLE}/START_TIMEThe timestamp when the snapshot began (in milliseconds)
Custom Metrics/VoltDB/Stats/Snapshot Status/{HOSTNAME}/{TABLE}/THROUGHPUTThe average number of bytes per second written to the file during the snapshot process
Custom Metrics/VoltDB/Stats/Snapshot Status/{HOSTNAME}/{TABLE}/TXNIDThe transaction ID of the snapshot

 

TABLE

Metrics related to every table, per partition

NameDescription
Custom Metrics/VoltDB/Stats/Table/{SITE_ID}/{PARTITION_ID}/{TABLE_NAME}/PERCENT_FULLThe percentage of the row limit currently in use by table rows in this partition
Custom Metrics/VoltDB/Stats/Table/{SITE_ID}/{PARTITION_ID}/{TABLE_NAME}/STRING_DATA_MEMORYThe total memory, in kilobytes, used for storing non-inline variable length data
Custom Metrics/VoltDB/Stats/Table/{SITE_ID}/{PARTITION_ID}/{TABLE_NAME}/TUPLE_ALLOCATED_MEMORYThe total size of memory, in kilobytes, allocated for storing inline data associated with this table in this partition
Custom Metrics/VoltDB/Stats/Table/{SITE_ID}/{PARTITION_ID}/{TABLE_NAME}/TUPLE_COUNTThe number of rows currently stored for this table in the current partition
Custom Metrics/VoltDB/Stats/Table/{SITE_ID}/{PARTITION_ID}/{TABLE_NAME}/TUPLE_DATA_MEMORYThe total memory, in kilobytes, used for storing inline data associated with this table in this partition
Custom Metrics/VoltDB/Stats/Table/{SITE_ID}/{PARTITION_ID}/{TABLE_NAME}/TUPLE_LIMITThe row limit for this table

For more info on stats please visit http://voltdb.com/docs/UsingVoltDB/sysprocstatistics.php

 

Custom Dashboard

 

Contributing

 

Always feel free to fork and contribute any changes directly via GitHub.

 

Support

 

For any support questions, please contact help@appdynamics.com.

 

Version:

1.0.0

Compatibility:

3.7 and above

Last Update:

5/14/2014