Products
Solutions
Resources
Customers
Company

Sign in
Account
The Hadoop monitoring extension captures metrics from Hadoop Resource Manager and/or Apache Ambari and displays them in Appdynamics Metric Browser.
This extension works only with the standalone machine agent.
Before the extension is installed, the prerequisites mentioned here need to be met. Please do not proceed with the extension installation if the specified prerequisites are not met.
MachineAgent/monitors
directoryResource Manger Monitor can be used if you have Hadoop Yarn. Please refer to Resource Manager Rest API for more details
<MachineAgent_Dir>/monitors/HadoopMonitor
The metricPrefix of the extension has to be configured as specified here. Please make sure that the right metricPrefix is chosen based on your machine agent deployment, otherwise this could lead to metrics not being visible in the controller.resourceManagerMonitor:
. The following instructions are applicable to this section onlyenabled
: This should be true if you want to collect metrics through AmbarimetricPrefix
: The metrics will be reported under this tree in the metric browser. You don't have to change thisservers
: The sub elements of this section defines the details of the Resource Manager Server. If you are monitoring multiple Resource Managers, then you can add the details one by one.uri
: The uri of the Resource Manager. You only need to change the scheme
, host
and port
in the uriusername
: The username to login to Resource Manager Host. Uncomment if you have authentication enabledpassword
: The password to login to Resource Manager Host. Uncomment if you have authentication enabledname
: Required only if there is more than one server. Add any name to identify a server, preferred name is the host name of the server.applications
: The extension will monitor the state of applications in the given monitoringTimePeriod
. It will report the number of running, finished, failed applications. You can add the applicationType and a regex that matches the application name to monitor.type
: The application Type eg. MAPREDUCEnames
: Includes all [.*]
by default. Can customize, for example to add all application names which start with app1 and app3 use ["app1.*","app3.*"]
monitoringTimePeriod
: (In Minutes) This is in reference with the above applications
element. It will monitor the state of the applications in the given time period.
Ambari Monitor can be used if you have installed the hadoop cluster using Ambari.
<MachineAgent_Dir>/monitors/HadoopMonitor
The metricPrefix of the extension has to be configured as specified here. Please make sure that the right metricPrefix is chosen based on your machine agent deployment, otherwise this could lead to metrics not being visible in the controller.ambariMonitor:
. The following instructions are applicable to this section onlyenabled
: This should be true if you want to collect metrics through AmbarimetricPrefix
: The metrics will be reported under this tree in the metric browser. You don't have to change thisservers
: The sub elements of this section defines the details of the Ambari Host. If you are monitoring multiple Ambari Installations, then you can add the details one by one.uri
: The uri of the Ambari Host. You only need to change the scheme
, host
and port
in the uriusername
: The username to login to Ambari Hostpassword
: The password to login to Ambari Host.name
: Uncomment only if there is more than one server. Add any name to identify a server, preferred name is the host name of the server.filters
: This section defines filters i.e meaning what to include. It supports multiple regular expressions, separated bt comma.To avoid setting the clear text password in the config.yml, please follow the process to encrypt the password and set the encrypted password and the key in the config.yml
java -cp "appd-exts-commons-1.1.2.jar" com.appdynamics.extensions.crypto.Encryptor myKey myPassword
passwordEncrypted
and encryptionKey
fields in config.yml
under the section servers:
instead of the element password
http://host:8088/ws/v1/cluster/metrics
, after replacing the host
and port
.http://host:8080/api/v1/clusters?fields=Clusters,hosts,services
, after replacing the host
and port
.-Dappdynamics.agent.maxMetrics=5000
if there is a metric limit reached error in the logs. If you dont see the expected metrics, this could be the cause.<MachineAgent>/conf/logging/log4j.xml
and update the level of the appender com.appdynamics
to debug Let it run for 5-10 minutes and attach the logs to a support ticketWorkbench is a feature that lets you preview the metrics before registering it with the controller. This is useful if you want to fine tune the configurations. Workbench is embedded into the extension jar.
To use the workbenchjava -jar /path/to/MachineAgent/monitors/HadoopMonitor/hadoop-monitoring-extension.jarThis starts an http server at
http://host:9090/
. This can be accessed from the browser.#Get the stats curl http://localhost:9090/api/stats #Get the registered metrics curl http://localhost:9090/api/metric-paths
config.yml
and validate it from the browser or the APIThe metrics will be reported under the tree Application Infrastructure Performance|$TIER|Custom Metrics|Hadoop Monitor|
Please refer to this screenshot to view the complete list of metrics reported by the Ambari Monitor.
Please refer to this screenshot to view the complete list of metrics reported by the ResourceManager Monitor.
Always feel free to fork and contribute any changes directly via GitHub.
For any questions or feature request, please contact AppDynamics Help.
Version | 2.5.1 |
Machine Agent Compatibility | 4.0+ |
Tested With Ambari Version | 2.2.1.1 |
Tested With ResourceManager Version | 2.7.0 |
Last Update | 2/19/18 |