Wednesday, July 7, 2010

Jboss Monitoring Using Zabbix

Monitoring of JBoss Application is required for delivering comprehensive fault management and proactive alarm notifications, checking for impending problems, triggering appropriate actions and gathering performance data for planning, analysis, and reporting.

Some of the components that can be monitored in JBoss are:
·                  JVM Heap & Non Heap Memory Usage
·                  Garbage Collection Information
·                  Class Loading and Unloading Information
·                  Running Thread Information
·                  Server Response Time
·                  Enterprise JavaBeans (EJBs)
·                  Metrics of all web applications
·                  Java Database Connectivity (JDBC) Pools
·                  Hardware and Operating system information (disk/cpu/memory/load, etc..)
·                  Other Jboss attributes

Open source Available tools which can be used for monitoring are Zabbix, Nagios and Zennos. In this blog, I have used Zabbix (Low cost effective solution) Open source distributed monitoring tool for Jboss and Linux System monitoring. Zabbix supports both polling and trapping techniques to collect data from monitored hosts. A flexible notification mechanism allows easy and quickly configure different types of notifications for pre-defined events.

Jboss Application server monitoring is logically divided into two sections:
 Java, which include JBoss and JVM Monitoring.
 Hardware & Operating System Monitoring.

Jboss and JVM monitoring
JMX stands for Java Management Extensions, and is a facility to allow for remote clients to connect to a JVM, and manage/monitor running applications in that JVM. This management is typically done through MBeans. MBeans are the heart of the JMX specification. 

In Jboss, JMX is being configured by enabling the JVM setting in "$JBOSS_HOME/bin/run.conf" file with the below mentioned options (stated JMX on 9999 port):

In order to monitor the Jboss information, JMX defines a method for Java developers to integrate their applications with existing network management software by dynamically assigning Java objects with management attributes and operations.

In Jboss there are two ways that can grab the jmx attribute values.
 Jboss Twiddle utility: A simple command line tool that allows for interaction with a remote JMX server instance. This tool is called twiddle (for twiddling bits via JMX) and is located in the bin directory of the Jboss distribution. The only issue with this tool is it provides very basic information which is not sufficient for monitoring all Jboss parameters.

 JMX Command line utility: This utility allows system administrators to retrieve JMX management data by querying Mbeans directly on Java applications such as memory use, class loaded, garbage collection counts, etc.

I have used Skajla-JMX command line utility (Open Source JMX command line utility specially designed for monitoring Jboss application ) allows the administrator to instrument their Java application with a Zabbix agent. Below mentioned is the monitoring architecture.

To capture JVM parameter, first check all available message bean parameters which user can capture using this JMX command line utility, below mentioned is the command:
#java -jar skajla-JMXClient.jar 9999 info

e.g Lets say we want to capture ‘PS Perm Gen init memory’, then below mentioned is the command
#java -jar skajla-JMXClient.jar 9999 admin jmxpassword 'java.lang:type=MemoryPool,name=PS Perm Gen' CollectionUsage init

To check the syntax of skajla-JMX client, do the following
# java -jar skajla-JMXClient.jar 
Usage :

JMX Command input parameter detail
JMX Server IP : IP Address of the jmx server  
JMX PORT      : Port on which JMX service is running
JMX USER      : Username required to connect to the JMX  
JMX PASSWORD  : Password required to connect to the JMX  
MESSAGE BEAN  : Target Bean for which attribute Value is required  
COMMAND       : Operation to run or attribute to fetch, Attributes begin with
a capital letter.  
ATTRIBUTE     : Optional, require in the cases when command output into multiple values

Skajla-JMX Command Line Utility and Zabbix Monitoring Architecture

Monitoring Network Architecture
In order to monitor the Jboss Server, Below mentioned is the minimal Server requirement (Fig. 2 Minimal Monitoring Network Architecture :
 Zabbix Server
 Zabbix Database Server
 Zabbix Web Server
 Monitoring Server (Depends on the Business Requirement)

Zabbix Agent Configuration for jboss Monitoring
Below mentioned is the required configuration, which we need to add in the zabbix agent configuration in  zabbix_agentd.conf file.
Step-1: Add the User Parameter to zabbix agentd configuration file.
User parameter has the following syntax:
Unique Item Key
Command to be executed to evaluate value of the key.

Example : To monitoring Perm Gen init memory, below mentioned is the required configuration
UserParameter=permgen_init, java -jar skajla-JMXClient.jar 9999 admin jmxpass 'java.lang:type=MemoryPool,name=PS Perm Gen' CollectionUsage init

Note: The advantage of using skajla-jmxclient is that it gives only the required value of queried MBean parameter, which eases to capture the information for monitoring.

You can easily get other parameter values which you want to monitor by just replacing values of ”java.lang:type=MemoryPool,name=PS Perm Gen with required parameters” in above mentioned userParameter configuration

Step-2: Restart the Zabbix agentd
Step-3: Configuration of Zabbix frontend
Step-A: Add monitoring items
Login the zabbix frontend
|_ Configuration
|_ Host
|_ Select monitored server
|_ Items->Create Item

To Configure the item, the key point which you have to take care is that,you must set the item key same as set in UserParameter in zabbix_agentd.conf and item type must be Zabbix agent
Step-B: Create Graphs
Login the zabbix frontend
|_ Select host
|_ Graph
|_ Create Graph

JBoss Monitoring Graphs (Also Defined in the Linux Jboss Template)


 Download Linux Jboss Template
 Download JMXClient Executable JAR of File:
 Download JMXClient Source Code: