Showing posts with label JMX. Show all posts
Showing posts with label JMX. Show all posts

Monday, November 17, 2014

Monitor cpu usage per thread in java...

There is a thread on SO , from which I discovered the TopThreads plugin for JConsole:

http://arnhem.luminis.eu/new_version_topthreads_jconsole_plugin/

jconsole -J-Djava.class.path=C:\Oracle\MIDDLE~1\JDK160~1\lib\jconsole.jar;C:\Oracle\MIDDLE~1\JDK160~1\lib\tools.jar;C:\Oracle\Middleware\wlserver_10.3\server\lib\wlfullclient.jar -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote -pluginpath "c:\pierre\downloads\topthreads-1.1.jar" -debug

in the jconsole output you should see "Plugin class net.luminis.jmx.topthreads.PluginAdapter loaded."

and here you are: AWESOME!!!


Surely you can script it with ThreadJMX MBean, to create a monitoring tool, but this JConsole UI is so nice...

Also great is this trick: run TOP, then SHIFT-H, take the decimal value of the threads who eat more CPU, convert to HEX and match it to the nid in a jstack threadDump
http://code.nomad-labs.com/2010/11/18/identifying-which-java-thread-is-consuming-most-cpu/

Wednesday, December 4, 2013

JMXTerm and WebLogic

Download the uberjar http://wiki.cyclopsgroup.org/jmxterm/download (I love that, no need to have complicated classpaths). This link worked for me.

On how to configure WebLogic for JMS see here.

This is a sample session:
java -jar jmxterm-1.0-alpha-4-uber.jar
#this will connect remotely
open myserver.acme.com:8888
#this will list all available beans
beans
#this will filter only com.bea stuff
domain com.bea
#this will set your current bean
bean com.bea:Name=wlsbjmsrpDataSource,ServerRuntime=osbpl1as,Type=JDBCDataSourceRuntime
#this will get an attrbute from current bean
get ActiveConnectionsCurrentCount


For a more complete example: Here a lot of useful commands.



Thursday, November 28, 2013

Java, JMX, WebLogic and DomainRuntime

I try to run the code:

http://docs.oracle.com/cd/E12840_01/wls/docs103/jmx/accessWLS.html#wp1111297

I get this error:
java.net.MalformedURLException: Unsupported protocol: t3
I add "C:\Oracle\Middleware\wlserver_10.3\server\lib\wljmxclient.jar" to the classpath

This time I receive a
"org.omg.CORBA.COMM_FAILURE: vmcid: SUN minor code: 211 completed: No" caused by java.io.IOException: End-of-stream at com.sun.corba.se.impl.transport.SocketOrChannelConnectionImpl.readFully(SocketOrChannelConnectionImpl.java:666)

This is apparently an issue in the JVM (see Doc ID 1598451.1 ) ... they say one can add -Dcom.sun.CORBA.connection.ORBHighWaterMark=300 ... personally I give up in frustration.

Friday, November 22, 2013

WebLogic JMX remote using JMX protocol and JMXTrans

Make sure this is enabled in your domain configuration:
a) [domain name]->[Configuration][General]->[Advanced]
[X] Platform MBean Server Enabled
[X] Platform MBean Server Used

b) [domain name]->[Security][General]
[X] Anonymous Admin Lookup Enabled




make sure each server listens on a different jmx port:
vi /opt/oracle/domains/osbpl1do/bin/setDomainEnv.sh

if [ "${SERVER_NAME}" = "osbpl1ms1" ] ; then
JAVA_OPTIONS="${JAVA_OPTIONS} -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=8888 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Djavax.management.builder.initial=weblogic.management.jmx.mbeanserver.WLSMBeanServerBuilder"
fi


if [ "${SERVER_NAME}" = "osbpl1as" ] ; then
JAVA_OPTIONS="${JAVA_OPTIONS} -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=8889 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Djavax.management.builder.initial=weblogic.management.jmx.mbeanserver.WLSMBeanServerBuilder"
fi



restart your servers

To monitor a DataSource Active Connections:

vi myds.json:

{
  "servers" : [ {
    "port" : "25006",
    "host" : "myhost.acme.com",
    "queries" : [ {
      "outputWriters" : [ {
        "@class" : "com.googlecode.jmxtrans.model.output.StdOutWriter",
        "settings" : {
        }
      } ],
      "obj" : "com.bea:ServerRuntime=acme-server-1,Name=acme-import.PickupPointDS,Type=JDBCDataSourceRuntime",
      "attr" : [ "ActiveConnectionsCurrentCount" ]
    }, {
      "outputWriters" : [ {
        "@class" : "com.googlecode.jmxtrans.model.output.StdOutWriter",
        "settings" : {
        }
      } ],
      "obj" : "java.lang:name=CMS Old Gen,type=MemoryPool",
      "attr" : [ "Usage" ]
    }, {
      "outputWriters" : [ {
        "@class" : "com.googlecode.jmxtrans.model.output.StdOutWriter",
        "settings" : {
        }
      } ],
      "obj" : "java.lang:name=ConcurrentMarkSweep,type=GarbageCollector",
      "attr" : [ "LastGcInfo" ]
    } ],
    "numQueryThreads" : 2
  } ]
}



./jmxtrans.sh start myds.json

you should get this

Result [attributeName=ActiveConnectionsCurrentCount, className=weblogic.jdbc.common.internal.DataSourceRuntimeMBeanImpl, typeName=Name=acme-import.PickupPointDS,ServerRuntime=acme-server-1,Type=JDBCDataSourceRuntime, values={ActiveConnectionsCurrentCount=0}, epoch=1385119401592]

To find the ObjectName (like: com.bea:ServerRuntime=acme-server-1,Name=acme-import.PickupPointDS,Type=JDBCDataSourceRuntime ) you connect to the Managed Server with WLST, serverRuntime(), then you cd to the MBean and you do "cmo", this will tell you the ObjectName. The attribute can be discovered wish "ls".

Saturday, November 2, 2013

JMXTrans and Graphite: getting started

You can achieve very decent monitoring of a Java application - including the JVM itself - with jmxtrans and graphite .

No more horrible hand-crafted tool to extract and chart JMS data.

A TRANS in Italian is a transexual, typically Brazilian - more knows as VIADOS - so hearing JMXTrans always puts me in a good mood because Italians like to make jokes about sex.

OK, let's be serious.
First read the excellent jmxtrans Wiki here.
As suggested by jmstrans site, I listened to the presentation by Coda Hale but frankly it didn't say much new, he speaks well but not very informative

Getting started:
download the tar.gz here

tar xvzf jmxtrans-242.tar.gz
cd jmxtrans-242/rpm
.... mmm when I run build.sh it asks me a release and version?????


I will download a rpm from here (instructions on the site are incomplete... not good...)

rpm -i jmxtrans-20121016.145842.6a28c97fbb-0.noarch.rpm

I verify the installation parameters:
less /etc/sysconfig/jmxtrans

# configuration file for package jmxtrans
export JAR_FILE="/usr/share/jmxtrans/jmxtrans-all.jar"
export LOG_DIR="/var/log/jmxtrans"
export SECONDS_BETWEEN_RUNS=60
export JSON_DIR="/var/lib/jmxtrans"
export HEAP_SIZE=512
export NEW_SIZE=64
export CPU_CORES=2
export NEW_RATIO=8
export LOG_LEVEL=debug


cd /usr/share/jmxtrans
./jmxtrans.sh
Usage: ./jmxtrans.sh {start|stop|restart|status} [filename.json]


I look into an example.json
{
  "servers" : [ {
    "port" : "1099",
    "host" : "w2",
    "queries" : [ {
      "outputWriters" : [ {
        "@class" : "com.googlecode.jmxtrans.model.output.StdOutWriter",
        "settings" : {
        }
      } ],
      "obj" : "java.lang:type=Memory",
      "attr" : [ "HeapMemoryUsage", "NonHeapMemoryUsage" ]
    }, {
      "outputWriters" : [ {
        "@class" : "com.googlecode.jmxtrans.model.output.StdOutWriter",
        "settings" : {
        }
      } ],
      "obj" : "java.lang:name=CMS Old Gen,type=MemoryPool",
      "attr" : [ "Usage" ]
    }, {
      "outputWriters" : [ {
        "@class" : "com.googlecode.jmxtrans.model.output.StdOutWriter",
        "settings" : {
        }
      } ],
      "obj" : "java.lang:name=ConcurrentMarkSweep,type=GarbageCollector",
      "attr" : [ "LastGcInfo" ]
    } ],
    "numQueryThreads" : 2
  } ]
}

(I still need to decide if I hate more JSON or XML... they both hurt the eye...)

I write a sample Java class to monitor:
vi Sample.java
public class Sample {
        public static void main(String[] args) {
            try {
                Thread.sleep(100000000);
            }
            catch (Exception e) {};
        }
}


javac Sample.java
java -Dcom.sun.management.jmxremote.port=1105 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false Sample

cp example.json sample.json
I edit port number and host to match 1105 and my localhost

./jmxtrans.sh start sample.json

this runs in background :
ps -ef | grep jmxtrans
root 2240 1 6 07:47 pts/1 00:00:15 /usr/bin/java -server -Djava.awt.headless=true -Djava.net.preferIPv4Stack=true -Djmxtrans.log.level=debug -Djmxtrans.log.dir=. -Xms512M -Xmx512M -XX:+UseConcMarkSweepGC -XX:NewRatio=8 -XX:NewSize=64m -XX:MaxNewSize=64m -XX:MaxTenuringThreshold=16 -XX:GCTimeRatio=9 -XX:PermSize=384m -XX:MaxPermSize=384m -XX:+UseTLAB -XX:CMSInitiatingOccupancyFraction=85 -XX:+CMSIncrementalMode -XX:+CMSIncrementalPacing -XX:ParallelGCThreads=1 -Dsun.rmi.dgc.server.gcInterval=28800000 -Dsun.rmi.dgc.client.gcInterval=28800000 -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=2101 -jar jmxtrans-all.jar -e -f sample.json -s 60


and if I do
tail -f jmxtrans.log
I get a line per minute:
[02 Nov 2013 07:47:22] [main] 0 INFO (com.googlecode.jmxtrans.JmxTransformer:134) - Starting Jmxtrans on : sample.json
[02 Nov 2013 07:47:23] [main] 1125 DEBUG (com.googlecode.jmxtrans.JmxTransformer:354) - Loaded file: /usr/share/jmxtrans/sample.json
[02 Nov 2013 07:47:23] [main] 1154 DEBUG (com.googlecode.jmxtrans.JmxTransformer:429) - Scheduled job: osb-vagrant.acme.com:1105-1383374843215-0158449004 for server: Server [host=osb-vagrant.acme.com, port=1105, url=null, cronExpression=null, numQueryThreads=2]
[02 Nov 2013 07:47:23] [ServerScheduler_Worker-1] 1167 DEBUG (com.googlecode.jmxtrans.jobs.ServerJob:31) - +++++ Started server job: Server [host=osb-vagrant.acme.com, port=1105, url=null, cronExpression=null, numQueryThreads=2]
[02 Nov 2013 07:47:23] [ServerScheduler_Worker-1] 1690 DEBUG (com.googlecode.jmxtrans.util.JmxUtils:102) - ----- Creating 3 query threads
[02 Nov 2013 07:47:24] [pool-1-thread-1] 1940 DEBUG (com.googlecode.jmxtrans.util.JmxUtils:195) - Executing queryName: java.lang:type=Memory from query: Query [obj=java.lang:type=Memory, resultAlias=null, attr=[HeapMemoryUsage, NonHeapMemoryUsage]]
[02 Nov 2013 07:47:24] [pool-1-thread-1] 2002 DEBUG (com.googlecode.jmxtrans.util.JmxUtils:209) - Finished running outputWriters for query: Query [obj=java.lang:type=Memory, resultAlias=null, attr=[HeapMemoryUsage, NonHeapMemoryUsage]]
[02 Nov 2013 07:47:24] [ServerScheduler_Worker-1] 2006 DEBUG (com.googlecode.jmxtrans.jobs.ServerJob:50) - +++++ Finished server job: Server [host=osb-vagrant.acme.com, port=1105, url=service:jmx:rmi:///jndi/rmi://osb-vagrant.acme.com:1105/jmxrmi, cronExpression=null, numQueryThreads=2]

Next step: install grafite and plot these metrics....

Saturday, August 25, 2012

WebLogic, detecting stuck threads with JMX and Java

Surprisingly, the "Health" reported in the Servers monitoring tab





is NOT the ServerRuntime.HealthState, but rather the

/serverRuntime/ThreadPoolRuntime/ThreadPoolRuntime.HealthState

the parent of this MBean is com.bea:Name=osbpl1ms1,Type=ServerRuntime, its type and name are ThreadPoolRuntime

which also contains a weblogic.health.HealthState

For this reason, to the code of the previous post I have added a new method:

    public static String getThisServerThreadPoolHealth() throws Exception {
 MBeanServerConnection connection = initConnection();
 ObjectName runtimeService = new ObjectName("com.bea:Name=RuntimeService,Type=weblogic.management.mbeanservers.runtime.RuntimeServiceMBean");
 String managedServerName = (String) connection.getAttribute(runtimeService, "ServerName");
 ObjectName serverRT = new ObjectName("com.bea:Name=" + managedServerName + ",Type=ServerRuntime");
 System.out.println("serverRT=" + serverRT);
 ObjectName serverTP = (ObjectName)connection.getAttribute(serverRT, "ThreadPoolRuntime");
 weblogic.health.HealthState tpHealthState = (weblogic.health.HealthState) connection.getAttribute(serverTP, "HealthState");
 return healthStateToString(tpHealthState.getState());
    }





Friday, August 24, 2012

WebLogic finding health state through JMX

This post is excellent and explains how to connect remotely (I don't like the static block thing).

This one covers better the "Local" case.

If you want to connect LOCALLY, things are explained here:

"Make Local Connections to the Runtime MBean Server.
Local clients can access a WebLogic Server instance's Runtime MBean Server through the JNDI tree instead of constructing a JMXServiceURL object"

The code is:
import javax.management.MBeanServer;
....
InitialContext ctx = new InitialContext();
MBeanServer server = (MBeanServer)ctx.lookup("java:comp/env/jmx/runtime"); 


Javadoc of the MBeanServer here:
http://docs.oracle.com/javase/6/docs/api/javax/management/MBeanServer.html


Here is the code for the "local" case (I have also removed the static initialization block):


package tests.monitoring;

import javax.management.MBeanServer;
import javax.management.MBeanServerConnection;
import javax.management.MalformedObjectNameException;
import javax.management.ObjectName;
import javax.management.remote.JMXConnector;
import javax.naming.InitialContext;
import javax.naming.NamingException;

public class ServerHealthStateMonitorLocal {

    private static MBeanServerConnection connection;
    private static JMXConnector connector;
    private static ObjectName service;
    private static String combea = "com.bea:Name=";
    private static String service1 = "DomainRuntimeService,Type=weblogic.management.mbeanservers.domainruntime.DomainRuntimeServiceMBean";


    public static void initConnection() throws NamingException, MalformedObjectNameException, NullPointerException {
 service = new ObjectName(combea + service1);
 InitialContext ctx = new InitialContext();
 MBeanServer server = (MBeanServer)ctx.lookup("java:comp/jmx/runtime");
  
 connection = server;
 System.out.println("connection = " + connection);
    }

    public static ObjectName[] getServerRuntimes() throws Exception {
 if (connection == null) {
     System.out.println("null connection");
 }
 
 return (ObjectName[]) connection.getAttribute(service, "ServerRuntimes");
    }

    public void printNameAndState() throws Exception {
 ObjectName arr[] = getServerRuntimes();
 for (ObjectName temp : arr)
     System.out.println("\n\t servers: " + temp);
 ObjectName domain = (ObjectName) connection.getAttribute(service, "DomainConfiguration");
 System.out.println("Domain: " + domain.toString());
 ObjectName[] servers = (ObjectName[]) connection.getAttribute(domain, "Servers");
 for (ObjectName server : servers) {
     String aName = (String) connection.getAttribute(server, "Name");
     try {
  ObjectName ser = new ObjectName("com.bea:Name=" + aName + ",Location=" + aName + ",Type=ServerRuntime");
  String serverState = (String) connection.getAttribute(ser, "State");
  System.out.println("\n\t Server: " + aName + "\t State: " + serverState);
  weblogic.health.HealthState serverHealthState = (weblogic.health.HealthState) connection.getAttribute(ser, "HealthState");
  int hState = serverHealthState.getState();
  if (hState == weblogic.health.HealthState.HEALTH_OK)
      System.out.println("\t Server: " + aName + "\t State Health: HEALTH_OK");
  if (hState == weblogic.health.HealthState.HEALTH_WARN)
      System.out.println("\t Server: " + aName + "\t State Health: HEALTH_WARN");
  if (hState == weblogic.health.HealthState.HEALTH_CRITICAL)
      System.out.println("\t Server: " + aName + "\t State Health: HEALTH_CRITICAL");
  if (hState == weblogic.health.HealthState.HEALTH_FAILED)
      System.out.println("\t Server: " + aName + "\t State Health: HEALTH_FAILED");
  if (hState == weblogic.health.HealthState.HEALTH_OVERLOADED)
      System.out.println("\t Server: " + aName + "\t State Health: HEALTH_OVERLOADED");
     } catch (javax.management.InstanceNotFoundException e) {
  System.out.println("\n\t Server: " + aName + "\t State: SHUTDOWN (or Not Reachable)");
     }
 }
    }

    public static void main(String[] args) throws Exception {
 printHealthStatus();
    }
    
    public static void printHealthStatus() throws Exception {
 ServerHealthStateMonitorLocal s = new ServerHealthStateMonitorLocal();
 initConnection();
 s.printNameAndState();
 connector.close();
    }

}





HealthState is described here, and the meaning of the integer values is

HEALTH_OK  0
HEALTH_WARN  1
HEALTH_CRITICAL  2
HEALTH_FAILED  3
HEALTH_OVERLOADED  4





The first time I get:



com.bea:Name=DomainRuntimeService,Type=weblogic.management.mbeanservers.domainruntime.DomainRuntimeServiceMBean
javax.management.InstanceNotFoundException: com.bea:Name=DomainRuntimeService,Type=weblogic.management.mbeanservers.domainruntime.DomainRuntimeServiceMBean
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1094)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:662)
at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:638)
at weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$12.run(WLSMBeanServerInterceptorBase.java:326)
at java.security.AccessController.doPrivileged(Native Method)
at weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.getAttribute(WLSMBeanServerInterceptorBase.java:324)
at weblogic.management.mbeanservers.internal.JMXContextInterceptor.getAttribute(JMXContextInterceptor.java:157)
at weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$12.run(WLSMBeanServerInterceptorBase.java:326)
at java.security.AccessController.doPrivileged(Native Method)
at weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.getAttribute(WLSMBeanServerInterceptorBase.java:324)
at weblogic.management.mbeanservers.internal.SecurityInterceptor.getAttribute(SecurityInterceptor.java:299)
at weblogic.management.jmx.mbeanserver.WLSMBeanServer.getAttribute(WLSMBeanServer.java:279)
at tests.monitoring.ServerHealthStateMonitorLocal.getServerRuntimes(ServerHealthStateMonitorLocal.java:34)



the connection is found, but the ObjectName
"DomainRuntimeService,Type=weblogic.management.mbeanservers.domainruntime.DomainRuntimeServiceMBean" is not available in the MBeanServer

the offending code is:

return (ObjectName[]) connection.getAttribute(service, "ServerRuntimes");

I try then in the Domain configuration to set "Platform MBean Server Enabled"
I restart the servers.
Still same error.



Then I try:

MBeanServer server = (MBeanServer)ctx.lookup("java:comp/jmx/runtime");
instead of
MBeanServer server = (MBeanServer)ctx.lookup("java:comp/env/jmx/runtime");

and still the same error...

The problem is that the Domain Runtime MBean Server is registered only on the Admin.... the code works on the Admin, but NOT on a Managed Server.

The issue is that we want to be able to retrieve the Server Health even if the Admin is not available (besides, it's a pain to store the weblogic credentials for a local call...).

Ok, my dear, then how do I get a reference to the ServerRuntime, if I don't have a reference of the DomainRuntime?

Here they explain it:

you need a ObjectName "RuntimeService,Type=weblogic.management.mbeanservers.runtime.RuntimeServiceMBean"

"Provides an entry point for navigating the hierarchy of WebLogic Server runtime MBeans and active configuration MBeans for the current server."

The utilization is quite rounbdaboutish and not at all intuitive, but here is something that definitely works:

package tests.monitoring;

import javax.management.MBeanServer;
import javax.management.MBeanServerConnection;
import javax.management.ObjectName;
import javax.naming.InitialContext;

public class ServerHealthStateMonitorLocal {

    public static MBeanServerConnection initConnection() throws Exception {
 MBeanServerConnection connection;
 System.out.println("initConnection");
 InitialContext ctx = new InitialContext();
 System.out.println("ctx");
 try {
     connection = (MBeanServer) ctx.lookup("java:comp/env/jmx/runtime");
 }
 catch (Exception e) {
     e.printStackTrace();
     try {
  connection = (MBeanServer) ctx.lookup("java:comp/jmx/runtime");
     }
     catch (Exception e2) {
  e2.printStackTrace();
  throw e2;
     }
 }

 System.out.println("connection=" + connection);
 return connection;
    }

    private static ObjectName getServerRuntime(MBeanServerConnection connection, String aName) throws Exception {
 System.out.println("getServerRuntime");
 ObjectName serverObjectName = new ObjectName("com.bea:Name=" + aName + ",Type=ServerRuntime");
 return (ObjectName) connection.getAttribute(serverObjectName, "ServerRuntime");
    }


    public static String getHealthOfThisServer() throws Exception {
 ServerHealthStateMonitorLocal s = new ServerHealthStateMonitorLocal();
 MBeanServerConnection connection = initConnection();
 String serverName = System.getProperty("weblogic.Name");
 return s.getHealthOfServer(serverName);
    }

    private String getHealthOfServer(String aName) throws Exception {
 String result = "NOT_FOUND";
 MBeanServerConnection connection = initConnection();
 ObjectName ser = getServerRuntime(connection, aName);
 String serverState = (String) connection.getAttribute(ser, "State");
 result = serverState;
 weblogic.health.HealthState serverHealthState = (weblogic.health.HealthState) connection.getAttribute(ser, "HealthState");
 int hState = serverHealthState.getState();
 result = result + "_" + hState;
 return result;
    }
    
    public static String getThisServerName() throws Exception {
 ServerHealthStateMonitorLocal s = new ServerHealthStateMonitorLocal();
 MBeanServerConnection connection = initConnection();
 ObjectName runtimeService = new ObjectName("com.bea:Name=RuntimeService,Type=weblogic.management.mbeanservers.runtime.RuntimeServiceMBean");
 String managedServerName = (String) connection.getAttribute(runtimeService, "ServerName");
 return managedServerName;
 
 /*
 ObjectName msServerRuntime = new ObjectName("com.bea:Name="+ managedServerName + ",Type=ServerRuntime");
   administrationURL = (String) mBeanServer.getAttribute(msServerRuntime, "AdminServerHost");
   adminPort = (Integer) mBeanServer.getAttribute(msServerRuntime, "AdminServerListenPort");
   System.out.println(administrationURL + adminPort);
 */  
    } 
    
    public static String getThisServerState() throws Exception {
 MBeanServerConnection connection = initConnection();
 ObjectName runtimeService = new ObjectName("com.bea:Name=RuntimeService,Type=weblogic.management.mbeanservers.runtime.RuntimeServiceMBean");
 System.out.println("runtimeService=" + runtimeService);
 ObjectName serverRT = (ObjectName) connection.getAttribute(runtimeService, "ServerRuntime");
 System.out.println("serverRT=" + serverRT);
 String serverState = (String) connection.getAttribute(serverRT, "State");
 return serverState;
    }

    public static String getThisServerHealth() throws Exception {
 MBeanServerConnection connection = initConnection();
 ObjectName runtimeService = new ObjectName("com.bea:Name=RuntimeService,Type=weblogic.management.mbeanservers.runtime.RuntimeServiceMBean");
 String managedServerName = (String) connection.getAttribute(runtimeService, "ServerName");
 ObjectName serverRT = new ObjectName("com.bea:Name=" + managedServerName + ",Type=ServerRuntime");
 System.out.println("serverRT=" + serverRT);
 weblogic.health.HealthState serverHealthState = (weblogic.health.HealthState) connection.getAttribute(serverRT, "HealthState");
 return healthStateToString(serverHealthState.getState());
    }
    
    private static String healthStateToString(int state) {
 String result = "HEALTH_UNKNOWN";
 switch (state) {
 case 0:
     result = "HEALTH_OK";
     break;
 case 1:
     result = "HEALTH_WARN";
     break;
 case 2:
     result = "HEALTH_CRITICAL";
     break;
 case 3:
     result = "HEALTH_FAILED";
     break;
 case 4:
     result = "HEALTH_OVERLOADED";
     break;

 default:
     break;
 }
 return result;
    }
    
    
}




where:

runtimeService=com.bea:Name=RuntimeService,Type=weblogic.management.mbeanservers.runtime.RuntimeServiceMBean
serverRT=com.bea:Name=osbpl2ms1,Type=ServerRuntime




Thursday, May 3, 2012

WebLogic monitoring with JMX and JConsole

This great post explains you everything


cd C:\Oracle\Middleware\wlserver_10.3\server\lib
java -jar wljarbuilder.jar
echo %JAVA_HOME%

C:\Oracle\MIDDLE~1\JDK160~1
cd %JAVA_HOME%
cd bin
jconsole -J-Djava.class.path=C:\Oracle\MIDDLE~1\JDK160~1\lib\jconsole.jar;C:\Oracle\MIDDLE~1\JDK160~1\lib\tools.jar;C:\Oracle\Middleware\wlserver_10.3\server\lib\wlfullclient.jar -J-Djmx.remote.protocol.provider.pkgs=weblogic.management.remote -debug



(make sure you enter it like that, with the -J-D which looks very weird)

when jconsole comes up, choose remote connection,
provide
service:jmx:iiop://yourhost:yourport/jndi/weblogic.management.mbeanservers.runtime

(your port is NOT the jmx port provided in -Dcom.sun.management.jmxremote.port=8888, but the listen port of the WLS)
and enter your weblogic username and password to authenticate


You can also find MBean Object Names with weblogic.Admin:


java weblogic.Admin -url t3://myserver.acme.com:8105 -username Pierluigi -password weblogic1 -pretty GET -type ClusterRuntime


and the MBean name is osbdv2do:ServerRuntime=osbdv2ms1,Name=osbdv2cl,Type=ClusterRuntime,Location=osbdv2ms1



---------------------------
MBeanName: "osbdv2do:ServerRuntime=osbdv2ms1,Name=osbdv2cl,Type=ClusterRuntime,Location=osbdv2ms1"
ActiveSingletonServices:
AliveServerCount: 2
CachingDisabled: true
CurrentMachine: myserver
CurrentSecondaryServer: -7551940697957911746S:myserver.acme.com:[8205,8205,-1,-1,-1,-1,-1]:myserver.acme.com:8105,myserver.acme.com:8205:osbdv2do:osbdv2ms2
DetailedSecondariesDistribution:
ForeignFragmentsDroppedCount: 0
FragmentsReceivedCount: 696
FragmentsSentCount: 805
HealthState: Component:null,State:HEALTH_OK,MBean:null,ReasonCode:[]
JobSchedulerRuntime:
MulticastMessagesLostCount: 2
Name: osbdv2cl
Parent: osbdv2ms1
PrimaryCount: 0
Registered: true
ResendRequestsCount: 1
SecondaryCount: 0
SecondaryDistributionNames:
SecondaryServerDetails: -7551940697957911746S:myserver.acme.com:[8205,8205,-1,-1,-1,-1,-1]:myserver.acme.com:8105,myserver.acme.com:8205:osbdv2do:osbdv2ms2
ServerMigrationRuntime: ServerMigrationRuntime
ServerNames: osbdv2ms2,osbdv2ms1
Type: ClusterRuntime
UnicastMessaging: UnicastMessagingRuntime




wlst jmx objectname for clusterruntime:

This is the way to find the OBJECT_NAME associated to a MBean with WLST:

http://docs.oracle.com/cd/E23943_01/web.1111/e13715/nav_edit.htm#i1001906

java weblogic.WLST

connect(...)
serverRuntime()
easeSyntax()
cd ClusterRuntime
cd osbdv2cluster
cmo

[MBeanServerInvocationHandler]com.bea:ServerRuntime=osbdv2ms1,Name=osbdv2cluster,Type=ClusterRuntime

so com.bea:ServerRuntime=osbdv2ms1,Name=osbdv2cluster,Type=ClusterRuntime is the OBJECT_NAME

you can extract attributes: ServerNames et AliveServerCount


You can test if your OBJECT_NAME is correct with "weblogic.Admin GET -mbean":

http://docs.oracle.com/cd/E13222_01/wls/docs81/admin_ref/cli30.html#1035148

java weblogic.Admin -url t3://myserver.acme.com:8105 -username pippo -password weblogic1 -pretty GET -mbean osbdv2do:ServerRuntime=osbdv2ms1,Name=osbdv2cl,Type=ClusterRuntime,Location=osbdv2ms1


To enable remote JMX connection, in $DOMAIN_HOME/bin/setDomainEnv.sh, almost at the bottom, add:

if [ "${SERVER_NAME}" = "osbdv2ms1" ] ; then
JAVA_OPTIONS="${JAVA_OPTIONS} -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=8888 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false"
fi


Here a good article on how to expose an interface through a MBean, including the HtmlAdaptorServer

Sunday, January 29, 2012

Counting JMS messages present in a JMS Queue

here we show 2 ways:

(I have a wlsbJMSServer JMSServer, a uoo JMSModule, a uooq JNDI name for the Queue)

WLST

connect(....)

cd serverRuntime:/JMSRuntime/AdminServer.jms/JMSServers/wlsbJMSServer/Destinations/uoo!uooq


the cmo is:

[MBeanServerInvocationHandler]com.bea:ServerRuntime=AdminServer,Name=uoo!uooq,Type=JMSDestinationRuntime,JMSServerRuntime=wlsbJMSServer

cmo.getMessagesCurrentCount()



Java

package com.acme.gu.jms;

import java.util.Properties;

import javax.jms.Destination;
import javax.jms.JMSException;
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;

import weblogic.management.runtime.JMSDestinationRuntimeMBean;
import weblogic.jms.extensions.JMSRuntimeHelper;

public class JMSJMX {
 public static void main(String[] args) throws NamingException, JMSException {
  JMSJMX jmsjmx = new JMSJMX();
  jmsjmx.countMessages();
 }
 
 public void countMessages() throws NamingException, JMSException {
  Properties env = new Properties();
  env.put(Context.PROVIDER_URL, "t3://localhost:7001");
  env.put(Context.SECURITY_PRINCIPAL, "weblogic");
  env.put(Context.SECURITY_CREDENTIALS, "weblogic1");
  env.put(Context.INITIAL_CONTEXT_FACTORY, "weblogic.jndi.WLInitialContextFactory");
  InitialContext ctx = new InitialContext(env);

  Destination queue = (Destination) ctx.lookup("uooq");

  JMSDestinationRuntimeMBean destMBean = JMSRuntimeHelper.getJMSDestinationRuntimeMBean(ctx, queue);
  
  System.out.println("count: " + destMBean.getMessagesCurrentCount());
 }
 
}




see also:


http://docs.oracle.com/cd/E11035_01/wls100/javadocs_mhome/weblogic/management/runtime/JMSDestinationRuntimeMBean.html


http://docs.oracle.com/cd/E12840_01/wls/docs103/javadocs/weblogic/jms/extensions/JMSRuntimeHelper.html



If you want to invoke the same code in OSB, you can use this variant:

package com.acme.gu.jms;

 /**
  * Method to be run inside a Container
  * @param jndis
  * @return
  * @throws NamingException 
  * @throws JMSException 
  */
 public static String countCurrentMessages(String[] jndis) throws NamingException, JMSException {
  StringBuffer result = new StringBuffer();
  InitialContext ctx = new InitialContext();

   
  for (int count = 0; count < jndis.length; count++) {
   Destination queue = (Destination) ctx.lookup(jndis[count]);
   JMSDestinationRuntimeMBean destMBean = JMSRuntimeHelper.getJMSDestinationRuntimeMBean(ctx, queue);
   result.append(Long.toString(destMBean.getMessagesCurrentCount()));
   if (count < jndis.length - 1) {
    result.append(",");
   }
  }


  return result.toString();

 }
And you invoke it with a Java Callout, passing in input a Sequence ('jndi1', 'jndi2') which is converted automatically in a String[] I am returning a CSV list (String), because a String[] as a return parameter in converted into a Java object, which is not easy to access in OSB. Make sure you associate a Service Account (static) to the Java Callout, because when you create a new InitialContext without associating credentials, OSB attaches an anonymous user and you get an error: Callout to java method "public static java.lang.String[] com.acme.gu.jms.JMSJMX.countCurrentMessages(java.lang.String[]) throws javax.naming.NamingException,javax.jms.JMSException" resulted in exception: javax.naming.NoPermissionException: User <anonymous> does not have permission on weblogic.management.adminhome to perform lookup operation. weblogic.jms.common.JMSException: javax.naming.NoPermissionException: User <anonymous> does not have permission on weblogic.management.adminhome to perform lookup operation. at weblogic.jms.extensions.JMSRuntimeHelper.getJMSDestinationRuntimeMBean(JMSRuntimeHelper.java:458) at com.acme.gu.jms.JMSJMX.countCurrentMessages(JMSJMX.java:49) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at stages.transform.runtime.JavaCalloutRuntimeStep$1.run(JavaCalloutRuntimeStep.java:173)


The alternative is to enable "Anonymous Admin Lookup Enabled" in "View Domain-wide Security Settings" in WebLogic console.

Tuesday, July 19, 2011

JMS Resources using JMX

here the original code

here the same code after some fixes

weblogic-wonders.com is a great site for WebLogic users... check out also http://middlewaremagic.com

Thursday, September 16, 2010

Log4j and JMX

A correct Logging mechanism / Error Reporting is one of the most vital parts of an IT solution, yet it's one of the most overlooked and let to the individual developer's taste.

This normally leads to frightening GREP nightmares in environments where the application is deployed in a cluster... usually several grepping monkeys (I have been one of them often :o( ) are assigned to the task of grooming the logs in search of error conditions.
 


The adoption of monitoring tools - often grepping logs based on a set of regexps - is normally something which happens very late in the project, and it entails a painful series of flashbacks and retrospectives to remember all the error conditions we have seen in the past, manually troubleshooted and not captured anywhere.

In most projects, logging is configured with a unique log4j.properties or log4j.xml (I prefer the xml).... which is edited by hand and ofter requires a restart of the appserver .... unless you use the propertyconfigurator, which is not something I particularly like. For instance, it becomes impossible to have any automated tests involving logs, because you don't have a way to configure them programmatically. But since logs are Cinderellas, nobody really want to test them.

Anyway I like the idea http://www.theserverlabs.com/blog/2010/04/22/dynamically-changing-log-level-with-weblogic-log4j-jmx-and-wlst/ of using JMX in conjunction with log4j. To be definitely explored in the next project, and that I have seen used in the past.

In the past I have used also a JSP console to change log levels at runtime, something like this:

http://ananthkannan.blogspot.com/2009/10/how-to-change-log-levels-on-fly-using.html

Saturday, September 4, 2010

JMX and Log4J

Tired of fighting with log4j.properties and restarting my application server,
giving also that I deeply dislike the idea of having a Thread dedicated to monitoring whether a log4j.properties file has changed or not, I decide to investigate in the intriguing connections between Log4J and JMX.


This seems a very authoritative article on the specific topic:
http://www.objectsource.com/j2eechapters/Ch06-Logging_Service.htm

and this http://java.sun.com/developer/technicalArticles/J2SE/jmx.html a very general comprehensive coverage of JMX....

I run the Hello example, funnily using JRMC it doesn't work, while with JConsole it works. Interestingly, they recommend to run JConsole remotely, not locally.

Here http://code.google.com/p/javatoolsforweblogic/source/browse/#svn/trunk/PVJMX  the Eclipse project.

Sunday, June 13, 2010

Never again without JMX

Simple tutorial to get started with exposing a Bean with JMX:
http://www.devdaily.com/blog/post/java/source-code-java-jmx-hello-world-application

and here
http://javatoolsforweblogic.googlecode.com/svn/trunk/JMXTest/

the Eclipse project in SVN (remember to set the JVM arguments as specified in the SimpleAgent javadoc)

Run SimpleAgent in Eclipse, run JConsole, connect to SimpleAgent and you can invoke the methods and inspect the attributes via JMX... fabulous!


It's great HtmlAdaptorServer to implement the solution depicted here.

To download the JDMK HtmlAdaptorServer go here: http://java.sun.com/products/jdmk/

Monday, November 2, 2009

Sitescope and Weblogic using JMX

a good introduction on the topic:

http://90kts.com/blog/2008/monitoring-weblogic-using-jmx-in-sitescope/

especially useful is the test using JConsole:

jconsole.exe -J-Dcom.sun.CORBA.transport.ORBTCPReadTimeouts=10:60000:500:10

and connecting to
service:jmx:rmi://jndi/iiop://yourserver:yourport/weblogic.management.mbeanservers.runtime

Anyway Sitescrotum UI is a POC (piece of crap), for instance if you enter the good JMX URL with the wrong weblogic username you still get "invalid URL" error...
frankly pathetic.

I could make Sitescrotum work only by adding
-Dcom.sun.CORBA.transport.ORBTCPReadTimeouts=10:30000:500:10

of course Sitescrotum doesn't give you any warning or error message...
also I didn't provide the optional domain name... also try hitting the "clear selection" and "reload configuration" many times...

Sitescrotum UI sucks!