WebLogic Server Request Performance View

On how to configure the deployment to add a "Context ID" using the wldf-resource/wldf-instrumentation-monitor tag: http://docs.oracle.com/cd/E24329_01/web.1211/e24426/config_context.htm#WLDFC253

How to educate yourself to be a Java Performance Engineer

Here is some articles that I found interesting and relevant:

https://en.wikipedia.org/wiki/Software_performance_testing
https://en.wikipedia.org/wiki/Stress_testing_(software)
https://en.wikipedia.org/wiki/Software_testing
 
http://www.slideshare.net/guru__123/loadrunner-presentation-5204243

http://www.correlsense.com/java-bytecode-instrumentation-an-introduction/
https://docs.oracle.com/javase/7/docs/api/java/lang/instrument/Instrumentation.html
https://docs.oracle.com/javase/7/docs/api/java/lang/instrument/package-summary.html

https://docs.oracle.com/javase/specs/jvms/se7/html/index.html
http://www.artima.com/insidejvm/ed2/jvm2.html
http://blog.jamesdbloom.com/JVMInternals.html

https://en.wikipedia.org/wiki/Java_class_file

https://docs.oracle.com/javase/8/docs/technotes/guides/vm/gctuning/

http://www.oracle.com/webfolder/technetwork/tutorials/obe/java/gc01/index.html

JVM Troubleshooting Guide

I am going through a couple of interesting documents:

This one contains many word of wisdom about strategic approach to parametrization of the JVM...

http://www.javacodegeeks.com/wp-content/uploads/2013/06/JVM_Troubleshooting_Guide.pdf

for instance it's valuable the advice to distinguish applications where mostly short-lived object are created (most Service Bus applications are like that) from Applications which keep long lived objects - for instance HTTP sessions. In the first case, you should adjust the Heap to give more space to the Young generation space. In the second case, give more space to the Tenured space.



This document also contains valuable tips and tools:

http://docs.oracle.com/javase/7/docs/webnotes/tsg/TSG-VM/html/toc.html

WebLogic Request Performance

A great demo from Jambay:


and here is the official Oracle documentation

By default you have a Diagnostic Module Module-FMWDFW containing whese watches:

UncheckedException = (SEVERITY = 'Error') AND ((MSGID = 'WL-101020') OR (MSGID = 'WL-101017') OR (MSGID = 'WL-000802') OR (MSGID = 'BEA-101020') OR (MSGID = 'BEA-101017') OR (MSGID = 'BEA-000802'))

Deadlock=((SEVERITY = 'Critical') OR (SEVERITY = 'Info')) AND ((MSGID = 'WL-000394') OR (MSGID = 'BEA-000394'))

StuckThread=(SEVERITY = 'Error') AND ((MSGID = 'WL-000337') OR (MSGID = 'BEA-000337'))

Linux disk performance benchmark

Suppose you have local and shared disks mounted on your box. You want to find out their relative performance, to assess for instance the performance loss in case you locate your WebLogic files (logs, filestores...) on one or the other.

$ df -kP

Filesystem 1024-blocks Used Available Capacity Mounted on
/dev/mapper/rootvg-rootlv 8125880 1953072 5753380 26% /
/dev/mapper/rootvg-optlv 30472188 24894656 4005004 87% /opt
/dev/mapper/rootvg-homelv 2031440 1939192 0 100% /home
/dev/mapper/rootvg-tmplv 3047184 161068 2728832 6% /tmp
/dev/mapper/rootvg-varlv 9141624 687588 7982244 8% /var
/dev/cciss/c0d0p1 497829 31487 440640 7% /boot
tmpfs 12342596 0 12342596 0% /dev/shm



$ /sbin/hdparm /dev/mapper/rootvg-optlv

/dev/mapper/rootvg-optlv: Permission denied

(hdparm has to be run with root privileges!)


iostat

Linux 2.6.18-308.1.1.el5 (myhostname)         04/13/2012

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           2.03    0.01    0.20    0.01    0.00   97.76

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
cciss/c0d0       10.14         8.85       305.48   16119821  556372354
cciss/c0d0p1      0.00         0.00         0.00       2351        156
cciss/c0d0p2     10.14         8.85       305.48   16117198  556372198
dm-0              4.94         0.63        38.96    1148082   70960475
dm-1             13.75         6.45       108.96   11742082  198439411
dm-2              5.78         1.29        45.76    2341482   83332872
dm-3              2.51         0.01        20.04      12186   36492920
dm-4             11.49         0.48        91.77     871650  167146504
dm-5              0.00         0.00         0.00       1072          0

You can also use Bonnie++ but I don't want to install any module.

A mini-framework for Performance Monitoring

Minimalistic set of 3 classes:

import java.util.Collection;

public class PerformanceMeasurement {
 
 RecordByTypeContainer recordByTypeContainer = new RecordByTypeContainer();

 public void record(String type, long elapsed) {
  RecordByType recordByType = recordByTypeContainer.getRecordByType(type);
  recordByType.addEntry(elapsed);
 }

 public long averageExecutionTime() {
  long sum = 0;
  for (RecordByType recordByType : recordByTypeContainer.getAllRecordByType()) {
   sum += recordByType.averageExecutionTime();
  }
  return recordByTypeContainer.size() > 0 ? sum / recordByTypeContainer.size() : 0;
 }

 public Collection getRecordByTypes() {
  return recordByTypeContainer.getAllRecordByType();
 }

 public void reset() {
  for (RecordByType recordByType : recordByTypeContainer.getAllRecordByType()) {
   recordByType.clear();
  }  
 }


}






import java.util.Collections;
import java.util.List;
import java.util.ArrayList;

public class RecordByType {
 private String type;
 private List measurements = Collections.synchronizedList(new ArrayList());

 public RecordByType(String type) {
  this.type = type;
 }

 public synchronized void addEntry(long elapsed) {
  measurements.add(elapsed);
 }

 public long averageExecutionTime() {
  long result = 0;
  synchronized (measurements) {
   for (Long item : measurements) {
    result += item;
   }  
  }

  return measurements.size() > 0 ? result / measurements.size() : 0;
 }

 public int numberOfRecordsAboveThreshold(long i) {
  int count = 0;
  synchronized (measurements) {
   for (Long item : measurements) {
    if (item > i)
     count++;
   }
  }
  return count;
 }

 public synchronized void clear() {
  measurements.clear();
 }

 public String getType() {
  return type;
 }

 public void setType(String type) {
  this.type = type;
 }


}




import java.util.Collection;
import java.util.HashMap;

@SuppressWarnings("serial")
public class RecordByTypeContainer extends HashMap {

 public RecordByType getRecordByType(String type) {
  RecordByType result = get(type);
  if (result == null) {
   result = new RecordByType(type);
   this.put(type, result);
  }
  return result;
 }
 
 public Collection getAllRecordByType() {
  return values();
 }

}

How to use it:

// create tooling
static final PerformanceMeasurement performanceMeasurement = new PerformanceMeasurement();


// reset to a know state
performanceMeasurement.reset();


// do all the tests each of which records the elapsed type

long now = System.currentTimeMillis();
// .... RUN YOUR TEST HERE
long elapsed = (System.currentTimeMillis() - now);
performanceMeasurement.record("TEST_TYPE", elapsed);


// print stats or do asserts

assertTrue("global averageExecutionTime", performanceMeasurement.averageExecutionTime() < 1600);


for (RecordByType recordByType : performanceMeasurement.getRecordByTypes()) {
 assertTrue("individual averageExecutionTime for type "  + recordByType.getType(), recordByType.averageExecutionTime() < 1600);

 assertTrue("execution threshold  for type " + recordByType.getType(), recordByType.numberOfRecordsAboveThreshold(2000) < 10);
}

Works like magic!