Saturday, October 31, 2015

Getting started with JSF and CDI

I have followed this tutorial
Clone this git repo . It's an Eclipse 4.3 OEPE for WLS 12.1.3 project.

Deploy the application to a running WLS and test with http://localhost:7001/JSFWeb

You can login using this username/pw: ADMIN/password

The Java code uses the annotations

PS in October 2015, 70 Palestinians (including many children and infants) have been cold-blooded killed by Israeli soldiers and some 2500 injured. Have you read this in the news? No? I am not surprised!

Thursday, October 29, 2015

javax.transaction.SystemException: SubCoordinator not available

If you see this "javax.transaction.SystemException: SubCoordinator not available" in the logs, AND you have recently moved your domain to new servers, it could very well be that you need to purge your TLOG DB tables (of file store).. in fact, IP address of previous servers is still hardcoded in the TLOG info, so those TX will still linger around failing recovery until the Abandon Timeout is hit (read the "Moving the Server" chapter here " the server identity may no longer match the information stored in the transaction logs"

Wednesday, October 28, 2015

installing WLS 12.2.1 on Linux RHEL 6.5

Download the generic installer fmw_12. from and put into /opt/install/
Download Java 8 JDK from
install this one:

rpm -i jdk-8u65-linux-x64.rpm

you should now find /usr/java/jdk1.8.0_65/bin/java

groupadd oracle
useradd oracle -g oracle
su - oracle
/usr/java/jdk1.8.0_65/bin/java -jar /tmp/fmw_12.

this fails because I don't have X11, so I have to do silent installation the error message is:

/usr/java/jdk1.8.0_111/bin/java -jar /opt/install/fmw_12.
Launcher log file is /tmp/OraInstall2017-01-02_09-52-00AM/launcher2017-01-02_09-52-00AM.log.
Extracting files.........
Starting Oracle Universal Installer

Checking if CPU speed is above 300 MHz.   Actual 2593.750 MHz    Passed
Checking monitor: must be configured to display at least 256 colors.  DISPLAY environment variable not set.    Failed <<<<
Checking swap space: must be greater than 512 MB.   Actual 3999 MB    Passed
Checking if this platform requires a 64-bit JVM.   Actual 64    Passed (64-bit not required)
Checking temp space: must be greater than 300 MB.   Actual 4663 MB    Passed

Some system prerequisite checks failed.
You must fulfill these requirements before continuing with the installation.

Continue? (yes [y] / no [n]) [n]
You have confirmed that the product cannot be installed on this platform.
Quitting the installation.

Exiting Oracle Universal Installer.
Log(s) for this session can be found in /tmp/OraInstall2017-01-02_09-52-00AM/launcher2017-01-02_09-52-00AM.log.

you should prepare this silent file ( see doc here )

vi installfmw.rsp

(see here for sample response files for WLS 12.2.1)

Response File Version=
#The oracle home location. This can be an existing Oracle Home or a new Oracle Home
#Set this variable value to the Installation Type selected. e.g. Fusion Middleware Infrastructure, Fusion Middleware Infrastructure With Examples.
INSTALL_TYPE=WebLogic Server
#Provide the My Oracle Support Username. If you wish to ignore Oracle Configuration Manager configuration provide empty string for user name.
#Provide the My Oracle Support Password
#Set this to true if you wish to decline the security updates. Setting this to true and providing empty string for My Oracle Support username will ignore the Oracle Configuration Manager configuration
#Set this to true if My Oracle Support Password is specified
#Provide the Proxy Host
#Provide the Proxy Port
#Provide the Proxy Username
#Provide the Proxy Password
#Type String (URL format) Indicates the OCM Repeater URL which should be of the format [scheme[Http/Https]]://[repeater host]:[repeater port]

as root:
vi /etc/oraInst.loc


chmod 777 /etc/oraInst.loc

mkdir -p /opt/oracle/inventory
chown oracle:oinstall /opt/oracle/inventory
mkdir /opt/oracle/fmw
chown oracle:oinstall /opt/oracle/fmw

su - oracle

/usr/java/jdk1.8.0_65/bin/java -jar /opt/install/fmw_12. -silent -responseFile /opt/install/installfmw.rsp -invPtrLoc /etc/oraInst.loc

no more stinky cmd prompt in Windows: use cmder instead!

Download the "full" edition from . Unzip. Run cmder.

If you install the version from the main page you get a "api-ms-win-crt-runtime-l1-1-0.dll missing" error.

It looks definitely more feature-rich than the STINKY cmd prompt from Windows.

Sunday, October 25, 2015

OSB Cluster

Create a new Domain, using the "OSB Production" template
Create it in "Production" mode
Create MS osb_server1 and osb_server2 and put them into a cluster.

You will notice that this deployment:

"ALSB Cluster Singleton Marker Application" is targeted to osb_server1

wlsbJMSServer_auto_1 and 2 are targeted to osb_server1 and 2

WseeJmsServer_auto_1 and 2 are targeted to osb_server1 and 2

wli.reporting.purge.queue is deployed to wlsbJMSServer_auto_1

all the reporting, email file ftp sftp transport JMS queues are created as Distributed Destinations and targeted to the cluster
(wlsb.internal.transport.task.queue.ftp_auto_1, wlsb.internal.transport.task.queue.sftp_auto_1, wlsb.internal.transport.task.queue.email_auto_1, wlsb.internal.transport.task.queue.file_auto_1, QueueIn_auto_1, wli.reporting.jmsprovider.queue_auto_1, wli.reporting.jmsprovider_error.queue_auto_1)

DefaultCallbackQueue-WseeJmsServer_auto_1/2 and DefaultQueue-WseeJmsServer_auto_1/2 are deployed to either server

see here

for File, FTP, email:
"the poller on only one managed server will poll for a given proxy service"

SOA Architect Questions

How would you monitor a WebLogic instance (hint: console, Nagios/ Oracle Enterprise Manager and or Grid Control/ HP Open View)

Need for SOA Governance?

Have you read some EAI books, and which EAI patterns do you remember?

Do you know which are the main internal variable in OSB?( Hint: body, inbound, outbound)

If a process is long running, with human interaction, how would you manage it? (hint asynchronous)

Explain me how you would handle errors in OSB (hint: eror handler stage level, proxy level, …)

How do you monitor performance and response time and Errors using the OSB console? (hint SLA -Alert, logging and reporting actions)

What is a fault in OSB? How is represented? (hint: $fault details, ….)

Which problem do you face when deploying an integration solution to a new environment, and how do you solve this (customization plans/files)

When is it necessary to apply compensation? (hint error handling, non transactional)

Which features are available in BPEL which are not in a general purpose programming language like Java?

How would you test an integration solution?

What is JNDI?

You have a CSV or EDI file, how do you tranform it in XML?

What is Oracle RAC? How do you configure WebLogic to take advantage of RAC (hint: Multidatasource)

Which options do you have to enforce data integrity in a DB (constraints, triggers, range check, not null, foreign keys)

What is a WSDL?

How can we reuse services?

How would you configure WebLogic domains without using manual operations (hint: WLST, domain templates)

Difference between a Proxy service and Business Service?

What are JMX Beans, what to they do …how can we use it

How would you create and monitor a JMS queue with scripts? (hint: WLST, eventually Java)

Can you start and stop a WebLogic server from WLST, and how? (hint: nmConnect, nmStart, Node Manager)

Which options do you have to target a JMS queue? (hint: cluster, individual WLS server, individual JMS server, subdeployments)

How do you ensure that operations can continue without disruption even if a WLS instance goes down
(hint: cluster, whole server migration, service migration, load balancing with failover)

How do you change the log properties? Which are the main logs you have for weblogic and osb?

Which metrics would you monitor in WLS for a JDBC Data Source?

How would you implement the following integration interface?
. Our source system is sending the requests via FTP (it writes into ftp) and the target destination is a database. (hint database and ftp adapters).
. we want the implementation not to manage the connection details to the ftp or the database (JCA)
. We want messages to be persisted (persistent stores, jms queues)
. control error (error handler strategies)
. monitor the execution (tracking actions logging messages) end to end (messages id)
. in case something is failing we want ensure the database integrity so if we update the database and before ending the process something fails we want the database to be back to the original status( XA)
. The source message has a different format than the target (hint: xquery, xslt)
. we want to ensure that messages are processed in order (sequences, unit of order)
. We need to ensure that the message we send and we receive is valid according to the input and output schemas (hint validate actions)
. While mapping the values some of them , for example currency or country codes are represented in a different format in each system. For example EURO and EUR, POUND and PND, ... (Hint: domain value mapping)
. We want the service to be reused and accessible from multiple systems (service registry and service repository)
. Finally we would like to have the schemas in a repository (metadata repository) "
Please explain how to create a queue in FIFO mode in Weblogic? (Hint :one consumer)

What is a Canonical Data model? (hint common data, reusability)

Would you mind to explain the difference between async and synch behaviour?

Can you explain me something about ORM (Object Relational Mapping)

How do you ensure that a request is persisted and executed even in case of system crash (hint: persistence, transactions)

What is a transactional resource? What is a XA interface?

How do you configure a UnitofOrder in Weblogic? A persistent store? A remotequeue?

Is it possible a Managed server continue providing services in spite of shutting down the admin server? How to?

WebLogic Administrator Questions

How can you see if your OS is running a 64 or 32 bit OS

Can you explain me the basic Linux directory structure (/var/log/, /etc, /opt. /dev, /users)

How can you see how much disk space is left

Can you see what processes are running

Name some scheduling techniques

What is VM ware, have you ever worked with it, name some vendors

how do you check the hostname of a machine

Can you explain me something about Domain Names, how can you override domain names

Name some infrastructure monitoring software (Nagios/ Big Brother/ HP Open View)

Would you mind to explain me what a Proxy service is?

Would you mind to explain me what Business service is?

What are JMX Beans, what to they do …how can we use it

Name some Monitoring Best Practices for Middleware environments

Name some Monitoring Products for Middleware environments

How is your knowledge of Oracle Enterprise Manager and or Grid Control

What is the difference between tablespace/ schema

What is the difference between active and passive FTP

What is the difference between SFTP and FTPS

Name some Configuration Management Systems, (Puppet / CFEngine) have you ever worked with it

Do an analysis of a problem in a 4 tier system…from frontend to backend…where do you start/ escalate

What can you tell about SMNP, (trap destination/ trap translator)
How would you create and monitor a JMS queue with scripts? (hint: WLST, eventually Java)

Can you start and stop a WebLogic server from WLST, and how? (hint: nmConnect, nmStart, Node Manager)

Which options do you have to target a JMS queue? (hint: cluster, individual WLS server, individual JMS server, subdeployments)

Why would you use a Distributed Queue? (hint: high availability, load balancing)

How do you ensure that operations can continue without disruption even if a WLS instance goes down
(hint: cluster, whole server migration, service migration, load balancing with failover)

How to you check for Stuck Threads in WLS (hint: thread dump)

How do you update a Deployment (deployment plan)

Which are some common JCA Adapter and how do you set them up (hint: FTPAdapter, FileAdapter)

How to verify which Patches are applied to WLS, and how to apply a new Patch ?

How do you change the log properties? Which are the main logs you have for weblogic and osb?

Which metrics would you monitor in WLS for a JDBC Data Source?

What does a 'lock' mean when you startup weblogic? How to solve it?

In linux what command tells you the dependencies of a .so file

How can you see what processes are holding which ports (hint netstat -antpu)

Can you explain something about certificates (SSL technology , hint : public private key)

Can you explain me something about ORM (Object Relational Mapping)

Name some storage technologies (RAID - vs Striped), what are the advantages disadvantages

What can you tell about ORACLE RAC technology

How do you configure a UnitofOrder in Weblogic?A persistent store? A remotequeue?

Is it possible a Managed server continue providing services in spite of shutting down the admin server? How to?

OSB developer questions

How do you check the linux distribution version

How can you see if your OS is running a 64 or 32 bit OS

Can you explain me the basic Linux directory structure (/var/log/, /etc, /opt. /dev, /users)

How can you see how much disk space is left

Can you see what processes are running

What is the difference between an abstract class and an interface

Can you name some JAVA open source frameworks

What is the difference between a script and a programming language (to fill time….idle talk)

Would you mind to explain me what a Proxy service is?

Would you mind to explain me what Business service is?

What do you use for mapping in OSB?

When do you use Xpath?

What is a XSD? What is a simple type? What is a complex type?

What is a Namespace?

How do you define a list of elements?

How do you call (invoke) a service?

How do you create a FILE adapter?

How do you test a HTTP service? (hint: OSB Console, SOAPUI…)

How do you monitor a BPEL execution? Which is the tool provided in SOA for doing it?

What is the structure of a WSDL
"Are you familiar with OSB and SOA suite, so In which case you would use OSB and in which one you would use SOA?

Tell me advantages and disadvantages."

Would you mind to tell me which are your prefered options for reporting and tracking information in OSB?

A Proxy or Business service points to endpoints (URIs) that will differ in each environment (DEV, TST, PROD…). How could you deploy your interface and change those endpoint URIs without modifying them manually? Which tool is provided by OSB?

Is is it possible to use java programs in OSB? How do you do it? (hint: java callout, custom xpath)

In case you want to implement an interface that processes in parallel, How do you do it in OSB?

How could you obtain a data from the database based on an id? Do you know how to do a lookup (hint: multiple options are available)?

Which are the main actions provided by OSB for invoking services?
"Imagine we are going to design the schemas for a renting agency
This schema is for defining the message that will be used when a customer rents a car
please provide me per each situation how you would implement it in XSD
- You can rent a car or a lorry, both contains similar fields (price, number of days to be rented, ...)
but they also have different elements specific to the type of vehicle. Please tell me the mode you would implement
it for reusing the common elements. (hint: extending or aggregating)"
"In the rent contract when you specify the vehicle you can choose lorry or car and it's mandatory to have one of the 2.
explain me how to represent that concept in XSD. (hint: choice, minoccur, maxoccur)"

In SOA/XSLT which function is used for getting a value from the database based on a value you pass?
"How can you map standard definitions like currency, country, ... that have a different representation in the Source system and in the target system?
for example EUR in system A and EU in system B represents Euro (hint: DVM lookup)"

How could you implement sequences in SOA? (hint: mediator sequencer)

Are you able to define custom XPATH functions?

How could you implement a synchronous BPEL process? and Asynchronous?

If an error occurs in a Message Flow, which options do you have to handle it?
"If you need to implement file poller Interface that transforms the message and then sends the message to a FTP.
How could you do it? and In case you want to raise an error if it takes longer than 90 seconds ? How could you do it? (hint: timeout, alert)"
"If you need the messages to be sent in order so the behaviour will be First Input and First Output,

How did you implement it? (hint: single consumer)"

How can you see what processes are holding which ports (hint netstat -antpu)

Can you explain something about certificates (SSL technology , hint : public private key)

Can you explain me something about ORM (Object Relational Mapping)

Would you mind to give me a sample of an scenario when you need to use Publish, CallBack and Route actions?
Explain the reason of choosing that action.

Can you explain me how to configure a transactional proxy service? Which are the main OSB actions involved?

Which properties are used? Which Qos will you use?"

Which kind of objects could be handled by OSB from a Java Callout? (hint: primitive types, arrays)

JKS keystore password vs key password

Something for me confusing (I get confused quite easily) is the relation between a JKS keystore password and a key password.
Some facts:
  • a .key file SHOULD always be protected by password.
  • a .jks store SHOULD always be protected by password.
  • not necessarily these 2 passwords should match

Once you have added a .key into a JKS, you can change the key's PW like this:
keytool -keypasswd -keystore keystore.jks -alias 

You can change the JKS PW like this:
keytool -storepasswd -new new_storepass -keystore keystore.jks

See also the excellent JKS guide and this very good FAQ
This is the keytool doc


I have installed only OSB (if you install SOA Suite, you will get more instances of :o) )


find . -name


So, which one are you supposed to use?

There is also a


which is invoked by everybody.

When using OSB, you should use the ./Oracle_OSB1/common/bin/
Remember that you should first change dir:

cd ./Oracle_OSB1/common/bin/
. ./

this gives me a "command not found /bin/java "

So I do

. ./
java weblogic.WLST

Liquibase in Action

Download the product from

Unzip it on your local drive

Type "liquibase" , you will get a pretty help. We learn that we have several areas:
Standard Commands (update, rollback..)
Diff commands
Maintenance Commands

We learn that some parameters are required (changeLogFile, username and pw, DB url) and other optional (drive, logfile...).

All the default parameters can be stored in More info here

driver: oracle.jdbc.OracleDriver
classpath: /opt/oracle/fmw11_1_1_5/wlserver_10.3/server/lib/ojdbc6.jar
url: jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(
username: pl1_soainfra
password: pl1_acme
mkdir data
I try first an export:
java -jar liquibase.jar --changeLogFile="./data/myexportfile.xml" --diffTypes="data" generateChangeLog
you must provide an extension (.xml) in the filename, otherwise you get "Liquibase generateChangeLog Failed: No serializer associated with the filename or extension '/data/myexportfile'"

A sample entry generated is:
    <changeSet author="soa (generated)" id="1395989865665-193">
        <insert tableName="BPM_MEASUREMENT_ACTION_SEQ">
            <column name="SEQ_NAME" value="ACTION_SEQ"/>
            <column name="SEQ_COUNT" valueNumeric="0"/>

The changelog file is a huge file not very easy to read. If I do 2 subsequent changelogs, the differences are many even if schema/data are the same... not very intuitive.

Now I try:
java -jar liquibase.jar updateSQL
--changeLogFile is required

I try again:
java -jar liquibase.jar --changeLogFile=./data/myexportfile.xml updateSQL

-- Changeset ./data/myexportfile.xml::1395989865665-193::soa (generated)

INSERT INTO PL1_SOAINFRA.DATABASECHANGELOG (ID, AUTHOR, FILENAME, DATEEXECUTED, ORDEREXECUTED, MD5SUM, DESCRIPTION, COMMENTS, EXECTYPE, LIQUIBASE) VALUES ('1395989865665-193', 'soa (generated)', './data/myexportfile.xml', SYSTIMESTAMP, 193, '7:f8ab8f75bd7c0bafd9cd43fd119e4340', 'insert', '', 'EXECUTED', '3.1.1');

so basically for each row in the DB (in this case it's a definition of a sequence, not a row) it produces the SQL and the reference to the changeset element.

Starting a Managed Server with WLST and NodeManager

1) factor out commonly used constants into a file:


#prefix for domain 



# home of certificates


2) prepare a NodeManager userConfigFile:



and a userKeyFile /opt/oracle/domains/osbdv1do/

3) Prepare a file (in reality it can start ANY server, not only the admin):

adminName = sys.argv[1]
domainName = sys.argv[2]
nmHost = sys.argv[3]

nmPort = '5556'
domHome='/opt/oracle/domains/' + domainName

print nmHost, nmPort, domainName, adminName

# -----------------------------------------------------------------------------
# connect to NodeManager
# -----------------------------------------------------------------------------
nmConnect( userConfigFile=domHome + '/', userKeyFile=domHome + '/', host=nmHost, port=nmPort, domainName=domainName, domainDir=domHome, nmType='plain' )

# -----------------------------------------------------------------------------
# start the server
# -----------------------------------------------------------------------------

    nmStart( adminName, domHome )

    exit('y', 4)


Using Oracle pre-built VM for Oracle VM Manager and Server

No need to install from scratch, you can use these 2 prebuilt VMs:

I download and "import appliance" OracleVMManager3.2.4-b524.ova into VirtualBox. Check that you have 4 GB ram and accept EULA.

Could not start the machine Oracle VM Manager 3.2.4-b524 because the following physical network interfaces were not found:
eth0 (adapter 1)
You can either change the machine's network settings or stop the machine.

"change network settings" and use "host only network". Assign static IP, netmask, gateway, DNS server, hostname = (maybe also could do for gateway and dns)

OVM should start.

Open firefox, log into https://localhost:7002/ovm/console with admin/Welcome1

If your screen gets locked, remember that ovm password is Welcome1.

Now you can import appliance OracleVMServer3.2.4-b525.ova in VirtualBox.

Could not start the machine Oracle VM Server 3.2.4-b525 because the following physical network interfaces were not found:
eth0 (adapter 1), eth1 (adapter 2)
You can either change the machine's network settings or stop the machine.

assign "host only" in the vm settings.

Assign static IP, netmask, gateway, DNS server, hostname =

one the OVM Server is up, you can connect from the OVM Manager opening a terminal windowd : ssh root@ (pw is ovsroot)

In reality one should use IPs in the range 192.168.56.*

References cool lab on how to setup a OVMManager/Server environment.

Deleting a non empty directory with find -delete

mkdir  /opt/oracle/Pipposcriptsstaging/PippoScripts-1.23
mkdir  /opt/oracle/Pipposcriptsstaging/PippoScripts-1.22

(put some content in both folders)

this will fail:

find /opt/oracle/Pipposcriptsstaging/  -type d -name "PippoScripts*" ! -name "PippoScripts-1.22" -delete

because find -delete can only remove empty folders (don't try the -maxdepth, -depth, -mindepth, -prune options, they won't work)

For some REALLY weird reason, this command too will fail:

find /opt/oracle/Pipposcriptsstaging/  -type d -name "PippoScripts*" ! -name "PippoScripts-1.22" -exec rm -rf '{}' \;
find: `/opt/oracle/Pipposcriptsstaging/PippoScripts-1.23': No such file or directory

but it actually deletes the PippoScripts-1.23 folder (WEIRD!)

The only way to make it work:

find /opt/oracle/Pipposcriptsstaging/PippoScripts-1.23 -type f  -delete

find /opt/oracle/Pipposcriptsstaging/PippoScripts-1.23 -type d  -delete

GIT: First steps with JGit

If you need a Java API for GIT, forget about JavaGit (it's just a wrapper around GIT CLI) and EGit - which internally uses JGit.

Useful snippets here

Download the library JAR here org.eclipse.jgit.jar, or if using maven consider using the dependency:

(the jar should end up in \org\eclipse\jgit\org.eclipse.jgit\ inside your stink-maven repo)

Looking at the examples, you must have a LOCAL repository.... there doesn't seem to be a way to query a remote repository, like one hosted on Stash, without first doing a git clone.... I look rather for a REST-based API, this one seems quite promising.

Missing JAVA_VENDOR and JAVA_HOME in WebLogic silent installation

in $DOMAIN_HOME/bin/ I keep finding:



if [ "${JAVA_VENDOR}" = "Oracle" ] ; then

(and JAVA_VENDOR is not declared, and *_JAVA_HOME is blank... so WLS fails when started complaining about missing JAVA_HOME....)

The secret is that you should declare (export) these 2 environment variables BEFORE you run the weblogic installation:
export JAVA_VENDOR=Sun
export JAVA_HOME=/usr/java/jdk1.7.0_55/
java -d64 -Xmx1024m -jar /software/wls1036_generic.jar -mode=silent -silent_xml=/software/weblogic_silent_install.xml

Why those 2 values cannot be part of the silent installer xml, no clue!

Here is the official doc of the silent installer

How to generate a OutOfMemoryError PERMGEN

Inspired by a very interesting presentation by Pierre-Hugues Charbonneau:

I have partially copied his code to generate a OOM PERMGEN:
import java.lang.reflect.InvocationHandler;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
import java.util.HashMap;
import java.util.Map;

public class ClassLoaderLeak {
    private static Map leakingMap = new HashMap ();
    public static void main(String[] args) {
 try {
     for (int i = 0; i < 100000; i++) {
  String dummyCLURLString = "file:" + i + ".jar";
  URL[] dummyCLURL = new URL[] { new URL(dummyCLURLString) };
  URLClassLoader urlClassLoader = new URLClassLoader(dummyCLURL);
  Class[] arg1 = new Class[] {Aclass.class}; 
  InvocationHandler arg2 = new InvocationHandler() {
      public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
   return null;
  Aclass aclass = (Aclass)Proxy.newProxyInstance(urlClassLoader, arg1, arg2);
  leakingMap.put(dummyCLURLString, aclass);
 } catch (Exception e) {



run it with "java -XX:+HeapDumpOnOutOfMemoryError -XX:MaxPermSize=64m -Xmx256m ClassLoaderLeak

you will see the PermGen space quickly fill up until you get a OOM error

You should see something like
java.lang.OutOfMemoryError: PermGen space  
Dumping heap to java_pid15744.hprof ...
Heap dump file created [154441412 bytes in 1.982 secs]

Now open Eclipse MAT, open the java_pid15744.hprof file and run the Java Basics / Classloader Explorer tool:

You will see several thousands (in my case, 27k) instances of URLClassLoader accumulated in the PERMGEN

ContextClassLoader, SystemClassLoader, CurrentClassLoader

If you run this code as a Java Application (no JEE container):

public class BootstrapCLTest {
    public static void main(String[] args) {
 BootstrapCLTest bootstrapCLTest = new BootstrapCLTest();

    private void go() {

 System.out.println("1 " + String.class.getClassLoader());
 System.out.println("2 " + ClassLoader.getSystemClassLoader ());
 System.out.println("3 " + this.getClass().getClassLoader());
 System.out.println("4 " + Thread.currentThread().getContextClassLoader());
 System.out.println("5 " + Thread.currentThread().getContextClassLoader().getParent());
 System.out.println("6 " + Thread.currentThread().getContextClassLoader().getParent().getParent());


you get this (parentesis are mine):

1 null (=bootstrap CL)
2 sun.misc.Launcher$AppClassLoader@1bd06bf (=application CL)
3 sun.misc.Launcher$AppClassLoader@1bd06bf (=application CL)
4 sun.misc.Launcher$AppClassLoader@1bd06bf (=application CL)
5 sun.misc.Launcher$ExtClassLoader@1061299 (=extension CL)
6 null (=bootstrap CL)

that is, in this simple case the SystemClassLoader == ApplicationClassLoader == ContextClassLoader

The first null is because the String class is loaded by the Bootstrap classloader, which is NOT represented as a ClassLoader object. The Bootstrap will then load the Extension CL, who in turn loads the Application CL.

Running a similar code in a JSP is more interesting, because here the JEE container (WebLogic in this case) will add its own CL:

 System.out.println("1 " + String.class.getClassLoader());
 System.out.println("2 " + ClassLoader.getSystemClassLoader ());
 System.out.println("3 " + this.getClass().getClassLoader());
 System.out.println("4 " + Thread.currentThread().getContextClassLoader());
 System.out.println("5 " + Thread.currentThread().getContextClassLoader().getParent());
 System.out.println("6 " + Thread.currentThread().getContextClassLoader().getParent().getParent());
 System.out.println("7 " + Thread.currentThread().getContextClassLoader().getParent().getParent().getParent());
 System.out.println("8 " + Thread.currentThread().getContextClassLoader().getParent().getParent().getParent().getParent());
 System.out.println("9 " + Thread.currentThread().getContextClassLoader().getParent().getParent().getParent().getParent().getParent());
 System.out.println("10 " + Thread.currentThread().getContextClassLoader().getParent().getParent().getParent().getParent().getParent().getParent());

the result is:

1 null
2 sun.misc.Launcher$AppClassLoader@fc5b01
3 weblogic.servlet.jsp.JspClassLoader@1bfdb6f finder: weblogic.utils.classloaders.CodeGenClassFinder@137e701 annotation:
4 weblogic.utils.classloaders.ChangeAwareClassLoader@1cc3dd4 finder: weblogic.utils.classloaders.CodeGenClassFinder@11f2691 annotation: _auto_generated_ear_@CLTestWeb
5 weblogic.utils.classloaders.FilteringClassLoader@1d16e3 finder: weblogic.utils.classloaders.CodeGenClassFinder@1c9a7b5 annotation: _auto_generated_ear_@CLTestWeb
6 weblogic.utils.classloaders.GenericClassLoader@ceb98d finder: weblogic.utils.classloaders.CodeGenClassFinder@217181 annotation: _auto_generated_ear_@
7 weblogic.utils.classloaders.FilteringClassLoader@1a308be finder: weblogic.utils.classloaders.CodeGenClassFinder@92f8f0 annotation:
8 weblogic.utils.classloaders.GenericClassLoader@1865594 finder: weblogic.utils.classloaders.CodeGenClassFinder@fcfe6b annotation:
9 sun.misc.Launcher$AppClassLoader@fc5b01
10 sun.misc.Launcher$ExtClassLoader@1353d27

that is, WebLogic inserts 2 layers of GenericClassLoader or ChangeAwareClassLoader plus FilteringClassLoader between the War Application and the System Classloader.

See also

Friday, October 23, 2015

Automate mouse click repetitions with Auto Mouse Click

First, assign the "find mouse position" to F6 (you don't want to have to manually find a X-Y coordinate...)... and don't forget to define F7 as Stop... all the rest is pretty straightforward. Click on AAutoMouseClick.exe, it doesn't require Admin privileges.

Tuesday, October 20, 2015

WebLogic: difference between self-tuning-thread-pool-size-max and shared-capacity-for-work-managers

Sets the maximum thread pool size of the self-tuning thread pool.

The self-tuning thread pool starts with the default size of 1. It grows and shrinks automatically as required. Setting this attribute changes the default max pool size. The active thread count will never increase beyond this value. This value defines the maximum number of threads permitted in the server. Note that the server will add threads only if it improves throughput. Measurements are taken every 2 seconds and the decision to increase or decrease the thread count is based on the current throughput measurement versus past values.

In WebLogic Server, all requests, whether related to system administration or application activity—are processed by a single thread pool. An administrator can throttle the thread pool by defining a maximum queue length. Beyond the configured value, WebLogic Server will refuse requests, except for requests on administration channels.
When the maximum number of enqueued requests is reached, WebLogic Server immediately starts rejecting:
• Web application requests.
• Non-transactional RMI requests with a low fair share, beginning with those with the lowest fair share.
If the overload condition continues to persist, higher priority requests will start getting rejected, with the exception of JMS and transaction-related requests, for which overload management is provided by the JMS and the transaction manager.
Throttle the thread pool by setting the Shared Capacity For Work Managers field in the Administration Console (see Environments > Servers > server_name > Configuration > Overload). The default value of this field is 65536.
Work Managers and Thread Pool Throttling
An administrator can configure Work Managers to manage the thread pool at a more granular level, for sets of requests that have similar performance, availability, or reliability requirements. A Work Manager can specify the maximum requests of a particular request class that can be queued. The maximum requests defined in a Work Manager works with the global thread pool value. The limit that is reached first is honored.

So, the first acts on the MAX NUMBER OF THREADS, the second on the MAX NUMBER OF REQUESTS WAITING IN A QUEUE.

My impression is that finding the right mix for your case might require some tests...

ulimit and lsof

PID of WLS is 29518
to count how many file descriptors are held by a process:
ls /proc/29518/fd | wc -l

funnily this doesn’t match the count reported by lsof ( one day I will discover why):

lsof -p 29518 | wc -l

to know the limit for the specific process:

cat /proc/29518/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 10485760 unlimited bytes
Max core file size 0 0 bytes
Max resident set unlimited unlimited bytes
Max processes 32000 32000 processes
Max open files 1024 1024 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 515304 515304 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us

which is different for the limit set by default for a user:

ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 515304
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 8192
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 1024
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

To set the limit for a process open by the current shell :


In Linux, this is a PER-PROCESS limit! You cannot directly limit the number of Files opened by a User! You can only set a max-processes-per-user with ulimit -u MAX .
to check the actual limits for a process:
cat /proc//limits

ORA-24756: transaction does not exist

Here is the stacktrace:

ORA-24756: transaction does not exist at 
oracle.jdbc.driver.T4CTTIoer.processError( at 
oracle.jdbc.driver.T4CTTIoer.processError( at 
oracle.jdbc.driver.T4CTTIoer.processError( at 
oracle.jdbc.driver.T4CTTIfun.processError( at 
oracle.jdbc.driver.T4CTTIfun.receive( at 
oracle.jdbc.driver.T4CTTIfun.doRPC( at 
oracle.jdbc.driver.T4CTTIOtxen.doOTXEN( at 
oracle.jdbc.driver.T4CXAResource.doTransaction( at 
oracle.jdbc.driver.T4CXAResource.doPrepare( at 

If you get this error, you must be using Oracle RAC and XA.

You must carefully craft the various timeouts in WebLogic and in Oracle DB.

See Oracle note "Troubleshooting ORA-24756 while Running an XA Program or MSDTC with the Oracle RDBMS (Doc ID 1076242.6)". THE rule is: global transaction timeout < session timeout < distributed_lock_timeout . See also "Recommended Timeout Configuration When Using an External Transaction Processing Monitor and XA with Oracle (Doc ID 338880.1)" .

Each MDB has a specific timeout, but this should not be involved in the issue (this in the weblogic.ejb.jar.xml DD )


So you should end up with this configuration in config.xml and the "$XA_DATASOURCE"jdbc.xml

> config/jdbc/<datasource>-jdbc.xml

> config/config.xml

Tuesday, October 13, 2015

WebLogic MemorySessionContext

We had some PROD OutOfMemory recently.... done a heap dump and seen that we were in this situation:

One instance of "weblogic.servlet.internal.session.MemorySessionContext" loaded by "sun.misc.Launcher$AppClassLoader @ 0x7818fbb60" occupies 732'427'648 (82.97%) bytes. The memory is accumulated in one instance of "java.util.Hashtable$Entry[]" loaded by "".

sun.misc.Launcher$AppClassLoader @ 0x7818fbb60

and this is the "Shortest Paths to the Accumulation Point" in Eclipse MAT:

java.util.Hashtable$Entry[383] @ 0x7b0455fd8
 1'552 732'427'400 
\table java.util.Hashtable @ 0x7823d6118
 48 732'427'488 
.\openSessions weblogic.servlet.internal.session.MemorySessionContext @ 0x7823d60e8
 48 732'427'648 
..+sessionContext weblogic.servlet.internal.WebAppServletContext @ 0x782310e88
 208 452'632 
..|+[0] java.lang.Object[1] @ 0x7812d7e28
 24 24 
..||\array java.util.concurrent.CopyOnWriteArrayList @ 0x7812d7de0
 24 96 
..||.\initializedContexts com.sun.faces.config.ConfigManager @ 0x7812d7dd0
 16 112 
..||..\CONFIG_MANAGER class com.sun.faces.config.ConfigManager @ 0x776ab8198
 64 100'560 
..||...\[17] java.lang.Object[2560] @ 0x78235eb90
 10'256 10'256 
..||....\elementData java.util.Vector @ 0x78235ae78
 32 10'288 
..||.....\classes weblogic.utils.classloaders.ChangeAwareClassLoader @ 0x78233c758
 104 9'648'032 
..||......+contextClassLoader$WorkQueueThread @ 0x783f0a9f0 [ORB=_it_orb_id_1,Pool=1]::id-10 Thread
 120 2'184 
..||......+contextClassLoader$WorkQueueThread @ 0x783f0b2d8 [ORB=_it_orb_id_1,Pool=1]::id-9 Thread
 120 2'184 
..||......+contextClassLoader$WorkQueueThread @ 0x783f0bb60 [ORB=_it_orb_id_1,Pool=1]::id-8 Thread
 120 2'192 
..||......+contextClassLoader$WorkQueueThread @ 0x783f12410 [ORB=_it_orb_id_1,Pool=1]::id-7 Thread
 120 2'192 
..||......+contextClassLoader$WorkQueueThread @ 0x783f12cd8 [ORB=_it_orb_id_1,Pool=1]::id-6 Thread
 120 2'192 
..||......+contextClassLoader$WorkQueueThread @ 0x783f13560 [ORB=_it_orb_id_1,Pool=1]::id-5 Thread
 120 2'304 
..||......+contextClassLoader$WorkQueueThread @ 0x783f13e60 [ORB=_it_orb_id_1,Pool=1]::id-4 Thread
 120 2'232 
..||......+contextClassLoader$WorkQueueThread @ 0x783f1a680 [ORB=_it_orb_id_1,Pool=1]::id-3 Thread
 120 2'160 
..||......+contextClassLoader$WorkQueueThread @ 0x783f1aef0 [ORB=_it_orb_id_1,Pool=1]::id-2 Thread
 120 2'304 
..||......+contextClassLoader$WorkQueueThread @ 0x783f1b7a0 [ORB=_it_orb_id_1,Pool=1]::id-1 Thread
 120 2'304 
..||......+contextClassLoader java.lang.Thread @ 0x7da0b6650 Keep-Alive-Timer »
 112 176 
..||......+classloader @ 0x78235b0b0 »
 40 232 
..||......\Total: 12 entries
..|+context weblogic.servlet.internal.WebAppRuntimeMBeanImpl @ 0x7824209f8 »

It turned out indeed that the session timeout was indeed waaay too long...

Monday, October 12, 2015

Minimalistic Java application to encode/decode in Base64

If your Outlook blocks all kind of attachments, you have here the FINAL resolution to the problem (Outlook doesn't check for Base64 text attachments)

import sun.misc.BASE64Decoder;
import sun.misc.BASE64Encoder;


public class Decoder {

    public static void main(String[] args) throws Exception {
        if (args.length < 3) {
            System.out.println("Usage: java EncodeFileWithBASE64 d|e <inputFile> <outputFile>");
        String op = args[0];
        String inputFile = args[1];
        String outputFile = args[2];

        if (op.equalsIgnoreCase("e")) {
            BASE64Encoder encoder = new BASE64Encoder();
                    new FileInputStream(inputFile),
                    new FileOutputStream(outputFile)
        } else {
            BASE64Decoder decoder = new BASE64Decoder();
            decoder.decodeBuffer(new FileInputStream(inputFile),
                    new FileOutputStream(outputFile));


If you put in in Eclipse you get an error, and this is how to solve it: (it's easier to ignore errors than using an Apache Commons library)

Go to Window-->Preferences-->Java-->Compiler-->Error/Warnings.
 Select Deprecated and Restricted API. Change it to warning.
 Change forbidden and Discouraged Reference and change it to warning. 
You can also use this online application

Saturday, October 10, 2015

Web Application for File Upload with streaming

This gives you an entire working example (worth 1000 user manual pages) .... I have tested it and put in (just git clone it and run it).

Here the user guide of Apache FileUpload module:

Create a webapp

Download commons-io jar and commons-fileupload jar and place them in WEB-INF/lib

Well actually rather than putting here snippets of code, here is the whole NON-Streaming example and here the STREAMING example

I have tested with a 4GB file and memory occupation on the server (and Garbage Collection) was really minimal... transfer speed was also decent (say 2 minutes for a 4GB file....).

Here you can download the war file directly.

Friday, October 9, 2015

WebLogic JNDIServiceLocator

Today there was a JNDI lookup failing across 2 clusters hosted on same machine but different NIC:
WLS_FW.JNDIServiceLocator BEA-000000
Error loading /somepath from
 at weblogic.workarea.StringWorkContext.writeContext(
 at weblogic.rjvm.MsgAbbrevOutputStream.writeContext(
 at weblogic.rjvm.MsgAbbrevOutputStream$NestedObjectOutputStream.writeContext(
 at weblogic.workarea.spi.WorkContextEntryImpl.write(
 at weblogic.workarea.WorkContextLocalMap.sendRequest(
 at weblogic.rmi.provider.WorkServiceContext.writeExternal(
 at weblogic.rjvm.MsgAbbrevOutputStream.writeObject(
 at weblogic.rjvm.MsgAbbrevOutputStream.marshalUserCustomCallData(
 at weblogic.rjvm.BasicOutboundRequest.transferThreadLocalContext(
 at weblogic.rmi.internal.BasicRemoteRef.getOutboundRequest(
 at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(
 at weblogic.rmi.cluster.ClusterableRemoteRef.invoke(
 at weblogic.jndi.internal.ServerNamingNode_12130_WLStub.lookup(Unknown Source)
 at weblogic.jndi.internal.WLContextImpl.lookup(
 at weblogic.jndi.internal.WLContextImpl.lookup(

The issue is still under investigation... these flags might help:

I could not find documentation of how internally WebLogic implements the RJVM communication protocol...

Lync copy and paste nightmare

Very unfortunately I have to use Lync as a chat platform. Some stuff is pretty cool, but there are 2 TERRIBLE issues:

one is that you select some text (ANY text), copy and paste (in any way you want) and what you paste is A BIG MESS including stuff who is miles away from your selection. I ended up ALWAYS pasting first into notepad, then selectin again only the stuff I need.

the other is that it's TERRIBLY slow (tens of seconds) at delivering messages....sometimes you get the notification that you got a message (often showing the WRONG username as author), then you have to wait patiently many seconds to get the text.

On the whole, I would rather have a less feature-rich chat, but more reliable and fast. This is typical Microsoft product, overloaded with features but clumsy and slow and full of bugs.

Sunday, October 4, 2015

Digging in JNDI

JNDI is a bit of a mystery for me...whenever I get an exception like:

javax.naming.NameNotFoundException: While trying to look up comp/env/jms/CF1 in /app/webapp/rocAdmin/1482669104.; remaining name "comp/env/jms/CF1" at weblogic.jndi.internal.BasicNamingNode.newNameNotFoundException( at weblogic.jndi.internal.ApplicationNamingNode.lookup( at weblogic.jndi.internal.WLEventContextImpl.lookup( at weblogic.jndi.internal.WLContextImpl.lookup( at at weblogic.jndi.internal.AbstractURLContext.lookup( at javax.naming.InitialContext.lookup( ...

I scratch my head ... So I wrote some code to dump the content of the InitialContext (it runs in a EJB timer to avoid having to create a client:
package com.pierre;

import javax.ejb.Schedule;
import javax.ejb.Stateless;
import javax.ejb.Timer;
import javax.naming.Binding;
import javax.naming.InitialContext;
import javax.naming.NamingEnumeration;
import javax.naming.NamingException;

public class PVTimer {

     * Default constructor.
    public PVTimer() {
 // TODO Auto-generated constructor stub

    @Schedule(second = "*/10", minute = "*", hour = "*", dayOfWeek = "*", dayOfMonth = "*", month = "*", year = "*", info = "MyTimer")
    private void scheduledTimeout(final Timer t) {
 System.out.println("@Schedule called at: " + new java.util.Date());
 try {
     InitialContext ic = new InitialContext();
     NamingEnumeration names = ic.listBindings("");
     printNames(names, "all");
     NamingEnumeration jms = ic.listBindings("jms");
     printNames(jms, "jms");
 } catch (Throwable t1) {


    private void printNames(NamingEnumeration names, String title)
     throws NamingException {
 int count = 1;
 while (names.hasMore()) {
     Binding next =;
     System.out.println(title + " " + count + " ToString=" + next.toString() + " Name=" + next.getName() + " Classname=" + next.getClassName() + " getObject=" + next.getObject());

and this is the result:

all 1 ToString=weblogic: weblogic.jndi.internal.WLEventContextImpl:EventContext (weblogic) Name=weblogic Classname=weblogic.jndi.internal.WLEventContextImpl getObject=EventContext (weblogic)
all 2 ToString=__WL_GlobalJavaApp: weblogic.jndi.internal.WLEventContextImpl:EventContext (__WL_GlobalJavaApp) Name=__WL_GlobalJavaApp Classname=weblogic.jndi.internal.WLEventContextImpl getObject=EventContext (__WL_GlobalJavaApp)
all 3 ToString=PVEarPVEjbProject_jarPVStateless_PVStatelessRemote: com.pierre.PVStateless_fcmphc_PVStatelessRemoteImpl:com.pierre.PVStateless_fcmphc_PVStatelessRemoteImpl@e964ee Name=PVEarPVEjbProject_jarPVStateless_PVStatelessRemote Classname=com.pierre.PVStateless_fcmphc_PVStatelessRemoteImpl getObject=com.pierre.PVStateless_fcmphc_PVStatelessRemoteImpl@e964ee
all 4 ToString=ejb: weblogic.jndi.internal.WLEventContextImpl:EventContext (ejb) Name=ejb Classname=weblogic.jndi.internal.WLEventContextImpl getObject=EventContext (ejb)
all 5 ToString=jms: weblogic.jndi.internal.WLEventContextImpl:EventContext (jms) Name=jms Classname=weblogic.jndi.internal.WLEventContextImpl getObject=EventContext (jms)
all 6 ToString=PVEarPVEjbProject_jarPVStateless_Home: weblogic.ejb.container.internal.StatelessEJBHomeImpl:weblogic.ejb.container.internal.StatelessEJBHomeImpl@f229af Name=PVEarPVEjbProject_jarPVStateless_Home Classname=weblogic.ejb.container.internal.StatelessEJBHomeImpl getObject=weblogic.ejb.container.internal.StatelessEJBHomeImpl@f229af
all 7 ToString=javax: weblogic.jndi.internal.WLEventContextImpl:EventContext (javax) Name=javax Classname=weblogic.jndi.internal.WLEventContextImpl getObject=EventContext (javax)
all 8 ToString=java:global: weblogic.jndi.internal.WLEventContextImpl:EventContext (java:global) Name=java:global Classname=weblogic.jndi.internal.WLEventContextImpl getObject=EventContext (java:global)
all 9 ToString=mejbmejb_jarMejb_EO: [4485985896511196818S:])/294 Name=mejbmejb_jarMejb_EO getObject=ClusterableRemoteRef(4485985896511196818S: [4485985896511196818S:])/294
all 10 ToString=_WL_internal_0Z7vb3dMvYJ62jEwg8odDkTCYHT0HwjvQwlqEb8FYeA1jZjPDZBGLEATyZk7rdEu: weblogic.jndi.internal.WLEventContextImpl:EventContext (_WL_internal_0Z7vb3dMvYJ62jEwg8odDkTCYHT0HwjvQwlqEb8FYeA1jZjPDZBGLEATyZk7rdEu) Name=_WL_internal_0Z7vb3dMvYJ62jEwg8odDkTCYHT0HwjvQwlqEb8FYeA1jZjPDZBGLEATyZk7rdEu Classname=weblogic.jndi.internal.WLEventContextImpl getObject=EventContext (_WL_internal_0Z7vb3dMvYJ62jEwg8odDkTCYHT0HwjvQwlqEb8FYeA1jZjPDZBGLEATyZk7rdEu)
jms 1 ToString=PVQueue: weblogic.jms.common.DestinationImpl:SystemModule-0!Queue-0 Name=PVQueue Classname=weblogic.jms.common.DestinationImpl getObject=SystemModule-0!Queue-0
jms 2 ToString=PVConnectionFactory: weblogic.jms.client.JMSXAConnectionFactory:weblogic.jms.client.JMSXAConnectionFactory@1a31048 Name=PVConnectionFactory Classname=weblogic.jms.client.JMSXAConnectionFactory getObject=weblogic.jms.client.JMSXAConnectionFactory@1a31048
jms 3 ToString=PVConnectionFactoryLocal: weblogic.jms.client.JMSXAConnectionFactory:weblogic.jms.client.JMSXAConnectionFactory@10414e6 Name=PVConnectionFactoryLocal Classname=weblogic.jms.client.JMSXAConnectionFactory getObject=weblogic.jms.client.JMSXAConnectionFactory@10414e6

(I have a stateless EJB "PVStateless" in the same module , implementing a PVStatelessRemote interface.... plus a JMS queue "PVQueue" with JNDI name "jms/PVQueue" and a JMS CF "PVConnectionFactory" with JNDI name "jms/PVConnectionFactory"....

Interesting javadoc and