Saturday, June 30, 2012

ORA-01654: unable to extend index in tablespace SYSTEM

Run this:

FROM dba_data_files db, dba_tablespaces t

I get this:


So I do
alter database datafile 'C:\ORACLEXE\ORADATA\XE\SYSTEM.DBF' resize 3500M;

( on Widows, the max filesize must be 4GB, I guess)

and you get a confirmation
"database datafile 'C:\ORACLEXE\ORADATA\XE\SYSTEM.DBF' altered."

javax.servlet.ServletException: java.lang.NoSuchMethodError: javax.el.ExpressionFactory.newInstance()Ljavax/el/ExpressionFactory

This happens because we had a wlfullclient.jar with javax.el package different from the one in JSTL libraries..... the only solution was to remove that package from wlfullclient.jar

What a pain.... modules in Java ... 8? or 7 already? should take care of this problem

Friday, June 29, 2012

ORA-24134 and ORA-12015, on Materialized View

If you create a Materialized View:


you get a
"ORA-23413: table "CMDB"."WLI_QS_REPORT_ATTRIBUTE" does not have a materialized view log"

so you do:

create materialized view log on WLI_QS_REPORT_ATTRIBUTE;

and now your MATERIALIZED VIEW refresh fast can be created.

Suppose now you create a VIEW on WLI_QS_REPORT_ATTRIBUTE:

create or replace view WLI_QS_REPORT_VIEW as select * from WLI_QS_REPORT_ATTRIBUTE;


you get:
ORA-12015: cannot create a fast refresh materialized view from a complex query
Reissue the command with the REFRESH FORCE or REFRESH COMPLETE option or create a simple materialized view.

If you want to create the Materialized View Fast Refresh, you should create it directly on the table, not on a view

OSB Another session operation is in progress. Please retry later.

28.2.2 Session Activation Error Message in Firefox

When activating an Oracle Service Bus session in Firefox, session activation takes longer than three minutes and you see an error message stating, "Another session operation is in progress. Please retry later."

The error message is harmless, and you can ignore it. The server is still processing the session activation, and you can wait for activation to finish.

The error message occurs when the Firefox network connection is set to "Manual proxy configuration" and the "HTTP Proxy" value is used for all listed protocols. The message is triggered by Firefox's auto-refresh occurring in conjunction with the proxy settings.

If you want to ensure the error message does not appear rather than ignoring it, change the Firefox proxy settings to another option, such as "Auto-detect proxy settings for this network" or "Automatic proxy configuration URL." To access these options in Firefox, choose Tools > Options > Advanced > Network > Settings.

This is not our case. I go to $DOMAIN_HOME/osb/config/sessions and there are 2 active sessions, deadlocking each other. You should see a folder for each username who was activating a session.

Shut down the Admin, and manually delete both of those folders in $DOMAIN_HOME/osb/config/sessions/ .

Then try again to create and activate a session. The first time is will silenty fail, the second it will work.

If you get this message:

"[DeploymentService:290036]Deployment for request id '1340973258338' is deferred since target 'osbpr1ms1' is disconnected."

it means you must probably restart some server - no worries.

Tuesday, June 26, 2012

Tomcat Web Application Security in 20 seconds

Your web.xml:

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns:xsi="" xmlns="" xmlns:web="" xsi:schemaLocation="" id="WebApp_ID" version="2.5">
      <web-resource-name>Entire Application</web-resource-name>
    <realm-name>OsbReports Application</realm-name>


in your $TOMCAT_HOME/conf/tomcat-users.xml:

<user username="tomcat" password="tomcat" roles="tomcat,manager-gui"/>

and you are done

Monday, June 25, 2012

Oracle DB dates

Official Doc on DATE datatype:

Use TO_CHAR(mydate, 'MM/DD/YYYY HH24:MI:SS') to display the Date

WARNING: when you export your tables in Oracle SQL Developer using the "INSERT" format, DATE columns are TRUNCATED to midnigth, with a to_date('22-JUN-12','DD-MON-RR') statement.

TO_DATE('2012-08-13 11.20.00', 'YYYY-MM-DD HH24:MI:SS')

to do the opposite conversion

Saturday, June 23, 2012

Ignore Stuck Threads for JCA FileAdapter

The default threading model for FileAdapter created long running threads who poll the file system. After 600 seconds they generate Health warning and error stacktraces. To get rid of this, without losing in general the capability to detect Stuck Threads in other parts of the Server, use a WorkManager and associate it to each Proxy Service using a FileAdapter.

WLST Snippet to create WorkManager :


cd('/SelfTuning/' + DOMAIN_NAME)
cd('/SelfTuning/' + DOMAIN_NAME + '/WorkManagers/WorkManagerIgnoreStuckThreads')
set('Targets',jarray.array([ObjectName('com.bea:Name=' + CLUSTER_NAME + ',Type=Cluster')], ObjectName))


For each of the FileAdapter JCA Proxy Services, edit the JCA Properties and assign 'WorkManagerIgnoreStuckThreads' to the Dispatch Policy.

See also the official Oracle doc on this topic (search for ignore-stuck-threads)

Thursday, June 21, 2012

Monitoring HAFileAdapter

if a file has already been processed but not purged from the FILEADAPTER_IN table,
the second time it's submitted you will see


where FILE_PROCESSED = "0"

and you will get a

java.sql.SQLIntegrityConstraintViolationException: ORA-00001: unique constraint (PL1_SOAINFRA.SYS_C0026314) violated

because the file is already in the DB.

So why the file is not being purged? Because it's still with FILE_PROCESSED = "0".

I see very often:




where the first ? is:
System.currentTimeMillis() - (oracle.tip.adapter.file.timeout.recoverunpicked.minutes * 60 * 1000)

(default is 3600 000 = 1 hour)

and in the log:
FileListDAO::Recovered [0] unpicked instances diff time=[1340299775855], recoverytime=[120000]

and also


where the first ? is:
System.currentTimeMillis() - (oracle.tip.adapter.file.timeout.recoverpicked.minutes * 60 * 1000)

and also


Tuesday, June 19, 2012

Enabling diagnostic log for FileAdapter (in general for oracle.soa.adapter)

The official doc here:

open em:
and identify the FileAdapter for each MS:

right click, Logs, Log configuration:

select "loggers with persistent log state level" (NOT Runtime Loggers)

search oracle.soa.adapter, and set to TRACE:32 FINEST, scroll down and enable the "persist" checkbox, click on Apply

do it for all MS in the cluster

the logs should go to

Monday, June 18, 2012

WebLogic and RAC

in fact the GridLink DataSource

contains an array of instances, one per Oracle RAC instance

plus a ONSClientRuntimeMBean to handle communication with the ONS (oracle notification service) .

One could monitor in WebLogic the status of each individual Oracle RAC Instance...

Weblogic 11 trace and log SQL statements on a Datasource

gone are the days of P6Spy...

Gone are the days of WLSpy : "The wlspy.jar has been deprecated"

Now there is JDBCSpy:

JDBCSpy in action

Official doc:

also this article is interesting

one day also JDBCSpy will die.... and se superseded by something else ... nothing lasts forever, apart human stupidity...

I tried to make SQLSpy work, no way.... maybe it works only with WebLogic drivers, and not Oracle drivers.
I try adding the ";spyAttributes=(log=(file)acmespy.log)" string to the URL, but it fails with a

java.sql.SQLRecoverableException: IO Error: NL Exception was generated
Caused by: NL Exception was generated

So I use the weblogic debug flag -Dweblogic.debug.DebugJDBCSQL or set it from the WebLogic console in the server/debug/weblogic/jdbc/sql/DebugJDBCSQL

and you get this

####<Jun 21, 2012 6:46:18 PM CEST> <Debug> <JDBCSQL> <hqchnesoa200> <osbpl1ms1> <[STUCK] ExecuteThread: '6' for queue: 'weblogic.kern
el.Default (self-tuning)'> <<WLS Kernel>> <> <11d1def534ea1be0:-115b5766:1380fddf0f3:-8000-0000000000000234> <1340297178066> <BEA-000
000> <[[weblogic.jdbc.wrapper.JTAConnection_weblogic_jdbc_wrapper_XAConnection_oracle_jdbc_driver_LogicalConnection-SOAXADataSource-1
830, oracle.jdbc.driver.LogicalConnection@5249d560]] prepareStatement(DELETE FROM FILEADAPTER_IN WHERE FILE_PROCESSED='2' AND FILE_R

Sunday, June 17, 2012

oracle sql developer keep connection alive

this man is a genious.

Book: My Holocaust, by Java Reich

I read this book because part of my family is Jewish, and because I am really annoyed by how the Holocaust is used these days to justify Israel imperialism in the Middle East - while no mention is made about the Palestinian Holocaust.

The style is definitely heavy: very long sentences, lot of blablabla about irrelevant facts; there is some juicy stuff, but it's buried under a lot of uninteresting bla.
I had to make an effort to read it all.

Still, an interesting book, who says all what needs to be said about the manipulation of the Holocaust.

At the end, a sentence sent the shivers down my spine:

"... yours was a transition Holocaust. Your Holocaust is history - buried in the archives and the tombs. It had a longer-than-average run for a human tragedy and atrocity, but finally and at long last, the Shoah's over, the curtain has fallen, the thumbs of the people have turned down. It's our turn now. A new universal Holocaust is coming, the horror of which has never been seen before and will never be seen again... Your Holocaust has been superseded, eclipsed. The blazing signs have been seared across the sky.
Blood.And fire. And column of smoke.
Total cremation. Everywhere ash."

I don't expect the 21st century to be a lot of fun... while climate crisis aggravates, and available resources (oir, water, soil, biodiversity) collapse, there will be a lot less support for human life on this planet... By the end of the century, I expect 5 of the 7 billion people living today to be eliminated in a war, in famine or epidemics or natural disasters. It will be a massacre compared to which WWII will be a sit-in comedy.
We have cut the branch on which we were sitting, and now we are in free, accelerated fall.

But, don't worry, keep driving your car, drinking Coke, eating meat, flying around, destroying the planet, today is just another day to enjoy the collective Potlatch that we insist upon calling "civilization". Tomorrow, there is no tomorrow.

Saturday, June 16, 2012


The right configuration for wlsbjmsrpDataSource is:

"Supports Global Transactions" ENABLED
"Logging Last Resource" ENABLED

If you set "One-Phase Commit" ENABLED, this leads to this error message:

Exception occurred during commit of transaction Name=NewJMSMessagePoller.ReportingMDB,Xid=BEA1-76E0B53E352F1F5F1C3A(131923607)
,Status=Rolled back. [Reason=javax.transaction.xa.XAException: JDBC driver does not support XA, hence cannot be a participant in two-
phase commit. To force this participation, set the GlobalTransactionsProtocol attribute to LoggingLastResource (recommended) or Emula
teTwoPhaseCommit for the Data Source = wlsbjmsrpDataSourceGL],numRepliesOwedMe=0,numRepliesOwedOthers=0,seconds since begin=0,seconds
back,assigned=osbpr1ms1),xar=WLStore_osbpr1do_FileStore_auto_1599741968,re-Registered = false),XAServerResourceInfo[wlsbjmsrpDataSour
SEmulateXAResourceImpl@42f8ac2b,re-Registered = false),SCInfo[osbpr1do+osbpr1ms1]=(state=rolledback),properties=({weblogic.transactio, weblogic.jdbc.remote.wlsbjmsrpDataSourceGL=t3://}),OwnerTransactionManager=ServerTM[ServerCoordinatorDescriptor=(, XA
Resources={WLStore_osbpr1do__WLS_osbpr1ms1, eis/activemq/Queue, wlsbjmsrpDataSourceGL_osbpr1do, eis/fioranomq/Topic, eis/jbossmq/Queu
e, WLStore_osbpr1do_FileStore_auto_1, WSATGatewayRM_osbpr1ms1_osbpr1do, SOADataSource_osbpr1do, eis/webspheremq/Queue, eis/AQ/aqSampl
e, WLStore_osbpr1do_NesoaCommonFileStore1, WLStore_osbpr1do_FileStorePerfStore1, eis/aqjms/Queue, NESOADataSource_osbpr1do, eis/tibjm
s/Queue, eis/sunmq/Queue, eis/pramati/Queue, eis/tibjms/Topic, WLStore_osbpr1do_WseeFileStore_auto_1, SOAXADataSource_osbpr1do, eis/A
pps/Apps, eis/wls/Queue, eis/tibjmsDirect/Topic, eis/wls/Topic, eis/aqjms/Topic, eis/tibjmsDirect/Queue},NonXAResources={})],Coordina weblogic.transaction.RollbackException: Could not prepare resourc
e 'wlsbjmsrpDataSourceGL_osbpr1do

Thursday, June 14, 2012

Wisdom of Wolves

I receive this and gladly publish. An IT team should be like a pack of wolves, highly cooperative and communicative. Most social animals are far better at coordinating each other than IT teams. Probably because they don't have managers :o) .

The Introduction from
Wisdom of Wolves
by Twyman Towery

The attitude of the wolf can be summed up simply: it is a constant visualization of success. The collective wisdom of wolves has been progressively programmed into their genetic makeup throughout the centuries. Wolves have mastered the technique of focusing their energies toward the activities that will lead to the accomplishment of their goals.

Wolves do not aimlessly run around their intended victims, yipping and yapping. They have a strategic plan and execute it through constant communication. When the moment of truth arrives, each understands his role and understands exactly what the pack expects of him.

The wolf does not depend on luck. The cohesion, teamwork and training of the pack determines whether the pack lives or dies.

There is a silly maxim in some organizations that everyone, to be a valuable member, must aspire to be the leader. This is personified by the misguided CEO who says he only hires people who say they want to take his job. Evidently, this is supposed to ensure that the person has ambition, courage, spunk, honesty, drive - whatever. In reality, it is simply a contrived situation, with the interviewee jumping through the boss's hoops. It sends warnings of competition and one-upmanship throughout the organization rather than signals of cooperation, teamwork and loyalty.

Everyone does not strive to be the leader in the wolf pack. Some are consummate hunters or caregivers or jokesters, but each seems to gravitate to the role he does best. This is not to say there are not challenges to authority, position and status - there are. But each wolf's role begins emerging from playtime as a pup and refines itself through the rest of its years. The wolf's attitude is always based upon the question, "What is best for the pack?" This is in marked contrast to us humans, who will often sabotage our organizations, families or businesses, if we do not get what we want.

Wolves are seldom truly threatened by other animals. By constantly engaging their senses and skills, they are practically unassailable. They are masters of planning for the moment of opportunity to present itself, and when it does, they are ready to act.

Because of training, preparation, planning, communication and a preference for action, the wolf's expectation is always to be victorious. While in actuality this is true only 10 percent of the time or less, the wolf's attitude is always that success will come-and it does.

Monday, June 11, 2012

WLST script to monitor for the presence of files in a directory

It will poll every second a directory, and write to a log file the name of all the files it can find...

Very useful to monitor the behaviour of a File Adapter...

#Checks at regular intervals for the presence of a file in a Directory
import os
import datetime
import time

dirToWatch = '/data/my/directory' 
while 1 == 1 :
     files = os.listdir(dirToWatch)
     if len(files) > 0:
        logfile = open("fileslog.log", "a")
        logfile.write(str( + " " + str(files) + "\n")

Sunday, June 10, 2012

PragmaticThinking and Learning, by Andy Hunt

This is a great book, IT is not mainly about IT, it's about how our brain work.

I dare to report (hope I won't be sued) a brilliant excerpt on CONTEXT SWITCHING:

Context Switching
How much disposable attention do you have?

That is, how many things are currently demanding your attention, and how many total
things can you possibly pay attention to at any given point in time?
The ratio is pretty unfavorable.
You can’t pay attention to as much as you think you can.
We can’t pay attention to too many different things at once, because
to change your focus from one item to another means you have to
switch context. And unfortunately, our brains just aren’t wired to
support context switching very well.
Multitasking takes a heavy toll on productivity.
One study  found that in general, multitasking
can cost you 20 to 40 percent of your productivity.
Right there, that cuts your eight-hour workday
down to five. Other studies bump that number
up closer to 50 percent, with a huge increase
in errors, as well.

Just to clarify, multitasking here refers to performing multiple concurrent
tasks at different levels of abstraction. Fixing a couple of
bugs while in the same area of code doesn’t count as multitasking,
nor does returning several similar phone calls or cooking a multicourse
meal. You get into trouble when you interrupt your codefixing
session by responding to an unrelated IM, email, or phone
call or take a quick peek at a news site.
It takes twenty minutes to reload context.
Unlike computers, our brains don’t have
a “save stack” or “reload stack” operation.
Instead, you have to drag all the bits of
memory back in, sort of one-by-one
. That
means that if you are deep into a task (such as debugging) and then
get interrupted, it can take you an average of twenty minutes to get
back into it. Twenty minutes. Consider how many interruptions
you may get in a given day; if each interruption requires twenty
minutes for you to recover, a good portion of your day is just plain
This could explain why programmers in general hate to be
interrupted—especially by nonprogrammers.

In today’s digital culture, this is part of a larger, dangerous phenomenon
known as cognitive overload. It’s a cocktail of stress, too
much multitasking, too many distractions, and the frequent flurry
of new data to deal with
. Scientists agree that trying to focus on
several things at once means you’ll do poorly at each of them
And if that wasn’t bad enough, a controversial study done in the
United Kingdom noted that if you constantly interrupt your task
to check email or respond to an IM text message, your effective IQ
drops ten points.
By comparison, smoking a marijuana joint drops your IQ a mere
four points (see Figure 8.4, on the next page).
Whatever you do, please don’t do both.

Saturday, June 9, 2012

Ubuntu server Windows installer on usb drive

Buy (or steal :o) ) a 32 GB USB Stick

Download Ubuntu Server ubuntu-12.04-server-amd64.iso:

Download LinuxLive USB Creator :

The USB creator is very self-explanatory

Bicycle Thieves in Lausanne...

third bicycle stolen in 3 months.... Lausanne is more dangerous than Lagos...

Here one of the greatest Italian movies of all times, directed by the great Vittorio de Sica:

Friday, June 8, 2012

Tomcat change heap size

cd /home/soa/software/apache-tomcat-7.0.27/bin

just before "if [ -z "$LOGGING_MANAGER" ]; then" insert:

JAVA_OPTS="$JAVA_OPTS -Xms536m -Xmx736m -XX:NewSize=256m -XX:MaxNewSize=256m -XX:PermSize=256m -XX:MaxPermSize=256m"

Oracle DbAdapter, FileAdapter, FTPAdapter automated customization

Put in SVN the Deployment Plan created manually in an environment - if makes no sense to create the Plan.xml programmatically, it's only asking for trouble.

In them, replace the actual environment-dependent values with tokens like ${DOMAIN_NAME}
the tokens are:

The JNDI names of all your DataSources will not change from environment to environment, so there is no point in replacing them.

create a file like this:







create a WLST script like this:

# Configure a new environment with the Plans.xml from SVN
# uses file.

from import FileInputStream
from shutil import copyfile
import os, sys
import re

#not used, keep only as a reference
def copyfileobj(fsrc, fdst, length=16*1024):
    """copy data from file-like object fsrc to file-like object fdst"""
    while 1:
        buf =
        if not buf:

def copyfileWithTokenSubstitution(filein, fileout, properties):
        input = open(filein)
        output = open(fileout, 'w')
        for s in input:
            rep = s
            for tokenname in properties.get("tokens").split(','):
                rep = rep.replace("${" + tokenname + "}", properties.get(tokenname))
#not used, keep only as a reference
def copyfile(src, dst):
    """Copy data from src to dst"""
    fsrc = None
    fdst = None
        fsrc = open(src, 'rb')
        fdst = open(dst, 'wb')
        copyfileobj(fsrc, fdst)
        if fdst:
        if fsrc:

propertyFileName = ''    

#loading properties
print 'Loading properties from ', propertyFileName
propInputStream = FileInputStream(propertyFileName)
configProps = Properties()

domainName = configProps.get('DOMAIN_NAME')


planArray = []
adapterArray = []

#create needed directories where to put *Plan.xml
for plan in plans.split(','):
    planFullPath = configProps.get(plan).replace("${DOMAIN_NAME}", domainName)

#create array of adapters
for adapter in adapters.split(','):
    adapterName = configProps.get(adapter)

#create dir if doesn't exist - fail if unable to create it
for planFullPath in planArray:    
    dirName = os.path.dirname(planFullPath)
    print "creating directory " + dirName
    if not os.path.exists(dirName):
#check for directory existence
for planFullPath in planArray:
    dirName = os.path.dirname(planFullPath)
    print "testing directory " + dirName
    if not os.path.exists(dirName):
        message = "directory " + dirName + " does not exist"
        print message
        raise Exception(message)


#copy all Plan.xml files to their final destination, with token substitution
for planFullPath in planArray:
    fileName = os.path.basename(planFullPath)
    print "copying " + fileName + " to " + planFullPath  
    copyfileWithTokenSubstitution(fileName, planFullPath, configProps)

#Connect to Admin Server    

#applying changes to the Adapters
    for index in range(len(planArray)):
        plan = planArray[index]
        adapter = adapterArray[index]
        adapterType = os.path.basename(adapter).split('.')[0]
        print 'Applying plan ' + plan + " to adapter " + adapter + " (adapter type is " + adapterType + ")"
        myPlan = loadApplication(adapter, plan)
        cd('/AppDeployments/' + adapterType + '/Targets')
        #updateApplication(appName, planPath);
        redeploy(adapterType, plan, targets = cmo.getTargets())

    message="unable to finish job"
    raise Exception(message)

print "job finished successfully"

Wednesday, June 6, 2012

OSB send html email

Ok this is what you hit in google

I have tried and it's simple:

create a Business Service

Service Type: Messaging Service
Request Message Type : text
Protocol : email
Endpoint URI:
SMTP Server : mysmtpserver

you can test the BS:

<b>ciao bello</b>
<i>ciao bello</i>

as long as you specify well formed HTML it should not pose any XML validation problem.

and make sure in the Transport Headers part you specify

Content-Type: text/html

If you go through a Proxy Service, it ca take care of that for you, but the problem is that it seems to precede it with a "xml" header:

<?xml version="1.0" encoding="ISO-8859-1"?> <html> <body> <b>ciao bello</b> <i>ciao bello</i> </body> </html>

Sunday, June 3, 2012

Puppet Labs!

I have been craving to learn Puppet for a while.... I am simply fed up of spending so much of my time harping on 7 different putty sessions and copying and pasting shell script and WLST code everywhere... and having to discover manually configuration of WebLogic, NICs, OS and chase changes that people make without telling me ...

Now is the time ... if not now, when? if not us, who? if not here, where?

Here is the Overview page

This is a cool hands-on minimalistic video on how to setup a Puppet client and server

WLST to delete JMS messages

this is the complicated version:

This is the "easy" one:


for i in range (1,5):
   connect('weblogic', 'weblogic1', 't3://acme-osbpp1ms' + str(i) + '')
   cd('osbpp1ms' + str(i) + '.jms')
   cd(jmsserver + str(i))
   cd(jmsmodule + '!' + jmsserver + str(i) + '@' + jmsqueue)

(replace values with your own parameters) Exception occured while downloading files, Posted content doesn't set it's Content-Length

in nodemanager.log :

<Jun 3, 2012 5:20:07 PM> <INFO> <osbpp1do> <osbpp1ms3> <Boot identity properties saved to "/opt/oracle/domains/osbpp1do/servers/osbpp1ms3/data/nodemanager/">
<Jun 3, 2012 5:20:07 PM> <INFO> <osbpp1do> <osbpp1ms3> <Startup configuration properties saved to "/opt/oracle/domains/osbpp1do/servers/osbpp1ms3/data/nodemanager/">
<Jun 3, 2012 5:20:07 PM> <INFO> <osbpp1do> <osbpp1ms3> <Rotated server output log to "/opt/oracle/domains/osbpp1do/servers/osbpp1ms3/logs/osbpp1ms3.out00116">
<Jun 3, 2012 5:20:07 PM> <INFO> <osbpp1do> <osbpp1ms3> <Server error log also redirected to server log>
<Jun 3, 2012 5:20:07 PM> <INFO> <osbpp1do> <osbpp1ms3> <Starting WebLogic server with command line: /opt/oracle/domains/osbpp1do/bin/ >
<Jun 3, 2012 5:20:07 PM> <INFO> <osbpp1do> <osbpp1ms3> <Working directory is '/opt/oracle/domains/osbpp1do'>
<Jun 3, 2012 5:20:07 PM> <INFO> <osbpp1do> <osbpp1ms3> <Rotated server output log to "/opt/oracle/domains/osbpp1do/servers/osbpp1ms3/logs/osbpp1ms3.out00117">
<Jun 3, 2012 5:20:07 PM> <INFO> <osbpp1do> <osbpp1ms3> <Server error log also redirected to server log>
<Jun 3, 2012 5:20:07 PM> <INFO> <osbpp1do> <osbpp1ms3> <Server output log file is '/opt/oracle/domains/osbpp1do/servers/osbpp1ms3/logs/osbpp1ms3.out'>
<Jun 3, 2012 5:20:11 PM> <INFO> <Successfully brought with netmask online on bond0:1>
<Jun 3, 2012 5:20:13 PM> <INFO> <Successfully removed from bond0:1.>
<Jun 3, 2012 5:20:13 PM> <INFO> <osbpp1do> <osbpp1ms3> <Server failed during startup so will not be restarted>
<Jun 3, 2012 5:20:13 PM> <WARNING> <Exception while starting server 'osbpp1ms3'> Server failed to start up. See server output log for more details.
at weblogic.nodemanager.server.AbstractServerManager.start(
at weblogic.nodemanager.server.ServerManager.start(
at weblogic.nodemanager.server.Handler.handleStart(
at weblogic.nodemanager.server.Handler.handleCommand(

in /opt/oracle/domains/osbpp1do/servers/osbpp1ms3/logs/osbpp1ms3.out:

<Jun 3, 2012 5:20:13 PM CEST> <Critical> <WebLogicServer> <BEA-000362> <Server failed. Reason:

There are 1 nested errors: Exception occured while downloading files
at weblogic.deploy.internal.targetserver.datamanagement.ConfigDataUpdate.doDownload(
at weblogic.deploy.internal.targetserver.datamanagement.Data.prepareDataUpdate(
at weblogic.t3.srvr.ServerServicesManager.startService(
at weblogic.t3.srvr.ServerServicesManager.startInStandbyState(
at weblogic.t3.srvr.T3Srvr.initializeStandby(
at weblogic.t3.srvr.T3Srvr.startup(
at weblogic.Server.main(
Caused by: Posted content doesn't set it's Content-Length
at weblogic.deploy.service.datatransferhandlers.MultipartParser.parseResponse(
at weblogic.deploy.service.datatransferhandlers.MultipartParser.<init>(
at weblogic.deploy.service.datatransferhandlers.MultipartParser.<init>(
at weblogic.deploy.service.datatransferhandlers.HttpDataTransferHandler.getDataAsStream(
at weblogic.deploy.service.datatransferhandlers.DataHandlerManager$RemoteDataTransferHandler.getDataAsStream(
at weblogic.deploy.internal.targetserver.datamanagement.ConfigDataUpdate.doDownload(
... 13 more


(btw it's not "it's Content-Length", but "its Content-Length"... lot of IT people are illiterate...)

Here I google and find the solution:

“As per KM 1294379.1, Released that there were 0sized files under the config directory of domain which were not being synchronized, removing those files fixed the problem.”

on the admin box (105):

find /opt/oracle/domains/osbpp1do/config -type f -size 0 -exec ls {} \; | xargs ls -ltr

-rw-r----- 1 soa soa 0 Nov 24 2011 /opt/oracle/domains/osbpp1do/config/fmwconfig/servers/osbpp1as/applications/OracleBamAdapter/config/BAMCommonConfig.xml.jmxori
-rw-r----- 1 soa soa 0 Nov 24 2011 /opt/oracle/domains/osbpp1do/config/fmwconfig/servers/osbpp1as/applications/OracleBamAdapter/config/BAMCommonConfig.xml
-rw-r----- 1 soa soa 0 Nov 24 2011 /opt/oracle/domains/osbpp1do/config/fmwconfig/servers/osbpp1ms1/applications/OracleBamAdapter/config/BAMCommonConfig.xml.jmxori
-rw-r----- 1 soa soa 0 Nov 24 2011 /opt/oracle/domains/osbpp1do/config/fmwconfig/servers/osbpp1ms1/applications/OracleBamAdapter/config/BAMCommonConfig.xml

-rw-r----- 1 soa soa 0 Jun 3 15:14 /opt/oracle/domains/osbpp1do/config/fmwconfig/servers/osbpp1ms4/applications/OracleBamAdapter/config/BAMCommonConfig.xml.jmxori
-rw-r----- 1 soa soa 0 Jun 3 15:14 /opt/oracle/domains/osbpp1do/config/fmwconfig/servers/osbpp1ms4/applications/OracleBamAdapter/config/BAMCommonConfig.xml

deleted those 2 files in bold. All works again.

Those 2 files were generated by a MS which was migrated to the same box of the Admin, and who left behind some config files... when you start any MS, they fetch the latest configuration files from the Admin, and if one of those files has length = 0, it fails! Bummer!

java.lang.NoClassDefFoundError: oracle/integration/platform/kernel/FabricMeshUtils

We often get this error while using OSB with JCA File Adapter:

java.lang.NoClassDefFoundError: oracle/integration/platform/kernel/FabricMeshUtils
        at oracle.tip.adapter.file.inbound.FileListDAO.(
        at oracle.tip.adapter.file.inbound.PollWork.(
        at oracle.tip.adapter.file.FileResourceAdapter.endpointActivation(

the class is in C:\Oracle\Middleware\jdeveloper\soa\modules\oracle.soa.fabric_11.1.1\fabric-runtime.jar

or in

(better use the latter...)

and in oracle.tip.adapter.file.inbound.FileListDAO there is clearly a dependency:
import oracle.integration.platform.blocks.FabricConfigManager;
import oracle.integration.platform.kernel.FabricMeshUtils;

When loading this class fails, you will see in the logs "Unable to retrieve database info"

After adding the fabric-runtime.jar file to $DOMAIN_HOME/lib, I still get:

While trying to lookup 'soa-infra:comp.ApplicationContext' didn't find subcontext 'soa-infra:comp'. Resolved ''
javax.naming.NameNotFoundException: While trying to lookup 'soa-infra:comp.ApplicationContext' didn't find subcontext 'soa-infra:comp'. Resolved ''; remaining name 'soa-infra:comp/ApplicationContext'
at weblogic.jndi.internal.BasicNamingNode.newNameNotFoundException(
at weblogic.jndi.internal.BasicNamingNode.lookupHere(
at weblogic.jndi.internal.ServerNamingNode.lookupHere(
at weblogic.jndi.internal.BasicNamingNode.lookup(
at weblogic.jndi.internal.WLEventContextImpl.lookup(
at weblogic.jndi.internal.WLContextImpl.lookup(
at javax.naming.InitialContext.lookup(
at oracle.integration.platform.kernel.FabricMeshUtils.getApplicationContext(
at oracle.tip.adapter.file.inbound.FileListDAO.<init>(
at oracle.tip.adapter.file.inbound.PollWork.<init>(
at oracle.tip.adapter.file.FileResourceAdapter.endpointActivation(

RESOLUTION: we decided to ignore this message, if you want to configure the FileAdapter for useCompression you can do so with the JCA Activation Properties.
I assume all works fine in SOA Suite, where the FabricMeshUtils is surely available in the classpath.

In fact, further investigation proves that the only side effect of this error is that the useCompression (default: false) flag can be set only using the JCA Activation Properties, and not with a DB value (in any case OSB doesn't use any DB to store configuration information).

WebLogic 11 Whole Server Migration

previous notes here

Here the doc:

Here an excellent crash course.

To provoke a server migration:
kill -9 PID
when NodeManager restarts the MS, kill again!

The second time, the NodeManager will not try to restart the MS locally, and will let che ClusterMaster decide another machine on which to start the MS

This is what is traced in the nodemanager.log

Successfully removed from bond0:4.
Server failed so attempting to restart (restart count = 1)
Sleeping for 30 seconds before attempting to restart server
Starting WebLogic server with command line: /opt/oracle/domains/osbpp1do/bin/
Working directory is '/opt/oracle/domains/osbpp1do'
Rotated server output log to "/opt/oracle/domains/osbpp1do/servers/osbpp1ms1/logs/osbpp1ms1.out00132"
Server error log also redirected to server log
Server output log file is '/opt/oracle/domains/osbpp1do/servers/osbpp1ms1/logs/osbpp1ms1.out'
Successfully brought with netmask online on bond0:4
Successfully removed from bond0:4.
Server failed during startup so will not be restarted

Server tribes migrating in the desert.

To discover on which machine it was started: open the console, MS, Control, Migration, "Current Machine:".

Now, I think something is weird.... let's say I have a cluster with
4 machines: m1, m2, m3 and m4,
4 managed servers: ms1, ms2, ms3, ms4

I kill ms1 on m1 and make it migrate to m2.
Now, I expect to be able to "fallback" from m2 to m1. I go to server/control/migration and, lo and behold, in the list "migrate to machine" I have only m2, m3, m4 (no m1)... incidentally, since m2 is the current machine, it makes no sense to migrate to m2...

Also, if you click on "Migratable Targets", the column "Current Hosting Server" reports the wrong information, m1 instead of m2.... but at least here, if you take the lock, select the ms1 and do "migrate", it gives you the option to migrate ms1 (migratable) back to hosting server ms1 (what does it mean "hosting server ms1"? even if I request migration, it fails with a Cannot migrate - destination server must not be the currently activate server - services are already deployed on that server.


How can I migrate a server with WLST?

Here the doc

Saturday, June 2, 2012

SQLIntegrityConstraintViolationException while the LeaseManager updates the ACTIVE table

####<May 31, 2012 11:14:06 AM CEST> <Debug> <ServerMigration> <hqchacme105> <osbpp1ms1> <[ACTIVE] ExecuteThread: '32' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <553e43a3c186ec6d:58114d2:1379f528b89:-7ffd-0000000000008622> <1338455646674> <BEA-000000> <Query: UPDATE ACTIVE SET TIMEOUT = ( (SYSDATE + (30/86400))) WHERE SERVER IN ('wlsserver.osbpp1ms1') AND INSTANCE ='-5605988896686436333/osbpp1ms1' AND DOMAINNAME='osbpp1do' AND CLUSTERNAME='osbpp1cl'>

####<May 31, 2012 11:14:07 AM CEST> <Debug> <ServerMigration> <hqchacme105> <osbpp1ms1> <[ACTIVE] ExecuteThread: '23' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <553e43a3c186ec6d:58114d2:1379f528b89:-7ffd-0000000000008626> <1338455647328> <BEA-000000> <<DatabaseLeasingBasis>: tryAcquire(service.SINGLETON_MASTER, 30000)>

####<May 31, 2012 11:14:07 AM CEST> <Debug> <ServerMigration> <hqchacme105> <osbpp1ms1> <[ACTIVE] ExecuteThread: '23' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <553e43a3c186ec6d:58114d2:1379f528b89:-7ffd-0000000000008626> <1338455647329> <BEA-000000> <Query: DELETE FROM ACTIVE WHERE (SYSDATE > TIMEOUT) AND SERVER = 'service.SINGLETON_MASTER' AND DOMAINNAME='osbpp1do' AND CLUSTERNAME='osbpp1cl'>

####<May 31, 2012 11:14:07 AM CEST> <Debug> <ServerMigration> <hqchacme105> <osbpp1ms1> <[ACTIVE] ExecuteThread: '23' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <553e43a3c186ec6d:58114d2:1379f528b89:-7ffd-0000000000008626> <1338455647331> <BEA-000000> <Query: INSERT INTO ACTIVE ( SERVER, INSTANCE, DOMAINNAME, CLUSTERNAME, TIMEOUT ) VALUES ( 'service.SINGLETON_MASTER' , '-5605988896686436333/osbpp1ms1' , 'osbpp1do' , 'osbpp1cl' , (SYSDATE + (30/86400)) )>

####<May 31, 2012 11:14:07 AM CEST> <Debug> <ServerMigration> <hqchacme105> <osbpp1ms1> <[ACTIVE] ExecuteThread: '23' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <553e43a3c186ec6d:58114d2:1379f528b89:-7ffd-0000000000008626> <1338455647333> <BEA-000000> <Attempted Query: INSERT INTO ACTIVE ( SERVER, INSTANCE, DOMAINNAME, CLUSTERNAME, TIMEOUT ) VALUES ( 'service.SINGLETON_MASTER' , '-5605988896686436333/osbpp1ms1' , 'osbpp1do' , 'osbpp1cl' , (SYSDATE + (30/86400)) )>

####<May 31, 2012 11:14:07 AM CEST> <Debug> <ServerMigration> <hqchacme105> <osbpp1ms1> <[ACTIVE] ExecuteThread: '23' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <553e43a3c186ec6d:58114d2:1379f528b89:-7ffd-0000000000008626> <1338455647334> <BEA-000000> <Unexpected exception

java.sql.SQLIntegrityConstraintViolationException: ORA-00001: unique constraint (PP1_SOAINFRA.SYS_C0013550) violated
at oracle.jdbc.driver.T4CTTIoer.processError(
at oracle.jdbc.driver.T4CTTIoer.processError(
at oracle.jdbc.driver.T4C8Oall.processError(
at oracle.jdbc.driver.T4CTTIfun.receive(
at oracle.jdbc.driver.T4CTTIfun.doRPC(
at oracle.jdbc.driver.T4C8Oall.doOALL(
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(
at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(
at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeUpdate(
at weblogic.jdbc.wrapper.PreparedStatement.executeUpdate(
at weblogic.cluster.singleton.DatabaseLeasingBasis.assumeLease(
at weblogic.cluster.singleton.DatabaseLeasingBasis.acquire(
at weblogic.cluster.singleton.LeaseManager.timerExpired(

this is the content of the ACTIVE table


"service.SINGLETON_MASTER" "2372471525479840300/osbpp1ms2" "osbpp1do" "osbpp1cl" 31-MAY-12

"wlsserver.osbpp1ms1" "-5605988896686436333/osbpp1ms1" "osbpp1do" "osbpp1cl" 31-MAY-12

"service.CLUSTER_MASTER" "2372471525479840300/osbpp1ms2" "osbpp1do" "osbpp1cl" 31-MAY-12

"wlsserver.osbpp1ms4" "2967680027205630303/osbpp1ms4" "osbpp1do" "osbpp1cl" 31-MAY-12

"wlsserver.osbpp1ms2" "2372471525479840300/osbpp1ms2" "osbpp1do" "osbpp1cl" 31-MAY-12

"wlsserver.osbpp1ms3" "1002143801186479830/osbpp1ms3" "osbpp1do" "osbpp1cl" 31-MAY-12

and the constraint is:


"SYS_C0013550" "Primary_Key" "" "" "" "" "" "ENABLED" "NOT DEFERRABLE" "VALIDATED" "GENERATED NAME" "" "" 26-APR-12 "PP1_SOAINFRA" "SYS_C0013550" "" ""

and the DDL is:


-- DDL for Table ACTIVE











probably the DELETE doesn’t delete the row....

This is what the official documentation says:

6.1.14 Harmless SQLIntegrityConstraintViolationException Can be Received in a SOA Cluster

The following SQLIntegrityConstraintViolationException can be received in a SOA cluster:

[TopLink Warning]: 2010.04.11 14:26:53.941--UnitOfWork(275924841)--Exception

[TOPLINK-4002] (Oracle TopLink - 11g Release 1 (

Internal Exception: java.sql.SQLIntegrityConstraintViolationException:

ORA-00001: unique constraint (JYIPS2RC4B49_SOAINFRA.SYS_C0035333) violated




This is not a bug. In a cluster environment, when the messages for the same group arrive on both the nodes, one node is bound to experience this exception for the first message. The application is aware of this exception and handles it properly. It does not break any functionality.

This exception can also come on a single node after you restart the server and send the message for the existing group. Again, this exception will be experienced on the very first message.

In summary, this exception is within the application design and does not impact any functionality. It is for this reason that you do not see this exception logged as severe in the soa-diagnostic logs.

Toplink does, however, log it in its server logs.

I hope this is our case... harmless....

Putty multitab

much, much better than having all those Putty windows open...

It has even a "send script" feature, by which you can run the same commands on multiple servers at the same time!