Sunday, September 30, 2012

tracert tutorial

Wikipedia pages to read if you want to become a network expert

interesting Linux Networking Manual
Basic intro to several commands

ntop tutorial

tcpdump tutorial

tcpdump -D
tcpdump -i eth0 v
tcpdump -i eth0 port http (to display only http traffic)
tcpdump -n (to display numeric IPs)
tcpdump -w session filename.log (captures data to a file)
tcpdump -r session filename.log (displays the content of the captured session)
This is a very concise excellent quick guide of the main commands:
To use wireshark to view packets captured with tcpdump:
tcpdump -i -s 65535 -w

Tcpdum for Windows available here

Friday, September 28, 2012

Wireshark tutorial

and also this one is very informative

Quick WLST script to create WLS domain using a template

set('ListenPort', 7001)
setOption('OverwriteDomain', 'true')

Thursday, September 27, 2012

WebLogic graceful shutdown waiting forever

thread A:

"[ACTIVE] ExecuteThread: '23' for queue: 'weblogic.kernel.Default (self-tuning)'" waiting for lock java.lang.Object@2eed24e4 WAITING java.lang.Object.wait(Native Method) weblogic.t3.srvr.GracefulShutdownRequest.waitForCompletion( weblogic.t3.srvr.ServerRuntime.shutdown( weblogic.server.RemoteLifeCycleOperationsImpl.shutdown( weblogic.server.RemoteLifeCycleOperationsImpl_WLSkel.invoke(Unknown Source)

thread B:

"[ACTIVE] ExecuteThread: '18' for queue: 'weblogic.kernel.Default (self-tuning)'" waiting for lock weblogic.deploy.internal.targetserver.BasicDeployment$AdminModeCallback@53110ccd WAITING java.lang.Object.wait(Native Method) weblogic.deploy.internal.targetserver.BasicDeployment$AdminModeCallback.waitForCompletion( weblogic.deploy.internal.targetserver.BasicDeployment.gracefulProductionToAdmin( weblogic.deploy.internal.targetserver.BasicDeployment.productionToAdminFromServerLifecycle($1.gracefulProductionToAdmin($4.transitionApp( weblogic.t3.srvr.ServerServicesManager.stopInternal( weblogic.t3.srvr.ServerServicesManager.stop( weblogic.t3.srvr.T3Srvr.suspend( weblogic.t3.srvr.T3Srvr.shutdown( weblogic.t3.srvr.T3Srvr.gracefulShutdown($

We find out that

stopInternal(servicesAfterAdminState.toArray(), flag);

 cycles over all services.

 To observe which service is stuck we have to enable a debug flag:

private static final DebugCategory debugSLC = Debug.getCategory("weblogic.slc");
private static final DebugLogger debugSLCWLDF = DebugLogger.getDebugLogger("DebugServerLifeCycle");

This flag is under t3.srvr in weblogic server debug tab
(read this fascinating post on the topic)

If you enable the flags, and do the graceful shutdown, you will see stuff like:

BEA-000000 calling stop on

BEA-149059 Module jejb_transport.jar of application JEJB Transport Provider is transitioning from STATE_ACTIVE to STATE_ADMIN on server osbdv1ms1.

BEA-149060 Module jejb_transport.jar of application JEJB Transport Provider successfully transitioned from STATE_ACTIVE to STATE_ADMIN on server osbdv1ms1

In fact, the call to DeploymentPostAdminServerService.stop()  invokes a

which again calls a transitionApps(AppTransition.GRACEFUL_PRODUCTION_TO_ADMIN)

You should also grep for "RunningToAdmin for app" (traced in BasicDeployment.productionToAdminFromServerLifecycle() )

so you should also enable the "DeploymentDebug" flag:
DebugDeploy DebugDeployment DebugDeploymentService DebugDeploymentServiceInternal DebugDeploymentServiceStatusUpdates DebugDeploymentServiceTransport DebugDeploymentServiceTransportHttp
having enabled these flags, you will see Sep 28, 2012 6:04:15 PM CEST Debug Deployment DmsThread-1 WLS Kernel 0000JcD_5_bE8Tk5ozx0iY1GPSV1000002 1348848255372 BEA-000000 Targets for app, DbAdapter
In practice, if you untarget the "JCA Transport Provider" the Server will shut down gracefully.

Coding Architects vs Non-Coding Architects

great reading!

Friday, September 21, 2012

Migratory birds over Geneva lake

one of the most breathtaking visions in my life. enjoy the extraordinary harmony of this miracle of life. camera is very shaky.... I am no Fellini or Pasolini...

Tuesday, September 18, 2012

Eclipse failed to create virtual machine

I have downloaded the Scala IDE based on Eclipse here

When I run it by double clicking on "Eclipse.exe", I get "Eclipse failed to create virtual machine".

Just open eclipse.ini, and add these 2 lines at the top:
C:\Program Files\Java\jre6\bin\javaw.exe
(your mileage might vary)

Another good reason to shun Eclipse - I didn't need a new one, I already had thousands.

Monday, September 17, 2012

ORA-00932: inconsistent datatypes: expected NUMBER got INTERVAL DAY TO SECOND

It looks like you are trying to do maths (+, -) with TIMESTAMP.
TIMESTAMP doesn't like that. you should CAST the TIMESTAMP to DATE:

rather than
bla - blu (where bla and blu are TIMESTAMP)
CAST (bla as DATE) - CAST (blu as DATE)
and you will get a NUMBER (multiply it by 3600 * 24 and you will turn it into seconds)

you will lose the millisecond info

Here you have the definition of the TIMESTAMP

In alternative, you should rather EXTRACT each component of the DIFFERENCE (day, hours, minutes, seconds) and do the match with them:

as explained here

start_time_in TIMESTAMP
, end_time_in TIMESTAMP
l_days NUMBER;
l_hours NUMBER;
l_minutes NUMBER;
l_seconds NUMBER;
l_milliseconds NUMBER;
SELECT extract(DAY FROM end_time_in-start_time_in)
, extract(HOUR FROM end_time_in-start_time_in)
, extract(MINUTE FROM end_time_in-start_time_in)
, extract(SECOND FROM end_time_in-start_time_in)
INTO l_days, l_hours, l_minutes, l_seconds
FROM dual;

l_milliseconds := l_seconds*1000 + l_minutes*60*1000 + l_hours*60*60*1000 + l_days*24*60*60*1000;
RETURN l_milliseconds;


Experimenting with Date/Timestamp conversions with Oracle PL/SQL

select CURRENT_TIMESTAMP from dual;

this returns 17-SEP-12 PM EUROPE/BERLIN
(with millisecond and TimeZone)

What is the difference between CURRENT_TIMESTAMP and SYSTIMESTAMP ?

0 0:0:0.000017

basically NONE.


How to convert a String such as
value = '2012-09-11 04:11:41.606+02:00'
to a Timestamp:

cast(to_timestamp_tz(value , 'YYYY-MM-DD hh24:mi:ss,FF3TZH:TZM') as TIMESTAMP)


you can do maths with TIMESTAMP:

select CURRENT_TIMESTAMP - cast(to_timestamp_tz('2012-09-11 04:11:41.606+02:00' , 'YYYY-MM-DD hh24:mi:ss,FF3TZH:TZM') as TIMESTAMP) from dual;


6 9:8:17.379691

which is a TIME INTERVAL, not a number.
How to covert a TIME INTERVAL to a NUMBER?



doesn't work, you get ORA-01722: invalid number


This function works pretty well to get the EPOCH ( it returns a decimal number representing the SECONDS, if you want the MILLISECONDS as in the Java getCurrentTimeMillis() you should multiply by 1000 and truncate)

create or replace

RETURN number

firstofjan70 timestamp;


firstofjan70 := timestamp '1970-01-01 00:00:00 +00:00';

extract(day from (time1 - firstofjan70 ))*86400+
extract(hour from (time1 - firstofjan70))*3600+
extract(minute from (time1 - firstofjan70))*60+
extract(second from (time1 - firstofjan70))
into sec
from dual;


A Oracle Consultant suggested to set the property
for every ConnectionPool

What is this?


Another useful property is:

"Set TimeToLiveTimeout to m seconds
Set PropertyCheckInterval to n seconds

if the network call is blocked, then just setting the TimeToLive timeout alone won't work. In that case, an additional step is needed -- setting the JDBC driver connection properties as follows:

for the Thin driver:
for the OCI driver: sqlnet.outbound_connection_timeout

There is a Oracle KB doc:
"How to Ensure that JDBC Queries are Always Timed Out [ID 559564.1]"

For more info on the TimeToLiveTimeout and PropertyCheckInterval :


set a timeout on Socket level for the jdbc thin driver: (in ms)

in reality: for jdbc versions < oracle.jdbc.ReadTimeout for jdbc versions >=


"In some deployments of Oracle RAC, you may need to set parameters in addition to the out of the box configuration of a data source in an Oracle RAC configuration. The additional parameters are:

Set oracle.jdbc.ReadTimeout=300000 (300000 milliseconds) for each data source."


The Oracle Consultant says:
"I would set the read and connect timeouts to slightly longer than the stuck thread time setting.
This ensures you get stuck thread warnings but the thread is released soon after "


Related Oracle KB docs:

ORACLE.NET.READ_TIMEOUT Property Does Not Work On 11g JDBC Driver [ID 1341966.1]

WebLogic Server: Intermittent Stuck Threads Caused Due to the 11.1.x Oracle JDBC Thin Driver [ID 1083794.1] (this one seems to be pretty old)

How to Ensure that JDBC Queries are Always Timed Out [ID 559564.1], the suggest using

  • for the Thin driver:
  • for the OCI driver:  sqlnet.outbound_connection_timeout

Sunday, September 16, 2012

Poll result: Women in IT are...

I can't agree more.... too few.... has it really got to be like that? Why so few women are in IT in Europe and US? In India they are a LOT more, up to 50%.... our current development team is composed almost entirely of women, and they are doing a very good job. The Administrators on the other hand are almost all men, and they are true slackers.

Saturday, September 15, 2012

Quick ways of doing code reviews and sanity checks in OSB

I have been asked to check that all log statements are done with "debug" level, as someone does it with "error" and then you end up with a lot of garbage in Production.

The stupid way is to click around.

The smart way is:

- export a sbconfig.jar with all the projects
- unzip the sbconfig.jar in a directory. You will notice that each Proxy is a XML file with .ProxyService extension
- open XMLSpy and open one of those .ProxyService xml file
- menu XML, Evaluate XPath
- enter //*:log[*:logLevel != 'debug' and *:expr/*:xqueryText != '$faultVar']
(normally we log with "error" level only the faultVar
- if you find NOTHING, then cool
- you can run the XPath on ALL files in a given folder (see "where" button in XMLSpy)

for more info:

The way I ahve done it, use Everything Search to identify all .ProxyService files, copy them into a folder, drag and drop them in XmlSpy so that they are all open.

Select "Open Files" in the XPath Evaluation tab, and run the little "evaluate XPath" button.

How much space is taken by a Oracle DB table

select sum(bytes)/1024/1024/1024 from user_segments where segment_name = 'WLI_QS_REPORT_ATTRIBUTE' ;

Friday, September 14, 2012

Every sperm is sacred

This is just too funny, I have to reblog it

There are Jews in the world.
There are Buddhists.
There are Hindus and Mormons, and then
There are those that follow Mohammed, but
I've never been one of them.

I'm a Roman Catholic,
And have been since before I was born,
And the one thing they say about Catholics is:
They'll take you as soon as you're warm.

You don't have to be a six-footer.
You don't have to have a great brain.
You don't have to have any clothes on. You're
A Catholic the moment Dad came,


Every sperm is sacred.
Every sperm is great.
If a sperm is wasted,
God gets quite irate.

Every sperm is sacred.
Every sperm is great.
If a sperm is wasted,
God gets quite irate.

Let the heathen spill theirs
On the dusty ground.
God shall make them pay for
Each sperm that can't be found.

Every sperm is wanted.
Every sperm is good.
Every sperm is needed
In your neighbourhood.

Hindu, Taoist, Mormon,
Spill theirs just anywhere,
But God loves those who treat their
Semen with more care.

Every sperm is sacred.
Every sperm is great.
If a sperm is wasted,...
...God get quite irate.

Every sperm is sacred.
Every sperm is good.
Every sperm is needed...
...In your neighbourhood!

Every sperm is useful.
Every sperm is fine.
God needs everybody's.
And mine!
And mine!

Let the Pagan spill theirs
O'er mountain, hill, and plain.
God shall strike them down for
Each sperm that's spilt in vain.

Every sperm is sacred.
Every sperm is good.
Every sperm is needed
In your neighbourhood.

Every sperm is sacred.
Every sperm is great.
If a sperm is wasted,
God gets quite iraaaaaate!

thanks to

Wednesday, September 12, 2012

Oracle DB monitor session. ORA-00942: table or view does not exist

apparently you need to have SYS

"What roles does your user have? (One way to find out is to do a "SELECT * FROM SESSION_ROLES;" in the SQL Worksheet or SQL*Plus.) The "Monitor Sessions" tool runs a query against the V$SESSION dynamic view and according to the Oracle 10g Database Reference document only user SYS or users with the SYSDBA role have access to the dynamic views.

I just tried using "Monitor Sessions" while connected as a user who has only the CONNECT and RESOURCE roles and got the ORA-00942 error. I connected as another user who has the DBA role and it ran without a problem."

So you want to become proficient in Scala, uh?

free course on "Functional Programming Principles in Scala" by
Martin Odersky

The first lecture will be available on Wednesday, September 19.

Sunday, September 9, 2012

Once every 7 minutes we find this in the Admin log:

java.lang.IllegalArgumentException: null id
at sun.reflect.GeneratedMethodAccessor783.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(
at java.lang.reflect.Method.invoke(
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(
at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(
at com.sun.jmx.mbeanserver.PerInterface.getAttribute(
at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(
at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(
at Method)
at Method)
at Method)
at Method)
at Source)
at weblogic.rmi.internal.BasicServerRef.invoke(
at weblogic.rmi.internal.BasicServerRef$
at weblogic.rmi.internal.BasicServerRef.handleRequest(

Who invokes this through JMX every 7 minutes?

And where is this action with a null ID?

The structure of the Proxy XML is:



Still investigating.....

Friday, September 7, 2012

OSB, number of files open

We run an environment with Whole Server Migration.

An important point is to make sure that we have enough resources (file handles) on a machine to be able to run 2 Managed Server - in case we migrate one.

This is how you discover how many files are used by the user "soa":

/usr/sbin/lsof -u soa | wc -l

It turned out that a single server running the Admin and 4 Managed Servers required some 9000 files. An important load was represented by the MANY Datasources we need to deploy.

This command:
ulimit -a

will tell you the hard limit imposed by the Linux Kernel:

ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 132607
max locked memory (kbytes, -l) 32
max memory size (kbytes, -m) unlimited
open files (-n) 8192
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 132607
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

apparently the limit is flexible, even with a limit of 8192 we were able to open 9050 files...

Thursday, September 6, 2012

java.sql.SQLException: ORA-01653: unable to extend table BLA by 8 in tablespace BLU

This query works like a breeze to find out the FAT tables:

once you discover the FAT tables, run a
truncate table BLA;

you might have to disable FK constraints (remember to reenable again)

Tuesday, September 4, 2012

A protocol for meetings

A meeting must have objectives. All relevant information should be circulated in advance to all participants of the meeting.
In every meeting there should be a minute taker, minutes should be circulated after the meeting. The meeting is not over
until all attendees have expressed their opinions, and all the objectives have been reached. Otherwise, pending points should be rescheduled for another meeting.

A meeting should not be called to read an email to all attendees. Email should be sent in advance, then eventually ask people what they think.
These emails should be concise, and with precise topics. Complex stuff should not be treated in emails, they have invented voice conversations for that.

It’s the management responsibility to identify tasks – which should be as much as possible atomic –prioritize them, and assign them to the technical resources.
Technical people can help in this process, but “task management” is strictly the management responsibility.
It’s the duty of the technical resource to provide regular (once a day, your mileage might vary) progress report to management.
Management has always the right to review priorities and reassign tasks, but bearing in mind that in general this should be avoided.
Excessive task switching is a known factor of burnout and poor work quality.

All management stuff should be discussed in management meeting ,to which technical resources do not need to attend.

Meetings are meant for people to exchange information and reach agreement,
they should not be the monopoly of a single people who takes all the bandwidth and prevents communication amongst other attendees.
As a rule of thumb, in a meeting with 10 people, no more than 30% of the time should be allocated to a single attendee.
If a single person talks for more than 3 minutes in a row in a meeting, this is a communication antipattern.

Long lasting verbal communication should be avoided, and if really needed backed up by schemas, writing etc. Not all people have the same receptivity
to verbal communication, and visual communication is to be preferred over pure verbal conceptualization.

Monday, September 3, 2012

Book review: In praise of Barbarians, Matt Davies

Excellent coverage of the Republican-Democrats equivalence in US (the only difference being which corporations sponsor them), and of the disasters that US foreign policies provoke around the world.

The author uses a quite sophisticated English, it makes a good reading if you want to refine your knowledge of the language.

Sunday, September 2, 2012

Book Review: State of Africa. by Martin Meredith

A breathtaking account of the major events and leaders in the recent (1952- to date) history of Africa.
All about all the conspiracies, negotiations, brutalities, setbacks on the European and American conquest of the black continent.

(I have lived 4 years in Africa... I really miss its variegated culture, its great sense of humour and hospitality... Europe and US suck big time in that respect)

Once you start reading it, it becomes really addictive. Great B&W pictures inside.

Saturday, September 1, 2012

The Arctic Ice Collapse 2012

Unless you live on the moon or in a cave - or you follow only mainstream media, which is equivalent to living in a Disneyland world of delusional fantasies and lies - you must have heard that the Arctic ice cap is collapsing at accelerated speed.

This can only mean one thing: this Planet will never be as before. Climate change will grow at exponential speed, due to the cumulative positive effect of the lack of albedo (reflectivity) of the Pole ice, the emission of Siberian methane etc.

Even if one refuses to acknowledge reality and seek refuge in a world of geekiness, reality will knock at our door: desertification, crop failures, floods, collapse of megapolis will be the result. If you add the coming oil crisis, with the supply of crude oil cut progressively to all secondary members of the Global Empire, and you have a recipe for a Perfect Storm.

Personally, I have no plan to escape, no island where to seek refuge. I will probably be washed away like the 95% of humanity. I am old, I am not afraid of death. I only pity those who are young today, they came onboard a doomed planet. Stupidity and greed doomed this planet.

I always refused to own a car, I have lived a very simple non consumeristic lifestyle, but even this way I have probably consumed 20 times more energy (=CO2) than my grand father... just fly once, and it's like the energy that in 1800 one person yould have consumed in a lifetime...

We shall go back to a middle age lifestyle, but with a deeply devastated ecosystem, unable to support the amount of life that was possible in the 1300.

The descent in the Maelstrom will not be fun. But you are not alone, in fact we are all in the same boat, Planet Earth.