Ignore Stuck Threads for JCA FileAdapter

The default threading model for FileAdapter created long running threads who poll the file system. After 600 seconds they generate Health warning and error stacktraces. To get rid of this, without losing in general the capability to detect Stuck Threads in other parts of the Server, use a WorkManager and associate it to each Proxy Service using a FileAdapter.

WLST Snippet to create WorkManager :

DOMAIN_NAME=osbpl1do
CLUSTER_NAME=osbpl1cl

edit()
startEdit()
cd('/SelfTuning/' + DOMAIN_NAME)
cmo.createWorkManager('WorkManagerIgnoreStuckThreads')
cd('/SelfTuning/' + DOMAIN_NAME + '/WorkManagers/WorkManagerIgnoreStuckThreads')
set('Targets',jarray.array([ObjectName('com.bea:Name=' + CLUSTER_NAME + ',Type=Cluster')], ObjectName))
cmo.setIgnoreStuckThreads(true)

activate()

For each of the FileAdapter JCA Proxy Services, edit the JCA Properties and assign 'WorkManagerIgnoreStuckThreads' to the Dispatch Policy.


See also the official Oracle doc on this topic (search for ignore-stuck-threads)

Enabling diagnostic log for FileAdapter (in general for oracle.soa.adapter)

The official doc here:

http://docs.oracle.com/cd/E21764_01/integration.1111/e10231/life_cycle.htm#BABBIBEE

open em: http://myadmin.acme.com:7001/em
and identify the FileAdapter for each MS:



right click, Logs, Log configuration:





select "loggers with persistent log state level" (NOT Runtime Loggers)




search oracle.soa.adapter, and set to TRACE:32 FINEST, scroll down and enable the "persist" checkbox, click on Apply




do it for all MS in the cluster


the logs should go to
${MW_HOME}/user_projects/domains/${DOMAIN_NAME}/servers/${SERVER_NAME}/logs/${SERVER_NAME}-diagnostic.log

WLST script to monitor for the presence of files in a directory

It will poll every second a directory, and write to a log file the name of all the files it can find...

Very useful to monitor the behaviour of a File Adapter...

#Checks at regular intervals for the presence of a file in a Directory
import os
import datetime
import time

dirToWatch = '/data/my/directory' 
while 1 == 1 :
     files = os.listdir(dirToWatch)
     if len(files) > 0:
        logfile = open("fileslog.log", "a")
        logfile.write(str(datetime.datetime.now()) + " " + str(files) + "\n")
        logfile.close()
        time.sleep(1)

Oracle DbAdapter, FileAdapter, FTPAdapter automated customization

Put in SVN the Deployment Plan created manually in an environment - if makes no sense to create the Plan.xml programmatically, it's only asking for trouble.

In them, replace the actual environment-dependent values with tokens like ${DOMAIN_NAME}
the tokens are:
tokens=DOMAIN_NAME,FTP_HOST,FTP_PASSWORD,FTP_USER

The JNDI names of all your DataSources will not change from environment to environment, so there is no point in replacing them.

create a adapters.properties file like this:

plan1=/opt/oracle/domains/${DOMAIN_NAME}/shared/apps/dbadapter/plan/DBAdapterPlan.xml
plan2=/opt/oracle/domains/${DOMAIN_NAME}/shared/apps/fileadapter/plan/FileAdapterPlan.xml
plan3=/opt/oracle/domains/${DOMAIN_NAME}/shared/apps/ftpadapter/plan/FTPAdapterPlan.xml

plans=plan1,plan2,plan3

adapter1=/opt/oracle/fmw11_1_1_5/osb/soa/connectors/DbAdapter.rar
adapter2=/opt/oracle/fmw11_1_1_5/osb/soa/connectors/FileAdapter.rar
adapter3=/opt/oracle/fmw11_1_1_5/osb/soa/connectors/FtpAdapter.rar

adapters=adapter1,adapter2,adapter3

username=weblogic
password=welcome1
url=t3://myhost.acme.com:7001

tokens=DOMAIN_NAME,FTP_HOST,FTP_PASSWORD,FTP_USER
DOMAIN_NAME=osbpl1do
FTP_HOST=myfthost.acme.com
FTP_PASSWORD=pippopassword
FTP_USER=pippouser

create a WLST script like this:

#############################################################################
#
# Configure a new environment with the Plans.xml from SVN
# uses adapters.properties file.
#
#############################################################################

from java.io import FileInputStream
from shutil import copyfile
import os, sys
import re

#not used, keep only as a reference
def copyfileobj(fsrc, fdst, length=16*1024):
    """copy data from file-like object fsrc to file-like object fdst"""
    while 1:
        buf = fsrc.read(length)
        if not buf:
            break
        fdst.write(buf)

def copyfileWithTokenSubstitution(filein, fileout, properties):
        input = open(filein)
        output = open(fileout, 'w')
        for s in input:
            rep = s
            for tokenname in properties.get("tokens").split(','):
                rep = rep.replace("${" + tokenname + "}", properties.get(tokenname))
            output.write(rep)
        input.close()
        output.close()
    
#not used, keep only as a reference
def copyfile(src, dst):
    """Copy data from src to dst"""
    fsrc = None
    fdst = None
    try:
        fsrc = open(src, 'rb')
        fdst = open(dst, 'wb')
        copyfileobj(fsrc, fdst)
    finally:
        if fdst:
            fdst.close()
        if fsrc:
            fsrc.close()



    
propertyFileName = 'adapters.properties'    

#loading properties
print 'Loading properties from ', propertyFileName
propInputStream = FileInputStream(propertyFileName)
configProps = Properties()
configProps.load(propInputStream)

domainName = configProps.get('DOMAIN_NAME')

plans=configProps.get("plans")
adapters=configProps.get("adapters")

planArray = []
adapterArray = []

#create needed directories where to put *Plan.xml
for plan in plans.split(','):
    planFullPath = configProps.get(plan).replace("${DOMAIN_NAME}", domainName)
    planArray.append(planFullPath)

#create array of adapters
for adapter in adapters.split(','):
    adapterName = configProps.get(adapter)
    adapterArray.append(adapterName)


#create dir if doesn't exist - fail if unable to create it
for planFullPath in planArray:    
    dirName = os.path.dirname(planFullPath)
    print "creating directory " + dirName
    if not os.path.exists(dirName):
        os.makedirs(dirName)
    
#check for directory existence
for planFullPath in planArray:
    dirName = os.path.dirname(planFullPath)
    print "testing directory " + dirName
    
    if not os.path.exists(dirName):
        message = "directory " + dirName + " does not exist"
        print message
        raise Exception(message)

    

#copy all Plan.xml files to their final destination, with token substitution
for planFullPath in planArray:
    fileName = os.path.basename(planFullPath)
    print "copying " + fileName + " to " + planFullPath  
    copyfileWithTokenSubstitution(fileName, planFullPath, configProps)

#Connect to Admin Server    
connect(configProps.get("username"),configProps.get("password"),configProps.get("url"))

#applying changes to the Adapters
edit()
try:
    for index in range(len(planArray)):
        startEdit()
        plan = planArray[index]
        adapter = adapterArray[index]
        
        adapterType = os.path.basename(adapter).split('.')[0]
        
        print 'Applying plan ' + plan + " to adapter " + adapter + " (adapter type is " + adapterType + ")"
        myPlan = loadApplication(adapter, plan)
        myPlan.save()
        save()
        activate(block='true')
        cd('/AppDeployments/' + adapterType + '/Targets')
        #updateApplication(appName, planPath);
        redeploy(adapterType, plan, targets = cmo.getTargets())
         

except:
    dumpStack()
    stopEdit('y')
    message="unable to finish job"
    raise Exception(message)

disconnect()
print "job finished successfully"

java.lang.NoClassDefFoundError: oracle/integration/platform/kernel/FabricMeshUtils

We often get this error while using OSB with JCA File Adapter:

java.lang.NoClassDefFoundError: oracle/integration/platform/kernel/FabricMeshUtils
        at oracle.tip.adapter.file.inbound.FileListDAO.(FileListDAO.java:188)
        at oracle.tip.adapter.file.inbound.PollWork.(PollWork.java:236)
        at oracle.tip.adapter.file.FileResourceAdapter.endpointActivation(FileResourceAdapter.java:222)
        at oracle.tip.adapter.sa.impl.fw.jca.AdapterFrameworkImpl.endpointActivation(AdapterFrameworkImpl.java:498)
        at oracle.tip.adapter.sa.impl.inbound.JCABindingActivationAgent.activateEndpoint(JCABindingActivationAgent.java:336)
        at oracle.tip.adapter.sa.impl.JCABindingServiceImpl.activate(JCABindingServiceImpl.java:113)

the class is in C:\Oracle\Middleware\jdeveloper\soa\modules\oracle.soa.fabric_11.1.1\fabric-runtime.jar

or in
C:\Oracle\Middleware\Oracle_SOA1\soa\modules\oracle.soa.fabric_11.1.1\fabric-runtime.jar

(better use the latter...)


and in oracle.tip.adapter.file.inbound.FileListDAO there is clearly a dependency:
import oracle.integration.platform.blocks.FabricConfigManager;
import oracle.integration.platform.kernel.FabricMeshUtils;



When loading this class fails, you will see in the logs "Unable to retrieve database info"

After adding the fabric-runtime.jar file to $DOMAIN_HOME/lib, I still get:




While trying to lookup 'soa-infra:comp.ApplicationContext' didn't find subcontext 'soa-infra:comp'. Resolved ''
javax.naming.NameNotFoundException: While trying to lookup 'soa-infra:comp.ApplicationContext' didn't find subcontext 'soa-infra:comp'. Resolved ''; remaining name 'soa-infra:comp/ApplicationContext'
at weblogic.jndi.internal.BasicNamingNode.newNameNotFoundException(BasicNamingNode.java:1139)
at weblogic.jndi.internal.BasicNamingNode.lookupHere(BasicNamingNode.java:247)
at weblogic.jndi.internal.ServerNamingNode.lookupHere(ServerNamingNode.java:182)
at weblogic.jndi.internal.BasicNamingNode.lookup(BasicNamingNode.java:206)
at weblogic.jndi.internal.WLEventContextImpl.lookup(WLEventContextImpl.java:254)
at weblogic.jndi.internal.WLContextImpl.lookup(WLContextImpl.java:411)
at javax.naming.InitialContext.lookup(InitialContext.java:392)
at oracle.integration.platform.kernel.FabricMeshUtils.getApplicationContext(FabricMeshUtils.java:57)
at oracle.tip.adapter.file.inbound.FileListDAO.<init>(FileListDAO.java:188)
at oracle.tip.adapter.file.inbound.PollWork.<init>(PollWork.java:236)
at oracle.tip.adapter.file.FileResourceAdapter.endpointActivation(FileResourceAdapter.java:222)
at oracle.tip.adapter.sa.impl.fw.jca.AdapterFrameworkImpl.endpointActivation(AdapterFrameworkImpl.java:498)
at oracle.tip.adapter.sa.impl.inbound.JCABindingActivationAgent.activateEndpoint(JCABindingActivationAgent.java:336)
at oracle.tip.adapter.sa.impl.JCABindingServiceImpl.activate(JCABindingServiceImpl.java:113)
at com.bea.wli.sb.transports.jca.binding.JCATransportInboundOperationBindingServiceImpl.activateService(JCATransportInboundOperationBindingServiceImpl.java:325)






RESOLUTION: we decided to ignore this message, if you want to configure the FileAdapter for useCompression you can do so with the JCA Activation Properties.
I assume all works fine in SOA Suite, where the FabricMeshUtils is surely available in the classpath.



In fact, further investigation proves that the only side effect of this error is that the useCompression (default: false) flag can be set only using the JCA Activation Properties, and not with a DB value (in any case OSB doesn't use any DB to store configuration information).

OSB and rejectedMessageHandlers in JCA File Adapter

I am completing a previous post on the same topic.

By default, the JCA File Adapter will write bad files into $DOMAIN_HOME/jca/Read/rejectedMessages, renaming them to a filename like:

INVALID_MSG_154330703_Read_20111109_103658_0898.dat


If you specify in your JCA file the property PhysicalErrorArchiveDirectory=/path/to/my/errorDirectory, the bad files will be written here and not to the $DOMAIN_HOME/jca/Read/rejectedMessages directory.



To retry a "failed" file (only those who are not directly rejected by the Adapter, but for instance those who fail validation in the Proxy Service), you can specify in the JCA Transport Configuration of the Proxy Service:

jca.retry.count = "3"
jca.retry.interval = "40"


If you want to change the destination, there is a property rejectedMessageHandlers to be set (see oracle.tip.adapter.file. package). In SOA Suite you would set it in bpel.xml, in OSB it's a bit tricky:

edit the JCA Proxy Service, JCA Transport Configuration, Dynamic EndPoint Properties, add "rejectedMessageHandlers" with value "file://path/to/your/dir"


If you fail format, you will get this message:

Error JCA_FRAMEWORK_AND_ADAPTER BEA-000000 Unrecognized Rejection handler
Unrecognized Rejection handler
The Rejection handler C:/acme/po/rejected is not recognized.
Please use on of the existing Rejection handlers: file://, queue://, bpel:// or wsif://


so your options are a AQ queue (queue refers to AQ, not to JMS!), a BPEL process, a Web Service.



Here you have a series of possible formats for the destination:

http://download.oracle.com/docs/cd/B14099_19/integrate.1012/b14058/life_cycle.htm

it says that you can send the file to a JMS queue or a DB, not necessarily to a Directory... I haven't tested this.


Here the Oracle Integration File Adapter Datasheet , but it doesn't say much about error handler.

Security and Authentication in JCA FTP File Adapter

I have setup a Credential Mapping to be able to do a Opaque FTP Put with the JCA FTP Adapter, as documented here

It maps a WebLogic user into a FTP EIS User.

I still get an authentication error:



####<13-Nov-2011 22:26:29 o'clock CET> <Error> <WliSbTransports> <pierrepc> <osb_server1> <[ACTIVE] ExecuteThread: '17' for queue: 'weblogic.kernel.Default (self-tuning)'> <<anonymous>> <BEA1-07F5BFB39FA8A589D5FB> <9218d6466d04a9d6:4f7d3afe:1339e75d3c7:-8000-00000000000000e4> <1321219589928> <BEA-381502> <Exception in JmsInboundMDB.onMessage: com.bea.wli.sb.transports.TransportException: <jca-transport-application-error xmlns="http://www.bea.com/wli/sb/transports/jca">
<jca-transport-error-message>Invoke JCA outbound service failed with application error</jca-transport-error-message>
<jca-runtime-fault-detail>
<eis-error-code>501</eis-error-code>
<eis-error-message>501 Syntax error</eis-error-message>
<exception>com.bea.wli.sb.transports.jca.JCATransportException: oracle.tip.adapter.sa.api.JCABindingException: oracle.tip.adapter.sa.impl.fw.ext.org.collaxa.thirdparty.apache.wsif.WSIFException: servicebus:/WSDL/Phoenix_PurchaseOrder/WriteToWMOS_FTP [ Put_ptt::Put(opaque) ] - WSIF JCA Execute of operation 'Put' failed due to: Error in logging in.
Error in logging in.
Unable to log in to the server.
; nested exception is:
BINDING.JCA-11439
Error in logging in.
Error in logging in.
Unable to log in to the server.
Please ensure userid and password specified to login to the server is correct.




To understand what is going on, I enable debug flag alsb-jca-framework-adapter-debug as explained here



All I get is :




Caused by: BINDING.JCA-11439
Error in logging in.
Error in logging in.
Unable to log in to the server.
Please ensure userid and password specified to login to the server is correct.

at oracle.tip.adapter.ftp.FTPClient.regularLogin(FTPClient.java:1630)
at oracle.tip.adapter.ftp.FTPClient.login(FTPClient.java:1593)
at oracle.tip.adapter.ftp.FTPAgent.login(FTPAgent.java:1170)
at oracle.tip.adapter.ftp.FTPAgent.preCall(FTPAgent.java:1632)
at oracle.tip.adapter.ftp.FTPAgent.validateOutputDir(FTPAgent.java:1230)
at oracle.tip.adapter.file.outbound.FileInteraction.validateDirectory(FileInteraction.java:2676)
at oracle.tip.adapter.file.outbound.FileInteraction.executeFileWrite(FileInteraction.java:547)
at oracle.tip.adapter.ftp.outbound.FTPInteraction.execute(FTPInteraction.java:251)
at oracle.tip.adapter.sa.impl.fw.wsif.jca.WSIFOperation_JCA.performOperation(WSIFOperation_JCA.java:529)
... 62 more





and there is no way to log the actual username/password it is using to login into the FTP server


I am using Filezilla Server, I disable the Miscellaneous/Don't show password in logs, and I enable logging on the server.

What I find is disconcerting:


(000033) 13-11-2011 21:46:48 - (not logged in) (127.0.0.1)> USER
(000033) 13-11-2011 21:46:48 - (not logged in) (127.0.0.1)> 501 Syntax error


while the normal sequence should be:


(000037) 13-11-2011 21:57:07 - (not logged in) (fe80::5950:1b46:fefe:aa61)> USER someuser
(000037) 13-11-2011 21:57:07 - (not logged in) (fe80::5950:1b46:fefe:aa61)> 331 Password required for someuser
(000037) 13-11-2011 21:57:07 - (not logged in) (fe80::5950:1b46:fefe:aa61)> PASS somepassword
(000037) 13-11-2011 21:57:07 - someuser (fe80::5950:1b46:fefe:aa61)> 230 Logged on


normally after USER I should see a username.... this means that the mapping fails.

The case is reported here


I notice that the log statement reports "anonymous". If you go to deployment, FtpAdapter, Security, Principal you can set "weblogic" and principal, and make sure that you also use weblogic in the Credential Mapping.
This is not enough.

In the $inbound of the OSB request I notice:


<con:security>
<con:transportClient>
<con:username><anonymous></con:username>
</con:transportClient>
</con:security>



so I create a weblogicServiceAccount for username weblogic, associate it to the Proxy JMS Service Account (hoping to make the OSB proxy "run as" the weblogic username, but I get this in the logs:


ServiceAccountRuntimeManagerImpl.getUsernamePasswordCredential = this: com.bea.wli.sb.svcacct.ServiceAccountRuntimeManagerImpl@17d3da

Could not find credentials on admin server: java.lang.IllegalArgumentException: com.bea.wli.sb.management.configuration.ServiceAccountRuntime is not an interface

com.bea.wli.sb.svcacct.ServiceAccountRuntimeCache@1e49c6b$ServiceAccountRuntimeCache.get(Acme_PurchaseOrder/weblogicServiceAccount)

ServiceAccountRuntimeCache.get(Acme_PurchaseOrder/weblogicServiceAccount) returned ServiceAccountRuntime[Acme_PurchaseOrder/weblogicServiceAccount]



and the $inbound still shows anonymous user


Anyway, at the end I do:

Deployments/FtpAdapter/Security/Credential Mapping/New/Unauthenticated User and I map it to the FTP user.... works like magic!

The alternative is to specify a username/password in the JCA Outbound Connection Pool instance properties (there are 55 properties, only 10 shown in the first page... they are in alphabetical order, so yuoi must go to the last page)

One day I will figure out how to attach a WebLogic principal to an incoming JMS request... security has always been my Achilles' heel

Anatomy of a JCA File Adapter parsing failure

If you submit a "broken" file to a File Adapter associated to a nXSD transformation, you might expect something like this to happen (see after).

To be noticed:

* eventually, the file is moved to the error directory

* The entry point to the translation is oracle.tip.pc.services.translation.xlators.nxsd.NXSDTranslatorImpl.translateFromNative


* The exception is a ORABPEL-11168
* The Proxy Service is not invoked, so there is no way to trace this error in OSB.

* To trace the error, you must implement a RejectedMessageHandler
* In alternative, you could "grep" the logs for ORABPEL-11168 with WLDF, but this solution really sucks

So implementing a RejectedMessageHandler seems to me the best way to trace the error. This requires more investigation.


<08-Nov-2011 22:39:57 o'clock CET> <Info> <JCA_FRAMEWORK_AND_ADAPTER> <pierrepc> <osb_server1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <9218d6466d04a9d6:5003231f:133830ed055:-8000-0000000000000031> <1320788397613> <BEA-000000> <Error while translating inbound file : po00000BROKEN.dat
ORABPEL-11168

Error while reading native data.
[Line=1, Col=1] Expected "," for the data starting at the specified position, while trying to read the data for "element with name OrderNumber", using "style" as "terminated" and "terminatedBy" as ",", but not found.
Ensure that ",", exists for the data starting at the specified position.

at oracle.tip.pc.services.translation.xlators.nxsd.NXSDStyleBasedReader.readTerminatedStyle(NXSDStyleBasedReader.java:338)
at oracle.tip.pc.services.translation.xlators.nxsd.NXSDTranslatorImpl.readTerminatedStyle(NXSDTranslatorImpl.java:2771)
at oracle.tip.pc.services.translation.xlators.nxsd.NXSDTranslatorImpl.readNativeData(NXSDTranslatorImpl.java:2841)
at oracle.tip.pc.services.translation.xlators.nxsd.NXSDTranslatorImpl.processScalarType(NXSDTranslatorImpl.java:3600)
at oracle.tip.pc.services.translation.xlators.nxsd.NXSDTranslatorImpl.processElement(NXSDTranslatorImpl.java:3800)
at oracle.tip.pc.services.translation.xlators.nxsd.NXSDTranslatorImpl.parseNXSD(NXSDTranslatorImpl.java:1327)
at oracle.tip.pc.services.translation.xlators.nxsd.NXSDTranslatorImpl.processGroup(NXSDTranslatorImpl.java:3705)
at oracle.tip.pc.services.translation.xlators.nxsd.NXSDTranslatorImpl.parseNXSD(NXSDTranslatorImpl.java:1334)
at oracle.tip.pc.services.translation.xlators.nxsd.NXSDTranslatorImpl.processComplexType(NXSDTranslatorImpl.java:3841)
at oracle.tip.pc.services.translation.xlators.nxsd.NXSDTranslatorImpl.parseNXSD(NXSDTranslatorImpl.java:1330)
at oracle.tip.pc.services.translation.xlators.nxsd.NXSDTranslatorImpl.processElement(NXSDTranslatorImpl.java:3806)
at oracle.tip.pc.services.translation.xlators.nxsd.NXSDTranslatorImpl.parseNXSD(NXSDTranslatorImpl.java:1327)
at oracle.tip.pc.services.translation.xlators.nxsd.NXSDTranslatorImpl.processGroup(NXSDTranslatorImpl.java:3705)
at oracle.tip.pc.services.translation.xlators.nxsd.NXSDTranslatorImpl.parseNXSD(NXSDTranslatorImpl.java:1334)
at oracle.tip.pc.services.translation.xlators.nxsd.NXSDTranslatorImpl.processComplexType(NXSDTranslatorImpl.java:3841)
at oracle.tip.pc.services.translation.xlators.nxsd.NXSDTranslatorImpl.parseNXSD(NXSDTranslatorImpl.java:1330)
at oracle.tip.pc.services.translation.xlators.nxsd.NXSDTranslatorImpl.processElement(NXSDTranslatorImpl.java:3806)
at oracle.tip.pc.services.translation.xlators.nxsd.NXSDTranslatorImpl.parseNXSD(NXSDTranslatorImpl.java:1327)
at oracle.tip.pc.services.translation.xlators.nxsd.NXSDTranslatorImpl.doTranslateFromNative(NXSDTranslatorImpl.java:846)
at oracle.tip.pc.services.translation.xlators.nxsd.NXSDTranslatorImpl.translateFromNative(NXSDTranslatorImpl.java:602)
at oracle.tip.adapter.file.inbound.InboundTranslatorDelegate.xlate(InboundTranslatorDelegate.java:314)
at oracle.tip.adapter.file.inbound.InboundTranslatorDelegate.doXlate(InboundTranslatorDelegate.java:121)
at oracle.tip.adapter.file.inbound.ProcessorDelegate.doXlate(ProcessorDelegate.java:388)
at oracle.tip.adapter.file.inbound.ProcessorDelegate.process(ProcessorDelegate.java:174)
at oracle.tip.adapter.file.inbound.ProcessWork.run(ProcessWork.java:349)
at weblogic.work.ContextWrap.run(ContextWrap.java:41)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:178)
>
####<08-Nov-2011 22:39:57 o'clock CET> <Info> <JCA_FRAMEWORK_AND_ADAPTER> <pierrepc> <osb_server1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <9218d6466d04a9d6:5003231f:133830ed055:-8000-0000000000000031> <1320788397614> <BEA-000000> <Since a translation exception was thrown, this indicates that it is a non-debatching scenario.>
####<08-Nov-2011 22:39:57 o'clock CET> <Info> <JCA_FRAMEWORK_AND_ADAPTER> <pierrepc> <osb_server1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <9218d6466d04a9d6:5003231f:133830ed055:-8000-0000000000000031> <1320788397615> <BEA-000000> <Failed to translate file : {C:\acme\ffmw\po\in\po00000BROKEN.dat}>
####<08-Nov-2011 22:39:57 o'clock CET> <Info> <JCA_FRAMEWORK_AND_ADAPTER> <pierrepc> <osb_server1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <9218d6466d04a9d6:5003231f:133830ed055:-8000-0000000000000031> <1320788397615> <BEA-000000> <Sending message to Adapter Framework for rejection to user-configured rejection handlers : {
file=C:\acme\ffmw\po\in\po00000BROKEN.dat, Exception=ORABPEL-11168

Error while reading native data.
[Line=1, Col=1] Expected "," for the data starting at the specified position, while trying to read the data for "element with name OrderNumber", using "style" as "terminated" and "terminatedBy" as ",", but not found.
Ensure that ",", exists for the data starting at the specified position.

}>
####<08-Nov-2011 22:39:57 o'clock CET> <Warning> <JCA_FRAMEWORK_AND_ADAPTER> <pierrepc> <osb_server1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <9218d6466d04a9d6:5003231f:133830ed055:-8000-0000000000000031> <1320788397616> <BEA-000000> <onReject: The resource adapter 'File Adapter' requested handling of a malformed inbound message. However, the following activation property has not been defined: 'rejectedMessageHandlers'. Please define it and redeploy. Will use the default Rejection Directory file://jca\Read\rejectedMessages for now.>
####<08-Nov-2011 22:39:57 o'clock CET> <Warning> <JCA_FRAMEWORK_AND_ADAPTER> <pierrepc> <osb_server1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <9218d6466d04a9d6:5003231f:133830ed055:-8000-0000000000000031> <1320788397616> <BEA-000000> <onReject: Sending invalid inbound message to Rejection Handler: >
####<08-Nov-2011 22:39:57 o'clock CET> <Info> <JCA_FRAMEWORK_AND_ADAPTER> <pierrepc> <osb_server1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <9218d6466d04a9d6:5003231f:133830ed055:-8000-0000000000000031> <1320788397618> <BEA-000000> <Handing rejected message to DEFAULT rejection handler: file://jca\Read\rejectedMessages since none of the configured rejection handlers [] succeeded.>
####<08-Nov-2011 22:39:57 o'clock CET> <Info> <JCA_FRAMEWORK_AND_ADAPTER> <pierrepc> <osb_server1> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <9218d6466d04a9d6:5003231f:133830ed055:-8000-0000000000000031> <1320788397622> <BEA-000000> <Copying file :C:\acme\ffmw\po\in\po00000BROKEN.dat to user-defined archive directory for files with errors :C:\acme\ffmw\po\error>



See also this Oracle documentation on the rejectedMessageHandler topic

Errors or faults arising after the message is posted to Service Infrastructure layer are not rejected. These faulted messages are handled by the Service Infrastructure components and will not be considered as rejected messages.

Adapters and other binding components (for example, WS Binding Component) reject messages which error out at the binding level, that is, before entering the Service Infrastructure layer. All rejected messages are stored in the Database with payload.

JCA File Adapter and Error files

Oracle implementation of the File Adapter is the class:
oracle.tip.adapter.file.inbound.FileActivationSpec

you customize the adapter in the .JCA file

One of its parameters is

PhysicalErrorArchiveDirectory

and it is documented here

For instance if you submit an empty file, the File Adapter will fail and move the file to the Error directory. In the logs I find:


<Warning> <JCA_FRAMEWORK_AND_ADAPTER> <BEA-000000> <onReject: The resource adapter 'File Adapter' requested handling of a malformed inbound message. However, the following activation property has not been defined: 'rejectedMessageHandlers'. Please define it and redeploy. Will use the default Rejection Directory file://jca\Read\rejectedMessages for now.>

<Warning> <JCA_FRAMEWORK_AND_ADAPTER> <BEA-000000> <onReject: Sending invalid inbound message to Rejection Handler: >


In case of error in the Proxy Service consuming the message, at proxy level you should set jca.retry.count and jca.retry.interval

<jca:endpoint-properties>
<jca:endpoint-property>
<jca:name>jca.retry.count</jca:name>
<jca:value>3</jca:value>
</jca:endpoint-property>
<jca:endpoint-property>
<jca:name>jca.retry.interval</jca:name>
<jca:value>40</jca:value>
</jca:endpoint-property>
</jca:endpoint-properties>

with these settings, you will see these messages:

Failed to send message {C:\acme\ffmw\po\in\po000000002.dat} to Adapter Framework due to Retriable Exception, the worker will sleep for the configured retryInterval[40000] msecs

Retry Interval will be bounded to [30000] msecs since we're using a global processor thread pool and the current retryInterval is[40000] msecs

Since Retry Interval is specified, the processor will disable for[30000] msecs to throttle

40 seconds after, you get:

The adapter will not process [C:\acme\ffmw\po\in\po000000002.dat] since it has been processed earlier

onReject: Sending invalid inbound message to Rejection Handler:

Handing rejected message to DEFAULT rejection handler: file://jca\Read\rejectedMessages since none of the configured rejection handlers [] succeeded.

Copying file :C:\acme\ffmw\po\in\po000000002.dat to user-defined archive directory for files with errors :C:\acme\ffmw\po\error

The adapter has handled the poisoned file [C:\acme\ffmw\po\in\po000000002.dat]




So, only if you set the retry count the file will eventually be moved to the error directory also on "application" error triggered by the Proxy Service.

Getting file info from JCA File Poller

logging $inbound/ctx:transport/ctx:request/tp:headers gives this:

<tran:headers xsi:type="jca:JCARequestHeadersXML" xmlns:jca="http://www.bea.com/wli/sb/transports/jca" xmlns:tran="http://www.bea.com/wli/sb/transports" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <tran:user-header name="jca.file.CreationTime" value="0"/>
  <jca:SOAPAction>Read</jca:SOAPAction>
  <jca:jca.file.FileName>po000000001.dat</jca:jca.file.FileName>
  <jca:jca.file.Directory>C:\acme\po\in</jca:jca.file.Directory>
  <jca:jca.file.Size>113</jca:jca.file.Size>
  <jca:jca.file.Batch>z_qjc5GlR6b6iHDgdbExXlDoQiEkb7g2BnTbpmrhd7E.</jca:jca.file.Batch>
  <jca:jca.file.BatchIndex>1</jca:jca.file.BatchIndex>
  <jca:jca.file.LastModifiedTime>1320482562574</jca:jca.file.LastModifiedTime>
</tran:headers>

Filename can be retrieved this way:

$inbound/ctx:transport/ctx:request/tp:headers/tp:headers/jca:jca.file.FileName/text()

where
ctx="http://www.bea.com/wli/sb/context"
tp="http://www.bea.com/wli/sb/transports"
jca="http://www.bea.com/wli/sb/transports/jca"

JCA File Adapter, File read vs Synchronous File Read, Sorting Files with ListSorter

In a nutshell, the Synchronous option doesn't create a File Poller Thread, but stops execution of an existing Thread (Message Flow) to read a file with a specified name.
In the parametrization of the File Adapter, the only difference is the existence of the "File Name" attribute. This is explained in detail here.

In the official documentation, this picture is wrong:

http://download.oracle.com/docs/cd/E15523_01/integration.1111/e10231/adptr_file.htm#BABEJGJB


it should be rather this:

Bear in mind that not all properties are shown in the JDeveloper wizard. You will have to hack the JCA file for some of them.
Their list is here.

A very interesting option is ListSorter :

ListSorter="oracle.tip.adapter.file.sorter.TimestampSorterAscending" allows you to process files in order of timestamp.

You can provide your own ListSorter: just implement a java.util.Comparator, you will receive 2 objects oracle.tip.adapter.file.FileInfo, so the available properties on which you can sort are:

fileName
compressedFileName
fileExtension
fullPath
compressedFullPath
timestamp
size
processed
isActive
fileList
batchId
processHeaderOnly
headerOnlyNoDelete
distributed
inputDirectory
compressedInputDirectory
rootDirectory
compressedRootDirectory
readOnly
creationTime
raw
primaryKey
originalPrimaryKey
clusteredFileList
fileType
singleThreaded
status
usePreciseTimestamp
poller
properties

However, bear in mind that the documentation says that you should have a SINGLE THREAD to make the sorting work


When files must be processed by Oracle File and FTP Adapters in a particular order, you must configure the sorting parameters. For example, you can configure the sorting parameters for Oracle File and FTP Adapters to process files in ascending or descending order by time stamps.
You must meet the following prerequisites for sorting scenarios of Oracle File and FTP Adapters:
• Use a synchronous operation
• Add the following property to the inbound JCA file:

property name="ListSorter" value="oracle.tip.adapter.file.inbound.listing.TimestampSorterAscending"
property name="SingleThreadModel" value="true"

High Availability File Adapter in OSB

While running the FileAdapter (eis/FileAdapter) in a cluster with 2 instances, I get this error

BINDING.JCA-11042 File deletion failed

the full error is

onFatalError: Adapter forced endpoint deactivation due to:
BINDING.JCA-11042
File deletion failed.

here https://forums.oracle.com/forums/thread.jspa?threadID=2157695 and here https://kr.forums.oracle.com/forums/thread.jspa?messageID=9757727 and here https://cn.forums.oracle.com/forums/thread.jspa?threadID=2252967 they report the same story.


OK, we need to use eis/HAFileAdapter and give a controlDir. But we are not having a SOAINFRA DB schema because we installed only OSB.

see High Availability in Inbound Operations here http://download.oracle.com/docs/cd/E15523_01/integration.1111/e10231/adptr_file.htm

it is not clear if you have an option to use a file as a coordinator, instead of a DB. What is controlDir for?

http://download.oracle.com/docs/cd/E14571_01/doc.1111/e17059.pdf here the HA FileAdapter documentation


This document is also quite cool, it says:

"Database-based mutex and locks are used to coordinate these operations in a File Adapter clustered topology. Other coordinators are available but Oracle recommends using the Oracle Database."

yet they don't mention the alternatives.... if you use a DB, then it must be a HA DB otherwise a DB failure would mean a File Adapter failure...



see also here for full solution

Stuck Threads when using JCA FileAdapter

you will notice that the Server Health has a Warning, and there are 5 stuck threads

one of this type:


"[STUCK] ExecuteThread: '7' for queue: 'weblogic.kernel.Default (self-tuning)'" TIMED_WAITING

java.lang.Thread.sleep(Native Method)

oracle.tip.adapter.file.inbound.PollWork.run(PollWork.java:369)

weblogic.work.ContextWrap.run(ContextWrap.java:41)

weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)

weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)

weblogic.work.ExecuteThread.run(ExecuteThread.java:178)



and 4 of this type:


"[STUCK] ExecuteThread: '4' for queue: 'weblogic.kernel.Default (self-tuning)'" waiting for lock oracle.tip.adapter.file.inbound.FilesToProcess@4bdddc TIMED_WAITING

java.lang.Object.wait(Native Method)

oracle.tip.adapter.file.inbound.FilesToProcess.dequeueToProcess(FilesToProcess.java:101)

oracle.tip.adapter.file.inbound.ProcessWork.run(ProcessWork.java:269)

weblogic.work.ContextWrap.run(ContextWrap.java:41)

weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)

weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)

weblogic.work.ExecuteThread.run(ExecuteThread.java:178)




according to Oracle documentation for the DB adapter this is normal. They don't mention File Adapter....


The solution is to create a WorkManager with the option “ignore stuck threads” and assign it as a dispatch policy to each Proxy Service using the FileAdapter

JCA File Adapter in OSB

This is a sample JCA file generated by JDeveloper

<adapter-config name="readfile" adapter="File Adapter" wsdlLocation="readfile.wsdl" xmlns="http://platform.integration.oracle/blocks/adapter/fw/metadata">

<connection-factory location="eis/FileAdapter" UIincludeWildcard="*.*"/>
<endpoint-activation portType="Read_ptt" operation="Read">
<activation-spec className="oracle.tip.adapter.file.inbound.FileActivationSpec">
<property name="DeleteFile" value="true"/>
<property name="MinimumAge" value="0"/>
<property name="PhysicalDirectory" value="c:/tmp/in"/>
<property name="Recursive" value="true"/>
<property name="PollingFrequency" value="60"/>
<property name="PhysicalArchiveDirectory" value="c:/tmp/archive"/>
<property name="IncludeFiles" value=".*\..*"/>
<property name="UseHeaders" value="false"/>
</activation-spec>
</endpoint-activation>

</adapter-config>

and this the associated WSDL

<wsdl:definitions
name="readfile"
targetNamespace="http://xmlns.oracle.com/pcbpel/adapter/file/PVTests/OSBJCATests/readfile"
xmlns:jca="http://xmlns.oracle.com/pcbpel/wsdl/jca/"
xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"
xmlns:tns="http://xmlns.oracle.com/pcbpel/adapter/file/PVTests/OSBJCATests/readfile"
xmlns:opaque="http://xmlns.oracle.com/pcbpel/adapter/opaque/"
xmlns:pc="http://xmlns.oracle.com/pcbpel/"
xmlns:plt="http://schemas.xmlsoap.org/ws/2003/05/partner-link/"
>
<plt:partnerLinkType name="Read_plt" >
<plt:role name="Read_role" >
<plt:portType name="tns:Read_ptt" />
</plt:role>
</plt:partnerLinkType>
<wsdl:types>
<schema targetNamespace="http://xmlns.oracle.com/pcbpel/adapter/opaque/"
xmlns="http://www.w3.org/2001/XMLSchema" >
<element name="opaqueElement" type="base64Binary" />
</schema>
</wsdl:types>
<wsdl:message name="Read_msg">
<wsdl:part name="opaque" element="opaque:opaqueElement"/>
</wsdl:message>
<wsdl:portType name="Read_ptt">
<wsdl:operation name="Read">
<wsdl:input message="tns:Read_msg"/>
</wsdl:operation>
</wsdl:portType>
</wsdl:definitions>

import in OSB first the WSDL, then the JCA (the JCA depends on the WSDL)

The file adapter uses a javax.resource.cci.ConnectionFactory

The Adapter can be configured to support XA Transactions, Local Transactions or No transactions.


to be continued...

Property setBatchSize is not defined for oracle.tip.adapter.file.inbound.FileActivationSpec

Invalid JCA transport endpoint configuration, exception: javax.resource.ResourceException: BINDING.JCA-12532 Cannot set JCA WSDL Property. Error while setting JCA WSDL Property. Property setBatchSize is not defined for oracle.tip.adapter.file.inbound.FileActivationSpec Please verify the spelling of the property.


Unfortunately in OSB 11.1.1.3 BatchSize is not supported (unlike in SOA Suite).
I think in 11.1.1.4 it is.

BEA-000337 StuckThreadMaxTime with OSB FileAdapter

20-Jan-2011 10:57:20 o'clock WET Error WebLogicServer BEA-000337 [STUCK] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "602" seconds working on the request "weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl@57134e4b", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
java.lang.Object.wait(Native Method)
oracle.tip.adapter.file.inbound.FilesToProcess.dequeueToProcess(FilesToProcess.java:101)
oracle.tip.adapter.file.inbound.ProcessWork.run(ProcessWork.java:269)
weblogic.work.ContextWrap.run(ContextWrap.java:41)
weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
weblogic.work.ExecuteThread.run(ExecuteThread.java:173)


I keep getting this error.

Besides, the number of Stuck Threads increases with time, all of them are weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl@blablabla

Another stack trace is:


oracle.tip.adapter.file.inbound.PollWork.run(PollWork.java:369)
weblogic.work.ContextWrap.run(ContextWrap.java:41)
weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
weblogic.work.ExecuteThread.run(ExecuteThread.java:178)



Here it solves the problem:

http://blog.easyteam.fr/2010/10/18/osb-%C2%AB-stuck-threads-%C2%BB-sur-la-consommation-de-fichiers/


basically it suggests to create a WorkManager "FilesPollingWorkManager" for the FileAdapter polling thread and setting "Ignore Stuck Threads" to true, and assigning this FilesPollingWorkManager as "Dispatch-Policy" to the File Polling Proxy Service

FileAdapter claiming that a message is malformed

20-Jan-2011 10:10:32 o'clock WET warning jca_framework_and_adapter BEA-000000 onReject: The resource adapter 'File Adapter' requested handling of a malformed inbound message. However, the following activation property has not been defined: 'rejectedMessageHandlers'. Please define it and redeploy. Will use the default Rejection Directory file://jca\Read\rejectedMessages for now.

20-Jan-2011 10:10:32 o'clock WET warning jca_framework_and_adapter BEA-000000 onReject: Sending invalid inbound message to Rejection Handler:


This happens when processing a 40000 lines file (5 MB of info).
The funny thing is that if I halve the number of lines, it works. And the second half of the file is exactly the same as the first. So I am sure it's NOT a problem with the data format, but rather with the message size.
No further clue is provided by the adapter.

Here look at 25.1.5 Logging for how to turn on debug flag (go to
C:\Oracle1\Middleware\user_projects\domains\soadev\alsbdebug.xml and set
alsb-jca-framework-adapter-debug to true, then restart the server)

but to no avail.

Where is file://jca\Read\rejectedMessages defined?
Here they give some clues, like setting

property name="rejectedMessageHandlers" to file://C:/orabpel/samples/test/errorTest
/rejectedMessages
but it's not accepted by FileAdapter.

see also here the official Oracle doc.

The adapter implementation is oracle.tip.adapter.file.inbound.FileActivationSpec

See here for a list of properties for the Adapter.


Here http://niallcblogs.blogspot.com/2010/09/oracle-file-adapter-osb-11g-debatching.html an interesting post on the OSB/Batch Reading topic.



At last I use these settings:

<adapter-config name="MyFileReader" adapter="File Adapter" wsdlLocation="MyFileReader.wsdl" xmlns="http://platform.integration.oracle/blocks/adapter/fw/metadata">
  
  <connection-factory location="eis/FileAdapter" UIincludeWildcard="*.txt" adapterRef=""/>
  <endpoint-activation portType="Read_ptt" operation="Read">
    <activation-spec className="oracle.tip.adapter.file.inbound.FileActivationSpec">
      <property name="UseHeaders" value="false"/>
      <property name="PhysicalDirectory" value="C:/acmepoc/myFiles/in"/>
      <property name="Recursive" value="true"/>
      <property name="PublishSize" value="200"/>
       <property name="DeleteFile" value="true"/>
      <property name="IncludeFiles" value=".*\.txt"/>
      <property name="PollingFrequency" value="10"/>
      <property name="MinimumAge" value="0"/>
    </activation-spec>
  </endpoint-activation>

</adapter-config>

and I get a beautiful OutOfMemoryException :



JCA_FRAMEWORK_AND_ADAPTER BEA-000000 InboundTranslatorDelegate caught Generic Exception , the Resource Adapter will ignore this
java.lang.OutOfMemoryError: Java heap space

InteractionSpec parameter has invalid value of oracle.tip.adapter.file.outbound.ChunkedInteractionSpec

I get this error message when I open this FileAdapter:

WTF???

Here it seems like oracle.tip.adapter.file.outbound.ChunkedInteractionSpec is a perfectly valid value...


The funny thing is that it deploys just fine.... it MIGHT be a problem in the JDev version number, the application was developed by another developer...


Anyway here it says that in 11.1.1.4 the ChunkSize property has been added to the File Adapter, so that you can be fed morcels of file without actually having to pull the info in chunked mode.

ORABPEL-11207

I was trying to write into a File using File Adapter and opaqueElement.... evidently there is a trick, I know you must transform the data in base64 binary format.... one day I will find out how to do it...

IO Failure in translator.
IO failure because the   translator failed to   to copy InputStream to OutputStream. .
Check the error stack and fix the cause of the error. Contact oracle support if error is not fixable.

 at oracle.tip.pc.services.translation.framework.XlatorHelper.copyStream(XlatorHelper.java:160)
 at oracle.tip.pc.services.translation.xlators.opaque.OpaqueTranslator.translateToNative(OpaqueTranslator.java:311)
 at oracle.tip.adapter.file.FileUtil.translate(FileUtil.java:1262)
 ... 64 more
Caused by: java.io.IOException: Error in encoded stream: needed 4 valid base64 characters but only got 3 before EOF, the 10 most recent characters were: "elloPierre"
 at com.sun.mail.util.BASE64DecoderStream.decode(BASE64DecoderStream.java:250)
 at com.sun.mail.util.BASE64DecoderStream.read(BASE64DecoderStream.java:148)
 at java.io.FilterInputStream.read(FilterInputStream.java:102)
 at oracle.tip.pc.services.translation.framework.XlatorHelper.copyStream(XlatorHelper.java:157)
 ... 66 more

File Adapter metadata with SOA Suite

( the bloody blog keeps messing up with this XML.... I thought that by 2010 these technical issues would not be there any more....)

You can easily get the filename from the properties, in Mediator go to "ASSIGN VALUES" and you will find how to copy Properties to the Expression output....

You might also copy the property to the same property in Mediator and then pass it around... see here


You might get this warning:

Warning: Assigning property/constant "$in.property.jca.file.FileName" to element "$out.payload/imp1:PreactivationFile/imp1:PreactivationLines/imp1:lotInformation/imp1:fileName". Please make sure target is single leaf node, otherwise non-leaf node will contain only string value which may generate non-valid xml as per the xsd.