Thursday, February 27, 2014

Oracle JCA DbAdapter db poller at work

I can see 4 threads
"[ACTIVE] ExecuteThread: '24' for queue: 'weblogic.kernel.Default (self-tuning)'" waiting for lock oracle.tip.adapter.db.InboundWork@5adcf919 TIMED_WAITING

 java.lang.Object.wait(Native Method)

 oracle.tip.adapter.db.InboundWork.run(InboundWork.java:609)

 oracle.tip.adapter.db.inbound.InboundWorkWrapper.run(InboundWorkWrapper.java:43)

 weblogic.work.ContextWrap.run(ContextWrap.java:41)

 weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)

 weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)

 weblogic.work.ExecuteThread.run(ExecuteThread.java:178)


and when it fails I see the full stack trace:
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:476)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:204)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:540)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:217)
at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:1079)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1466)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3752)
at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:3887)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeUpdate(OraclePreparedStatementWrapper.java:1508)
at weblogic.jdbc.wrapper.PreparedStatement.executeUpdate(PreparedStatement.java:172)
at oracle.tip.adapter.db.inbound.DestructivePollingStrategy.poll(DestructivePollingStrategy.java:434)
at oracle.tip.adapter.db.InboundWork.runOnce(InboundWork.java:699)
at oracle.tip.adapter.db.InboundWork.run(InboundWork.java:578)
at oracle.tip.adapter.db.inbound.InboundWorkWrapper.run(InboundWorkWrapper.java:43)
at weblogic.work.ContextWrap.run(ContextWrap.java:41)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:178)


I note down the 4 threads name (12,13,14,24) and execute a poll on a record.
Why 4 threads? Because we have 4 Proxy Services using the oracle.tip.adapter.db.DBActivationSpec to poll a table, and each is configured in the JCA file with NumberOfThreads=1:
  <endpoint-activation portType="SSS_Batching_Service_ptt" operation="receive">
    <activation-spec className="oracle.tip.adapter.db.DBActivationSpec">
      <property name="DescriptorName" value="SSS_Batching_Service.SssBatchOrder"/>
      <property name="QueryName" value="SSS_Batching_ServiceSelect"/>
      <property name="MappingsMetaDataURL" value="NCube_Batching_Service-or-mappings.xml"/>
      <property name="PollingStrategy" value="LogicalDeletePollingStrategy"/>
      <property name="MarkReadColumn" value="STATUS"/>
      <property name="MarkReadValue" value="PROCESSED"/>
      <property name="MarkReservedValue" value="R${weblogic.Name-2}-${IP-2}"/>
      <property name="MarkUnreadValue" value="CREATED"/>
      <property name="PollingInterval" value="60"/>
      <property name="MaxRaiseSize" value="10"/>
      <property name="MaxTransactionSize" value="10"/>
      <property name="NumberOfThreads" value="1"/>
      <property name="ReturnSingleResultSet" value="false"/>
    </activation-spec>
  </endpoint-activation>


See here Polling Thread: By default, as a performance best practice, the Oracle Database Adapter uses one thread to poll the database (NumberOfThreads=1 property in the activation spec). Because the adapter never releases that thread, which is by design, you may see a stuck thread stack trace in the server log. If you set the NumberOfThreads to more than one, you may see stack traces for all of those threads. You can ignore stuck thread stack traces.

Analyzing the logs, it's evident that THE SAME POLLING THREAD IS USED IN OSB TO PROCESS THE MESSAGE.

We have an issue here: if the Processing Time for a batch is > the Polling Period, the Polling Period is not guaranteed.

Something else to explore is to understand the behaviour of the component if we set NumberOfThreads to something > 1.

No comments: