Friday, June 8, 2012

Oracle DbAdapter, FileAdapter, FTPAdapter automated customization

Put in SVN the Deployment Plan created manually in an environment - if makes no sense to create the Plan.xml programmatically, it's only asking for trouble.

In them, replace the actual environment-dependent values with tokens like ${DOMAIN_NAME}
the tokens are:
tokens=DOMAIN_NAME,FTP_HOST,FTP_PASSWORD,FTP_USER

The JNDI names of all your DataSources will not change from environment to environment, so there is no point in replacing them.

create a adapters.properties file like this:

plan1=/opt/oracle/domains/${DOMAIN_NAME}/shared/apps/dbadapter/plan/DBAdapterPlan.xml
plan2=/opt/oracle/domains/${DOMAIN_NAME}/shared/apps/fileadapter/plan/FileAdapterPlan.xml
plan3=/opt/oracle/domains/${DOMAIN_NAME}/shared/apps/ftpadapter/plan/FTPAdapterPlan.xml

plans=plan1,plan2,plan3

adapter1=/opt/oracle/fmw11_1_1_5/osb/soa/connectors/DbAdapter.rar
adapter2=/opt/oracle/fmw11_1_1_5/osb/soa/connectors/FileAdapter.rar
adapter3=/opt/oracle/fmw11_1_1_5/osb/soa/connectors/FtpAdapter.rar

adapters=adapter1,adapter2,adapter3

username=weblogic
password=welcome1
url=t3://myhost.acme.com:7001

tokens=DOMAIN_NAME,FTP_HOST,FTP_PASSWORD,FTP_USER
DOMAIN_NAME=osbpl1do
FTP_HOST=myfthost.acme.com
FTP_PASSWORD=pippopassword
FTP_USER=pippouser


create a WLST script like this:

#############################################################################
#
# Configure a new environment with the Plans.xml from SVN
# uses adapters.properties file.
#
#############################################################################

from java.io import FileInputStream
from shutil import copyfile
import os, sys
import re

#not used, keep only as a reference
def copyfileobj(fsrc, fdst, length=16*1024):
    """copy data from file-like object fsrc to file-like object fdst"""
    while 1:
        buf = fsrc.read(length)
        if not buf:
            break
        fdst.write(buf)

def copyfileWithTokenSubstitution(filein, fileout, properties):
        input = open(filein)
        output = open(fileout, 'w')
        for s in input:
            rep = s
            for tokenname in properties.get("tokens").split(','):
                rep = rep.replace("${" + tokenname + "}", properties.get(tokenname))
            output.write(rep)
        input.close()
        output.close()
    
#not used, keep only as a reference
def copyfile(src, dst):
    """Copy data from src to dst"""
    fsrc = None
    fdst = None
    try:
        fsrc = open(src, 'rb')
        fdst = open(dst, 'wb')
        copyfileobj(fsrc, fdst)
    finally:
        if fdst:
            fdst.close()
        if fsrc:
            fsrc.close()



    
propertyFileName = 'adapters.properties'    

#loading properties
print 'Loading properties from ', propertyFileName
propInputStream = FileInputStream(propertyFileName)
configProps = Properties()
configProps.load(propInputStream)

domainName = configProps.get('DOMAIN_NAME')

plans=configProps.get("plans")
adapters=configProps.get("adapters")

planArray = []
adapterArray = []

#create needed directories where to put *Plan.xml
for plan in plans.split(','):
    planFullPath = configProps.get(plan).replace("${DOMAIN_NAME}", domainName)
    planArray.append(planFullPath)

#create array of adapters
for adapter in adapters.split(','):
    adapterName = configProps.get(adapter)
    adapterArray.append(adapterName)


#create dir if doesn't exist - fail if unable to create it
for planFullPath in planArray:    
    dirName = os.path.dirname(planFullPath)
    print "creating directory " + dirName
    if not os.path.exists(dirName):
        os.makedirs(dirName)
    
#check for directory existence
for planFullPath in planArray:
    dirName = os.path.dirname(planFullPath)
    print "testing directory " + dirName
    
    if not os.path.exists(dirName):
        message = "directory " + dirName + " does not exist"
        print message
        raise Exception(message)

    

#copy all Plan.xml files to their final destination, with token substitution
for planFullPath in planArray:
    fileName = os.path.basename(planFullPath)
    print "copying " + fileName + " to " + planFullPath  
    copyfileWithTokenSubstitution(fileName, planFullPath, configProps)

#Connect to Admin Server    
connect(configProps.get("username"),configProps.get("password"),configProps.get("url"))

#applying changes to the Adapters
edit()
try:
    for index in range(len(planArray)):
        startEdit()
        plan = planArray[index]
        adapter = adapterArray[index]
        
        adapterType = os.path.basename(adapter).split('.')[0]
        
        print 'Applying plan ' + plan + " to adapter " + adapter + " (adapter type is " + adapterType + ")"
        myPlan = loadApplication(adapter, plan)
        myPlan.save()
        save()
        activate(block='true')
        cd('/AppDeployments/' + adapterType + '/Targets')
        #updateApplication(appName, planPath);
        redeploy(adapterType, plan, targets = cmo.getTargets())
         

except:
    dumpStack()
    stopEdit('y')
    message="unable to finish job"
    raise Exception(message)

disconnect()
print "job finished successfully"



11 comments:

xiaowen said...

Hi, very good post. But have a question. from my understanding your script should replace the variable ( like FTP_HOST) in PLAN.XML with the value in your adapter.properties file. This PLAN.XML should be store in SVN, and the target plan.xml is in planFullPath. your script below, copyfileWithTokenSubstitution(fileName, planFullPath, configProps). loading the script from same path in server. how do you read the PLAN.XML check out from SVN?

thanks

Kevin

vernetto said...

I am not 100% sure if this answer your question: the property adapter.properties file contains 3 variables plan1, plan2, plan3 pointing to the xxxplan.xml files on disk.... you must check them out of SVN and copy them to disk...

xiaowen said...

Hi, Thanks for your quick reply. like you said, after check out the xxxplan.xml, i need manually copy them to disk. But to where, is it the same location as current runtime version plan.xml?

vernetto said...

ooooh sorry I am completely stupid, I had not understood your question. The "original" plan.xml should be in the same directory where the .py file is.

xiaowen said...

Is this "original" plan.xml checked out from svn? Is the path of plan1 runtime version plan.xml in the admin server?

xiaowen said...

I think I understand now. i misunderstood this line fileName = os.path.basename(planFullPath). It's just use the same filename, not the actual file in that location. thanks, i will give a go tomorrow. thanks for our time.

xiaowen said...

Hi,Sorry to take so long time to test your scripts. It's half way working. Maybe i missed something. I run the script and gor successful message, but the adapter resource is not showing in my DbAdapter(DbAdapter->Configuration->Outbound Connection Pools)

here is the log


[wlst] Applying plan /oracle/Middleware/user_projects/domains/hb6d_domain/shared/jcaAdapter/plan/DBPlan.xml to adapter /oracle/Middleware/Oracle_SOA1/soa/connectors/DbAdapter.rar (adapter type is DbAdapter)
[wlst] Loading application from /oracle/Middleware/Oracle_SOA1/soa/connectors/DbAdapter.rar ...
[wlst]
[wlst]
[wlst]
[wlst] Loaded application from /oracle/Middleware/Oracle_SOA1/soa/connectors/DbAdapter.rar and the deployment plan from /oracle/Middleware/user_projects/domains/hb6d_domain/shared/jcaAdapter/plan/DBPlan.xml
[wlst] The WLSTPlan object is assigned to variable wlstPlan_connectors.
[wlst]
[wlst] Saving all your changes ...
[wlst] Saved all your changes successfully.
[wlst] Activating all your changes, this may take a while ...
[wlst] The edit lock associated with this edit session is released
[wlst] once the activation is completed.
[wlst] Activation completed
[wlst] Redeploying application DbAdapter ...
[wlst]
[wlst] .Completed the redeployment of Application with status completed
[wlst] Current Status of your Deployment:
[wlst] Deployment command type: redeploy
[wlst] Deployment State : completed
[wlst] Deployment Message : no message
[wlst] Done executing the script.
[wlst] Disconnected from weblogic server: AdminServer
[wlst]

BUILD SUCCESSFUL
Total time: 14 seconds

Could you help,please

cheers

Kevin

vernetto said...

in your logs everything looks correct, we also run the script a million times and it was always working. My only suggestion is to run the script manually, one line at a time, and examine each variable and the result of each action....

xiaowen said...

The new resource was added in the Plan.xml. But if i did not manually "update" DbAdapter, it's not showing in the console. ????

vernetto said...

maybe you need to uncomment the line with "updateApplication"... not sure why, we don't do "updateApplication" and it works here... please try that and kindly let me know...
appName should be DbAdapter.rar, I guess, kindly read the documentation:

http://docs.oracle.com/cd/E13222_01/wls/docs91/config_scripting/reference.html

xiaowen said...

Hi, I know what's happened now. It's weblogic console bug. The scripts is working and create new Adapter configuration in the Plan.xml and also in the JNDI tree. BUT not showing the new configuration in the console(DbAdapter->Configuration->Outbound Connection Pools). it's appeared in (DbAdapter->Control->Outbound Connection Pools).
I did not deploy my osb code, because i did not see they are in console(DbAdapter->Configuration->Outbound Connection Pools). So just ignore this in the console, everything is working fine.