A Missing Class during Business Processing
In our development environment a BPEL Fault began occurring with the following error in the SOA Managed Server logs:
<bpelFault><faultType>0</faultType><remoteFault xmlns="http://schemas.oracle.com/bpel/extension"><part name="summary"><summary>oracle.fabric.common.FabricInvocationException: java.lang.NoClassDefFoundError: oracle/tip/adapter/api/MutatorsAsProperties</summary></part><part name="detail"><detail>oracle/tip/adapter/api/MutatorsAsProperties</detail></part><part name="code"><code>null</code></part></remoteFault></bpelFault>
************************
java.lang.NoClassDefFoundError: oracle/tip/adapter/api/MutatorsAsProperties
************************
In this environment the OSB and SOA Managed Servers are both running on the same VM/Domain, so in this case they both use the same ./bin/setDomainEnv.sh script which is magically generated during installation.
Running both OSB and SOA on the same node has issues that occur when a patch to one or both products changes which classes are available in the libraries used by the managed servers. I suspected the issue was a recent OSB patch had caused setDomainEnv to use an OSB copy of a JAR needed by SOA, but how can we prove it?
First, let's identify the location (jar file) containing the class we want.
We can find it here in the SOA Product home:
[oracle@dev-01 ~]$ find /path/to/products/Oracle_SOA1/ -name "*.jar" | xargs grep MutatorsAsProperties
Binary file ./soa/modules/oracle.soa.adapter_11.1.1/jca-binding-api.jar matches
Try the same search in the OSB home and there's no matches:
[oracle@dev-01 ~]$ find /path/to/products/Oracle_OSB1/ -name "*.jar" | xargs grep MutatorsAsProperties | wc -l
0
From this investigation we learn that the SOA server needs the jca-binding-api.jar in its classpath. So let's check the server logs to see what's happening. Here’s a copy of the classpath from the SOA1.out log:
CLASSPATH=/app/oracle/product/oracle_common/modules/oracle.jdbc_11.1.1/ojdbc6dms.jar:/app/oracle/product/Oracle_SOA1/soa/modules/user-patch.jar:/app/oracle/product/Oracle_SOA1/soa/modules/soa-startup.jar::/app/oracle/product/Oracle_OSB1/lib/osb-server-modules-ref.jar:/app/oracle/product/patch_wls1036/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/app/oracle/product/patch_ocp371/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/usr/java/latest/lib/tools.jar:/app/oracle/product/wlserver_10.3/server/lib/weblogic_sp.jar:/app/oracle/product/wlserver_10.3/server/lib/weblogic.jar:/app/oracle/product/modules/features/weblogic.server.modules_10.3.6.0.jar:/app/oracle/product/wlserver_10.3/server/lib/webservices.jar:/app/oracle/product/modules/org.apache.ant_1.7.1/lib/ant-all.jar:/app/oracle/product/modules/net.sf.antcontrib_1.1.0.0_1-0b2/lib/ant-contrib.jar:blah blah blah
Wow. It’s a long list, and we check through it we don't see jca-binding-api.jar anywhere in there. There are a lot of JARs that make up the SOA product, many of them are included in the classpath inside the Manifests of other JARs so they do not need to be explicitly written in the classpath that the server shows.
So, we could try adding it manually to this list, but then maybe another class will be missing. Our best bet is to work out which OSB class is being loaded in place of the SOA. Let’s cut down the CLASSPATH list to just ones coming from the OSB Home:
[oracle@dev-ofm-01 logs]$ grep CLASSPATH soa1.out | tr ":" "\n" | grep Oracle_OSB1
- /app/oracle/product/Oracle_OSB1/lib/osb-server-modules-ref.jar
- /app/oracle/product/Oracle_OSB1/soa/modules/oracle.soa.common.adapters_11.1.1/oracle.soa.common.adapters.jar
- /app/oracle/product/Oracle_OSB1/lib/version.jar
- /app/oracle/product/Oracle_OSB1/lib/alsb.jar
- /app/oracle/product/Oracle_OSB1/3rdparty/classes
- /app/oracle/product/Oracle_OSB1/lib/external/log4j_1.2.8.jar
One JAR immediately stands out as the culprit of our problems (because it references SOA) -- ./Oracle_OSB1/soa/modules/oracle.soa.common.adapters_11.1.1/oracle.soa.common.adapters.jar
We know from our earlier searching that the MutatorsAsProperties class was not found in the OSB home. Let’s now check if there is a corresponding JAR in the SOA home that includes the jca-bindings-api.
Yes, there is:
/app/oracle/product/Oracle_SOA1/soa/modules/oracle.soa.common.adapters_11.1.1/oracle.soa.common.adapters.jar
Let’s peek inside the JAR and we see only a manifest file with a classpath to include other JARs in the SOA Home:
Manifest-Version: 1.0
…
Class-Path: ../oracle.soa.adapter_11.1.1/jca-binding-api.jar ../oracle
.soa.adapter_11.1.1/adapter_xbeans.jar ../oracle.soa.fabric_11.1.1/bp
m-infra.jar ../oracle.soa.fabric_11.1.1/oracle-soa-client-api.jar ../
maverick-all.jar com.bea.alsb.client_1.4.0.0.jar osb_soa_client.jar
We found it! So, it seems that our SOA Managed Server should be using ./Oracle_SOA1/soa/modules/oracle.soa.common.adapters_11.1.1/oracle.soa.common.adapters.jar
While our OSB Managed Server should use:
./Oracle_SOA1/soa/modules/oracle.soa.common.adapters_11.1.1/oracle.soa.common.adapters.jar
Update the setDomainEnv.sh with some logic to reflect this.
Checking the setDomainEnv.sh script in the domain home we find the lines:
POST_CLASSPATH="/u01/app/oracle/product/Oracle_OSB1/soa/modules/oracle.soa.common.adapters_11.1.1/oracle.soa.common.adapters.jar${CLASSPATHSEP}}${POST_CLASSPATH}"
export POST_CLASSPATH
Let’s change this as follows (The parts in bold should be checked for your own environment):
if [ "${SERVER_NAME}" == "dev01_osb1" ] ; then
POST_CLASSPATH="/u01/app/oracle/product/Oracle_OSB1/soa/modules/oracle.soa.common.adapters_11.1.1/oracle.soa.common.adapters.jar${CLASSPATHSEP}${ALSB_HOME}/soa/modules/oracle.soa.adapter_11.1.1/adapter_xbeans.jar${CLASSPATHSEP}${ALSB_HOME}/soa/modules/oracle.soa.adapter_11.1.1/jca-binding-api.jar${CLASSPATHSEP}${POST_CLASSPATH}"
export POST_CLASSPATH
else
POST_CLASSPATH="/u01/app/oracle/product/Oracle_SOA1/soa/modules/oracle.soa.common.adapters_11.1.1/oracle.soa.common.adapters.jar${CLASSPATHSEP}${POST_CLASSPATH}"
export POST_CLASSPATH
fi
Restart both the SOA and OSB VMs and check the CLASSPATH output during startup. We should see the OSB Server using the OSB version of the JAR and the SOA Server using the SOA copy.
Re-create the conditions where the class originally called and the error happened and now the process does not encounter any fault.
