Problems accessing datasource in Karaf 4 from Apache Camel - apache-camel

I've created a datasource in Karaf 4 (ServiceMix 7) that works from the karaf console - I can list tables, execute queries and so on.
My issue is when I try to use it from my Camel route.
Excerpt from my blueprint:
...
<reference filter="(osgi.jndi.service.name=jdbc/erp)" id="erpDataSource" interface="javax.sql.DataSource"/>
...
<to id="erpSelectQuery" uri="jdbc:erpDataSource"/>
...
It finds my dataSource but the blueprint can't start due to:
"java.lang.IllegalArgumentException: connectionFactory must be specified"
My datasource was created using:
jdbc:ds-create -dbName erp -dt DataSource -dn mysql -u erp -dc com.mysql.jdbc.Driver -p pre jdbc/erp
I'm at loss here

I have never done it via the jdbc command syntax I followed the guides from the Ops4J Wiki On Datasource creation which I like for one reason alone, this method creates a simple text file that can be administered by not just a Java developer, i.e. it is easier to modify and troubleshoot.
For the sake of not subjecting my answer to link rot I will just outline the procedure here.
Create a datasource configuration file(simple text file) in /servicemixhome/etc with the following naming convention org.ops4j.datasource-give_your_datasource_a_name.cfg .
In the config file configure the appropriate settings an example of mine looks like this:
osgi.jdbc.driver.class = com.mysql.jdbc.Driver
databaseName=dhData
user=foo
url=jdbc:mysql://192.199.199.199:3306/dhData
password=somepassword
dataSourceName=myDSName
Make sure you installed the ops4j required features:
feature:install pax-jdbc-mysql pax-jdbc-config
Now list the datasources using the following syntax:
karaf#root()> service:list javax.sql.DataSource
This will echo something like the list below back.
[javax.sql.DataSource]
----------------------
osgi.jdbc.driver.class = com.mysql.jdbc.Driver
databaseName=dhData
user=foo
url=jdbc:mysql://192.199.199.199:3306/dhData
password=somepassword
dataSourceName=myDSName
Provided by :
OPS4J Pax JDBC Config (216)
At this point you can reference the datasource usign an osgi filter in the blueprint.xml with the following syntax:
<reference filter="(&(objectClass=javax.sql.DataSource)(dataSourceName=myDSName ))" id="myData" interface="javax.sql.DataSource"/>
Then to reference this as property of a bean for example you could do the following:
<bean class="foo.bar" id="ImsCbrEventsBean">
<property name="dataSource" ref="myData"/>
</bean>
Keep in mind this creates a singular connection to a database and you should really create a connection pool.
This can be done by installing the pax-jdbc-pool-dbcp2 feature or any of the other connection pools but use only one at a time, then modifying the datasource config file to carry appropriate information like the example below:
osgi.jdbc.driver.name = mysql
databaseName=dhData
user=foo
url=jdbc:mysql://192.199.199.199:3306/dhData
password=somepassword
dataSourceName=myDSName
jdbc.pool.maxTotal=32
jdbc.pool.blockWhenExhausted=true
jdbc.pool.lifo=false
jdbc.pool.maxIdle=24
jdbc.pool.maxWaitMillis=5000
jdbc.pool.minEvictableIdleTimeMillis=1800000
jdbc.pool.minIdle=16
jdbc.pool.numTestsPerEvictionRun=3
jdbc.pool.softMinEvictableIdleTimeMillis=-1
jdbc.pool.testOnBorrow=true
jdbc.pool.testOnCreate=true
jdbc.pool.testOnReturn=true
jdbc.pool.testWhileIdle=true
jdbc.pool.timeBetweenEvictionRunsMillis=3600000

Related

JPA/Wildfly/MsSql: GenericJDBCException: Unable to acquire JDBC Connection

currently my team and I are facing a strange problem. By now we have been spending 4 days searching for the solution.
We are developing a Java EE Web Application. We have been using Wildfly 14. Using Wildfly 14, there was no problem. Now we have upgraded Wildfly to Version 18 and afterwars to 19 and are now facing the following problem. As database we have an MS SQL.
The datasource looks like this:
<datasource jndi-name="java:/label" pool-name="Label">
<connection-url>jdbc:sqlserver://localhost:1433;DatabaseName=label</connection-url>
<driver>mssql-jdbc-8.2.2.jre8.jar</driver>
<security>
<user-name>label</user-name>
<password>label</password>
</security>
</datasource>
And the persistence.xml is the following:
<?xml version="1.0" encoding="UTF-8"?>
<persistence version="2.0"
xmlns="http://java.sun.com/xml/ns/persistence"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">
<persistence-unit name="label-pu"
transaction-type="JTA">
<jta-data-source>java:/label</jta-data-source>
<properties>
<property name="hibernate.dialect"
value="org.hibernate.dialect.SQLServerDialect" />
</properties>
</persistence-unit>
We have a Bean in which we inject an EntityManager using the persistence-unit. When calling the bean's method we are getting an exception. The datasource is working. We checked that in the wildfly ui.
#Dependent
public class ConstructionService {
#PersistenceContext(unitName = "label-pu")
private EntityManager em;
public List<ConstructionRecordEntity> retrieveRecordForConstruction(int contructionId) {
String sqlSelectEmployeeId = "SELECT lfdnr, " + //
" erwkz1, " + //
" erwbe, " + //
" erwdat " + //
" FROM [label].[dbo].[aderw] where adnr = " + contructionId;//
return em//
.createNativeQuery(sqlSelectEmployeeId, ConstructionRecordEntity.class)//
.getResultList();
}
In the following you can see the stacktrace we are getting.
15:11:24,506 WARN [org.camunda.bpm.engine.rest.exception] (default task-1) ENGINE-REST-HTTP500 javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Unable to acquire JDBC Connection
at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:154)
at org.hibernate.query.internal.AbstractProducedQuery.list(AbstractProducedQuery.java:1515)
at org.hibernate.query.Query.getResultList(Query.java:132)
at de._._.ConstructionService.retrieveRecordForConstruction(ConstructionService.java:26)
...
Caused by: org.hibernate.exception.GenericJDBCException: Unable to acquire JDBC Connection
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:47)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:113)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:99)
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.acquireConnectionIfNeeded(LogicalConnectionManagedImpl.java:109)
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.getPhysicalConnection(LogicalConnectionManagedImpl.java:136)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.connection(StatementPreparerImpl.java:50)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$5.doPrepare(StatementPreparerImpl.java:149)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$StatementPreparationTemplate.prepareStatement(StatementPreparerImpl.java:176)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.prepareQueryStatement(StatementPreparerImpl.java:151)
at org.hibernate.loader.Loader.prepareQueryStatement(Loader.java:2082)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:2012)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1990)
at org.hibernate.loader.Loader.doQuery(Loader.java:949)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:351)
at org.hibernate.loader.Loader.doList(Loader.java:2787)
at org.hibernate.loader.Loader.doList(Loader.java:2770)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2604)
at org.hibernate.loader.Loader.list(Loader.java:2599)
at org.hibernate.loader.custom.CustomLoader.list(CustomLoader.java:338)
at org.hibernate.internal.SessionImpl.listCustomQuery(SessionImpl.java:2243)
at org.hibernate.internal.AbstractSharedSessionContract.list(AbstractSharedSessionContract.java:1069)
at org.hibernate.query.internal.NativeQueryImpl.doList(NativeQueryImpl.java:170)
at org.hibernate.query.internal.AbstractProducedQuery.list(AbstractProducedQuery.java:1506)
... 306 more
Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000457: Unchecked throwable in managedConnectionReconnected() cl=org.jboss.jca.core.connectionmanager.listener.TxConnectionListener#c18c68f[state=DESTROYED managed connection=org.jboss.jca.adapters.jdbc.local.LocalManagedConnection#51667cf2 connection handles=0 lastReturned=1594905084472 lastValidated=1594904382991 lastCheckedOut=1594905084468 trackByTx=false pool=org.jboss.jca.core.connectionmanager.pool.strategy.OnePool#66e3a014 mcp=SemaphoreConcurrentLinkedQueueManagedConnectionPool#685b64a3[pool=Label] xaResource=LocalXAResourceImpl#671cf884[connectionListener=c18c68f connectionManager=bec01f1 warned=false currentXid=null productName=Microsoft SQL Server productVersion=15.00.2000 jndiName=java:/label] txSync=null]
at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:159)
at org.jboss.as.connector.subsystems.datasources.WildFlyDataSource.getConnection(WildFlyDataSource.java:64)
at org.hibernate.engine.jdbc.connections.internal.DatasourceConnectionProviderImpl.getConnection(DatasourceConnectionProviderImpl.java:122)
at org.hibernate.internal.NonContextualJdbcConnectionAccess.obtainConnection(NonContextualJdbcConnectionAccess.java:35)
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.acquireConnectionIfNeeded(LogicalConnectionManagedImpl.java:106)
... 325 more
Caused by: javax.resource.ResourceException: IJ000457: Unchecked throwable in managedConnectionReconnected() cl=org.jboss.jca.core.connectionmanager.listener.TxConnectionListener#c18c68f[state=DESTROYED managed connection=org.jboss.jca.adapters.jdbc.local.LocalManagedConnection#51667cf2 connection handles=0 lastReturned=1594905084472 lastValidated=1594904382991 lastCheckedOut=1594905084468 trackByTx=false pool=org.jboss.jca.core.connectionmanager.pool.strategy.OnePool#66e3a014 mcp=SemaphoreConcurrentLinkedQueueManagedConnectionPool#685b64a3[pool=Label] xaResource=LocalXAResourceImpl#671cf884[connectionListener=c18c68f connectionManager=bec01f1 warned=false currentXid=null productName=Microsoft SQL Server productVersion=15.00.2000 jndiName=java:/label] txSync=null]
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.reconnectManagedConnection(AbstractConnectionManager.java:1055)
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:792)
at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:151)
... 329 more
Caused by: javax.resource.ResourceException: IJ000461: Could not enlist in transaction on entering meta-aware object
at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.managedConnectionReconnected(TxConnectionManagerImpl.java:571)
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.reconnectManagedConnection(AbstractConnectionManager.java:977)
... 331 more
Caused by: javax.transaction.SystemException: Error enlisting resource in transaction=Local transaction (delegate=TransactionImple < ac, BasicAction: 0:ffffc0a83801:34f7831:5f103dd7:8e1 status: ActionStatus.ABORT_ONLY >, owner=Local transaction context for provider JBoss JTA transaction provider)
at org.jboss.jca.core.connectionmanager.listener.TxConnectionListener$TransactionSynchronization.checkEnlisted(TxConnectionListener.java:957)
at org.jboss.jca.core.connectionmanager.listener.TxConnectionListener.enlist(TxConnectionListener.java:394)
at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.managedConnectionReconnected(TxConnectionManagerImpl.java:564)
... 332 more
Caused by: java.lang.Throwable: Failed to enlist
at org.jboss.jca.core.connectionmanager.listener.TxConnectionListener$TransactionSynchronization.enlist(TxConnectionListener.java:1001)
at org.jboss.jca.core.connectionmanager.listener.TxConnectionListener.enlist(TxConnectionListener.java:379)
... 333 more
I apreciate any hint.
We found out, that this is related to Camunda, when we first thought is was a pure Java EE Problem.
Using the mentioned Bean in a Rest Service works perfectly. Only when we inject the Bean into a Camunda JavaDelegate there is a problem.
We could solve this by adding
<system-properties>
<property name="com.arjuna.ats.arjuna.allowMultipleLastResources" value="true"/>
</system-properties>
to Wildfly Standalone.
It is working but we are getting a warning. Actually, this is necessary if you are using xa-datasource to have a transaction that surrounds multiple datasources. However, we do not have xa-datasource and we do not need a transaction that surrounds everything.
As far as we understand Camunda uses JTA to make the process jobs transaction safe. So when we are making a db query inside the JavaDelegate, there is a transaction inside a transaction. The first transaction is instantiated by Camunda the second one is instantiated by us by using JPA to query the database. Nevertheless, I do not understand. This scenario should be very common. So other people should have experienced this behavior, too? Or maybe it is related to the MS SQL Database?
The other thing that I do not understand is, that using JPA inside JavaDelegate worked with camunda 7.10 and in the latest version 7.13 it stopped working.
So, we found a workaround and we are not stuck anymore. Nonetheless, this workaround is not satisfying as we do not understand entirely what is going on underneath and we keep looking for an appropriate solution.
I was working with WildFly server long time ago so there is just my amateur thought:
There is something bad happened with transaction.
The issue point is connected with resource enlistement:
http://javadox.com/org.jboss.ironjacamar/ironjacamar-core-impl/1.2.7.Final/org/jboss/jca/core/connectionmanager/listener/TxConnectionListener.TransactionSynchronization.html#enlist()
According to the documentation, the database connection is also an "Transactional Resource". So, yes, we do not have an database connection for some reason.
May be someone close transaction for some reason? Try to check transaction status with injection of UserTransaction or with debugging of TxConnectionListener class (see the log)
If transaction is ok, then just debug from the point of error. Try to find the check that leads to an error. You can find the proper library to debug inside WildFly directory (with Far Manager, Double Commander and so on) and connect it to the IDE to be able to make a break point in the proper place.

Apache Camel "transacted" does not work well with sql component`s "outputType=StreamList"

In my transacted camel route I need to:
Call oracle package to set value for variable in this package;
Execute sql statement which is using variable from package;
Note that package variable is only visible in connection from which it was set - so I need to use "transacted" here.
Here is a sample code which demonstrates the problem:
from("direct-vm:process")
.transacted()
.to("sql:call my_pack.set_v1('10')")
.to("sql:select my_pack.get_v1 from dual?outputType=StreamList")
.split(body()).streaming()
.log("${body}")
.end();
Result for above code will be: GET_V1=null
If I comment ".transacted()" I will get: GET_V1=10
If I remove "StreamList" option from sql and un-comment ".transacted()": GET_V1=10
Question: is it not possible for "transacted" to work with sql component`s "StreamList" option?
Additional info:
If I start above route in multiple threads, like this:
Map<String, String> map = new HashMap<>();
map.put("10", "10");
map.put("20", "20");
map.put("30", "30");
map.put("40", "40");
map.put("50", "50");
map.forEach((key, values) -> {
from("timer://runOnce?repeatCount=1")
.setHeader("key", constant(key))
.setHeader("value", constant(values))
.inOnly("seda:processParallel");
});
from("seda:processParallel?concurrentConsumers=5")
.to("direct:process");
from("direct:process")
//.transacted()
.to("sql:call my_pack.pset_v1(:#value)?dataSource=generalDataSource")
.to("sql:select :#key key, my_pack.get_v1 value from dual?outputType=StreamList")
.split(body()).streaming()
.to("log:row")
.end();
I will get inconsistent results:
KEY=**20**, VALUE=**50**
KEY=**50**, VALUE=**40**
KEY=**40**, VALUE=**20**
KEY=**10**, VALUE=**30**
KEY=**30**, VALUE=**10**
Transaction manager configures as shown below:
#Bean
public DataSourceTransactionManager dataSourceTransactionManager(DataSource dataSource) {
DataSourceTransactionManager dataSourceTransactionManager = new DataSourceTransactionManager();
dataSourceTransactionManager.setDataSource(dataSource);
return dataSourceTransactionManager;
}
if it is only for same database , you dont need transacted() which is XA transaction
usually it convers different resources for example one JMS , another Database
Can you show us how you have defined your transactionManager ? In particular, did you bind this txManager to your datasource ?
<bean id="txManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="generalDataSource"/>
</bean>

Spring boot Auto connection to database [duplicate]

I have a nice little Spring Boot JPA web application. It is deployed on Amazon Beanstalk and uses an Amazon RDS for persisting data. It is however not used that often and therefore fails after a while with this kind of exception:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet successfully received from the server was 79,870,633 milliseconds ago.
The last packet sent successfully to the server was 79,870,634 milliseconds ago. is longer than the server configured value of 'wait_timeout'.
You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem.
I am not sure how to configure this setting and can not find information on it on http://spring.io (a very good site though). What are some ideas or pointers to information?
I assume that boot is configuring the DataSource for you. In this case, and since you are using MySQL, you can add the following to your application.properties up to 1.3
spring.datasource.testOnBorrow=true
spring.datasource.validationQuery=SELECT 1
As djxak noted in the comment, 1.4+ defines specific namespaces for the four connections pools Spring Boot supports: tomcat, hikari, dbcp, dbcp2 (dbcp is deprecated as of 1.5). You need to check which connection pool you are using and check if that feature is supported. The example above was for tomcat so you'd have to write it as follows in 1.4+:
spring.datasource.tomcat.testOnBorrow=true
spring.datasource.tomcat.validationQuery=SELECT 1
Note that the use of autoReconnect is not recommended:
The use of this feature is not recommended, because it has side effects related to session state and data consistency when applications don't handle SQLExceptions properly, and is only designed to be used when you are unable to configure your application to handle SQLExceptions resulting from dead and stale connections properly.
The above suggestions did not work for me.
What really worked was the inclusion of the following lines in the application.properties
spring.datasource.testWhileIdle = true
spring.datasource.timeBetweenEvictionRunsMillis = 3600000
spring.datasource.validationQuery = SELECT 1
You can find the explanation out here
Setting spring.datasource.tomcat.testOnBorrow=true in application.properties didn't work.
Programmatically setting like below worked without any issues.
import org.apache.tomcat.jdbc.pool.DataSource;
import org.apache.tomcat.jdbc.pool.PoolProperties;
#Bean
public DataSource dataSource() {
PoolProperties poolProperties = new PoolProperties();
poolProperties.setUrl(this.properties.getDatabase().getUrl());
poolProperties.setUsername(this.properties.getDatabase().getUsername());
poolProperties.setPassword(this.properties.getDatabase().getPassword());
//here it is
poolProperties.setTestOnBorrow(true);
poolProperties.setValidationQuery("SELECT 1");
return new DataSource(poolProperties);
}
I just moved to Spring Boot 1.4 and found these properties were renamed:
spring.datasource.dbcp.test-while-idle=true
spring.datasource.dbcp.time-between-eviction-runs-millis=3600000
spring.datasource.dbcp.validation-query=SELECT 1
whoami's answer is the correct one. Using the properties as suggested I was unable to get this to work (using Spring Boot 1.5.3.RELEASE)
I'm adding my answer since it's a complete configuration class so it might help someone using Spring Boot:
#Configuration
#Log4j
public class SwatDataBaseConfig {
#Value("${swat.decrypt.location}")
private String fileLocation;
#Value("${swat.datasource.url}")
private String dbURL;
#Value("${swat.datasource.driver-class-name}")
private String driverName;
#Value("${swat.datasource.username}")
private String userName;
#Value("${swat.datasource.password}")
private String hashedPassword;
#Bean
public DataSource primaryDataSource() {
PoolProperties poolProperties = new PoolProperties();
poolProperties.setUrl(dbURL);
poolProperties.setUsername(userName);
poolProperties.setPassword(password);
poolProperties.setDriverClassName(driverName);
poolProperties.setTestOnBorrow(true);
poolProperties.setValidationQuery("SELECT 1");
poolProperties.setValidationInterval(0);
DataSource ds = new org.apache.tomcat.jdbc.pool.DataSource(poolProperties);
return ds;
}
}
I have similar problem. Spring 4 and Tomcat 8. I solve the problem with Spring configuration
<bean id="dataSource" class="org.apache.tomcat.jdbc.pool.DataSource" destroy-method="close">
<property name="initialSize" value="10" />
<property name="maxActive" value="25" />
<property name="maxIdle" value="20" />
<property name="minIdle" value="10" />
...
<property name="testOnBorrow" value="true" />
<property name="validationQuery" value="SELECT 1" />
</bean>
I have tested. It works well! This two line does everything in order to reconnect to database:
<property name="testOnBorrow" value="true" />
<property name="validationQuery" value="SELECT 1" />
In case anyone is using custom DataSource
#Bean(name = "managementDataSource")
#ConfigurationProperties(prefix = "management.datasource")
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}
Properties should look like the following. Notice the #ConfigurationProperties with prefix. The prefix is everything before the actual property name
management.datasource.test-on-borrow=true
management.datasource.validation-query=SELECT 1
A reference for Spring Version 1.4.4.RELEASE
As some people already pointed out, spring-boot 1.4+, has specific namespaces for the four connections pools. By default, hikaricp is used in spring-boot 2+. So you will have to specify the SQL here. The default is SELECT 1. Here's what you would need for DB2 for example:
spring.datasource.hikari.connection-test-query=SELECT current date FROM sysibm.sysdummy1
Caveat: If your driver supports JDBC4 we strongly recommend not setting this property. This is for "legacy" drivers that do not support the JDBC4 Connection.isValid() API. This is the query that will be executed just before a connection is given to you from the pool to validate that the connection to the database is still alive. Again, try running the pool without this property, HikariCP will log an error if your driver is not JDBC4 compliant to let you know. Default: none
For those who want to do it from YAML with multiple data sources, there is a great blog post about it: https://springframework.guru/how-to-configure-multiple-data-sources-in-a-spring-boot-application/
It basically says you both need to configure data source properties and datasource like this:
#Bean
#Primary
#ConfigurationProperties("app.datasource.member")
public DataSourceProperties memberDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#Primary
#ConfigurationProperties("app.datasource.member.hikari")
public DataSource memberDataSource() {
return memberDataSourceProperties().initializeDataSourceBuilder()
.type(HikariDataSource.class).build();
}
Do not forget to remove #Primary from other datasources.

Spark - jdbc write fails in Yarn cluster mode but works in spark-shell

I am using Spark 1.6.2, Hadoop 2.6, Scala 2.10.5 and Java 1.7
I am using JDBC to read data from MSSQL and this works without any problem:
val hqlContext = new HiveContext(sc)
val url = "jdbc:sqlserver://1.1.1.1:1111;database=CIQOwnershipProcessing;user=OwnershipUser;password=Ownership123"
val driver = "com.microsoft.sqlserver.jdbc.SQLServerDriver";
val df1 = hqlContext.read.format("jdbc").options(
Map("url" -> url, "driver" -> driver,
"dbtable" -> "(select * from OwnershipStandardization_PositionSequence_tbl) as ps")).load()
And, while writing back dataframe to MSSQL, I am using the JDBC write as shown below. This works fine in Spark-shell but fails when I do spark-submit in Yarn-Cluster mode. What am I missing ?
val prop = new java.util.Properties
df1.write.mode("Overwrite").jdbc(url, "CIQOwnershipProcessing.dbo.df_sparkop",prop)
This is how my spark-submit command looks like. As you can see, I am passing the SQLJDBC jar path too. And, I have also specified the jdbc jar path in "spark.executor.extraClassPath" property in spark-defaults.conf on all nodes of the cluster. Since the JDBC read is working, I doubt if it has anything to do with the classpaths.
spark-submit --class com.spgmi.csd.OshpStdCarryOver --master yarn --deploy-mode cluster --conf spark.yarn.executor.memoryOverhead=2048 --num-executors 1 --executor-cores 2 --driver-memory 3g --executor-memory 8g --jars $SPARK_HOME/lib/datanucleus-api-jdo-3.2.6.jar,$SPARK_HOME/lib/datanucleus-core-3.2.10.jar,$SPARK_HOME/lib/datanucleus-rdbms-3.2.9.jar,/usr/share/java/sqljdbc_4.1/enu/sqljdbc41.jar --files $SPARK_HOME/conf/hive-site.xml $SPARK_HOME/lib/spark-poc2-17.1.0.jar
The error thrown in the Yarn-Cluster mode is:
17/01/05 10:21:31 ERROR yarn.ApplicationMaster: User class threw
exception: java.lang.InstantiationException:
org.apache.spark.sql.execution.datasources.jdbc.DriverWrapper
java.lang.InstantiationException:
org.apache.spark.sql.execution.datasources.jdbc.DriverWrapper
at java.lang.Class.newInstance(Class.java:368)
at org.apache.spark.sql.execution.datasources.jdbc.DriverRegistry$.register(DriverRegistry.scala:46)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$2.apply(JdbcUtils.scala:53)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$2.apply(JdbcUtils.scala:52)
at org.apache.spark.sql.DataFrameWriter.jdbc(DataFrameWriter.scala:278)
at com.spgmi.csd.OshpStdCarryOver$.main(SparkOshpStdCarryOver.scala:175)
at com.spgmi.csd.OshpStdCarryOver.main(SparkOshpStdCarryOver.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:558)
I was facing the same issue. I resolved it by setting connection property in prop.
val prop = new java.util.Properties
prop.setProperty("driver","com.mysql.jdbc.Driver")
now pass this prop in
df1.write.mode("Overwrite").jdbc(url, "CIQOwnershipProcessing.dbo.df_sparkop",prop)
Your problem feels very similar to SPARK-14204 and SPARK-14162 -- although that bug was supposed to be fixed in Spark 1.6.2 (?!)
With a Type 4 JDBC driver you should not have to explicitly mention the "driver" property; the JAR should automatically register the URL prefix that it supports (here jdbc:sqlserver:).
But because of the bug, the Spark JDBC module may not use that "registration" to find the driver that implicitly matches the URL.
In other words: for reading, you force the "driver" property and the connection works; for writing, you don't force it, and it does not work. Aha!

Apache CXF 2.7.11 on WebSphere 8.5

I have an application that exposes web services for clients via CXF. This side of things works perfectly.
The application also needs to act as a client itself and contact other servers, this is where I am running into problems.
With "Parent First" classloading I get this:
Caused by: javax.xml.ws.WebServiceException: Error: Maintain Session is enabled but none of the session properties (Cookies, Over-written URL) are returned.
at org.apache.axis2.jaxws.ExceptionFactory.createWebServiceException(ExceptionFactory.java:173) ~[org.apache.axis2.jar:na]
at org.apache.axis2.jaxws.ExceptionFactory.makeWebServiceException(ExceptionFactory.java:70) ~[org.apache.axis2.jar:na]
at org.apache.axis2.jaxws.ExceptionFactory.makeWebServiceException(ExceptionFactory.java:118) ~[org.apache.axis2.jar:na]
at org.apache.axis2.jaxws.BindingProvider.setupSessionContext(BindingProvider.java:355) ~[org.apache.axis2.jar:na]
at org.apache.axis2.jaxws.BindingProvider.checkMaintainSessionState(BindingProvider.java:322) ~[org.apache.axis2.jar:na]
at org.apache.axis2.jaxws.client.proxy.JAXWSProxyHandler.invokeSEIMethod(JAXWSProxyHandler.java:393) ~[org.apache.axis2.jar:na]
at ...
With "Parent last" classloading the application can't even expose its own services:
[23/06/15 15:33:12:985 BST] 000002d3 servlet E com.ibm.ws.webcontainer.servlet.ServletWrapper service Uncaught service() exception thrown by servlet cxf: java.lang.VerifyError: JVMVRFY013 class loading constraint violated; class=org/apache/cxf/jaxb/attachment/JAXBAttachmentUnmarshaller, method=getAttachmentAsDataHandler(Ljava/lang/String;)Ljavax/activation/DataHandler;, pc=0
at java.lang.J9VMInternals.verifyImpl(Native Method)
at java.lang.J9VMInternals.verify(J9VMInternals.java:85)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:162)
I have tried disabling WebShere's own JAXWS Engine via the WAR's manifest.mf and no matter what I try with "Parent last" classloading I always get some error like the above. A different class depending on what JAR I have moved or replaced, but always a verify error.
I have also gone through the official Apache documentation, various IBM guides, countless blog and forum posts to no avail. I am at my wit's end with this
The same WAR runs perfectly on Tomcat, JBoss and WebLogic.
This is a complete list of all thirdparty JAR files:
activation-1.1.jar
antisamy-1.4.3.jar
aopalliance-1.0.jar
asm-3.3.1.jar
batik-css-1.7.jar
batik-ext-1.7.jar
batik-util-1.7.jar
bcprov-jdk15-1.46.jar
bsh-core-2.0b4.jar
commons-beanutils-1.7.0.jar
commons-beanutils-core-1.7.0.jar
commons-codec-1.3.jar
commons-collections-3.2.jar
commons-configuration-1.5.jar
commons-dbutils-1.6.jar
commons-digester-1.8.jar
commons-fileupload-1.3.1.jar
commons-httpclient-3.1.jar
commons-io-2.4.jar
commons-jexl-2.1.1.jar
commons-lang-2.4.jar
commons-logging-1.1.1.jar
cxf-api-2.7.11.jar
cxf-rt-bindings-soap-2.7.11.jar
cxf-rt-bindings-xml-2.7.11.jar
cxf-rt-core-2.7.11.jar
cxf-rt-databinding-jaxb-2.7.11.jar
cxf-rt-frontend-jaxws-2.7.11.jar
cxf-rt-frontend-simple-2.7.11.jar
cxf-rt-transports-http-2.7.11.jar
cxf-rt-ws-addr-2.7.11.jar
cxf-rt-ws-policy-2.7.11.jar
dom4j-1.6.1.jar
esapi-2.0.1.jar
FastInfoset-1.0.2.jar
geronimo-javamail_1.4_spec-1.7.1.ja
hamcrest-all-1.3.jar
hsqldb-1.8.0.10.jar
httpclient-4.3.6.jar
httpcore-4.3.3.jar
jaxen-1.1-beta-8.jar
jaxrpc-api-1.1.jar
jaxrpc-impl-1.1.3_01.jar
jaxrpc-spi-1.1.3_01.jar
joda-time-2.2.jar
js-1.7R2.jar
log4j-1.2.16.jar
logback-classic-0.9.21.jar
logback-core-0.9.21.jar
mail-1.4.7.jar
mailapi-1.4.3.jar
nekohtml-1.9.12.jar
not-yet-commons-ssl-0.3.9.jar
opensaml-2.6.1.jar
openws-1.5.1.jar
quartz-1.8.6.jar
saaj-api-1.3.5.jar
saaj-impl-1.3.jar
serializer-2.7.1.jar
slf4j-api-1.6.0.jar
slf4j-log4j12-1.6.0.jar
spring-aop-3.2.6.RELEASE.jar
spring-beans-3.2.6.RELEASE.jar
spring-context-3.2.6.RELEASE.jar
spring-core-3.2.6.RELEASE.jar
spring-expression-3.2.6.RELEASE.jar
spring-web-3.2.6.RELEASE.jar
stax2-api-3.1.4.jar
velocity-1.7.jar
vuelinkcore-20.2.3.jar
vueservlet-20.2.3.jar
woodstox-core-asl-4.2.1.jar
wsdl4j-1.6.3.jar
xml-apis-ext-1.3.04.jar
xml-resolver-1.2.jar
xmlsec-1.5.6.jar
xmltooling-1.4.1.jar
xom-1.1.jar
Does anyone know how to get Apache CXF 2.7.11 on WebSphere 8.5 to be able to act as a server and as a client?
We had the same problem using Was 8.5 (jdk 1.7_64), CXF, JAXB & xmlbeans:
JAXB is the default xml/java binding used by CXF. Was 8.5 uses endorsed JAXB api definition version 2.2.2 (in <WebSphere-dir>\AppServer\endorsed_apis\jaxb-api.jar) and standard implementation (in JRE rt.jar).
Xmlbeans 2.4.x holds inside org.w3c.* classes already present in Was (<WebSphere-dir>\AppServer\java_1.7_64\jre\lib\xml.jar).
In the end we solved so:
first following the instructions here:
http://www.ibm.com/developerworks/websphere/library/techarticles/1001_thaker/1001_thaker.html
then deleting from our deploy the following jar:
activation-*,
stax-api-* (but not stax2-api!),
jaxb-api-*,
jaxb-impl-*,
xercesImpl-*,
xml-apis-*
last deleting all org.w3c classes inside xmlbeas-2.x.jar
This is a complete list of all thirdparty JAR files we are successfully using:
cxf-*-2.7.11.jar
dom4j-1.6.1.jar
ehcache-2.8.2.jar
ehcache-core-2.5.1.jar
jettison-1.1.jar
neethi-3.0.3.jar
ognl-3.0.6.jar
opensaml-2.6.1.jar
openws-1.5.1.jar
spring-*-3.2.13.RELEASE.jar
stax2-api-3.1.1.jar
woodstox-core-asl-4.2.1.jar
wsdl4j-1.6.3.jar
wss4j-1.6.10.jar
xml-resolver-1.2.jar
xmlbeans-2.3.0-now3c.jar
xmlpull-1.1.3.1.jar
xmlschema-core-2.1.0.jar
xmlsec-1.5.4.jar
xmltooling-1.4.1.jar
xpp3_min-1.1.4c.jar
xstream-1.4.7.jar
We hope this is helpful.
PARENT_LAST:
Maybe you have a third party library in your deployment with the javax.activation.DataHandler class. Try to remove the activation-1.1.jar from your deployment.
This post can be usefull for you: LinkageError whilst trying to invoke CXF/SOAP webservice

Resources