spring #Transactional JDBC Template MySQL DB not rolling back - spring-aop

I am trying to implement the Spring #transactional with jdbc templates from the service layer calling
2 insert methods in a DAOImpl and using simplejdbctemplate to the insert and i see in the logs that spring creates a new transaction on my service method and my first insert suceeds and the second insert fails and even though it says it is rolling back on the same connection the first insert never get rolled back from my Mysql DB.(i am using innodb engine).
Here is my service method.
#Service
#TransactionConfiguration(transactionManager="txManager")
public class NewWizardService{
ApplicationContext ctx = new ClassPathXmlApplicationContext("dataSourcesConfig.xml");
UserDAO userDAO = (UserDAO)ctx.getBean("userDAO");
#Transactional(rollbackFor=Throwable.class, readOnly=false)
public void createNewFirm()
{
userDAO.insert1();
userDAO.insert2();
}
}
Here is my datasource and transaction manager spring configuration.
<bean id="txManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource"/>
</bean>
<bean id="dbcpDataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName"><value>${jdbc.driverClassName}</value></property>
<property name="url"><value>${jdbc.url}</value></property>
<property name="username"><value>${jdbc.username}</value></property>
<property name="password"><value>${jdbc.password}</value></property>
</bean>
<tx:annotation-driven transaction-manager="txManager" proxy-target-class="true"/>
Here is my log trace.
2012-03-28 16:56:31,460 DEBUG [org.springframework.jdbc.datasource.DataSourceTransactionManager] - Creating new transaction with name [com.CAIS.wizardService.NewWizardServiceImpl.createNewFirm]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT; '',-java.lang.RuntimeException
2012-03-28 16:56:31,654 DEBUG [org.springframework.jdbc.datasource.DataSourceTransactionManager] - Acquired Connection [org.apache.commons.dbcp.PoolableConnection#7d627b8b] for JDBC transaction
2012-03-28 16:56:31,660 DEBUG [org.springframework.jdbc.datasource.DataSourceTransactionManager] - Switching JDBC Connection [org.apache.commons.dbcp.PoolableConnection#7d627b8b] to manual commit
2012-03-28 16:56:31,663 DEBUG [org.springframework.jdbc.core.JdbcTemplate] - Executing prepared SQL update
2012-03-28 16:56:31,663 DEBUG [org.springframework.jdbc.core.JdbcTemplate] - Executing prepared SQL statement [insert into client (fullName, createDate, createUser) values (?, ?, ?)]
2012-03-28 16:56:31,663 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] - Fetching JDBC Connection from DataSource
2012-03-28 16:56:31,664 DEBUG [org.springframework.jdbc.datasource.DriverManagerDataSource] - Creating new JDBC DriverManager Connection to [jdbc:mysql://localhost:3306/cais]
2012-03-28 16:56:31,816 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] - Registering transaction synchronization for JDBC Connection
2012-03-28 16:56:31,833 DEBUG [org.springframework.jdbc.core.JdbcTemplate] - SQL update affected 1 rows
2012-03-28 16:56:31,840 DEBUG [org.springframework.jdbc.core.JdbcTemplate] - SQLWarning ignored: SQL state '01000', error code '1265', message [Data truncated for column 'createDate' at row 1]
2012-03-28 16:56:31,842 DEBUG [org.springframework.jdbc.core.JdbcTemplate] - Executing prepared SQL update
2012-03-28 16:56:31,842 DEBUG [org.springframework.jdbc.core.JdbcTemplate] - Executing prepared SQL statement [insert into client (fullName, createDate, createUser) values (?, ?, ?)]
2012-03-28 16:56:31,918 DEBUG [org.springframework.beans.factory.support.DefaultListableBeanFactory] - Finished creating instance of bean 'Sybase'
2012-03-28 16:56:31,918 INFO [org.springframework.jdbc.support.SQLErrorCodesFactory] - SQLErrorCodes loaded: [DB2, Derby, H2, HSQL, Informix, MS-SQL, MySQL, Oracle, PostgreSQL, Sybase]
2012-03-28 16:56:31,918 DEBUG [org.springframework.jdbc.support.SQLErrorCodesFactory] - Looking up default SQLErrorCodes for DataSource [org.springframework.jdbc.datasource.DriverManagerDataSource#6cf84b0a]
2012-03-28 16:56:31,920 DEBUG [org.springframework.jdbc.support.SQLErrorCodesFactory] - Database product name cached for DataSource [org.springframework.jdbc.datasource.DriverManagerDataSource#6cf84b0a]: name is 'MySQL'
2012-03-28 16:56:31,920 DEBUG [org.springframework.jdbc.support.SQLErrorCodesFactory] - SQL error codes for 'MySQL' found
2012-03-28 16:56:31,920 DEBUG [org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator] - Unable to translate SQLException with Error code '1406', will now try the fallback translator
2012-03-28 16:56:31,920 DEBUG [org.springframework.jdbc.support.SQLStateSQLExceptionTranslator] - Extracted SQL state class '22' from value '22001'
2012-03-28 16:56:31,921 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] - Returning JDBC Connection to DataSource
2012-03-28 16:56:31,923 DEBUG [org.springframework.jdbc.datasource.DataSourceTransactionManager] - Initiating transaction rollback
2012-03-28 16:56:31,923 DEBUG [org.springframework.jdbc.datasource.DataSourceTransactionManager] - Rolling back JDBC transaction on Connection [org.apache.commons.dbcp.PoolableConnection#7d627b8b]
2012-03-28 16:56:31,934 DEBUG [org.springframework.jdbc.datasource.DataSourceTransactionManager] - Releasing JDBC Connection [org.apache.commons.dbcp.PoolableConnection#7d627b8b] after transaction
2012-03-28 16:56:31,934 DEBUG [org.springframework.jdbc.datasource.DataSourceUtils] - Returning JDBC Connection to DataSource
Thanks in advance.

use #Transactional instead of #TransactionConfiguration because it is for transactional tests.
#Transactional is fine for a single txManager, but if you have more than one, you can change your tx anno. like #Transactional("productTxManager") or #Transactional("orderTxManager").

Your <tx:annotation-driven transaction-manager="txManager" proxy-target-class="true"/> needs to be defined in your webapp-serlet.xml.
You will find an explaination here : Spring #Transactional annotations ignored

Related

Kafka JDBC sink connector - USE statement is not supported to switch between databases

I am using Kafka JDBC sink connector to sink data to Azure SQL server. I have tested the connector with one database and it worked fine but when I added more databases, I started seeing the following error:
USE statement is not supported to switch between databases. Use a new connection to connect to a different database.
Config:
tasks.max: 1
topics: topic_name
connection.url: jdbc:sqlserver://server:port;database=dbname;user=dbuser
connection.user: dbuser
connection.password: dbpass
transforms: unwrap
transforms.unwrap.type: io.debezium.transforms.ExtractNewRecordState
transforms.unwrap.drop.tombstones: false
auto.create: true
value.converter: org.apache.kafka.connect.json.JsonConverter
value.converter.schemas.enable: true
insert.mode: upsert
delete.enabled: true
pk.mode: record_key
Stack:
2020-12-10 11:56:36,990 ERROR WorkerSinkTask{id=NAME-sqlserver-jdbc-sink-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted. Error: java.sql.SQLException: com.microsoft.sqlserver.jdbc.SQLServerException: USE statement is not supported to switch between databases. Use a new connection to connect to a different database.
(org.apache.kafka.connect.runtime.WorkerSinkTask) [task-thread-NAME-sqlserver-jdbc-sink-0]
org.apache.kafka.connect.errors.ConnectException: java.sql.SQLException: com.microsoft.sqlserver.jdbc.SQLServerException: USE statement is not supported to switch between databases. Use a new connection to connect to a different database.
at io.confluent.connect.jdbc.sink.JdbcSinkTask.put(JdbcSinkTask.java:87)
at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:560)
at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:323)
at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:226)
at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:198)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:185)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:235)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.sql.SQLException: com.microsoft.sqlserver.jdbc.SQLServerException: USE statement is not supported to switch between databases. Use a new connection to connect to a different database.
I have identified the issue, I thought in the beginning that the issue is due to having multiple databases inside the db server but turned out that the topic name has prefix.dbo.table_name in it. instead of just table_name.
Hence, the connector is detecting prefix.dbo as another database.
The solution is to use transform dropPrefix.
Example, to save data from topic hello.dbo.table1,hello.dbo.table2 to table1 and table2 in the database, use the following config:
tasks.max: 1
topics: hello.dbo.table1, hello.dbo.table2
connection.url: jdbc:sqlserver://server:port;database=dbname;user=dbuser
connection.user: dbuser
connection.password: dbpass
transforms: dropPrefix,unwrap
transforms.dropPrefix.type: org.apache.kafka.connect.transforms.RegexRouter
transforms.dropPrefix.regex: hello\.dbo\.(.*)
transforms.dropPrefix.replacement: $1
transforms.unwrap.type: io.debezium.transforms.ExtractNewRecordState
transforms.unwrap.drop.tombstones: false
auto.create: true
value.converter: org.apache.kafka.connect.json.JsonConverter
value.converter.schemas.enable: true
insert.mode: upsert
delete.enabled: true
pk.mode: record_key
If it doesn't work as a single connector then you'll need to create one connector per database.

Destroying connection that could not be successfully matched : Error in jboss while connecting with Database

Modified: I am using Jboss 7 server and oracle 10g for my web Application but when I starting the server application working properly but after 2 or 3 hours it is becoming slow.
I guessed like.
1. connection is not happening between database and jboss server properly so that it is not fetching data, so cause of this it becoming slow.
Datasource information in my standalone-full.xml file in jboss server is below:
<datasources>
<datasource jndi-name="java:/TTKConnectionDataSource" pool-name="TTKConnectionDataSourcePool" enabled="true" use-java-context="true" use-ccm="false">
<connection-url>jdbc:oracle:thin:#10.1.0.112:1521:vidaltest</connection-url>
<driver-class>oracle.jdbc.OracleDriver</driver-class>
<driver>oracle</driver>
<transaction-isolation>TRANSACTION_READ_COMMITTED</transaction-isolation>
<pool>
<min-pool-size>5</min-pool-size>
<max-pool-size>150</max-pool-size>
<prefill>true</prefill>
<use-strict-min>true</use-strict-min>
<flush-strategy>FailingConnectionOnly</flush-strategy>
</pool>
<security>
<user-name>appln</user-name>
<password>appln</password>
</security>
<validation>
<valid-connection-checker class-name="org.jboss.jca.adapters.jdbc.extensions.oracle.OracleValidConnectionChecker"/>
<validate-on-match>true</validate-on-match>
<background-validation-millis>300000</background-validation-millis>
<stale-connection-checker class-name="org.jboss.jca.adapters.jdbc.extensions.oracle.OracleStaleConnectionChecker"/>
<exception-sorter class-name="org.jboss.jca.adapters.jdbc.extensions.oracle.OracleExceptionSorter"/>
</validation>
<timeout>
<idle-timeout-minutes>10</idle-timeout-minutes>
</timeout>
</datasource>
And error what I am getting during slow down of server :
11:31:04,689 WARN [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (http-web-10) IJ000612: Destroying connection that could not be successfully matched: org.jboss.jca.core.connectionmanager.listener.TxConnectionListener#323ff8fa[state=DESTROYED managed connection=org.jboss.jca.adapters.jdbc.local.LocalManagedConnection#62b2c9f connection handles=0 lastUse=1494391408674 trackByTx=false pool=org.jboss.jca.core.connectionmanager.pool.strategy.OnePool#547ac048 pool internal context=SemaphoreArrayListManagedConnectionPool#46e5e24c[pool=TTKConnectionDataSourcePool] xaResource=LocalXAResourceImpl#1e6c0ff1[connectionListener=323ff8fa connectionManager=488aa6d1 warned=false currentXid=null] txSync=null]
11:37:09,075 INFO [com.ttk.action.claims.TTKListener] (http-web-20) Session is created sessionCreated
11:37:09,078 ERROR [org.apache.struts.actions.DispatchAction] (http-web-20) Request[/LoginAction] does not contain handler parameter named 'mode'. This may be caused by whitespace in the label text.
11:46:35,964 WARN [org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory] (http-web-10) Destroying connection that is not valid, due to the following exception: oracle.jdbc.driver.T4CConnection#5b8b1ccb: java.sql.SQLException: pingDatabase failed status=-1
at org.jboss.jca.adapters.jdbc.extensions.oracle.OracleValidConnectionChecker.isValidConnection(OracleValidConnectionChecker.java:74) [ironjacamar-jdbc-1.0.12.Final.jar:1.0.12.Final]
05: Connection error occured: org.jboss.jca.core.connectionmanager.listener.TxConnectionListener#38b370[state=NORMAL managed connection=org.jboss.jca.adapters.jdbc.local.LocalManagedConnection#a8c7e2d connection handles=0 lastUse=1494390662156 trackByTx=false pool=org.jboss.jca.core.connectionmanager.pool.strategy.OnePool#547ac048 pool internal context=SemaphoreArrayListManagedConnectionPool#46e5e24c[pool=TTKConnectionDataSourcePool] xaResource=LocalXAResourceImpl#dafc1c4[connectionListener=38b370 connectionManager=488aa6d1 warned=false currentXid=null] txSync=null]: java.sql.SQLException: pingDatabase failed status=-1
when I am restarting server than connection is fine up to 2 or 3 hours maximum after that server again becoming slowdown ,Please suggest me what are the possibilities to overcome on this issue.
Thanks in advance.
sorry for disturbance, but again I am facing this problem actually what is happening ? when my server become slowdown then it's not allowing me to login its keep on buffering because its not connecting with my database as i think or may be its not getting connection object. and after some time of buffering its giving warning message what i mentioned above "Destroying connection that could not be successfully matched" . than after that if i will try to login the application than its taking time to get login or last option i need to restart my server that is not that much preferable ever.
The WARN messages are not unusual when something outside JBoss closes a connection.
That warning indicates that JBoss got the Oracle JDBC driver to ping the database to ensure that the connection still worked, and it reported that it didn't, so JBoss destroyed the connection. JBoss would then create a new one and give that to the application, so it should in theory not cause any actual problems.
Method "org.jboss.jca.adapters.jdbc.extensions.oracle.OracleValidConnectionChecker.isValidConnection()" throws this exception. Method isValidConnection() internally calls OracleConnection's pingDatabase() [1] method. It the database is closed it returns -1, refer [2]. This issue is a known issue with Oracle driver which is seen when their are issues with network or database and is not related to JBoss
This is a known type of error indicating some problem with your Oracle database or occasionally with the network.
I would ask you to check the network connectivity is stable between the JBoss node and Database.
[1] https://docs.oracle.com/cd/E18283_01/appdev.112/e13995/oracle/jdbc/OracleConnection.html#pingDatabase__
[2] https://docs.oracle.com/cd/E18283_01/appdev.112/e13995/oracle/jdbc/OracleConnection.html#DATABASE_CLOSED

Apache Camel SQL component is not closing resultset?

I am using Camel sql component to run select queries against Oracle DB. The war file is deployed on Jboss EAP 6.1. In the config file under configuration I have track-statements set to true (i.e. true). So Jboss checks if the result sets are closed when a connection is returned back to the pool.
I am configuring sql component as follows:
//datasource is injected here
#Resource(mappedName = "java:/jdbc/OracleDS")
private DataSource dataSource;
...
...
//SQL component
SqlComponent sqlComponent = new SqlComponent();
sqlComponent.setDataSource(dataSource);
camelCtx.addComponent("sql", sqlComponent);
The sql component is used in a recipientList as follows:
from("activemq:"+queue)
...
.recipientList(simple(getFromConfig(sqlStmtName)),"false")
..
The sql statement is nothing special. Just a regular select statement from a single table.
In Jboss logs I see this warning.
2015-04-16 16:23:07,169 WARN [org.jboss.jca.adapters.jdbc.WrappedConnection] (ajp-8009-2|R:r2K4crKLnPRT-0br|ctvc|1.0) Closing a result set you left open! Please close it yourself.: java.lang.Throwable: STACKTRACE
at org.jboss.jca.adapters.jdbc.WrappedStatement.registerResultSet(WrappedStatement.java:1357)
at org.jboss.jca.adapters.jdbc.WrappedStatement.getResultSet(WrappedStatement.java:740)
at org.apache.camel.component.sql.SqlProducer$2.doInPreparedStatement(SqlProducer.java:127)
at org.apache.camel.component.sql.SqlProducer$2.doInPreparedStatement(SqlProducer.java:90)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:589)
at org.apache.camel.component.sql.SqlProducer.process(SqlProducer.java:90)
at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:573)
at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:506)
at org.apache.camel.processor.MulticastProcessor.process(MulticastProcessor.java:215)
at org.apache.camel.processor.RecipientList.sendToRecipientList(RecipientList.java:167)
at org.apache.camel.processor.RecipientList.process(RecipientList.java:120)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:72)
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:398)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
...
...
I checked SqlProducer.java (line 127). Since sql component uses Spring JDBCtemplate I would assume that resultset would be closed by the template.
Is there anything I need to do to close the resultset?
Yes that is a bug in Apache Camel. I have logged a ticket to get this fixed in future releases
https://issues.apache.org/jira/browse/CAMEL-8715

wildfly-8.0.0.Final: XA END / un-enlist for database connection called to early

While trying to port an EE application from JBoss5 to WF8 the following problem occurred:
EJBs (#Required) using JPA to do some data changes.
Some changes get already written to the database, others reside in the session cache.
After the top EJB call returns, a Hibernate Session flush is triggered in beforeCompletion.
Then more changes are flushed to the database, but I run in a reproduceable database locking problem.
After some time an update of a row fails with lock wait timeout.
This row has been inserted prior during the EJB call.
There should be exactly one xa transaction active processing all data changes.
But the database shows two active session, one is an xa transaction with sessionId -2 (orphaned), the other session is a local transaction.
After analyzing all database communication with the help of the JDBC driver logging I found the following behaviour:
a connection is enlisted and xa start called
the same connection is used for several select / insert / update statements
after return of the top EJB call on the same connection xa end and connection un-enlist is called
the same connection is used for session flush (beforeCompletion) but with local transaction because the connection is no longer associated with the xa transaction, so locks can be expected.
Shouldn't xa end be called AFTER beforeCompletion / session flush!?!
Where is the bug? In wildfly? In arjuna?
Environment:
wildfly 8.0.0.Final
Windows 7 64-bit
MS SQL Server 2012 database
Latest MS JDBC driver
XA datasource

Querydsl generates invalid SQL on basic aggregate query

I am using QueryDSL and Hibernate to query a SqlServer2008 database.
I have a Sessions table that contains a sessionId column. I want to obtain the max sessionId.
I run the following query:
QSessions session = QSessions.sessions;
HibernateSQLQuery query = new HibernateSQLQuery(sessionFactory.openSession(), new SQLServer2005Templates());
query.from(session).list(session.sessionId.max());
My hibernate.cfg.xml:
<hibernate-configuration>
<session-factory>
<property name="hibernate.connection.url">jdbc:sqlserver://SVR;databaseName=DB</property>
<property name="hibernate.connection.username">X</property>
<property name="hibernate.connection.password">X</property>
<property name="hibernate.connection.driver_class">com.microsoft.sqlserver.jdbc.SQLServerDriver</property>
<property name="hibernate.dialect">org.hibernate.dialect.SQLServer2008Dialect</property>
<mapping class="com.coveo.data.Sessions" />
</session-factory>
</hibernate-configuration>
When I run the query, I can see that Hibernate attempts to run the following SQL query:
17:36:10.011 [main] DEBUG org.hibernate.SQL - select max(sessions.*.sessionId) as col__1 from sessions
This is not valid SQL. I would have expected
select max(sessions.sessionId) as col__1 from sessions
When Hibernate attempts to execute the query, I get the following error:
com.microsoft.sqlserver.jdbc.SQLServerException: Incorrect syntax near '*'.
I get the same syntax error when attempting to run the invalid SQL query using SQL Management Studio.
Why does Querydsl require a dialect? Isn't Hibernate in charge of generating the SQL?
Why is the generated SQL invalid?
EDIT: I get the same behavior when using .avg(), .min() or another aggregate function.
I ended up switching to jOOQ.
So far, the experience has been very positive. The API is somewhat similar, the documentation is better and it doesn't generate invalid SQL.
There are a few issues here.
You use HibernateSQLQuery which is meant to execute SQL queries through the Hibernate API, please use HibernateQuery for Hibernate ORM queries
You use the Q-type generated for ORM queries in SQL queries. Querydsl should handle this usage better and I created a ticket for it https://github.com/mysema/querydsl/issues/463
Querydsl supports both ORM and SQL based queries for Hibernate and you accidently mixed them.
For normal Hibernate queries your example would be
QSessions session = QSessions.sessions;
HibernateQuery query = new HibernateQuery(sessionFactory.openSession());
query.from(session).list(session.sessionId.max());
I assume that QSessions is based on the Sessions entity.
jOOQ is a very good alternative for the SQL level, but if you need to access also JPA and NoSQL based storages, Querydsl offers more.

Resources