We have an application using a Weblogic pooled non-XA SQL Server JDBC driver. Here is the flow of what is causing the issue.
Get connection from datasource.
Set auto commit to false.
Create CallableStatement.
Set parameters.
Call executeUpdate.
Inside stored proc call, the database raises an error with SET XACT_ABORT as ON
Application sees error code in out param of CallableStatement.
Closes CallableStatement
Calls rollback on the connection.
Exception is thrown:
2013-Oct-17 16:45:20.500
EDT||XXXXXXX|asyncDelivery9|XXX|XXX|XXXX-a3436e48-1672-4a26-88c0-50b9954f6a42|ERROR|XXXXX|rollBack|The
server failed to resume the transaction. Desc:4200000012.
com.microsoft.sqlserver.jdbc.SQLServerException: The server failed to
resume the transaction. Desc:4200000012.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:216)
~[sqljdbc4.jar:na]
at com.microsoft.sqlserver.jdbc.TDSTokenHandler.onEOF(tdsparser.java:254)
~[sqljdbc4.jar:na]
at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:84)
~[sqljdbc4.jar:na]
at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:39)
~[sqljdbc4.jar:na]
at com.microsoft.sqlserver.jdbc.SQLServerConnection$1ConnectionCommand.doExecute(SQLServerConnection.java:1756)
~[sqljdbc4.jar:na]
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:5696)
~[sqljdbc4.jar:na]
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1715)
~[sqljdbc4.jar:na]
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectionCommand(SQLServerConnection.java:1761)
~[sqljdbc4.jar:na]
at com.microsoft.sqlserver.jdbc.SQLServerConnection.rollback(SQLServerConnection.java:1964)
~[sqljdbc4.jar:na]
at weblogic.jdbc.wrapper.PoolConnection_com_microsoft_sqlserver_jdbc_SQLServerConnection.rollback(Unknown
Source) ~[wlfullclient.jar:12.1.1.0]
Connection is closed.
Then the next time the application attempts to get a connection from the data source, the following error is thrown:
java.sql.SQLException: Failed to setAutoCommit to true for pool connection: First we got The server failed to resume the transaction. Desc:4200000012., then we got The server failed to resume the transaction. Desc:4200000012.
at weblogic.jdbc.wrapper.PoolConnection.init(PoolConnection.java:70) ~[wlfullclient.jar:12.1.1.0]
at weblogic.jdbc.common.internal.RmiDataSource.getPoolConnectionObj(RmiDataSource.java:639) ~[wlfullclient.jar:12.1.1.0]
at weblogic.jdbc.common.internal.RmiDataSource.getPoolConnection(RmiDataSource.java:478) ~[wlfullclient.jar:12.1.1.0]
at weblogic.jdbc.common.internal.RmiDataSource.getConnectionInternal(RmiDataSource.java:558) ~[wlfullclient.jar:12.1.1.0]
at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:518) ~[wlfullclient.jar:12.1.1.0]
at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:511) ~[wlfullclient.jar:12.1.1.0]
In the Weblogic log, the following is logged:
<Oct 15, 2013 3:07:50 PM EDT> <Warning> <JDBC> <BEA-001153> <Forcibly releasing inactive/harvested connection "weblogic.jdbc.wrapper.PoolConnection_com_microsoft_sqlserver_jdbc_SQLServerConnection#238607" back into the data source connec
tion pool "MASTERDB", currently reserved by: reclaimed because of init failure before user access. Not because of inactivity..>
It seems to me that the transaction on the connection was never ended. But a rollback was called which I would have thought would end the transaction. And now the next time Weblogic attempts to return the connection, it errors when initializing it. Any ideas as to what the issue is? We are not explicitly setting auto commit back to true at the end, could that be the issue?
Found this suggestion for same similar problem here.
"Make sure you have the datasource/pool configured to test-on-reserve, and seconds-to-trust = 0."
This should solve the issue of eliminating those "bad" connections from pool before returning to application.
Related
We had the error that the Microsoft Distributed Transaction Coordinator (MS DTC) has cancelled a transaction of BizTalk which causes some stuck messages in the Receive Port, it has never happened before for years but it just occurs recently in 1 minute which causes few stuck messages. Is there anyway to make the Receive Location retries on error? What caused the DTC issue? I notice all the instances of stuck messages having in active running service instances, is it an issue? and how can I fix it?
Error from BizTalk:
There was a failure executing the receive pipeline:
"CargoWise.eHub.Gateway.Pipelines.Rcv_ResolveInboundGatewayMessage,
CargoWise.eHub.Gateway.Pipelines, Version=3.0.0.0, Culture=neutral,
PublicKeyToken=4f570df270576350" Source: "Unknown " Receive Port:
"Gateway_SelectInboxMessageByStatus_CACustoms" URI:
"mssql://ehubtransactions.db.wisegrid.net//eHubTransactions?InboundId=SelectInboxMessagesByStatus&Category=CACustoms"
Reason: 0x8004d00e
Error from SQL Server:
Microsoft Distributed Transaction Coordinator (MS DTC) has stopped this transaction
Error: 8522, Severity: 18, State: 1
It is a WCF-SQL receive port which pulls a message from database and its pipeline will dissemble and add properties to the message. The pipeline components don't access to database. However, there is no error in the query as it will update the db status to 1:
UPDATE top (1) Inbox WITH (ROWLOCK)
SET Status = 1 WHERE PK = #PK
AND Status = 0
SELECT PK,MessageTrackingID,Content
FROM InboxMessage WITH (ROWLOCK)
WHERE PK = #PK and InsertUTC = #InsertUTC
I'm trying to get my web application up and running. It uses Kentico. I connected to the admin page which is taking me through the initial database ans site setup. It ran through "Creating database objects". I can see the tables are now in the database. The "Next" button hasn't become active.
This error is in the Creating database objects log:
at CMS.DataEngine.SqlInstallationHelper.ImportDefaultData(IDataConnection connection, String dataFolder, LogMessage log, String defaultErrorMessage)
at CMS.DataProviderSQL.DataConnection.BulkInsert(DataTable sourceData, String targetTable, BulkInsertSettings insertSettings)
at CMS.DataEngine.AbstractDataConnection.HandleError(String queryText, Exception ex)
The statement has been terminated.
Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
Caused exception:
BulkInsert to table 'BadWords_Word'
Query:
[DataConnection.HandleError]:
ERROR: An error occurred:
data\badwords_word
objects/proc_personas_reevaluateallcontacts
...
objects/cms_resource
How can I get past this?
Increase the timeout for your connection string in your web.config file to something like 600 then try again.
When I am generating scripts for my database it fails. The database is of 60gb and it fails somewhere 10GB.
Taking backup is not an option as I have connected to remote and remote server does not have enough space.
Below is the error, The error is too long so i have just paste the first few lines(He has the same problem I guess):
Microsoft.SqlServer.Management.SqlScriptPublish.SqlScriptPublishException:
An error occurred while scripting the objects. --->
System.Data.SqlClient.SqlException: Timeout expired. The timeout
period elapsed prior to completion of the operation or the server is
not responding. ---> System.ComponentModel.Win32Exception: The wait
operation timed out --- End of inner exception stack trace --- at
System.Data.SqlClient.SqlInternalConnection.OnError(SqlException
exception, Boolean breakConnection, Action1 wrapCloseInAction)
As a solution to above link I tried Scripting only Data and then Only Schema which also didn't work in my case.
Getting below error from code deployed on WAS at:
org.springframework.messaging.MessageHandlingException:
error occurred in message handler [org.springframework.integration.aggregator.AggregatingMessageHandler#0];
nested exception is org.springframework.jdbc.CannotGetJdbcConnectionException:
Could not get JDBC Connection;
nested exception is com.ibm.websphere.ce.cm.ConnectionWaitTimeoutException:
Connection not available, Timed out waiting for 180000
Detailed Trace:
Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is com.ibm.websphere.ce.cm.ConnectionWaitTimeoutException: Connection not available, Timed out waiting for 180000
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:630)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:695)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:727)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:752)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:762)
at org.springframework.integration.jdbc.JdbcMessageStore.getMessageGroup(JdbcMessageStore.java:431)
at org.springframework.integration.aggregator.AbstractCorrelatingMessageHandler.handleMessageInternal(AbstractCorrelatingMessageHandler.java:388)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:78)
... 157 more
Caused by: com.ibm.websphere.ce.cm.ConnectionWaitTimeoutException: Connection not available, Timed out waiting for 180000
at com.ibm.ws.rsadapter.AdapterUtil.toSQLException(AdapterUtil.java:1684)
at com.ibm.ws.rsadapter.jdbc.WSJdbcDataSource.getConnection(WSJdbcDataSource.java:686)
at com.ibm.ws.rsadapter.jdbc.WSJdbcDataSource.getConnection(WSJdbcDataSource.java:636)
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111)
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77)
... 165 more
Caused by: com.ibm.websphere.ce.j2c.ConnectionWaitTimeoutException: Connection not available, Timed out waiting for 180000
at com.ibm.ejs.j2c.FreePool.createOrWaitForConnection(FreePool.java:1729)
at com.ibm.ejs.j2c.PoolManager.reserve(PoolManager.java:3329)
at com.ibm.ejs.j2c.PoolManager.reserve(PoolManager.java:2610)
at com.ibm.ejs.j2c.ConnectionManager.allocateMCWrapper(ConnectionManager.java:1500)
at com.ibm.ejs.j2c.ConnectionManager.allocateConnection(ConnectionManager.java:1012)
at com.ibm.ws.rsadapter.jdbc.WSJdbcDataSource.getConnection(WSJdbcDataSource.java:669)
... 168 more
Looking to your logs with the com.ibm.ejs.j2c.PoolManager, I'd recommend you to go to WAS support. Looks like your the Connection Pool is very small for your use-case. Especially having your clue:
It happens when i post too many requests on server.
I'm trying now to startup a liferay 6.1.1 ga2 server with sqlserver database.
I created user and database with windows commande line "sqlcmd -S localhost" as the following:
1> create database liferayportal;
2> create login lrUser with password='abc123';
3> create user lrUser for login lrUser;
4> exec sp_addrolemember 'liferayportal' , 'lrUser';
the portal-ext.properties:
jdbc.default.driverClassName=net.sourceforge.jtds.jdbc.Driver
jdbc.default.url=jdbc:jtds:sqlserver://localhost/liferayportal
jdbc.default.username=lrUser
jdbc.default.password=abc123
And this is a summary of the stacktrace
09:28:36,657 WARN [com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1][BasicResourcePool:1841] com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask#6245a4 -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (3). Last acquisition attempt exception:
java.sql.SQLException: Network error IOException: Connection refused: connect
at net.sourceforge.jtds.jdbc.ConnectionJDBC2.<init>(ConnectionJDBC2.java:410)
.
.
.
09:28:36,702 WARN [com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0][BasicResourcePool:1841] com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask#c8aeb3 -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (3). Last acquisition attempt exception:
java.sql.SQLException: Network error IOException: Connection refused: connect
at net.sourceforge.jtds.jdbc.ConnectionJDBC2.<init>(ConnectionJDBC2.java:410)
.
.09:28:36,717 WARN [com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#7][BasicResourcePool:1841] com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask#fab5b1 -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (3). Last acquisition attempt exception:
java.sql.SQLException: Network error IOException: Connection refused: connect
at net.sourceforge.jtds.jdbc.ConnectionJDBC2.<init>(ConnectionJDBC2.java:410)
.
.
.
09:28:36,743 WARN [com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#4][BasicResourcePool:1841] com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask#12524b0 -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition attempts (3). Last acquisition attempt exception:
java.sql.SQLException: Network error IOException: Connection refused: connect
at net.sourceforge.jtds.jdbc.ConnectionJDBC2.<init>(ConnectionJDBC2.java:410)
.
.
.
I would check whether the server has the port related to MSSQL open for remote access (normally the port is 1433) if not, you should set MSSQL to work on that port, and allow remote access and verify that your firewall allows the access to that port.