db.DbTransaction getConnection: null
java.sql.SQLException: Transient error, please try again.
at
com.google.appengine.api.rdbms.RdbmsApiProxyClient$ApiProxyBlockingInterface.makeSyncCall(RdbmsApiProxyClient.java:108)
at com.google.appengine.api.rdbms.RdbmsApiProxyClient$ApiProxyBlockingInterface.openConnection(RdbmsApiProxyClient.java:71)
at com.google.cloud.sql.jdbc.internal.SqlProtoClient.openConnection(SqlProtoClient.java:58)
at com.google.cloud.sql.jdbc.Driver.connect(Driver.java:66)
at com.google.cloud.sql.jdbc.Driver.connect(Driver.java:26)
at java.sql.DriverManager.getConnection(DriverManager.java:620)
at java.sql.DriverManager.getConnection(DriverManager.java:222)
at db.DbTransaction.getConnection(DbTransaction.java:44)
When i restarted the instance of Google Cloud SQL, it is resolved. What the reason behind it and how can i solve it. I have used servlet based connection pooling in this application. Whether the cause of exception have an relation to connection pooling or have any issue related to An App Engine instance cannot have more than 30 concurrent connections to Google Cloud SQL, so a leak will eventually cause new connections to fail (https://developers.google.com/appengine/docs/java/cloud-sql/).
Related
We have a website running on a server. We have a "production" instance and a "staging" instance each having its own database. The MSSQL Server is running locally on the same server.
Today, suddenly the "production" website went down. Looking at the logs, the following exception showed up:
System.Data.Entity.Core.EntityException: The underlying provider failed on Open. ---> System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
At the same time, the "staging" website was working just normally.
While trying to figure out what was happening, I tried all sorts of things like re-creating both the app pool and the IIS app. I also hooked up the "production" IIS app to the same app pool of the "staging" app, still the same issue. Restarted the server too of course.
Also, I ran the executable of the "production" website directly (as a console app) and it worked normally. So it's a problem that happens only when running under IIS.
One last thing I tried, is that I reconfigured the "staging" website to use the "production" database, and to my utter shock it worked normally. Because I thought the problem was the "production" database itself.
I just have no idea whatsoever about what's going on here. Any help is very much appreciated.
If all the connections in the connection pool are used, it is almost certainly because your application is opening database connections and failing to close them.
Since you are using Entity Framework, it's probably because your application is failing to dispose of the DbContext object.
It's nothing to do with the production database as such; probably the increased activity on your production site vs your staging site is making the application bug manifest itself more quickly.
We are experiencing what I believe to be an issue with connection pooling when using Amazon RDS database instances to back our web application running on ECS.
Using https://github.com/npgsql/Npgsql.EntityFrameworkCore.PostgreSQL ("Npgsql.EntityFrameworkCore.PostgreSQL": "1.1.0").
The error specifically occurs when we roll-back the RDS instance to a previous point in time. The connection string remains unchanged, but I suspect the Npgsql driver believes it still has active connections open, even though they have been terminated server side (by the rollback).
Restarting our application obviously fixes the problem as it creates new connections after launch, but is an undesirable solution.
Is this a bug within Npgsql? Should it be timing out on a connection that has been forcibly closed? Maybe there is something we can put in the connection string to force it to retry? (Although I'd like to avoid turning off connection pooling completely if possible.)
I'm using jmeter to stress test a GAE web service which uses CloudSQL and I'm getting intermittent communications link failure exceptions.
I've tried using direct connections and a connection pool, and I see exceptions in either scenario. The exceptions increase as the number of requests per second increase.
Note that we are using the highest tier of cloud sql, D32 and the tests are well under the max 3200 connections.
Here's a stack trace for reference:
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.GeneratedConstructorAccessor48.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:33)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1117)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:350)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2413)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2450)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2235)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:818)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46)
at sun.reflect.GeneratedConstructorAccessor46.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:33)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:404)
at com.mysql.jdbc.GoogleNonRegisteringDriver$JdbcWrapper.getInstance(GoogleNonRegisteringDriver.java:276)
at com.mysql.jdbc.GoogleNonRegisteringDriver.connect(GoogleNonRegisteringDriver.java:246)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:215)
Update: I changed the connection pool settings to maxActive = 5 and maxIdle = 5 and the intermittent communications link exceptions went away. Note that I've tried commons dbcp and tomcat dbcp. I'm now seeing the following exceptions in the logs:
Caused by: java.sql.SQLException: java.lang.SecurityException: Unable to access gatherPerformanceMetrics
Caused by: java.sql.SQLException: java.lang.SecurityException: Unable to access includeThreadDumpInDeadlockExceptions
Caused by: java.sql.SQLException: java.lang.SecurityException: Unable to access nullNamePatternMatchesAll
From https://cloud.google.com/appengine/docs/java/cloud-sql/#Java_Size_and_access_limits
"Each App Engine instance cannot have more than 12 concurrent connections to a Google Cloud SQL instance."
Can you tell more about the test set-up? How many requests is jmeter sending to appengine and how many connections does the app instance open for each of those requests?
To everyone who are looking for why you might be getting "com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure" on a connection.
Make sure your IP is allowed if you are calling from a test server!
I was testing at a friends house, and this unhelpful error kept showing up.
I'm administering a web-based application that is set-up to pull in data from a variety of databases; SQL, Oracle, Mainframe, etc.
I was given credentials to access an Oracle DB, and am establishing a connection through the web-based app server via JDBC. The JDBC connection requires me to provide a Database URL and JDBC driver for the connection. I also built in a SQL statement to pull only the information I needed from the Oracle DB into my web-based app.
Things were running smoothly with this set-up, until just recently. I now receive the following error when trying to establish a connection from my web-based app to the Oracle DB:
Failure trying to get a pooled connection to
[jdbc:oracle:thin:#<SERVER NAME>:1521:cqdb]java.sql.SQLException: Protocol violation
The Oracle DBA I work with is not very helpful in resolving in helping me troubleshoot this issue. Without his help, I really don't even know where to start with troubleshooting.
Any suggestions on where to start? I can provide additional information if needed.
*Additional information. This is what is in my STDOUT file in relation to the error. I can keep digging as well:
07:31:08,565 WARN QuartzScheduler_Worker-5 WEB-APP.api.Aggregator:979 - Exception during aggregation. Reason: Failure trying to get a pooled connection to [jdbc:oracle:thin:#SERVER-NAME:1521:cqdb]java.sql.SQLException: Protocol violation
WEB-APP.tools.GeneralException: Failure trying to get a pooled connection to [jdbc:oracle:thin:#SERVER-NAME:1521:cqdb]java.sql.SQLException: Protocol violation
at WEB-APP.api.Aggregator.aggregateAccounts(Aggregator.java:1897)
at WEB-APP.api.Aggregator.execute(Aggregator.java:1222)
at WEB-APP.task.ResourceIdentityScan.execute(ResourceIdentityScan.java:76)
at WEB-APP.api.TaskManager.runSync(TaskManager.java:643)
at WEB-APP.scheduler.JobAdapter.execute(JobAdapter.java:116)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:529)
Caused by: WEB-APP.connector.ConnectorException: Failure trying to get a pooled connection to [jdbc:oracle:thin:#SERVER-NAME:1521:cqdb]java.sql.SQLException: Protocol violation
at WEB-APP.connector.JDBCConnector.getConnection(JDBCConnector.java:833)
at WEB-APP.connector.JDBCConnector.iterateObjects(JDBCConnector.java:649)
at WEB-APP.connector.JDBCConnector.iterateObjects(JDBCConnector.java:90)
at WEB-APP.connector.ConnectorProxy.iterateObjects(ConnectorProxy.java:109)
at WEB-APP.api.Aggregator.iterateObjects(Aggregator.java:2673)
at WEB-APP.api.Aggregator.aggregateAccounts(Aggregator.java:1818)
... 6 more
Caused by: WEB-APP.tools.GeneralException: Failure trying to get a pooled connection to [jdbc:oracle:thin:#SERVER-NAME:1521:cqdb]java.sql.SQLException: Protocol violation
at WEB-APP.tools.JdbcUtil.getPooledConnection(JdbcUtil.java:1178)
at WEB-APP.tools.JdbcUtil.getConnection(JdbcUtil.java:823)
at WEB-APP.connector.JDBCConnector.getConnection(JDBCConnector.java:830)
... 11 more
07:33:03,983 ERROR http-8080-2 WEB-APP.server.Authenticator:229 - WEB-APP.connector.AuthenticationFailedException: [LDAP: error code 49 - 80090308: LdapErr: DSID-0C090334, comment: AcceptSecurityContext error, data 52e, vece
**Additional Information:
Oracle JDBC Driver version 14, JRE ver 1.6.0_23-b05. I don't have the Oracle DB version. Awaiting response from our Oracle DBA.
***Additional Information:
This issue was resolved. Our Oracle DBA did something on his end to correct the connection issue. He hasn't explained what he did, yet. Thanks for your help. Sorry for not getting you all the info you needed up front.
I have an MVC3 application hosted by third party hosting provider. The site has been running well for the past 3 months without any problems. Today suddenly the Application started throwing following Exception as recorded in my logs part of which is shown below.
System.Data.ProviderIncompatibleException: The provider did not return
a ProviderManifestToken string. --->
System.Data.SqlClient.SqlException: Timeout expired. The timeout
period elapsed prior to completion of the operation or the server is
not responding.
The message is self explanatory and I first thought I should increase the connect timeout, but then the exception was still thrown suggesting the other part (Server Not Responding). I contacted my hosting provider and he said there was nothing wrong on his part. So I am stuck with a down website and don't know what to do.
Any ideas why the provider is throwing the exception listed above. Also, is it possible for me to remotely connect to the database on the hosting server with limited authority. Any tools for that ? I don't have an exposure in database subject, except for application programming.
This occurs due to the Timeout, the default timeout is 30 seconds, for time out there are 2 common reasons.
Long running tasks or uncommitted transactions. Refer to the Timeout expired to know about this.