I am developing a Servlet applicaiton. It obtains a database connection from the connection pool supported by the Tomcat container to query and update database data.
I run into a problem. The Servlet gets a database connection and then add a new table row or delete a table row. After that, it commits the change. Later, a connection is obtained to execute queries. I find that the data returned from the queries using the second connection do not reflect the change made with the first database connection.
Isn't it strange? The changes made with the first database connection have been committed successfully. Why the new rows inserted do not appear in the later query? Why the rows deleted still appear in the later query?
Does it relate to the setting of transaction level?
Can anyone help?
03-12: More Information (#1):
I use MySQL Community Server 5.6.
My servlet runs on Tomcat 7.0.41.0.
The Resource element in the conf/server.xml is as follows:
<Resource type="javax.sql.DataSource"
name="jdbc/storewscloud"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/myappdb"
maxActive="100"
minIdle="10"
maxWait="10000"
initialSize="10"
removeAbandonedTimeout="60"
removeAbandoned="true"
logAbandoned="true"
username="root"
password="xxxxxxxxxx"
/></li>
I do not use any cache explicitly.
Every time the servlet gets a database connection, it turns the auto-commit mode of the connection off.
When the servlet is invoked, a database connection is obtained. The servet uses it to update data in the database. After that, it commits the changes. Then, it uses Apache HttpClients to invoke the same servlet to do some other thing which also obtains a database connection and execute query. The later query returns 'old' data. If I refresh the web page, the latest data are shown. It looks like some party, mysql jdbc driver or connection object, cache the data somewhere. I have no clue.
03-12: More Information (#2):
I did an experiment getting a connection without using the connection pool. The result is correct. So, the problem is caused by the connection pool.
To make the query return right data using the 2nd connection from the pool, I need to not only commit the data changes using the 1st connection from the pool but also CLOSE the 1st connection.
It seems that the data changes made are not completely saved in the database even the commit() is called until the close() is called.
Why?
I found that there is a new version of C3P0 connection pool released recently. I gave it a try. It works! The problems I had do not occur. Therefore, I use it to replace the bundled connection pool of the Tomcat server. For those who encounter the same problem as I do, C3P0 maybe a solution for you too.
C3P0 Project URL
Related
I have a Symfony command line task that has a habit of dropping the mysql connection.
Its a data import task. Which fetches data from multiple connections. Its not one big query but a few smaller ones.
It seems to drop the connection the first time it is ran. About half way through the script. However the second time its ran (from the beginning) it always completes the task.
Its not timing out on the query as the error response I get is that the connection has been dropped and it runs ok on its own. So im thinking that its some kind of timeout issue that is avoided when its ran the second time due to query caching speeding up the script.
So my question is how do I refresh the database connection?
[Doctrine\DBAL\DBALException]
SQLSTATE[HY000]: General error: 2013 Lost connection to MySQL server during query
A different approach is to check if Doctrine is still connected to the MySQL server through the ping() method in the connection. If the connection is lost, close the active connection since it is not really closed yet and start a new one.
if(FALSE == $em->getConnection()->ping()){
$em->getConnection()->close();
$em->getConnection()->connect();
}
I guess you mean to connect to the database if the connection is lost for some reason. Given an EntityManager, you can do it the following way:
$success = $_em->getConnection()->connect();
With getConnection, you are retrieving the connection object doctrine uses (Doctrine\DBAL\Connection), which exposes the connect method.
You can call connect anytime, as it checks wether a connection is already established. If this is the case, it returns false.
There is also a isConnected method to check wether a connection is established. You could use that to see where exactly the connection is dropping to get a clearer picture of what is happening.
I have 3 servers set up for SQL mirroring and automatic failover using a witness server. This works as expected.
Now my application that connects to the database, seems to have a problem when a failover occurs - I need to manually intervene and change connection strings for it to connect again.
The best solution I've found so far involves using Failover Partner parameter of the connection string, however it's neither intuitive nor complete: Data Source="Mirror";Failover Partner="Principal" found here.
From the example in the blog above (scenario #3) when the first failover occurs, and principal (failover partner) is unavailable, data source is used instead (which is the new principal). If it fails again (and I only tried within a limited period), it then comes up with an error message. This happens because the connection string is cached, so until this is refreshed, it will keep coming out with an error (it seems connection string refreshes ~5 mins after it encounters an error). If after failover I swap data source and failover partner, I will have one more silent failover again.
Is there a way to achieve fully automatic failover for applications that use mirroring databases too (without ever seeing the error)?
I can see potential workarounds using custom scripts that would poll currently active database node name and adjust connection string accordingly, however it seems like an overkill at the moment.
Read the blog post here
http://blogs.msdn.com/b/spike/archive/2010/12/15/running-a-database-mirror-setup-with-the-sqlbrowser-service-off-may-produce-unexpected-results.aspx
It explains what is happening, the failover partner is actually being read from the sql server not from your config. Run the query in that post to find out what is actually being used as the failover server. It will probably be a machine name that is not discoverable from where your client is running.
You can clear the application pool in the case a failover has happened. Not very nice I know ;-)
// ClearAllPools resets (or empties) the connection pool.
// If there are connections in use at the time of the call,
// they are marked appropriately and will be discarded
// (instead of being returned to the pool) when Close is called on them.
System.Data.SqlClient.SqlConnection.ClearAllPools();
We use it when we change an underlying server via SQL Server alias, to enforce a "refresh" of the server name.
http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlconnection.clearallpools.aspx
The solution is to turn connection pooling off Pooling="false"
Whilst this has minimal impact on small applications, I haven't tested it with applications that receive hundreds of requests per minute (or more) and not sure what the implications are. Anyone care to comment?
Try this connectionString:
connectionString="Data Source=[MSSQLPrincipalServerIP,MSSQLPORT];Failover Partner=[MSSQLMirrorServerIP,MSSQLPORT];Initial Catalog=DatabaseName;Persist Security Info=True;User Id=userName; Password=userPassword.; Connection Timeout=15;"
If you are using .net development, you can try to use ObjAdoDBLib or PigSQLSrvLib and PigSQLSrvCoreLib, and the code will become simple.
Example code:
New object
ObjAdoDBLib
Me.ConnSQLSrv = New ConnSQLSrv(Me.DBSrv, Me.MirrDBSrv, Me.CurrDB, Me.DBUser, Me.DBPwd, Me.ProviderSQLSrv)
PigSQLSrvLib or PigSQLSrvCoreLib
Me.ConnSQLSrv = New ConnSQLSrv(Me.DBSrv, Me.MirrDBSrv, Me.CurrDB, Me.DBUser, Me.DBPwd)
Execute this method to automatically connect to the online database after the mirror database fails over.
Me.ConnSQLSrv.OpenOrKeepActive
For more information, see the relevant links.
https://www.nuget.org/packages/ObjAdoDBLib/
https://www.nuget.org/packages/PigSQLSrvLib/
https://www.nuget.org/packages/PigSQLSrvCoreLib/
Can jdbc connections which are closed due to database un-availability be recovered.
To give back ground I get following errors in sequence. It doesn't look to be manual re-start. The reason for my question is that I am told that the app behaved correctly without
the re-start. So if the connection was lost, can it be recovered, after a DB re-start.
java.sql.SQLException: ORA-12537: TNS:connection closed
java.sql.SQLRecoverableException: ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
IBM AIX RISC System/6000 Error: 2: No such file or directory
java.sql.SQLRecoverableException: ORA-01033: ORACLE initialization or shutdown in progress
No. The connection is "dead". Create a new connection.
A good approach is to use a connection pool, which will test if the connection is still OK before giving it to you, and automatically create a new connection if needed.
There are several open source connection pools to use. I've used Apache's JDCP, and it worked for me.
Edited:
Given that you want to wait until the database comes back up if it's down (interesting idea), you could implement a custom version of getConnection() that "waits a while and tries again" if the database doesn't respond.
p.s. I like this idea!
The connection cannot be recovered. What can be done is to failover the connection to another database instance. RAC and data guard installations support this configuration.
This is no problem for read-only transactions. However for transactions that execute DML this can be a problem, especially if the last call to the DB was a commit. In case of a commit the client cannot tell if the commit call completed or not. When did the DB fail; before executing the commit, or after executing the commit (but not sending back the acknowledgment to the client). Only the application has this logic and can do the right thing. If the application after failing over does not verify the state of the last transaction, duplicate transactions are possible. This is a known problem and most of us experienced it buying tickets or similar web transactions.
I have a DBCP connection pool in Tomcat. The problem is that when the connection is lost briefly the appliction is broken because DBCP won't try to reconnect again later when there is a connection. Can I get DBCP to reconnect automatically?
There are 2 ways to "solve" this, though both have some issues:
You can use a "validationQuery" (see below) to have a test query run before you go (generally something like 'select 1 from dual' which will be used to test connections before/after you get/give them to the pool. This adds an extra call per connection request from the pool. See: http://wiki.apache.org/commons/DBCP
Instead of doing this per query, you can have the idleEvictorThread do it by setting testWhileIdle, though in some versions that thread can cause deadlocking under high-load. See: http://commons.apache.org/dbcp/configuration.html for more details on that and other options
Don't think DBCP does that, but BoneCP (http://jolbox.com) can be configured to automatically replay any transactions when the DB or network goes down. It's completely transparent to your application.
I have a JSP/Spring application using Hibernate/JPA connected to a database. I have an external program that check if the web server is up every 5 minutes.
The program call a specific URL to check if the web server is still running. The server returns "SUCCESS". Obviously if the server is now, nothing is returned. The request timesout and an alert is raised to inform the sysadmin that something is wrong...
I would like to add another layer to this process: I would like the server to return "ERROR" if the database server is down. Is there a way using Hibernate to check if the database server is alive and well?
What I tought to do was to take an object and try to save it. This would work, but I think it's probably too much for what I want. I could also read(load) an object from the database. But since we use second-level caching for all our objects, the object will be loaded from the cache, and not the database.
What I'm looking for is something like:
HibernateUtils.checkDatabase()
Does such a function exist in Hibernate?
You could use a native query, e.g.
Query query = sess.createNativeQuery("select count(*) from mytable").setCacheable(false);
BigDecimal val = (BigDecimal) query.getSingleResult();
That should force hibernate to use a connection from the pool. If the db is down, I don't know exactly what will be the error returned by hibernate given that it can't get the connection from the pool. If the db is up, you get the result of your query.