We are trying to connect to Google Cloud SQL from Eclipse using the Database Development perspective. To do so I'm trying to add a new Database Connection, which I was able to do successfully for a local MySQL instance running on my machine.
The motivation for doing this is that we currently run our JUnit tests against the local instance. However, we are switching to Hibernate and want to make sure that all of our configuration files work with Cloud SQL. As a general guide I've been using:
https://developers.google.com/appengine/articles/using_hibernate
We're diverging slightly in that we're using hibernate.cfg.xml instead of persistence.xml, but I don't think this will actually have a bearing on the current issue of simply connecting to the database. From another answer as well as some Google documentation I'm aware that I can't use the com.google.appengine.api.rdbms.AppEngineDriver, because that needs to be run from an AppEngine instance. Instead I'm trying to follow the directions here:
https://developers.google.com/cloud-sql/docs/external
and am using com.mysql.jdbc.Driver.
I have assigned my Cloud SQL instance an ip address and have added my current ip address to the whitelist, as described here:
https://developers.google.com/cloud-sql/docs/access-control#appaccess
My driver is the Connector/J driver I've been using successfully with the local instance, and the url I'm using is:
jdbc:google:rdbms://my-app:my-cloud-sql-instance/myDatabase
which I got based on:
https://developers.google.com/appengine/articles/using_hibernate
After adding the connection and setting the information I click Test Connection, which worked successfully on my local instance. However, this throws the following error:
java.lang.Exception: Connection failed with unspecified error.
at org.eclipse.datatools.connectivity.DriverConnectionBase.internalCreateConnection(DriverConnectionBase.java:110)
at org.eclipse.datatools.connectivity.DriverConnectionBase.open(DriverConnectionBase.java:54)
at org.eclipse.datatools.connectivity.drivers.jdbc.JDBCConnection.open(JDBCConnection.java:73)
at org.eclipse.datatools.enablement.internal.mysql.connection.JDBCMySQLConnectionFactory.createConnection(JDBCMySQLConnectionFactory.java:28)
at org.eclipse.datatools.connectivity.internal.ConnectionFactoryProvider.createConnection(ConnectionFactoryProvider.java:83)
at org.eclipse.datatools.connectivity.internal.ConnectionProfile.createConnection(ConnectionProfile.java:359)
at org.eclipse.datatools.connectivity.ui.PingJob.createTestConnection(PingJob.java:76)
at org.eclipse.datatools.connectivity.ui.PingJob.run(PingJob.java:59)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)
Obviously this isn't very helpful.
I've tried fiddling with the url, tried a number of users (none of which require passwords, so I'm leaving the password fields blank), and different versions of the driver for different versions of MySQL. Nothing has worked.
There are perhaps more deep-seated issues with doing it this way, such as how I will easily switch between test and deployment versions of my hibernate.cfg.xml, and I don't have good answers. I was just planning on editing them by hand back to the AppEngineDriver, which means I might run into further configuration issues at that point even if the JUnit tests are passing. Nevertheless, I think getting a connection set up to Cloud SQL that will allow JUnit testing will be a step in the right direction. I'd appreciate any input!
You should use jdbc:mysql://<cloudsql-instance-ip>:3306/<database-name> to connect from an external network. The connection string you are using is to connect from Google App Engine.
Related
I Created SQL Server Database in Azure which is serverless and tried to access it using my SQL Server Management Studio in my local but I couldn't get it work.
It always gives me this message:
I tried to whitelist also my IP in Azure but still I get the same result.
Is there a possible way to make it connect?
Is the database currently online or paused?
I'll repeat the text from #David Browne's link:
If a serverless database is paused, then the first login will resume the database and return an error stating that the database is unavailable with error code 40613. Once the database is resumed, the login must be retried to establish connectivity. Database clients with connection retry logic should not need to be modified.
So;
Assuming the database is paused, this is normal operation
Please read docs
You need to retry after the database starts OR manually pre-start it using the Powershell provided in the link below
https://learn.microsoft.com/en-us/azure/sql-database/sql-database-serverless#connectivity
And yes, you also need to whitelist your IP address as you have already done.
Obviously this flavour of SQL is unsuitable for some types of applications - there is more information in the link - I suggest you read the whole thing.
I made ASP.NET MVC web application, uploaded the files, also the database, but I get the following error when browsing it.
The network path was not found
I'm using Entity Framework and this is the connection string in my web.config file
<connectionStrings>
<add name="[mydatabase]Entities" connectionString="metadata=res://*/Models.Model1.csdl|res://*/Models.Model1.ssdl|res://*/Models.Model1.msl;provider=System.Data.SqlClient;provider connection string="data source=sql.[somedomain].net;initial catalog=[mydatabase name];User ID=[myUsername];Password=[myPassword];MultipleActiveResultSets=True;App=EntityFramework"" providerName="System.Data.EntityClient" />
</connectionStrings>
I've uploaded many websites using IP Address eg. xxx.xxx.x.xxx as data source, but this is the first time to use server name eg. sql.[somedomain].net -I cannot get the server ip-, so I'm not sure if this caused the error, or if I should make something special to make it work.
So, I'm asking if there is something I should do to use server name as data source, if not then what else may cause this error.
This answer doesn't really help.
Thanks in advance.
Update
If I ping the server sql.[somedomain].net, I get this result
Ping request could not find host sql.[somedomain].net. Please check the name and try again.
And if I nslookup it, I get this result
*** Unknown can't find sql.[somedomain].net: Non-existent domain
So does that mean -for sure- that the server is not accessible. And is there anything I can do beside contacting the hosting technical support?
Solved
It was the Hosting Provider error/misconfiguration. After 3 days of searching and contacting the customer support, they realized it was their issue. I'm leaving this question to tell future viewers to Host Only with reliable/well-known Hosting Providers no matter what.
The info you got from tech support seems fairly contradictory. It's common to disable remote access for database servers, but if that's the case, then using the domain to connect doesn't really help you.
If you're trying to connect to this database from a published MVC project, residing with the same provider as supplies your database, then you should have no issues connecting as you're no longer "remote". However, unless your DNS is also hosted at the same provider, using the domain may make the connection appear remote, as it's going outside to come back in. The safest bet is to simply use the IP address of the database server in the connection string.
Also, pay attention to the IP address you have for the database server. If it's in the 10.* or 172.* range, it's a local IP, but if it's something else, it's most likely an outside IP address. Trying to connect to such an address, may also make the connection appear remote as you're going outside the network to come back in. Also, while disallowing remote access to a database server is a good idea, you can generally safely allow remote access to certain IPs. You most likely don't have control over that directly, but you can check with your provider to see if they can add an explicit rule for your web server's IP so that even if the connection is coming through as remote, it'll still work.
As far as working locally in development, you'll just have to use a local database. You may already have that covered, but your question wasn't entirely clear on that aspect.
I also faced the same issue. I used the forward slash instead of back slash for instance name (clustername\instancename). Once I changed it to back slash. It worked fine.
I'm using the AWS Toolkit in Visual Studio 2013 to attempt to launch a new instance on Amazon RDS. I get through the wizard for creating the new instance and after clicking finish, there is a delay, and then a message appears saying:
Error launching DB instance: DB Security Groups can only be associated with VPC DB Instances using API version 2012-01-15 through 2012-09-17.
Launching different types of instances (SQL Server SE vs MySQL) doesn't seem to help, nor does selecting different versions of the platforms (SQL Server 2008 vs 2012). The only thing that gets it to go through is unchecking the box for "default" in the DB Security Groups area. However, I feel like something is going on here that shouldn't be happening.
Can anyone explain why this is happening and how I can resolve it other than by not setting a default security group? Thank you.
If you created your AWS account recently, you will be using a VPC by default.
It sounds like the API the plugin is trying to use hasn't been updated. The latest version is 1.5.6, and looking at the history it seems like some of these features were added in 1.5.0.
I finally solved it! Since I couldn't use the API that the VS 2013 plugin uses, I had to manually add my IP to the Security Group created for my Elastic Beanstalk.
Go to the console, ec2's security groups configuration
Find the one which description matches your Beanstalk (e.g.: Security Group created for Beanstalk Environment to give access to RDS instances)
Hit Inbound, Edit and add a new rule for All Traffic (I guess HTTP should be enough, but just in case).
In Source, select My IP and Save.
I'm using Phil Sturgeons Rest as an API between the server and mobile APP. Locally this works brilliantly, on the server it bounces back with an issue when using API Keys authentication:
Unable to connect to your database server using the provided settings.
Filename: core/Loader.php
This is while $config['rest_enable_keys'] = TRUE; is enabled, when I change the variable to false the API service works just fine. Both locally and the server are running the same table layout and both have the same API key in the keys table. Rest of the site works fine doesn't show any database connection errors.
Changing the line $config['rest_database_group'] = 'default'; prompts a different error, which would lead me to believe that it is infact connecting to the correct database, just can't establish the keys table?
You have specified an invalid database connection group.
There's no difference in the code between the server and locally yet locally I get the correct error message of {"status":false,"error":"Invalid API Key."}
Could this be an issue with using multiple database connections? I checked the environment and it seems to be correct but this wouldn't explain why it would work locally.
Thanks a lot to nithin-meppurathu who helped me debug the core of the rest library to find the problem.
The problem lied in the multiple database connections, due to switching what is used depending on the environment it was always being defaulting back to the 'default' database environment. Changing $config['rest_database_group'] = 'default'; to my ENVIRONMENT variable I was able to get it pointing to the correct place.
Context: The Cloud
We have a java-based web application that we normally host on our own servers. Recently we used Amazon Web Services (AWS EC2) cloud to host an instance.
This "cloud setup" matches our typical "on site" setup: one server for the app server, another server for the database server. (Several app servers point to the same database server)
The problem
In this cloud setup, we receive intermittent "connection reset by peer errors" between the database and the jdbc driver, where at (seemingly) random intervals and at random points in the codebase, the database connection fails.
Here are a few error excerpts for the log
Stack Trace Example 1:
at com.participate.pe.genericdisplay.client.taglib.GenDisplayViewTag.doStartTag(GenDisplayViewTag.java:77)
... 75 more
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The connection is closed.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:170)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.checkClosed(SQLServerConnection.java:304)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.getMetaData(SQLServerConnection.java:1734)
at org.jboss.resource.adapter.jdbc.WrappedConnection.getMetaData(WrappedConnection.java:354)
Stack Trace Example 2
at java.lang.Thread.run(Thread.java:619)
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Connection reset
at com.microsoft.sqlserver.jdbc.SQLServerConnection.terminate(SQLServerConnection.java:1368)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.terminate(SQLServerConnection.java:1355)
at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:1532)
at com.microsoft.sqlserver.jdbc.TDSReader.readPacket(IOBuffer.java:3274)
at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:4437)
at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:4389)
at com.microsoft.sqlserver.jdbc.SQLServerConnection$1ConnectionCommand.doExecute(SQLServerConnection.java:1457)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:4026)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1416)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectionCommand(SQLServerConnection.java:1462)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.setAutoCommit(SQLServerConnection.java:1610)
at org.jboss.resource.adapter.jdbc.BaseWrapperManagedConnection.checkTransaction(BaseWrapperManagedConnection.java:429)
Technical Environment
Jboss 4.2.2.GA (Jboss-Web 2.0/ Tomcat 6)
MSSQL 2005 2.0 jdbc driver
Some points
We have never seen this problem in
our own environment (i.e. own data centers) running the application for several years
This led me to conclude "something funny is going on with Amazon network environment". I may be wrong/missing something/etc.
This problem only occurs with our application. We have other java and php applications which have not had this problem. The other java application uses a different jdbc driver (jtds, afaik)
It doesn't seem like a simple connection timeout
Questions
-Has anyone seen this before?
-If it's an EC2 "known issue", can we configure our way around the problem (i.e. make sure everything is on its own subnet or virtual private cloud (vpc) ?
-Any jdbc driver settings to get past this problem?
** Update **
I've extended and increased the bounty on this question.
On extra bit of information: the two virtual servers (database and application server) were on different subnets--i.e. one hop between the two servers.
In a non-cloud environment we have "zero hops" bewtewn the two servers.
Our hosting admins said we had no control over the subnets of our EC2 instances. This made me wonder if virtual private cloud would help.
thanks in advance
will
Not sure if this is related or not. We experienced something similar with an app that we were running in the EC2 environment. Same symptom, that the database connection would intermittently close. We were using MSSQL 1.2 driver. Also, we would see the errors usually after a delay or idle time with the connection. Our assumption (never proven) was that something in the network layer was closing the connection and the client wasn't detecting it, so it became stale.
We were able to work around it because we were using commons connection pools, and had the pool recreate the connection on failure. We eventually moved the application out of EC2 and didn't see the issue again.
Just a word of caution on usind DBCP/connection pool features to mitigate the issue - the more you enable 'testOnBorrow' and other features, the more you can introduce latency or other performance changing affects on the system. I don't know if DBCP still does this or not, but a few years ago it would generate actual test queries to test the connection - full stack, database responses - not just at the network layer. The above link from Brian brings back horrific memories from the early 2000s on surrounding re-try logic for JDBC connection management.
Anyway, it's tough to really root cause this, other than gather evidence and eliminate the 'seemingly random' to a specific set of conditions:
You could try to throw up a Wireshark/PCAP trace, find when it happens, and send the results to both Amazon and Microsoft to see if they can root cause it
You could try the above with certain test harnesses to isolate the problem (JMeter tests to get concurrency up), bounce the network connection, watch for recovery, etc
You could try alternative versions of SQL Server to discount a SQL Server/JDBC driver bug that has since been fixed.
If DNS is used in connection strings, could use IP addresses to validate nslookup issues
I'm not a SQL Server expert, but another route for research could be within the related products domain - e.g. see if anyone experienced similar issues with TFS/Sharepoint (e.g. such as http://nickhoggard.wordpress.com/2009/12/07/further-experiences-with-tfs-2010-beta-2-on-amazon-ec2/ )
I have seen this issue in both the EC2 environment and the Windows Azure environment. I think connection retry logic needs to be a standard part of your design when working in a distributed computing environment.
This article is for SQL Azure - but I think it equally applies to EC2 and all drivers.
I can also confirm that this happens and will spin up a lower priority investigation since it's not production critical.
Our production servers are in our data center. We use developer laptops to run our applications. Neither of these get this issue once we configured c3p0 connection pool timeouts and test period (see article: http://www.codefin.net/2007/05/hibernate-and-mysql-connection-timeouts.html).
However, we do have a development staging server that is in EC2 and it does indeed happen there. If I find something that seems to work, I'll ping back. Also, I'm using mysql. I see that you are using MS SQL Server so it is across database vendors.