Play Siena failing to connect to MySQL on GAE - google-app-engine

I am using play framework 1.2.7, gae module 1.6.0 and siena module 2.0.7 (also tested 2.0.6). This is a simple project that should run in play deployed on App Engine and connect to a MySQL database in Google Cloud SQL. My project runs fine locally but fails to connect to the database in production. Looking at the logs it looks like it is using the postgresql driver instead of the mysql one.
Application.conf
# db=mem
db.url=jdbc:google:mysql://PROJECT_ID:sienatest/sienatest
db.driver=com.mysql.jdbc.GoogleDriver
db.user=root
db.pass=root
This is the crash stack trace
play.Logger niceThrowable: Cannot connected to the database : null
java.lang.NullPointerException
at com.google.appengine.runtime.Request.process-a3b6145d1dbbd04d(Request.java)
at java.util.Hashtable.put(Hashtable.java:432)
at java.util.Properties.setProperty(Properties.java:161)
at org.postgresql.Driver.loadDefaultProperties(Driver.java:121)
at org.postgresql.Driver.access$000(Driver.java:47)
at org.postgresql.Driver$1.run(Driver.java:88)
at java.security.AccessController.doPrivileged(AccessController.java:63)
at org.postgresql.Driver.getDefaultProperties(Driver.java:85)
at org.postgresql.Driver.connect(Driver.java:231)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:215)
at play.modules.siena.GoogleSqlDBPlugin.onApplicationStart(GoogleSqlDBPlugin.java:103)
at play.plugins.PluginCollection.onApplicationStart(PluginCollection.java:525)
at play.Play.start(Play.java:533)
at play.Play.init(Play.java:305)
What is going on here? I am specifying the correct driver and url schema and it's using postgresql driver. Google Cloud SQL API access is enabled, the app is allowed to connect to the mysql instance, I am not using db=mem, ... I am stuck and can't figure out how to move forward! :-((
UPDATE: I thought I found the solution, but that was not the case. If I keep the %prod. prefix and create a war normally (or just don't define any DB properties), then the application will use Google DataStore instead of the Cloud SQL. If I create the war file adding --%prod at the end (or just delete the %prod. prefix in the application.conf), then it will keep failing to connect to the database showing the same initial error.
Any ideas please?

After being stuck for so long on this I just found the solution in no time after posting the question. Quite stupid actually.
The production environment properties in the application.conf file must be preceded by %prod. so the database config should read
%prod.db.url=jdbc:google:mysql://PROJECT_ID:sienatest/sienatest
%prod.db.driver=com.mysql.jdbc.GoogleDriver
%prod.db.user=root
%prod.db.pass=root
And everything runs fine.
EDIT: This is NOT the solution. The problem went away, but the app is using the DataStore instead of the Cloud SQL

At the end I ended doing a slight modification in play siena module source code and recompiling it.
In case anyone is interested, you will need to remove/comment/catch exception in this code around line 97 in GoogleSqlDBPlugin class:
// Try the connection
Connection fake = null;
try {
if (p.getProperty("db.user") == null) {
fake = DriverManager.getConnection(p.getProperty("db.url"));
} else {
fake = DriverManager.getConnection(p.getProperty("db.url"), p.getProperty("db.user"), p.getProperty("db.pass"));
}
} finally {
if (fake != null) {
fake.close();
}
}
For some reason the connection fails when initiated with DriverManager.getConnection() but it works when initiated with basicDatasource.getConnection(); which apparently is the way used by the module in the rest of the code. So if you delete the above block, and recompile the module everything will work as expected. If you are compiling with JDK 7, you will also need to implement public Logger getParentLogger() throws SQLFeatureNotSupportedException in the ProxyDriver inner class at the end of GoogleSqlDBPlugin file.
Strangely, I digged into the DriverManager.getConnection() and it looked like some postgresql driver is registered somehow, because otherwise I can't see why DriverManager.getConnection() would call to org.postgresql.Driver.connect().

Related

Idea doesn't upgrade sources while in local debug for appEngine app

I created Spring Boot + Google App Engine application. For development purpose I use IntelliJ IDEA and Google Cloud Tools plugin. I'm currently using only localDebug, which means I don't deploy anything on Google cloud. The configuration for debug is below:
I created a simple service to be sure if my code is updated on change or not:
static int i = 10;
#GetMapping(value = "/test")
public String test() {
return Integer.toString(++i);
}
Unfortunately when I change my code (e.g. from i = 10 to i = 100) and restart the app (I mean press on Rerun (Ctrl+F5) or Stop (Ctrl+F2) + Run my code doesn't apply on server, which means Idea doesn't rebuild the sources on server start. As you see on the screenshot above I even tried to add Build Project step to Before launch, which didn't work.
So to apply changes I need to execute from command line mvn appengine:run -> press Ctrl+C to stop it, switch to IDEA and start debug again which is a pain in the ass.
Another option is to use Hot Reload (Update application, Ctrl+F10). It recompiles only changed classes and reloads resources. This is a cool feature, but unfortunately it doesn't work in a lot of cases which makes me unable to use it as a reliable reload.
Is there anything I can do to force IDEA compile my sources? Is it a bug I should report to plugin developer. Or maybe appengine uses some additional remote sources that require explicit call of maven?
I finally found a solution. As I understood the Google cloud plugin just complies the classes into target/classes but when it starts the appEngine, the engine expected unpacked .war to be present under target/demo-0.0.1-SNAPSHOT.
E.g. if because if I delete both directories I get the error below:
To solve the issue I needed to compile those sources:
In toolbar Run -> Edit configuration
Select Google App Engine Standard Local server
In before launch add Build Artifact -> demo:war exploded where demo is the name of your App.

GORM Cloud SQL Connection on App Engine Using Go

I'm trying to connect to a Cloud SQL database using GORM in golang.
db, _ = gorm.Open("mysql", "user:pass#cloudsql(connection:name:example)/")
if err != nil {
log.Println(err)
//panic(err)
}
When I attempt to serve the app
goapp serve appengine/
I get a runtime error
ERROR 2017-02-19 20:48:05,436 http_runtime.py:396] bad runtime process port ['\r\n']
Which I found was related to the database migration
db.AutoMigrate(&models.Event{})
If I remove the AutoMigrate, the runtime process port error goes away. However whenever I access a route (ie /events) that does a database query, the connection gets dropped, a 404 page is thrown, and an error message is logged sql: database is closed
When I run the app locally by building the package go build && ./appname and using a local MySQL server, it works fine.
Can someone please tell me how to connect to a Cloud SQL database using Go's GORM framework and App Engine?
This is due to the call to log.New in this file: https://github.com/jinzhu/gorm/blob/master/logger.go#L15
This anwser explain why dev_appserver.py gets it: https://stackoverflow.com/a/24112953/4266494
To disable this, you can either disable all GORM logging:
db.LogMode(false)
Or use an adapter the logger output: https://github.com/benguild/GAEBridge/blob/master/log/debugLevel.go
db.SetLogger(NewDebugLogger(nil)) // on application scope
db.SetLogger(NewDebugLogger(appengine.NewContext(req))) // on request scope
I'm setting a new logger with the real context
This is the only workaround I found to avoid crashes while keeping some logs, it could be awesome if one of you had a real one.

NullPointerException with JDBC and App Engine

I have been struggling with a problem in Google App Engine, using Java, for several days.
Many times (about 50% of the time) when I try to request the connection to a Cloud Sql instance, the connection returns a null value, resulting in several NullPointerException messages when trying to invoke Cloud Sql queries (when invoking .prepareCall(stored_proc)).
I have the latest App Engine Java SDK, in a project service, shared with other services built in Python which consume this Java backend.
Could it be possible that after certain time the instance/s could crash (I am just testing at this point, so I am using default scalation)?
This is the code that returns null:
Class.forName("com.mysql.jdbc.GoogleDriver");
url = "jdbc:google:mysql://project:instance/database?user=root";
log.info(url);
return DriverManager.getConnection(url);
This is part of my configuration file:
<application>app</application>
<module>mod</module>
<version>1</version>
<threadsafe>true</threadsafe>
<use-google-connector-j>true</use-google-connector-j>
I tried several suggestions from other posts, but with no success at all.
Any suggestion will be welcome, thanks in advance.
I was facing the same problem while using Google Cloud SQL and App engine.
I solved the problem by managing the connection pool my self. I realised that when you request a new connection for each request and close it on completion of the thread. The other requests would get back a null resulting to NullPointException.
I decided to do the following and it work for me for like 2 years now.
Open a connection and keep it to a static class that has a number of connections;
Every time i want to find a connection to the database, i would first check if the is an available connection for me to use.
Incase a Query killed the connection, thus means i needed to request another extra connection just to for the sack of connection drops.
I will add this as an answer, since it is not exactly what Chrispinus mentioned, although he gave me a good idea for teh solution.
I went deeper in the code and found that some of the methods were not closing the database connection. I had assumed all of them were doing that, but looking at each method, I found I was wrong.
So, although it sounds obvious, check connections are being closed (or managed, as Chrispinus says) properly.

Having issues getting WordPress running under GAE + Cloud SQL

I tried to set up WordPress under Google App Engine earlier tonight (following the instructions here: https://developers.google.com/appengine/articles/wordpress).
It runs fine locally, but when I push to remote I get a database error (visible at https://wp-dot-frontiermediag.appspot.com/). If we throw on a /wp_admin/install.php you get:
This either means that the username and password information in your
wp-config.php file is incorrect or we can't contact the database server
at :/cloudsql/frontiermediag:fmwp. This could mean your host's database
server is down.
Here's the relevant code in wp-config:
/** MySQL hostname */
if(isset($_SERVER['SERVER_SOFTWARE']) && strpos($_SERVER['SERVER_SOFTWARE'],'Google App Engine') !== false) {
define('DB_HOST', ':/cloudsql/frontiermediag:fmwp');
}else{
define('DB_HOST', 'localhost');
}
frontiermediag:fmwp is showing "Status Runnable" in Developers Console > Cloud SQL.
I did this once before and it worked so I'm not sure what I'm missing here. I thought it might have been because I'm using WP 3.8.1. but rolled back to 3.5.1 and same thing's happening.
Any ideas? frontiermediag is listed as an authorized application on the :fmwp ACL.
This situation happened to me earlier.However, I edited my Cloud SQL instance , and set "Preferred Location" as "Follow App Engine App" from Google Developers Console. This database connection problem was solved in my case.
I tried the instructions with wordpress 3.5.1 and the instructions seem to work for me. The code snippet you have above seems right and I am not sure what could be wrong without looking at rest of your code. Can you try the instructions from the beginning one more time with 3.5.1?
I had this issue, because "Follow App Engine App" doesn't seem to be an option for second generation instances in my case, and so the instance connection name includes the region setting.
Look at the instance details, and under properties, find "Instance connection name". That is the text that should follow :cloudsql/.

Install Orchard to an existing database

In our team, we use a common database server. When running the Orchard Setup, I point the connection string to an existing Orchard Database and got this error message:
Setup failed: The requested service 'Orchard.IWorkContextAccessor' has not been registered. To avoid this exception, either register a component to provide the service, check for service registration using IsRegistered(), or use the ResolveOptional() method to resolve an optional dependency.
The only way that can make setup successful for me is to point to an empty database. I attempted a workaround by doing so first, and then edit connection string in App_Data\Sites\Default\Settings.txt, but then I got the same error.
I had this problem as well, also tried copying the settings.txt file which did not work. This time I copied all the contents of the App_data directory over and it worked for me. Also, this latest time I was using the latest commit from codeplex.
I deleted cache.dat and restarted the website in IIS, this fixed it for me

Resources