How to connect a GAE app and a GCE app to the same datastore locally? - google-app-engine

Question

I am running into an issue similar to this one. I have a GAE app and a GCE app that seem to work fine once in the cloud but I am having trouble getting my local environment working so that both of them access the same datastore.
I have setup the local datastore as described in the link above except my code looks like this (I had to build it this way in order to get it working in the cloud):
print("Connecting to datastore - " + datasetId);
// Get credentials form GCE if not local.
Credential computeEngineCredential = DatastoreHelper.getComputeEngineCredential();
if(computeEngineCredential != null)
{
print("Compute Engine Credetianls are not null! Access token: " + computeEngineCredential.getAccessToken());
}
DatastoreOptions options = DatastoreHelper.getOptionsfromEnv().credential(DatastoreHelper.getComputeEngineCredential()).dataset(datasetId).build();
print("Connection Host: " + options.getHost());
print("Connection Dataset: " + options.getDataset());
datastore = DatastoreFactory.get().create(options);
When I run the GCE app locally and try to connect to the running GAE datastore I get the following (I have replaced the actual data set id with "myDatasetId" in the output below):
Connecting to datastore - "myDatasetId"
Connection Host: http://localhost:8888
Connection Dataset: "myDatasetId"
com.google.api.services.datastore.client.DatastoreFactory makeClient
WARNING: Not using any credentials
There was a problem running query: Error:
runQuery Error 404 Error
404
com.google.api.services.datastore.client.DatastoreException: Error 404 Error
404
at com.google.api.services.datastore.client.RemoteRpc.makeException(RemoteRpc.java:114)
at com.google.api.services.datastore.client.RemoteRpc.call(RemoteRpc.java:80)
at com.google.api.services.datastore.client.Datastore.runQuery(Datastore.java:109)
My GAE console prints this out (I can access the admin console in the 8888 port just fine):
com.google.appengine.tools.development.AbstractModule startup
INFO: The admin console is running at http://localhost:8888/_ah/admin
com.google.appengine.tools.development.LocalResourceFileServlet doGet
WARNING: No file found for:
/datastore/v1beta2/datasets/"myDatasetId"/runQuery
I have verified the dataset ID in the GCE app and the GAE app match. I have been able to successfully run each app localy on their own and they both are able to properly connect to the local Datastore (this is while using the gcd.cmd tool for the GCE app). Based on the answer in the link above, it sounds like this is possible, am I doing something wrong?
Update
Not sure if this is related but I am getting the following error when starting up the GCD Tool:
SEVERE: Unable to load the App Engine dev agent. Security restrictions
will not be completely emulated. java.lang.RuntimeException:
Unexpected exception during cast.
at com.google.apphosting.utils.clearcast.ClearCast$CasterImpl.cast(ClearCast.java:385)
at com.google.apphosting.utils.clearcast.ClearCast.staticCast(ClearCast.java:252)
at com.google.apphosting.utils.clearcast.ClearCast.staticCast(ClearCast.java:263)
at com.google.appengine.tools.development.agent.AppEngineDevAgent.premain(AppEngineDevAgent.java:61)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(Unknown
Source)
at sun.instrument.InstrumentationImpl.loadClassAndCallPremain(Unknown
Source) Caused by: java.lang.IllegalAccessException: Class
com.google.apphosting.utils.clearcast.ClearCast$CasterImpl can not ac
cess a member of class
com.google.appengine.tools.development.agent.$Proxy0 with modifiers
"public"
at sun.reflect.Reflection.ensureMemberAccess(Unknown Source)
at java.lang.reflect.AccessibleObject.slowCheckMemberAccess(Unknown
Source)
at java.lang.reflect.AccessibleObject.checkAccess(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at com.google.apphosting.utils.clearcast.ClearCast$CasterImpl.cast(ClearCast.java:383)
... 9 more
If this initialized properly, could I somehow connect my GAE App to the GCD Tool datastore? So confused.

Answer 1

There's no officially supported way to share Datastore data between the Java Development Server (dev_appserver.sh) and the local Cloud Datastore tool (gcd.sh).
However, if your app is written in Java, you may be able to use the workaround described here and point dev_appserver.sh to the data file generated by gcd.sh by specifying the -Ddatastore.backing_store=<project dir>/WEB-INF/appengine-generated/local_db.bin option.

Answer 2

Thanks for replying this was very useful. I was able to point my local GAE to the local_db.bin that the gcd tool uses through eclipse by providing the "-Ddatastore.backing_store" as a VM argument as you suggested.
However, they still seem to have a different views of the datastore. The admin viewer for the GAE app running on the default 8888 port only shows the data added by the GAE app, on the other hand, the gcd one running on the gcd tool port (8080) shows the data added by the GCE app.
I assumed this was just a visibility issue on the admin site so I tried to access the GAE data through my GCE app but the query is unsuccessful, it doesn't seem to find the Entity kind and thus returns no results. I was able to verify that once deployed GCE is able to access the data the GAE app wrote to the datastore with the same query, due to this I am assuming it is not a namespace issue but more of an issue on where the data is held. Even though they are both pointing to the same local_db.bin file it seems the data is still split somewhere.
Am I supposed to run the dev_appserver.cmd directly form the command line maybe? If so, How do I do this for an EAR project (currently on eclipse).