I was looking to check out the experimental backup/restore feature of the app-engine datastore, so I followed the instructions and enabled "Datastore Administration" as explained in the link.
After doing so, instead of seeing the Datastore administration controls in the dashboard I am getting a "This webpage is not available" on chrome and "Server not found" on Firefox.
I understand this is experimental but still any help from the app-engine team would be appreciated in terms of what's going on with this feature :)
UPDATE: I tried this on another app (with an empty datastore) and the Datastore Administration page comes up normally.
A few things to check:
Are you using the new high availability datastore? The docs don't specify but it's likely this experimental feature is not compatible with the deprecated master/slave datastore.
If your app is using Java, you have to deploy a non-default Python app to use the backup/restore feature (search for "a note for java developers" on the page).
How large is your datastore? It may take awhile to enable the feature on very large datastores.
How long is your app's name? The error you are seeing is DNS related. The Datastore Admin feature seems to make a request to:
ah-builtin-python-bundle-dot-latest-dot-[YOUR APP NAME].appspot.com
at one point. Due to the length of that subdomain name, if your app name is longer than roughly 23 characters, you'll get a DNS error because DNS is limited to 63 characters per domain section:
http://en.wikipedia.org/wiki/Domain_Name_System#Domain_name_syntax
In my limited testing, I was able to get to a longer domain name by adding the ah-builtin-python-bundle... domain name directly to my system's etc/host file:
http://en.wikipedia.org/wiki/Hosts_(file)
Related
On an App Engine application, deployed on standard environment with java 8 activated, I get the exception below if I use both Objectify 6.0 AND the tag <url-stream-handler>urlfetch</url-stream-handler> in the appengine-web.xml file.
com.google.cloud.datastore.DatastoreException: Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
Any idea how I can bypass that?
Things get back to normal if I remove the urlfetch tag (and activate the billing) or if I downgrade Objectify to v5.
I suspect what you're running into is one of the differences between the Java 7 and the Java 8 Standard Environments. There are substantial benefits to avoiding the urlfetch path, as Issuing HTTP(S) Requests notes (e.g. HTTP/2). In particular, the metadata server, which does automatic authentication for the Google Cloud SDKs when running on App Engine, isn't reachable using urlfetch (see that page).
I think your best option here is probably to enable billing, if you're comfortable with that. From the docs page:
Despite the requirement to enable your application for billing, your app won't incur any more cost than the same application running on Java 7.
If you'd still like to give urlfetch a try, I think the error you're encountering can be fixed by manually supplying credentials (e.g. bundling them in your app), but that won't be as secure or performant.
The best option is to use the
<appengine-web-app>
<url-stream-handler>native</url-stream-handler>
<appengine-web-app>
and enable billing. Without billing you can only use the legacy urlfetch and you have to face all these problems.
you can set credentials explicitly, like it is shown in the example below:
Resource credentialsCyberpower = resourceLoader.getResource("classpath:yourservice-datastore-access.json");
GoogleCredentials credentials = GoogleCredentials.fromStream(credentialsCyberpower.getInputStream())
.createScoped(Lists.newArrayList("https://www.googleapis.com/auth/cloud-platform"));
DatastoreOptions options =
DatastoreOptions.newBuilder().setProjectId("XXXXXX").setCredentials(credentials).build();
Datastore datastore = options.getService();
ObjectifyService.init(new ObjectifyFactory(datastore));
generate yourservice-datastore-access.json in IAM service accounts. working with Objectify 6.0.5
If a particular computer is making tons of accounts or flooding my server with other requests, could parse-server automatically check this behaviour and block the specified IP address?
Built-in rate limiting would also be a nice alternative, although it doesn't really solve the problem if the person continues to spam.
I am hosting on google app engine by the way.
I don't know about Parse itself, but from App Engine side you have DoS protection service controlled via dos.yaml file in your project that lets you blacklist IP blocks—sounds like that may help. It's not "automatic", though; you still need to manually update this file and issue appcfg.py update_dos <PROJECT_DIR> for changes to take effect.
I don't believe that this is a feature out of the box - see advanced options here: https://github.com/ParsePlatform/parse-server.
You'd need to look at controlling access to the Google App Engine (or another host - such as Microsoft Azure Web App) using a firewall (you can easily do this with Azure. I'm not familiar with Google App Engine, but imagine similar functionality is available.
However, I don't believe that a firewall is necessary - just better app security. Disable anonymous users - Parse Server Security
I'm running an app on a VM instance (instance-1) and would like myproject.appspot.com requests to be served by instance-1.
I read https://cloud.google.com/appengine/docs/java/modules/routing but it wasn't clear. Is there a way to say "send all traffic to my one instance"?
If I go to my (ephemeral) external IP address for that instance, I can see the server. But, that won't work for an oAuth2 domain (no IP addresses allowed), so I need it to go through the named domain.
I'd be ok if I could use something constant like instance-1-dot-myproject.appspot.com but would prefer the base myproject.appspot.com to say "any instances? great! use that."
I think you want to use Managed VMs. They give you the flexibility of Google Compute Engine but work more like the PaaS that is Google App Engine.
You don't create the Google Compute Engine VM instances yourself, however, Managed VMs will spin them up on demand, using the Docker image you provide as the container of the code, data, etc.
Note that as of 29 Sep 2015, per the docs:
Beta
This is a Beta release of Managed VMs. This feature is not covered by any SLA or deprecation policy and may be subject to backward-incompatible changes.
When will SSL support for custom domain be available within the developer console, instead of having to go over to Google Apps? I read somewhere it says Q3 of 2015. But seems like people have some ways of getting that to work already. Is it a private beta feature?
Building some apps for a client and they can't get their Google Apps account to work and we already have the custom sub-domain mapped to the GAE, and just need the last piece. Help!
This is the tracking issue, indeed Q3 seems to be more likely: https://code.google.com/p/googleappengine/issues/detail?id=10794
FWIW, I detailed a bit my solution to get things working via Google Apps in this Q&A: AppEngine subdomains to modules without wildcard mapping, maybe it can help with your customer's problems in the meantime.
Recently I migrated my application to the High-Replication datastore using the migration tool.
The Datastore Admin page now isnt diplaying correctly:
Error: Server Error
The server encountered an error and could not complete your request.
If the problem persists, please report your problem and mention this
error message and the query that caused it.
When I visit the Datastore Admin in the old application (the one I migrated from) I get the following notifications:
This application is hidden by an alias. The application lrweb2012 has
an alias that causes it to receive all traffic normally directed at
this one. Unless you are looking for historical data, you probably
want to see that application.
Datastore Admin is disabled. Because lrweb2012 has an alias that
shadows this application the Datastore Admin tool cannot be used.
Can anyone experianced suggest what is going on, and how I may go about resolving this issue?
Cheers,
You migrated your app, so you probably want to be looking at the admin for lrweb2012-hrd (or wherever you migrated it to) instead of the old setup.
Use the drop-down at the top-left corner to pick a different app.