We've just created a new Loadbalancer which works perfectly well with all our services. The only issue we have is with the Blobstore.
The initial blobservice request works - returning a URL for the upload. But executing that request (/_ah/upload/) fails with a 404. My understanding is that it should be redirected to the blobstore service which will then send a response with the upload id. This works fine without the load balancer.
I can't see a way to add the blobstore as a service, so I don't know what to do to make sure the request is routed to the correct place.
Any help is gratefully appreciated!
This is currently a known limitation with Blobstore when used in an HTTPS load balancer in App Engine.
With the App Engine to Blobstore service, the generated URL for upload is shared to Google Front End. When the Load Balancer is introduced, the App Engine URL is returned, and causes a 302 redirect. The returned URL is used as a relative path to connect via the load balancer, and as it is not aligned to the one expected by the GFE, a 404 response is thrown.
Blobstore is a legacy API and has been deprecated, and with the limitations it has, it is ideal to look for alternatives.
You can consider using Cloud Storage, as mentioned in this overview. It is possible to achieve the upload behavior through Cloud Storage signed URLs which bypasses the App Engine Upload component. It would be necessary to implement a further component to notify on completion, such as a Pub/Sub for new object creation, if that information is needed.
Besides Cloud Storage, you can check other replacement options for Blobstore in this documentation.
Related
I have an API running in Google App Engine and I want to introspect HTTP POST body to reject poorly formed calls before it hits my API running in app engine. Is there a way to front an App Engine application whereby I can introspect the URI, the POST body and return a 400 without it actually being processed by my code running in App Engine? Please let me know, thanks.
If you want to inspect HTTP, then you will need to write a proxy application to do so. Neither the Google Cloud Load Balancer nor App Engine support hooking HTTP requests.
Proxies have the MITM (Man In The Middle) problem in that HTTP traffic is often encrypted. You will not be able to inspect that traffic unless your proxy is the endpoint. If you implement your proxy with Apache or Nginx then you can use the ModSecurity project.
Basically, you want to implement a WAF. Google offers Cloud Armor which supports inspecting traffic via Cloud Armor rules.
Current App:
Goole App Engine on a custom domain: myapp.com
Google Cloud Storage bucket on a custom subdomain (it uses DNS): images.myapp.com
I have SSL certificate with the images subdomain
My question is: Is possible to have this configuration with SSL?
Things I have tried:
Load balancer. I cannot figure out how to redirect everything that is not /images/* to the GAE (it seems it only works with backends)
dispatch.yaml. I think it only works for the services you deploy
In App Engine Settings I have added images.myapp.com, but as the redirection of images.myapp.com is done at DNS level, the secure layer is lost
Any suggestion or idea is more than welcome.
Thank you!
The limitation comes from the Cloud Storage bucket presented as a website under a custom domain - you can't use SSL with that. From You want your content served through HTTPS:
SSL is not currently supported by the Cloud Storage webservers; thus,
you can only use a CNAME redirect with HTTP and not with HTTPS. If you
wish to serve content through HTTPS from your bucket, we recommend you
use a third-party Content Delivery Network with Cloud Storage.
Alternatively, you can serve your static website content from
Firebase Hosting instead of Google Cloud Storage.
If the content of your site meets the Code and static data storage quota you can serve your website through GAE, as static content (through a google CDN), which can be mapped to a custom domain and use SSL. See:
Serving Static Files
Hosting a static website on Google App Engine
If your content exceeds the mentioned quota then you can still serve it through your app, but dynamically, with your app accessing the content stored on GCS and serving it - more expensive as you'll be using instance hours for it.
We need to migrate from one app engine project to another (due to the constraints put in place for changing region).
The ideal solution would just be to proxy all requests through to the new server however we are using Google Cloud Endpoints which are intercepted by the server and delivered as POST requests.
We can't redirect as we have mobile apps relying on the API.
Does anyone have a solution (rather than proxying every API method we have) to proxy to a new server?
I would write a ServletFilter on the old app that intercepts /_ah/spi/* and forwards it to the new app, also on /_ah/spi/*. Keep in mind that you'll have to keep the existing Endpoints code in place, or the proxy will delete your configuration and not forward anything.
I'm developing a Java app in GAE, which offers an API through Google Cloud Endpoints.
Basically it receives requests in the endpoints and uses a number of web services from different providers, stores some data and returns some data through the endpoints...
I understand that my app is conceptually a backend, because it doesn't provide any web page, but only the endpoints, don't you think so?
But there's no way to create only a backend, without being associated to any frontend app, is there? At least Google Plugin for Eclipse only allow you to "Generate App Engine Backend", from an existing app, and moreover this app must be an Android project...
I'm using it as a frontend and there's no problem, but apart from the conceptual issue, I've read that backends are kind of optimized to be backends, with more memory and CPU...
I think you're just confused because the Cloud Endpoints documentation uses the word 'backend' to refer to the entire cloud-hosted server implementation. It doesn't specifically refer to the use of GAE backend instances. Endpoint requests can be served by frontend or backend instances, based on how you set them up and the url being accessed.
From the App Ending docs:
"When an application is called to serve a web request, it must issue a response within 60 seconds"
"App Engine Backends are instances of your application that are exempt from request deadlines and have access to more memory (up to 1GB) and CPU (up to 4.8GHz) than normal instances."
So unless you're requests are doing something crazy, you don't need to use a backend. In the google-plugin-for-eclipse, "generate appengine backend" is talking about creating a backend for your android app... a server for your android app to contact (in this case your android app is the frontend and you're appengine app is the backend). In the example app you can remove the web side (index.html) to the appengine application and you'll have no web frontend. Index.html is using the gapi javascript library to make endpoints calls to your appengine service.
I would like to know if there is a way to upload a big file (>500MB) tp Google Cloud Storage passing through a Google App Engine application: I suppose that it is not possible because of the GAE servlet limitation (execution time <=60s).
Is it correct or do you know some new trick?
What do you mean by "passing through" an application?
You can use createUploadUrl and set a destination Google Storage bucket in the UploadOptions and the blob will be written to Google storage rather than blobstore, and your callback will be invoked when the upload is complete.
When you upload a file, it's between your client and the Cloud Storage. There should be no request to your front-end instance hanging until the upload is complete.
That being said, if for some reason your front-end should get involved as a "pass-through" link, there is no limit on a servlet that runs on the backend instance. You can link your upload widget to your backend instance, i.e. call myBackend1.myApp.appspot.com. You can read on how to configure the backend in GAE documentation.
Another option is to use Google Cloud Storage's direct upload mechanism:
https://developers.google.com/storage/docs/reference-methods#postobject
Did you already consider that as an option?