I was unable to access my Backend Instance at custom domain.
For example, I have an app and I access the Normal Instance sucessfully at:
http://www.[my_app_id].appspot.com or http://[my_app_id].appspot.com
And I have a backend config name=test and I accessed Backend Instance successfully at:
http://test.[my_app_id].appspot.com
In admin interface, the "Instances" link show the instances of Backend and Normal Instance separately. The content show is the same, but is easy to see when a request go to the Backend Instance and when go to Normal Instance.
Then I configured the wildcard "test" in Google Apps to access my Backend Instance at a custom URL:
I continue access the Normal Instance sucessfully at:
http://www.[my_domain].com or http://[my_domain].com
But request at
http://test.[my_domain].com
redicted to the Normal Instance instead of Backend Instance.
The doc's said it should work but I cann't at this moment and I need uses custom domain because my app is multitenancy.
What I do wrong?
Your backed is really supposed to be accessed by the front end, as I understand it.
So when your application front end makes a request to it's back end (e.g. via a URL), it'll work as it's all done internally.
Have you set your back end to be publicly accessible?
https://developers.google.com/appengine/docs/python/backends/overview#Public_and_Private_Backends
Backends are private by default, since they typically function as a component inside an application, rather than acting as its public face. Private backends can be accessed by application administrators, instances of the application, and by App Engine APIs and services (such as Task Queue tasks and Cron jobs) without any special configuration. Backends are not primarily intended for user-facing traffic, but you can make a backend public for testing or for interacting with an external system.
I don't know why the redirection is not working, but perhaps you should modify your question to show what problem it is you are trying to solve here and get an answer to that instead?
Related
In Google App Engine, I have 3 services, 1 for front end, 2 for back end.
Is there a way to block http calls to my backend services for accounts not from my company's domain (and the service account of the front end), but allow everyone http access to my front end service?
I know there is the firewall option, but this is restricted to IP addresses, I would prefer user based
If it matters all services are python3
There's currently no option to filter traffic to specific App Engine services within a single application/project:
App Engine Firewall filters by source IP ranges but can only be set for the whole app, not per service.
Identity-Aware Proxy can filter access by user account as you'd prefer but also applies to the whole app. Also, it only supports user account and can't be used with service accounts.
One option you may have would be to split your app in 2 different projects. Keep the front-end in one project open to the world and restrict access to the backend services in your other project via firewall rules.
I have seen the following being used in task queues in GAE. Maybe it would help.
If u were using python 2, in standard environment, i think u could have used login handler element in app.yaml file.
You could have added following lines to your app.yaml file:
handlers:
- url: /.*
script: worker.app
login: admin
This prevents other users from accessing this service.
But the same login handler is not available for python3, according to Google Docs.
Just found following in Google Docs:
If a task performs sensitive operations (such as modifying data), you might want to secure the handler URL to prevent a malicious external user from calling it directly. You can prevent users from accessing task URLs by restricting access to App Engine administrators. Task requests themselves are issued by App Engine and can always target a restricted URL.
You can restrict a URL by adding the login: admin element to the handler configuration in your app.yaml file.
You can also call your backend services through cloud tasks or task queues (both are almost the same i guess), in case this only work for cloud tasks.
Find the code usage here:
https://github.com/GoogleCloudPlatform/python-docs-samples/tree/6f5f3bcb81779679a24e0964a6c57c0c7deabfac/appengine/standard/taskqueue/counter
Find details about handler here.
https://cloud.google.com/appengine/docs/standard/python/config/appref#handlers_element
Find details about Cloud task and queue here:
https://cloud.google.com/appengine/docs/standard/python/taskqueue/push/creating-handlers
I want to access datastore (and storage) data of an AppEngine project via google-cloud-datastore and google-cloud-storage with an Python program on my own server.
This works with my AppEngine staging server, by creating a service account and giving it owner access (to the project).
Doing the same thing with the production AppEngine instance fails with
google.api_core.exceptions.PermissionDenied: 403 Missing or insufficient permissions.
Part of the problem might be, that I might be using the wrong project to create the service account with. There are more than one project with the same name in my cloud console. How do I identify the correct one?
How do I get more details about the problem?
First, note that the Datastore and the Cloud Storage are 2 different products with 2 different accessing methods.
The Datastore is closely tied to the GAE project - each project has its own datastore. The external access procedure in general is captured in How do I use Google datastore for my web app which is NOT hosted in google app engine?.
When switching the project (staging to production in your case) there are 2 things to keep in mind:
as you observed, you need to change the project you're accessing.
you also need to change the credentials you load and use for access to match the project you select, as each project has it own service account key configured in the above-mentioned procedure
For the google-cloud-datastore library both of these are simultaneously configured via the datastore.Client() call parameters (emphasis mine):
class google.cloud.datastore.client.Client(project=None,
namespace=None, credentials=None, _http=None, _use_grpc=None)
project (str) – (Optional) The project to pass to proxied API methods.
credentials (Credentials) – (Optional) The OAuth2 Credentials to use for this client. If not passed (and if no _http object is passed),
falls back to the default inferred from the environment.
The Cloud Storage is completely independent from GAE, the GAE project/credentials you use (if any) have no bearing on bucket/object access restrictions whatsoever. There's nothing you need to do from the google-cloud-storage library perspective when switching from one GAE project to another
To eliminate the confusion created by multiple projects having the same name just go to the IAM & admin Settings page, select the respective projects from the drop-down list on the top blue bar and rename them using meaningful names (click in the Project name box to edit the name, then click SAVE). Then re-check if you're using the right keys for the desired project.
Maybe this is a really basic question, but how do you architect your system such that your single page application is hosted on premise with some hostname, say mydogs.com but you want to host your application services code in the cloud (as well as database). For example, let's say you spin up an Amazon EC2 Container Service using docker and it is running NodeJS server. The hostnames will all have ec2_some_id.amazon.com. What system sits in from of the Amazon EC2 instance where my angularjs app connects to? What architecture facilitate this type of app? Especially AWS based services.
One of the important aspects setting up the web application and the backend is to server it using a single domain avoiding cross origin requests (CORS). To do this, you can use AWS CloudFront as a proxy, where the routing happens based on URL paths.
For example, you can point the root domain to index.html while /api/* requests to the backend endpoint running in EC2. Sample diagram of the architecture is shown below.
Also its important for your angular application to have full url paths. One of the challenges having these are, for routes such as /home /about and etc., it will reload a page from the backend for that particular path. Since its a single page application you won't be having server pages for /home and /about & etc. This is where you can setup error pages in CloudFront so that, all the not found routes also can be forwarded to the index.html (Which serves the AngularJS app).
The only thing you need to care about is the CORS on whatever server you use to host your backend in AWS.
More Doc on CORS:
https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS
Hope it helps.
A good approach is to have two separated instances. It is, an instance to serve your API (Application Program Interface) and another one to serve your SPA (Single Page Application).
For the API server you may want more robust service because it's the one that will suffer the most receiving tons of requests from all client instances, so this one needs to have more performance, band, etc. In addition, you probably want your API server to be scalable when needed (depends on the load over it); maybe not, but is something to keep in mind if your application is supposed to grow fast. So you may invest a little bit more on this one.
The SPA server in the other hand, is the one that will only serve static resources (if you're not using server side rendering), so this one is supposed to be cheaper (if not free). Furthermore, all it does is to serve the application resources once and the application actually runs on client and most files will end up being cached by the browser. So you don't need to invest much on this one.
Anyhow, you question about which service will fit better for this type of application can't be answered because it doesn't define much about that you may find the one that sits for the requisites you want in terms of how your application will be consumed by the clients like: how many requests, downloads, or storage your app needs.
Amazon EC2 instance types
I am work on a web application as an interface with Google Cloud Storage(GCS).
I am using a backend service to retrieve the list of files I stored on GCS and their URL with the JSON API and return that to my web application. However, I was not really able to load the files through those URL, which always came back with 403 forbidden.
I am not sure how GCS authentication work behind the scene and whether it is possible to directly grant access to web application. I am not sure how could I attach application authentication information via http request. What I know is I can do that via the backend service but for the reason of simplicity I wonder if it is possible to get around with that. One of the thing I tried is adding the web application domain(which will be sent via referrer in http request) into ACL to that bucket, which doesn't work at all.
And thanks to what #Brandon pointed out below. I am ok to grant anyone whoever have access to the application to view the content of the GCS since it is an internal app and I have already checked their authentication when I first serve the web application.
====
Solution
I ended up using the signedUrl that expire in 5 minutes and I highly recommend interact with gcs using gcloud (Their python document is really good). Thanks again for the thorough answer!
You have a user on a web browser who wants to download an object that only your application's service account has read access for. You have a few options:
Expand access: make these object publicly readable. Probably not the best choice if this info is sensitive, but if it's not, this is the easiest solution.
Give your app's credentials to the user so that they can authenticate as your app. This is a REALLY bad idea, and I probably shouldn't even list it here.
When a user wants to download a file, have them ask your app for it, and then have your app fetch the file and stream its contents to the user. This is the easiest solution for the client-side code, but it makes your app responsible for streaming file contents, which isn't really great.
When a user wants to download a file, have them ask your app for permission, and reply to them with some sort of token they can use to fetch the data directly from GCS.
#4 is what you want. Your users will ask your app for a file, your app will decide whether they are allowed to access that file via whatever you're doing (passwords? IP checks? Cookies? Whatever.) Then, your app will respond with a URL the user can use to fetch the file directly from GCS.
This URL is called a "signed URL." Your app uses its own private key to add a signature to a URL that indicates which object may be downloaded by the bearer and for long the URL is valid. The procedure for signing URLs is somewhat tricky, but fortunately the gcloud storage libraries have helper functions that can generate them.
I am developing a GWT application on google app engine and I am looking for the best approach to initialize objects (like singleton, list, shared resources etc).
I guess I am looking for something like "Spring application context file"
any ideas?
What you're looking for is here:
http://code.google.com/appengine/docs/java/config/appconfig.html#Using_a_ServletContextListener
Basically, you're going to make a Servlet Context Listener, which is a part of the servlet API designed for exactly what you're referring to. If you're running this locally, it will run when you start your server. In the app engine environment it should run for every warm up request (to avoid, this, you can use "Always ON", which will be set here: http://code.google.com/appengine/docs/adminconsole/instances.html#Always_On)
Besides ServletContextListener you can also use <load-on-startup> to mark your normal servlet to be invoked during a warmup request.