I'm looking for a way to tell Google App Engine (Java) not to serve static resources if they're accessed from a specific region/country. What I'd like to achieve is users to be redirected to a special page saying that the website is currently not available for their country when they try to access it from a disallowed country.
I know it's possible to identify a request origin by means of special headers, but that doesn't apply to static content, which is directly managed by the Google CDN.
In a way you almost answered your own question: the only control you have over the static content (served by the Google CDN) is whatever your app's config file allows you to specify for static content.
And neither app.yaml nor appengine-web.xml support any sort of per-IP address conditioning (neither for static content nor for anything else)
So the only way I can see of achieving what you're looking for on GAE is to serve everything through your app (where you can obtain the request's origin and enforce some policy based on it) instead of through the Google CDN as static content.
Related
We want to map subdomains to static sites hosted in folders within a GCP bucket. From a non-technical perspective, this is identical to what Shopify and other website builders do with their sites for trial customers.
For example:
1. Navigate to foo.rootdomain.com
2. Display site stored inside at <Shared Google Bucket>/foo/index.html
3. Continue displaying foo.rootdomain.com in URL address bar
These subdomains/subdirectories will be created on-demand programatically when customers request them. For example, a customer could request "baz.rootdomain.com", and then we will upload their static site to the shared Google bucket's /baz subdirectory. These subdomains should all be SSL'd. We should be able to support tens to hundreds of thousands of these subdomains, all on the same root domain. That is:
abc.rootdomain.com --> <Shared Google Bucket>/abc/index.html
anothersubdomain.rootdomain.com --> <Shared Google Bucket>/anothersubdomain/index.html
foobar.rootdomain.com --> <Shared Google Bucket>/foobar/index.html
<ANY_TEXT>.rootdomain.com --> <Shared Google Bucket>/<ANY_TEXT>/index.html
I started by looking into Google's load balancers and URL maps to handle these, but AFAICT, these can only map to explicit resources rather - so they could be mapped to a backend bucket which redirects to a specific bucket, but couldn't be mapped to a parameterized bucket based on the subdomain. This SO answer explains how to map wildcard routes to specific resources, which doesn't work for this use-case.
I also looked into hosting a service at Google's App Engine and doing routing via dispatch.yaml because subdomains need to be listed out explicitly and there is a limit of 20 routing rules, so we'd need to keep creating them for every 20 subdomains, which wouldn't scale.
It looks like in this question the author had the same problem and was able to solve it by "creating separated VM instance that actually proxying requests to the google bucket". I looked up 'GCP vm proxy requests to google bucket' and wasn't able to figure out what this means, or how to do it. This doc page talks about using proxies to keep files private, which isn't what we're trying to do, and was very confusing to read through in general.
How can we approach this issue?
How do I direct the www. subdomain to just domain.tld without www? I'm used to firebase doing this automatically. Should I look into configuring the app.yaml, dispatch.yaml, or another method?
What you're describing is called a "naked domain", and this is described in the documentation on Custom Domains. The documentation provides the steps for mapping a custom domain to your app and updating the DNS records at your domain registrar once your service has already been mapped to your custom domain in App Engine.
To redirect your requests, you can use wildcard mappings with services in App Engine by using the dispatch.yaml file. You can find instructions on how to do that here. If you would like to know more about routing requests, you can take a look at this documentation as well which also highlights creating a dispatch file. Handlers are limited to handle URLs by executing application code, or by serving static files uploaded with the code, such as images, CSS, or JavaScript. Therefore, they cannot directly redirect one URL to another.
You would need to handle your URL by running a script that executes code that will redirect your URL.
The comment shows a complete example as the script runs main.py which then redirects the URL
I'd like to be able to access the files in a Cloud Storage Bucket from my App Engine App without making the objects or the bucket itself Publicly Readable. While I'm aware of a bunch of options out there that allow access to bucket objects with authentication (client libraries, signed urls, etc.), the complicating factor is that I'd like to be able to access the files with path that is similar to the folder structure of the bucket in question.
For example, if I make my bucket publicly readable, I can access objects with the public link: https://storage.googleapis.com/MY_BUCKET/FOLDER_IN_MY_BUCKET/FILE_IN_FOLER.txt. This url mimics the internal folder structure of the bucket. However, there doesn't appear to be a comparable url if the bucket is not publicly readable. My App Engine App service account has been added as a storage admin for the bucket I need but I'm not sure if there's a url that I can use to access the buckets objects. An object's mediaLink won't work because generation information is appended to the end, and selfLink results in a 404 error.
The need for a url like this is because the bucket contains several thousand objects. Downloading them using a client library to the the App Engine's persistent storage kind of defeats the purpose of using cloud storage in my case. Obtaining signed urls for all of them when a request is made would be time consuming and then I'd have to manage thousands of signed urls somewhere.
Is there a way to read from the cloud storage bucket with a predictable url, like the public url, while also still authenticating the request?
Rather than trying to vend thousands of signed URLs in the response you can create a 'redirect' endpoint in your app engine app.
e.g. user does a 'GET' against www.myapp.com/fetch/<bucket>/<object>
Your app engine code handling this endpoint authorizes the user to make sure they should have access, pulls the bucket/object out of the URL, then generates a signed URL granting access to the resource and returns a 302 redirect to that URL.
The URL you mention can be accessed without making the bucket or file public given that your browser is authenticated with an account having access to those resources:
https://storage.cloud.google.com/MY_BUCKET/FOLDER_IN_MY_BUCKET/FILE_IN_FOLDER
Regarding the access to the file through a different application (for example App Engine), you can always use the client libraries for your preferred language. You can test how the API works in the documentation, just by defining the bucket parameter as MY_BUCKET and the object parameter as FOLDER_IN_MY_BUCKET/FILE_IN_FOLDER. You should use this same structure when applying it to the client library of your choice.
I'm writing an application that takes a user's Evernote notes and displays them in a website inline. By its very nature, people accessing the resources attached to a note will not be logged in. I'm looking at the bottom of this page and saw how to pass authentication credentials via POST and get the resource. This is exactly what I need.
My question is how does this work in the real world? If I pass authentication tokens to the Javascript client (not secure in the first place), I can't get the resource because of Access-Control-Allow-Origin restrictions. The only other way I can think of doing this is saving all of the resources to my server and serving them from there, but that's not ideal (Google App Engine).
Ideas?
Yea, Evernote does not support CORS yet. You can do it in a chrome extension or get it on the server side.
It seems like static files can only be served from the same domain as the app.
I could create a new app for hosting static files but I'm a little nervous that that would a violation of the terms of service.
You don't need to serve your static content from a different app, you just need to use a different hostname. App Engine makes it pretty easy to have many different hostnames that point to the same app.
With wildcard subdomains you don't even have to create a DNS entry. If your app lives at myapp.appspot.com, you can also reach it through any subdomain, like static.myapp.appspot.com. If you're using your own domain, you'll need to configure it manually.
"4.4. You may not develop multiple Applications to simulate or act as a single Application or otherwise access the Service in a manner intended to avoid incurring fees"
"You may not develop" but, you can "Enable Billing" for multiple application. An example:
1) mysite.appspot.com
2) mysite-static.appspot.com
3) mysite-data-service.appspot.com
Section 4.4 of the terms of service prohibit one from splitting one logical app into two pieces - so hosting your dynamic content and static content in two separate GAE apps would violate the terms.
However, you could host static files on another web hosting service - anything from a simple shared hosting solution all the way up to a big CDN. This approach enables your site to serve static content from domains other than your app's domain.