I have two services default and taskworker that get deployed to app engine's standard environment.
I communicate from default to taskworker via cloud tasks on a few exposed HTTP handlers. E.g: background/check_emails.
I also have a cron job running every minute for background/check_emails.
My deployments of default and taskworker are straightforward e.g:
runtime: python37
service: taskworker
handlers:
- url: /background/.*
script: auto
runtime: python37
handlers:
- url: /static
static_dir: static/
- url: /favicon.ico
static_files: static/img/favicon.ico
upload: static/img/favicon.ico
- url: .*
script: auto
Given that I want to continue getting external traffic to "default" and restrict traffic of "taskworker" from everyone except 1) cron job 2) cloud task http requests:
What are my options?
p.s: I'm not very firewall savvy, and the app engine rules for the project seem to affect the whole project, I do not know how to do a service-based firewall.
You can check for a few headers in your handler to ensure the request is indeed from the Task clouds.
https://cloud.google.com/tasks/docs/creating-appengine-handlers#reading_request_headers
X-AppEngine-QueueName is one of them but you can see the doc for more.
The document specifically says "If your request handler finds any of the headers listed above, it can trust that the request is a Cloud Tasks request."
As of now a "service.based" firewall is not supported for App Engine Standard, the App Engine firewall act upon all the services that comprise your application and it doesn't support granularity for each service.
But there is a Feature Request, please go to the following link in order to check the status of it, notice that there is no ETA or a guarantee that this will be implemented.
As suggested on the Feature Request link shared, the current workaround would consist of migrating your application to App Engine Flexible environment and configure the respective firewall rules on a VPC networks of the Compute Engine instances where your app will reside.
Related
I deployed a Streamlit app to App Engine and configured a custom domain https://datanerd.tech that I purchased through Google Domains.
The problem I'm running into is that when I type in the browser the naked domain datanerd.tech, I am not getting auto re-directed to a secure connection via https://datanerd.tech. Instead, it stays with http://datanerd.tech even though a secure option is available.
This is my app.yaml file:
runtime: custom
env: flex
service: default
I'm unable to use the following in the yaml file to force a secure connection because I need a flexible environment.
handlers:
- url: /.*
secure: always
redirect_http_response_code: 301
script: auto
I'm not as experienced with setting up web servers, and so I'm not sure if this is a Google issue or a Streamlit issue.
EDIT
Here are the rules I have set up for connecting my google domain to App Engine:
App Engine configuration
Google Domain configuration
EDIT 2:
I attempted domain forwarding from my Google Domain account using https://datanerd.tech and the url no longer worked due to "To many redirects". I even tried cycling the different options of 'Redirect Type', 'Path Forwarding', and 'SSL' with no luck.
I was NOT able to solve this issue using App Engine. I was able to get a partial solution for www.datanerd.tech using #NoCommandLine's solution.
Instead, Since I wasn't dependent on App Engine; I switched services to Cloud Run and was able to map the custom domain with no issues.
Thanks again for your patience #NoCommandLine.
Login to Google Domains and select your domain
Click on 'DNS' on the (left hand side) of the page and on the right hand side of the page, look for the Domain Forward section
Add an entry to forward http://datanerd.tech to https://datanerd.tech
Update
This is what shows up for https://datanerd.tech. http://datanerd.tech also works
I have a python app engine that handles api results and it's stateful. However it seems that after a few hours of inactivity (no requests), the server shuts off, resetting all states, and when a new request is made, it's listening again.
But the states are reset. I want the server to actively remain unchanged 24/7 and not reset/restart as I want to maintain states.
I have configured as per documentation but it's still restarting, I am not sure what's wrong
Here is my app.yaml:
runtime: python37
entrypoint: python main.py
manual_scaling:
instances: 1
In App Engine the general recomendation is to create stateless applications as mentioned on the documentation
Your app should be "stateless" so that nothing is stored on the instance.
As an alternative for the application not to get restarted you can deploy it on Compute Engine, As that service is a Virtual Machine you can have total control of the states.
I am trying to force https for all traffic to an app hosted on Google Cloud AppEngine. https works, but despite following the instructions for rewriting http traffic to https, it's still possible to access the site with http, which causes problems.
I have added this to the app.yaml:
handlers:
- url: /.*
script: _go_app
secure: always
redirect_http_response_code: 301
but it doesn't seem to make any difference.
I am using the julienschmidt router and then this to handle all routes:
log.Fatal(fmt.Println(http.ListenAndServe(":8080", router)))
I have looked at using http.ListenAndServeTLS but this takes extra parameters and I can't work out what the values of those should be in the Google AppEngine context.
log.Fatal(fmt.Println(http.ListenAndServeTLS(":8443", "cert.pem", "key.pem", router)))
Where are "cert.pem" and "key.pem"?
I've read that I don't need to explicitly serve TLS in my app, because AppEngine will handle it for me, so even if I knew what the parameters were, I'm not sure it would help in forcing https.
http://sapling.appspot.com
https://sapling.appspot.com
http://sapling.money
https://sapling.money
All of the above work, but I don't seem able to force either of the http versions to https.
If you are using the secure:always handler and the requests are not being automatically redirected, then you are probably using App Engine Flex?
App Engine Flex does not support handlers, you can see this in the Flexible app.yaml documentation.
Instead, you can check in your code if a request was sent via HTTP or HTTPs and redirect. This is done with the App Engine specific header X-Forwarded-Proto.
The implementation is configured on your end and there is a brief paragraph on the subject.
You can also view similar Stack posts with the same answers 1
I hope this helps!
You are correct for including the secure: always element in your app.yaml as this will force HTTPS for your app's handlers.
However, I believe the following App Engine Documentation for "Securing Your App" found here may be of some use to you. As indicated in that link, you can convert HTTP URLs to HTTPS by simply replacing the periods between each resource with a -dot- instead. You may see the example provided below.
http://[SERVICE_ID].[MY_PROJECT_ID].appspot.com
https://[SERVICE_ID]-dot-[MY_PROJECT_ID].appspot.com
For additional information about HTTPS URLs and resource targeting you may see how requests are routed here.
Hope this helps!
You can use the Strict-Transport-Security header to instruct the browser to prefer https over http for a given page or an entire domain as outlined in this document. In order to add HTTP Strict-Transport-Security headers (HSTS) to your app, you must implement the headers within your app's code, not within your app's config file (app.yaml or appengine-web.xml).
It is also a good idea to enable HSTS preloading if you register your application with Google's HSTS preload list. Firefox and Chrome will never load your site over a non-secure connection.
I have a frequent cron job in my app on google app engine standard, and it's using tons of instance hours to perform a quick task. I find that the instance hours problem goes away if I switch the app to F1 in app.yaml, but the web front-end needs more power (been using F4_1G).
It seems like a simple solution would be to use App Engine Modules to run the cron job on F1 while keeping the rest of the app on F4_1G, but the documentation is short on actual code. Can somebody please show how this can be accomplished?
This doesn't actually require code changes, it's controlled by your projects configuration (yaml) files.
You create a service (formerly module) by specifying it in a separate .yaml file, deploying the service, and then telling your cron job to run on that service.
Let's assume you want to create a service called "lightweight".
Start by copying your existing app.yaml to lightweight.yaml,
add (or modify) the "service" line to read "service: lightweight", and update the application instance to f1.
Optionally, clean up the handlers so that only the ones you need for your cron instance are present
eg, lightweight.yaml:
application: yourapp
service: lightweight
version: 0-4
runtime: python27
api_version: 1
threadsafe: true
instance_class: F1
handlers:
- url: /mycronjob
script: main.app
login: admin
Then, in your cron.yaml, specify the service as your target.
cron:
- description: example
url: /mycronjob
schedule: every 5 minutes
target: lightweight
Once that is done, deploy lightweight.yaml and cron using gcloud or appcfg.
Once deployed, your cron job will run on the lightweight service, using an f1 instance. You can also access the lightweight service directly in your browser lightweight.yourapp.appspot.com
Google is now offering, in beta, a managed security service for automated deployment of SSL certificates to GAE apps, which are currently signed by Let's Encrypt. The managed security service is a really good idea, particularly for Windows users, who cannot easily generate SSL certificates through the Let's Encrypt service.
I currently have one custom domain mapping to my GAE app plus its www alias (which means that both 'customdomain.com' and 'www.customdomain.com' map to my app). Also, I have enabled managed security for both of them.
Unfortunately, an HTTPS connection is enforced only if one accesses my app using the www alias (www.customdomain.com). If one does not determine a subdomain (customdomain.com), the connection served is insecure. Of course, one can enforce an HTTPS connection (by adding 'https://' before 'customdomain.com').
Why is that happening? Is that a service bug?
As of September 2017, Google Cloud Platform has introduced Managed SSL which should implement SSL to your custom domain by default.
Below is an extract from this instructional post:
Now, when you build apps on App Engine, SSL is on by default — you no
longer need to worry about it or spend time managing it. We’ve made
using HTTPS simple: map a domain to your app, prove ownership, and App
Engine automatically provisions an SSL certificate and renews it
whenever necessary, at no additional cost. Purchasing and generating
certificates, dealing with and securing keys, managing your SSL cipher
suites and worrying about renewal dates — those are all a thing of
the past.
You can specify the paths that you wish to be secured inside the app.yaml:
handlers:
- url: /.*
script: main.app
secure: always