Does App Engine cache responses server side for either dynamic or static requests if I set Cache-Control headers? Documentation doesn't seem to clarify this either way https://cloud.google.com/appengine/docs/standard/php/how-requests-are-handled
I have an API that responds highly cachable responses, so it'd be nice to leverage any edge caching.
You can set the cache in your app.yaml file for static files
- url: /static
static_dir: static
expiration: 10m
You can set a a default cache in your app.yaml file
application: my-app
version: 1
runtime: python27
api_version: 1
threadsafe: yes
module: default
default_expiration: "1h"
instance_class: F2
For caching json/response data from request handlers you can use Memcache
Related
I have two application that I want to deploy with Google Cloud App Engine.
One of them is react front end, and I want to serve this through www.videoo.io
Second one is back-end, which will be served via api.videoo.io
Frontend yaml file react.yaml :
runtime: nodejs16
env: standard
handlers:
- url: /static
static_dir: static
secure: always
- url: www.videoo.io/*
service: frontend
script: auto
secure: always%
API yaml file, api.yaml :
runtime: python37
entrypoint: gunicorn -b :$PORT videoo.wsgi
service: "videoo-api"
env: standard
handlers:
- url: api.videoo.io/*
service: backend
script: auto
secure: always%
Is this the correct way to achieve this ?
What is the best strategy to serve these two separate applications that will interactively communicate (Frontend will make calls to API to get object information that is stored Django app) ?
Here is also my domain name information in Google App Engine settings :
You are on the right path. You are using the microservices architecture which is basically deploying individual apps as parts (services) under a single project.
Your frontend service seems to be your default so you don't need a service name for it. Every GAE App needs a default service.
Rename react.yaml to app.yaml (since it will be your default service) and update the contents to
runtime: nodejs16
env: standard
handlers:
- url: /static
static_dir: static
secure: always
- url: /.*
script: auto
secure: always
Also rename your api.yaml to backend.yaml since that is what you called your service (not sure if this is required but I do that to easily track of what is controlling my service). Update the contents of the file to
service: backend
runtime: python37
entrypoint: gunicorn -b :$PORT videoo.wsgi
env: standard
handlers:
- url: /.*
script: auto
secure: always
You'll need a dispatch.yaml file to route traffic to the different services. Something like
dispatch:
# Send all api traffic to the backend service.
- url: "api.videoo.io/*"
service: backend
# Send all other traffic to the default (frontend).
- url: "*/*"
service: default
Final step is that during your deploy, you will deploy the 2 services in addition to your dispatch.yaml file. The dispatch.yaml file has to be in your project root folder
gcloud app deploy app.yaml dispatch.yaml <path_to_backend.yaml>
If part of my app.yaml file looks like this:
handlers:
- url: /favicon\.ico
static_files: favicon.ico
upload: favicon\.ico
- url: /static
static_dir: public
- url: /.*
secure: always
redirect_http_response_code: 301
script: auto
automatic_scaling:
min_idle_instances: automatic
max_idle_instances: automatic
min_pending_latency: automatic
max_pending_latency: automatic
max_concurrent_requests: 1
min_instances: 1
max_instances: 10
Then is my static content also affected by the scaling parameters for the app? Example, it would run with the same max_concurrent_requests restriction per node, or not?
My assumption is that serving /static would be a completely different layer independent from the instances running for your app in GAE. I was trying to find an architecture diagram confirming this kind of decoupling (maybe a diagram with nginx running with a LB to the GAE Application Instance nodes).
Ideally, a clear answer would be qualified with a reference to Google Cloud documentation material.
Closest related doc I found was this, but it does not clearly answer my question:
Storing and Serving Static Files
Your understanding of the static file serving architecture is correct. App Engine will handle the static file request directly without letting the requests get to the language runtime.
Because of that, these requests will not be affected by the scaling settings the same way as the "regular" requests would. The max_concurrent_requests is a good example of that.
I have requested an update to the documentation page you referenced to add this information there.
I have a flask + react application that is deployed on Google App Engine. Recently, I discovered that each time I deployed a new version to the GAE, my site would go down for a few hours, and several web pages cannot load correctly. I checked the console, the web application is trying to get the static files from the last version, which resulted in a 404 Error. Can anyone help me to find what the problem is?
Here is my app.yaml file:
runtime: python37
env: standard
default_expiration: "5m"
entrypoint: gunicorn -b :$PORT main:app --timeout 150
instance_class: F4
automatic_scaling:
max_instances: 5
min_instances: 1
min_pending_latency: "5s"
target_cpu_utilization: 0.75
inbound_services:
- warmup
handlers:
- url: /static/js/(.*)
static_files: build/static/js/\1
upload: build/static/js/(.*)
- url: /static/css/(.*)
static_files: build/static/css/\1
upload: build/static/css/(.*)
- url: /static/media/(.*)
static_files: build/static/media/\1
upload: build/static/media/(.*)
- url: /(.*\.(json|ico))$
static_files: build/\1
upload: build/.*\.(json|ico)$
- url: /
static_files: build/index.html
upload: build/index.html
- url: /.*
script: auto
I am here to answer my own question. I seem to find the problem and how to solve it.
The main problem seems to be a caching issue. For the app.yaml settings, although the default expiration time is set to 5m, the url with path don't have the expiration set. For example, page www.example.com/about will have a different caching time than the js package. This means when a new build folder is deployed, the js packages have been changed, but the www.example.com/about page generated by your backend application is still the old version, and it will try to request the js package from the previous build foler. Thus, causing the 404 error.
The way to solve this is to set the expiration time for your response generated by your backend application. I am using the Flask environment, so the code for that is (credited to this answer)
response["Cache-Control"] = "no-cache, no-store, must-revalidate" # HTTP 1.1.
response["Pragma"] = "no-cache" # HTTP 1.0.
response["Expires"] = "0" # Proxies.
the web application is trying to get the static files from the last version
So are these files that were removed in your new version that you just deployed?
In general, it sounds like your problem has to do with stale browser caches. I wouldn't remove static assets from your deployed app right away specifically for this reason.
I see you're using ReactJS. Are you using the features that combine all the js and css into a single file whose name contains a hash? This should help with cache-busting.
The part that's confusing is that you said it would go down for a few hours. You have default_expiration: "5m" in your app.yaml so a few hours sounds a bit extreme. Are you not doing a hard reload (or even a full reload) when you are trying to check out your changes in your browser?
I'm running a static site on GAE and using a custom domain (let's call it example.com) with SSL certificates enabled. I'd like to canonicalize URLs to https://www.example.com/. That means catching any requests to myproject.appspot.com, plain HTTP, and/or the naked domain, and redirecting to www over HTTPS.
I understand that it's not possible to put redirect logic in app.yaml, but ideally I'd like to keep the static file serving logic there, and only have app code for the redirect. (As opposed to doing the static serving in app code as well.)
Here's what I have so far:
Contents of the file app.yaml:
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /
static_files: www/index.html
upload: www/index.html
- url: /(.*)
static_files: www/\1
upload: www/(.*)
Contents of the file dispatch.yaml:
dispatch:
- url: "myproject.appspot.com/*"
module: canonicalizer
Contents of the file canonicalizer.yaml:
module: canonicalizer
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: canonicalizer.app
Contents of the file canonicalizer.py:
import webapp2
def get_redirect_uri(handler, *args, **kwargs):
return 'https://www.example.com/' + kwargs.get('path')
app = webapp2.WSGIApplication([
webapp2.Route('/<path:.*>',
webapp2.RedirectHandler,
defaults={'_uri': get_redirect_uri, '_code': 302}),
], debug=True)
As you can see, I've only attempted to implement redirecting myproject.appspot.com so far. I haven't been able to get it working; myproject.appspot.com still serves content rather than redirecting to the custom domain.
I saw a similar SO question and used it as a basis for my code above. I followed it fairly closely, so I'm not sure if it's outdated or missing details.
I'm not very familiar with webapp2. Also open to solutions in a different framework or even different programming language.
As sllopis said in their answer, an HTTP to HTTPS redirect can be implemented via a secure: always element.
The rest of what I wanted to do needed to be done in app code. The code in my answer was on the right track, but I had some confusion about how services work in GAE and about dispatch.yaml. Here's my final code:
<application root>/app.yaml
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /
static_files: www/index.html
upload: www/index.html
secure: always
redirect_http_response_code: 301
- url: /(.*)
static_files: www/\1
upload: www/(.*)
secure: always
redirect_http_response_code: 301
<application root>/dispatch.yaml
dispatch:
- url: "*.appspot.com/*"
service: canonicalizer
- url: "example.com/*"
service: canonicalizer
<application root>/canonicalizer/app.yaml
service: canonicalizer
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: canonicalizer.app
<application root>/canonicalizer/canonicalizer.py
import webapp2
def get_redirect_uri(handler, *args, **kwargs):
return 'https://www.justinforcentral.com/' + kwargs.get('path')
app = webapp2.WSGIApplication([
webapp2.Route('/<path:.*>',
webapp2.RedirectHandler,
defaults={'_uri': get_redirect_uri, '_code': 301}),
], debug=False)
This allows all the redirects to be done while still maintaining the ability to route the static site via static_files handlers.
As an aside, I also didn't realize that simply doing gcloud app deploy . from the application root only deploys the default service. To deploy this whole thing I had to run gcloud app deploy . dispatch.yaml canonicalizer.
Mapping Custom Domains to GAE
App Engine allows applications to be served via a custom domain, such as example.com, instead of the default appspot.com address. You can create a domain mapping for your App Engine app so that it uses a custom domain.
You will need to do the following:
Verify that you are the owner of your domain through Webmaster Central
Ensure that your domain has been verified.
Delegate the ownership of your domain to other users or service accounts, if needed.
Map your domain to your App Engine app.
Fill out the form with the listed resource records, including their type and canonical name (CNAME).
Add this information to the DNS configuration of your domain registrar.
Securing Custom Domains with SSL
By default, when you map your custom domain to your app, App Engine issues a managed certificate for SSL for HTTPS connections. Securing your custom domains with SSL offers more information about this.
Handling URL requests that do not use HTTPS
Any URL handler can use the secure setting, including script handlers and static file handlers. If secure is set to always, requests for a URL that match this handler that do not use HTTPS are automatically redirected to the HTTPS URL with the same path. Query parameters are preserved for the redirect.
Example in the app.yaml file:
handlers:
- url: /youraccount/.*
secure: always
script: auto
Conclusion
As a result, after following these steps, you should have a custom domain properly mapped to your App Engine site that uses SSL certificates to secure the custom domain.
Moreover, by adding the secure:always handler in your app.yaml file, any URL requests made against your App Engine site will be automatically redirected to the HTTPS URL with the same path.
Update - Redirect all URLs with Google App Engine
Credits to How to redirect all URLs with Google App Engine:
app.yaml
handlers:
- url: /.*
script: main.py
main.py
import webapp2
class MainPage(webapp2.RequestHandler):
def get(self):
self.redirect("https://example.com", True)
app = webapp2.WSGIApplication([
('/', MainPage),
], debug=True)
Then, you can adjust this code to your needs.
I would like to host PrivateBin from a GCP App Engine, but I don't know how to configure the app.yaml file. If somebody has done this before, I would appreciate the help. :)
I have tried two options, but neither of them worked:
runtime: php55
api_version: 1
threadsafe: yes
handlers:
- url: /.*
script: index.php
&
runtime: php55
api_version: 1
handlers:
- url: /(.*.(ico$|jpg$|png$|gif$|htm$|html$|css$|js$|xml$))
static_files: \1
upload: (.*.(ico$|jpg$|png$|gif$|htm$|html$|css$|js$|xml$))
application_readable: true
- url: /(.+)
script: \1
- url: /
script: index.php
How do you intend to store your data?
https://github.com/PrivateBin/PrivateBin/blob/master/INSTALL.md#using-a-database-instead-of-flat-files
Looks like private bin uses the filesystem by default, which wont work on GAE Standard. You could do that in GAE flex, but the directory where you are reading and writing to would need to be a mounted Google Cloud Storage Bucket or something for the data to be persistent (as well as be available to multiple instances) https://cloud.google.com/storage/docs/gcs-fuse
The better route seems to be to configure privatebin to use a SQL db, which would be Google Cloud SQL:
https://cloud.google.com/sql/docs/
https://cloud.google.com/php/getting-started/using-cloud-sql-with-mysql