Google App Engine -- Deploying a new version will make my site go down - reactjs

I have a flask + react application that is deployed on Google App Engine. Recently, I discovered that each time I deployed a new version to the GAE, my site would go down for a few hours, and several web pages cannot load correctly. I checked the console, the web application is trying to get the static files from the last version, which resulted in a 404 Error. Can anyone help me to find what the problem is?
Here is my app.yaml file:
runtime: python37
env: standard
default_expiration: "5m"
entrypoint: gunicorn -b :$PORT main:app --timeout 150
instance_class: F4
automatic_scaling:
max_instances: 5
min_instances: 1
min_pending_latency: "5s"
target_cpu_utilization: 0.75
inbound_services:
- warmup
handlers:
- url: /static/js/(.*)
static_files: build/static/js/\1
upload: build/static/js/(.*)
- url: /static/css/(.*)
static_files: build/static/css/\1
upload: build/static/css/(.*)
- url: /static/media/(.*)
static_files: build/static/media/\1
upload: build/static/media/(.*)
- url: /(.*\.(json|ico))$
static_files: build/\1
upload: build/.*\.(json|ico)$
- url: /
static_files: build/index.html
upload: build/index.html
- url: /.*
script: auto

I am here to answer my own question. I seem to find the problem and how to solve it.
The main problem seems to be a caching issue. For the app.yaml settings, although the default expiration time is set to 5m, the url with path don't have the expiration set. For example, page www.example.com/about will have a different caching time than the js package. This means when a new build folder is deployed, the js packages have been changed, but the www.example.com/about page generated by your backend application is still the old version, and it will try to request the js package from the previous build foler. Thus, causing the 404 error.
The way to solve this is to set the expiration time for your response generated by your backend application. I am using the Flask environment, so the code for that is (credited to this answer)
response["Cache-Control"] = "no-cache, no-store, must-revalidate" # HTTP 1.1.
response["Pragma"] = "no-cache" # HTTP 1.0.
response["Expires"] = "0" # Proxies.

the web application is trying to get the static files from the last version
So are these files that were removed in your new version that you just deployed?
In general, it sounds like your problem has to do with stale browser caches. I wouldn't remove static assets from your deployed app right away specifically for this reason.
I see you're using ReactJS. Are you using the features that combine all the js and css into a single file whose name contains a hash? This should help with cache-busting.
The part that's confusing is that you said it would go down for a few hours. You have default_expiration: "5m" in your app.yaml so a few hours sounds a bit extreme. Are you not doing a hard reload (or even a full reload) when you are trying to check out your changes in your browser?

Related

How to define envs for GAE?

I created web app that consist with two separate parts: server and client. And I want to deploy it to google cloud with app engine.
I already did it, but the main problem is unworking env_variables that I need for correct work of application that I defined in .yaml files.
I don't understand why the env variables doesn't work, If I do it in accordance to docs.
I wrote .yaml file to each part. Here the file structure:
./
--client/
----source_files...
----client.yaml
--server/
----source_files...
----api.yaml
--dispatch.yaml
Here the content of each file.
client.yaml:
runtime: nodejs16
service: default
handlers:
- url: /(.*\..+)$
static_files: build/\1
upload: build/(.*\..+)$
- url: /.*
static_files: build/index.html
upload: build/index.html
env_variables:
API_LINK: "https://gcloudezample-11111.lm.r.appsport.com"
api.yaml:
runtime: nodejs16
service: api
env: standard
env_variables:
MONGO_DB_PWD: "db_password"
dispatch.yaml:
dispatch:
- url: '*/api'
service: api
Then I deployed this parts in .yaml files order above.
The result is working frontend but failed api requests, because the wrong request URL: http://localhost:3001/v1/my-route
Screenshot
It's a common misunderstanding of how a website works. Take 2 sec and think about it: What are doing your frontend part?
Let me help
handlers:
- url: /(.*\..+)$
static_files: build/\1
upload: build/(.*\..+)$
- url: /.*
static_files: build/index.html
upload: build/index.html
It serves only static files. Therefore, only static file means no processing, no instances, no variable definition in the runtime context because only static files are served.
The runtime is on the client browser that run the JS code.
Note: the counterpart great advantage, that the static page serving is free (in term of resource usage because there is no instance up and running to serve your files; the egress is billed)
So now, how to solve this?
Simply build your static files with the API_LINK value INSIDE the static code.

Why does Google App Engine Standard Edition for Java create a __static__ folder and double the number of files deployed to a project?

I am deploying a web application to Google App Engine Standard edition using Java 8.
It is a documentation web site plus servlets that includes a huge amount of static web content (built with Jekyll).
I recently started getting this error trying to deploy the app:
ERROR: (gcloud.app.deploy) INVALID_ARGUMENT: This deployment has too many files.
New versions are limited to 10000 files for this app.
- '#type': type.googleapis.com/google.rpc.BadRequest
fieldViolations:
- description: This deployment has too many files.
New versions are limited to 10000 files for this app.
field: version.deployment.files[...]
My src/main/webapp folder has 5016 files.
But after running:
mvn clean install appengine:stage
my target/appengine-staging folder has 10113 files.
I see that the files are in 2 locations, both
target/appengine-staging
and
target/appengine-staging/__static__
Why do they appear twice?
Is this normal?
Or am I doing something wrong?
How can I avoid this duplication?
Or do I have to limit my files to under 5000?
Thanks for any advice.
Note: The generated app.yaml looks like this:
runtime: java8
instance_class: F1
inbound_services:
- warmup
derived_file_type:
- java_precompiled
threadsafe: True
auto_id_policy: default
beta_settings:
'source_reference': 'https://...'
api_version: 'user_defined'
handlers:
- url: (/.*/)
static_files: __static__\1index.html
upload: __NOT_USED__
require_matching_file: True
login: optional
secure: always
- url: (/)
static_files: __static__\1index.html
upload: __NOT_USED__
require_matching_file: True
login: optional
secure: always
- url: (/.*)
static_files: __static__\1
upload: __NOT_USED__
require_matching_file: True
login: optional
secure: always
- url: /.*
script: unused
login: optional
secure: always
skip_files: app.yaml

GCloud Error: Not Found The requested URL was not found on this server

I followed this steps to deploy my angular 8 application in Google Cloud
https://medium.com/#karthiksagar/how-to-deploy-angular-8-app-on-google-cloud-platform-gcp-cdd79e5cc5cf
My app.yaml is as follows
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /
static_files: sofbox-angular/index.html
upload: sofbox-angular/index.html
- url: /
static_dir: sofbox-angular
I have placed the app.yaml file inside dist/ directory as mentioned.
I am able to access the main url- http://www.myridemate.com/
but when clicked on blog the link is not accessible - http://www.myridemate.com/blog
It gives below error
Error: Not Found The requested URL /blog was not found on this server.
Tried adding extra handler for /blog in app.yaml and it didn't work.
This same code is working in my local and also in hostgator - https://www.myridemate.alexvijayraj.com/blog
Not sure what I am missing here. Is there some extra configuration required in app.yaml or anything in GCP?
Error Message in App Engine Logs:
{
"time": "2020-09-17T03:58:28.682252Z",
"severity": "WARNING",
"logMessage": "Static file referenced by handler not found: sofbox-angular/blog"
}
The answer in this thread worked for me. Angular 6 routes not found on Google App Engine
Remember to remove the two handlers that are already on the .yaml file (-url: ...)
handlers:
- url: /(.*\.(js|css|svg|png)(|\.map))$
static_files: dist/\1
upload: dist/(.*)(|\.map)
- url: /.*
static_files: dist/index.html
upload: dist/.*

Google Endpoints, the API explorer works locally but does not work after deployment

worked perfectly locally but the explorer is stuck in "Loading.." when deployed. Steps I took so far:
cleared browser cache
checked the logs, I have no errors and i DONT see a
"/_ah/spi/BackendService.getApiConfigs 200 " request, so thats a
issue and I dont know how to fix.
The link:
https://.appspot.com/_ah/spi/BackendService.getApiConfigs
results in: {"state": "APPLICATION_ERROR", "error_message": "Not
Found"}
Please point me in the right direction to debug this. thanks in advance
my app.yaml file
application: appID
version: 1
runtime: python27
api_version: 1
threadsafe: true
handlers:
# Endpoints handler
- url: /_ah/spi.*
script: the_api.APPLICATION
- url: /favicon\.ico
static_files: favicon.ico
upload: favicon\.ico
- url: /js
static_dir: static/js
- url: /.*
script: the_web_page.app
secure: always
libraries:
- name: webapp2
version: "2.5.2"
- name: endpoints
version: 1.0
- name: jinja2
version: latest
- name: pycrypto
version: latest
I had a simple error. To fix I changed the line
- url: /_ah/spi.*
in the app.yaml file to
- url: /_ah/spi/.*
what I find hard to explain is why the first worked in localhost.
I also had the yellow "Loading" status in my API Explorer on deployment and I was struggling to understand why - I had not changed any of my code but suddenly the API explorer stopped working.
All I did was to clear out my internet explorer temporary files, cookies and website data and it worked again.
This happens to me whenever I use Opera.i don't know if this is what you are using but try another browser.
Make sure you are using HTTPS. Endpoints in production only work using a secure connection.
It suddenly stopped working.
When I click on the Endpoint, it just change the url in the address bar of the browser but doesn't load it.
So I copied the url from the address bar and loaded the url directly and it worked.

How do i run FuelPHP on Google App Engine

Hi i have setup an app engine account. everything works ok with standard php. i have a local install of fuelphp, this works with no issues when running the development server of google app engine. however once i move it to the production application, i get an error stating the cache directory is unwritable of non existent. i read through some articles and it mentioned the application sandbox on GAE is not writable, so i went and manually change the configurations to point to an external path for the cache, the issues is this persist. i even tried to remove the usage of cache, ( false in config ) and hardcoded remove by commenting out. this just crashed everyting.
app.yaml
env_variables:
FUEL_ENV: 'production'
handlers:
url: /(.*.(htm$|html$|css$|js$))
static_files: public/\1
upload: public/(.*.(htm$|html$|css$|js$))
application_readable: true
url: /(.*.(ico$|jpg$|png$|gif$))
static_files: public/\1
upload: public/(.*.(ico$|jpg$|png$|gif$))
application_readable: true
url: /assets
static_dir: public/assets
url: /favicon.ico
static_files: public/favicon.ico
upload: public/favicon.ico
url: /.*
script: public/index.php
Log file patch: https://github.com/isatan/FuelPHP4Gae_patch
Deploy SQL instance, load database schema and setup your production config.php to connect to in via unix socket.

Resources