Our website is hosted on Google App Engine: https://www.boutir.com/
Without any code/dns/config changes today, the static files like .js/.css suddenly fail to load. The network inspector shows that the file is pending forever. Occasionally though the files would load successfully. How do we solve the issue?
It is interesting to note that if we use the PROJECT_ID.appspot.com domain, there would be no such issues.
Our app.yaml looks something like this:
runtime: python27
handlers:
- url: /js
static_dir: pages/js
secure: always
Similar issue reported in the past was observed to be related to the DNS configuration of the Custom Domain. Particularly since you confirm that the Static Contents are being served as expected with the appspot domain.
With that being said, there was an internal GCP issue reported earlier suggesting that requests for App Engine endpoints that serve static content may be failing with Customers seeing ”204 No Response” with 0 bytes data, or request hangs indefinitely. I have information that this issue is now resolved, however.
Related
I am deploying a Next.js app on GAE and none of the images in the public folder get served, they all return 500 with the following error after reviewing logs
EROFS: read-only file system, unlink '/workspace/.next/cache/images/zUb0XWtfOUy8jJS-olwIcsJpEse7wovGYRmT3B79mCc=/0.1641776976898.S-2U16B+8WquUZAAJsKsf1k9FpDiuBJX2T6gFU5eXy4=.svg
Here is how my app.yaml file looks like:
---
handlers:
- url: /.*
secure: always
script: auto
manual_scaling:
instances: 1
runtime: nodejs16
I run next build to get a production build, however none of the images load. Please help
It looks like GAE (Google App Engine) does not allow for writing, you can read them but its probably trying to perform some transcoding in the request, e.g: convert to svg
One potential way maybe to host the images in an external URL or use a host like imgix or cloudinary
https://nextjs.org/docs/api-reference/next/image#domains
We have a web application (frontend) using React created from Create React App. The app is running on Google Cloud Platform App Engine Standard. Our web application is code splitted. Each page is then loaded on user navigation.
It's working really well. The issue we have is for example user A is on the app home page. We deploy a fix that change the chunk file name. The user A then try to access another page and then got the error Loading chunk * failed. The url to get the file is now returning a 404 because the file has been replaced by some new chunk files.
It's a frequent problem as I can see during my research but I didn't find a solution that apply for Google App Engine.
Here's an article that explain the problem / solution: https://mitchgavan.com/code-splitting-react-safely/
I would like to use the solution "Solution 1: Keep old files on your server after a deployment" but I can't see how to do this using GCP ...
Here's the app.yaml file
service: frontend
runtime: nodejs14
env: standard
instance_class: F1
handlers:
- url: /(.*\..+)$
static_files: build/\1
upload: build/(.*\..+)$
- url: /.*
static_files: build/index.html
upload: build/index.html
We have the following dispatch file (* for masked url)
dispatch:
- url: "*"
service: frontend
- url: "www.*"
service: frontend
Haven't tried this before but see if it makes sense/works.
We have a blog article about downloading your source code from GAE. It contains an explanation of where your source is stored when you deploy (a staging bucket), how long it stays there and how you can modify how long it stays before Google automatically deletes it.
When you deploy to GAE, gcloud only deploys files that have changed (it doesn't deploy those that haven't). Since you now have 'new' files because new hashes were generated, the older files no longer exist on your local machine. I do not know if Google will automatically delete those files from the staging location in bullet 1 above.
My proposal here is that you follow the steps in the blog article (from bullet 1) and alter (change) how long the files are retained in your staging bucket. Another option is to check the retention policy tab and see if you can change the rule so the files don't get deleted. If you're able to alter how long the files remain or the retention policy, it just might solve your problem. Let me know if this works
I am using Google App Engine Standard Environment for a node.js app.
I have a folder named public for my static files. This contains files such as chunk-XIAWB55V.js. I want these files to be heavily cached as the hashed filename will change if the contents change.
I added a static file handler to my app.yaml to redirect requests starting /static to my public folder, and I confirmed it worked by checking the logs: requests to e.g. https://<redacted>.com/static/javascript/build/prod/chunk-Z4M5HAC7.js are in the request log and are not hitting my app. The docs imply that a default cache of 10 minutes should be applied. However the browser devtools show that the actual headers are cache-control: no-cache, must-revalidate and a request (which gets a 304) is sent every time.
I have tried adding a expiration param to my app.yaml but it did not make a difference. I also tried setting the headers in app.yaml but it did not make a difference. My current file looks like:
handlers:
- url: /static
static_dir: public
secure: always
expiration: "1d"
http_headers:
Cache-Control: public
Vary: Accept-Encoding
- url: /.*
secure: always
script: auto
I am still getting responses like:
I am using Identity Aware Proxy in front of App Engine and I know from e.g. this question that IAP and caching can mix badly but that seems to refer simply to possible IAP bypass. My static files are not sensitive and I am happy to accept that risk.
GCP support have suggested this is an edge case due to the use of IAP. A workaround I have implemented which I can recommend for anyone else in this situation is to remove the static route and serve the files from your app instead. While this increases load on the app, the high max age means they will not be requested often.
There's some documentation of this at : This page describes best practices for using Identity-Aware Proxy (IAP) https://cloud.google.com/iap/docs/concepts-best-practices -- TL;DR, App Engine does some caching for static_files that interacts poorly with IAP. That page has some instructions you can apply if you want to protect your static_files.
How do we stop App Engine from using caches of previously loaded files?
According to Google's documents here under "App Caching", they suggest that App Engine caches files that are loaded. They seem to suggest that it is related to the presence of their main() function. However, we removed all references to main() and find that App Engine is still using cached files.
We also tried this HTML below per this answer, but are still getting cached results in the browser. (When we return to the home page - the tables which load JSON are not getting the new file results - they are receiving the old versions of the files.)
<meta http-equiv="cache-control" content="no-cache, no-store, must-revalidate">
<meta http-equiv="Pragma" content="no-cache">
<meta http-equiv="Expires" content="0">
Here is our app.yaml
application: OurApp
version: 0.5
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /stylesheets
static_dir: stylesheets
handlers:
- url: /scripts
static_dir: scripts
- url: /.*
script: main.app
Any suggestions for how to prevent this file load caching behavior in App Engine?
If the files you are trying to update are static files, such as those inside the /stylesheets or /scripts folders from your project, you should take a look at this doc for static cache expiration for GAE projects in the standard environment for Pyhton. According to it, "files are likely to be cached by the user's browser, as well as by intermediate caching proxy servers such as Internet Service Providers". But I've found a way to flush static files cached by your app on Google Cloud.
Head to your Google Cloud Console and open your project. Under the left hamburger menu, head to Storage -> Browser. There you should find at least one Bucket: your-project-name.appspot.com. Under the Lifecycle column, click on the link with respect to your-project-name.appspot.com. Delete any existing rules, since they may conflict with the one you will create now.
Create a new rule by clicking on the 'Add rule' button. For the object conditions, choose only the 'Newer version' option and set it to 1. Don't forget to click on the 'Continue' button. For the action, select 'Delete' and click on the 'Continue' button. Save your new rule.
This new rule will take up to 24 hours to take effect, but at least for my project it took only a few minutes. Once it is up and running, the version of the files being served by your app under your-project-name.appspot.com will always be the latest deployed, solving the problem. Also, if you are routinely editing your static files, you should remove any expiration element from handlers related to those static files and the default_expiration element from the app.yaml file, which will help avoid unintended caching by other servers.
The section of the doc that you cite has to do with how python files on the server side are handled. It's a server-side optimization, and is entirely separate from how a browser caches HTML from your application.
You can specify additional HTTP headers (e.g., Cache-Control) in app.yaml. See the Static File Handlers section in the doc. You can also add headers dynamically via the self.response.header. This is discussed, with a small example, in the Responses section of the doc.
I'm getting an error "This webpage has a redirect loop" when loading my Google Apps Engine url: http://my-application-id.appspot.com (my real application ID in the url of course)
Google Apps Engine requires an app.yaml file in the root of the application directory to work correctly. Here are the contents of my file....
application: my-application-id
version: 1
runtime: php
api_version: 1
handlers:
- url: /.*
script: install.php
Is my app.yaml file setup correctly? The install.php file for my application works much the same way as a wordpress install.php file. You access it from a web browser and go through the basic setup. If my app.yaml file is correct, why am I still getting the "This webpage has a redirect loop" error displayed in my browser?
Also tried other browsers, clearing cookies, etc. The application installation screen loads perfectly in my SDK but not live on Google Apps Engine. What am I doing wrong?
I figured it out. There were 2 problems.
On app.yaml, in my case, the last line needed to say "index.php" instead of "install.php" Second, in my case, my configuration.ini file had port "80" set for MYSQL when in fact it needed to be "3306"