I have a Java based web application running on google appengine that depends on data in the datastore. When I update this backend data and deploy. I can see the data change immediately if I access the url 1-dot-myapp.appspot.com. I cannot get the default version of the url (myapp.appspot.com) to update on another device unless I access the full specific version of the url.
How can I force the default version of the application to update on deployment?
Thank you
I went back and looked at my cookies information. 1-dot-myapp.appspot.com only has a _ga cookie, the entry for myapp.appspot.com has 3 cookie values: application cache, ACID, and _ga. I was surprised that 1-dot-myapp.appspot.com did not have an application cache value in the cookie. So now I guess my question is. How can I force the application cache to renew as desired.
What I came up with was to either remove reference to my manifest file from my html or to rename my manifest file. So when ever I want to for the client browser to update the cache I redploy with a newly named manifest file. The manifst file is renamed with a version number like manifest2.mf. Then my build modifies all references for manifest to the newly name manifest file i.e. manifest2.mf. My html files and my appengine.xml file then us manifest2.mf . these changes seem to force the client browsers to update their cache.
Related
I am building a react app, which consists in a Single Page Application, hosted on Amazon S3.
Sometimes, I deploy a change to the back-end and to the front-end at the same time, and I need all the browser sessions to start running the new version, or at least those whose sessions start after the last front-end deploy.
What happens is that many of my users still running the old front-end version on their phones for weeks, which is not compatible with the new version of the back-end anymore, but some of them get the updates by the time they start the next session.
As I use Webpack to build the app, it generates bundles with hashes in their names, while the index.html file, which defines the bundles that should be used, is uploaded with the following cache-control property: "no-cache, no-store, must-revalidate". The service worker file has the same cache policy.
The idea is that the user's browser can cache everything, execpt for the first files they need. The plan was good, but I'm replacing the index.html file with a newer version and my users are not refetching this file when they restart the app.
Is there a definitive guide or a way to workaround that problem?
I also know that a PWA should work offline, so it has to have the ability to cache to reuse, but this idea doesn't help me to perform a massive and instantaneous update as well, right?
What are the best options I have to do it?
You've got the basic idea correct. Why your index.html is not updated is a tough question to answer to since you're not providing any code – please include your Service Worker code. Keep in mind that depending on the logic implemented in the Service Worker, it doesn't necessarily honor the HTTP caching headers and will cache everything including the index.html file, as it seems now is happening.
In order to have the app work also in offline mode, you would probably want to use a network-first SW strategy. Using network-first the browser tries to load files from the web but if it doesn't succeed it falls back to the latest cached version of the particular file it tried to get. Another option would be to choose what is called a stale-while-revalidate strategy. That first gives the user the old file (which is super fast) and then updates the file in the background. There are other strategies as well, I suggest you read through the documentation of the most widely used SW library Workbox (https://developers.google.com/web/tools/workbox/modules/workbox-strategies).
One thing to keep in mind:
In all other strategies except "skip SW and go to the network", you cannot really ensure the user gets the latest version of the index.html. It is not possible. If the SW gives something back from the cache, it could be an old version and that's that. In these situations what is usually done is a notification to the user that a new version of the app has been donwloaded in the background. Basically user would load the app, see the version that was available in the cache, and SW would then check for updates. If an update was found (there was a new index.html and, because of that, new service-worker.js), the user would see a notification telling that the page should be refreshed. You can also trigger the SW to check for an update from the server manually from your own JS code if you want. In that situation, too, you would show a notification to the user.
Does this help you?
I pushed an HTML static file containing an Angular SPA as catch-all handler for my custom domain with this settings:
- url: /(api|activate|associate|c|close_fb|combine|import|password|sitemap)($|/.*)
script: gae.php
- url: /.*
static_files: public/static/app/v248/es/app.html
upload: public/static/app/v248/es/app.html
expiration: "1h"
That worked fine, but if I push a new app.html it doesn't update. I've tried to change the local path, deploy a new app version, even replacing the catch-all handler with a custom php endpoint, but it doesn't work, the response still is the first version of app.html I uploaded.
Other people has had the same problem (CSS File Not Updating on Deploy (Google AppEngine)), and it looks like is related to Google CDN cache but, as far as I know, there isn't any way to flush it.
There is a way to flush static files cached by your app on Google Cloud.
Head to your Google Cloud Console and open your project. Under the left hamburger menu, head to Storage -> Cloud Storage -> Browser. There you should find at least one Bucket: your-project-name.appspot.com. Under the Lifecycle column, click on the link with respect to your-project-name.appspot.com. Delete any existing rules, since they may conflict with the one you will create now.
Create a new rule by clicking on the 'Add A Rule' button. For the action, select "Set storage to nearline". For the object conditions, choose only the 'Number of newer versions' option and set it to 1. Click on the 'Continue' button and then click 'Create'.
This new rule will take up to 24 hours to take effect, but at least for my project it took only a few minutes. Once it is up and running, the version of the files being served by your app under your-project-name.appspot.com will always be the latest deployed, solving the problem. Also, if you are routinely editing your static files, you should remove any expiration element from handlers related to those static files and the default_expiration element from the app.yaml file, which will help avoid unintended caching by other servers.
When performing changes in static files in an App Engine application, changes will not be available immediately, due to cache, as you already imagined. The cache in Google Cloud cannot be manually flushed, so instead I would recommend you to change the expiration time to a shorter period (by default it is 10 minutes) if you want to test how it works, and later setting an appropriate expiration time according to your requirements.
Bear in mind that you can change the static cache expiration time both for all static files or for just the ones you choose, just by setting the proper element in the app.yaml file.
2020 Update:
For my application I found that App Engine started failing to detect my latest app deployments once I reached 50 Versions in my Versions list.
See (Burger Menu) -> App Engine -> Versions
After deleting a bunch of old versions on next deploy it picked up my latest changes immediately. Not sure if this is specific to my account or billing settings but that solved it for me.
I had my static files over a service in Google Cloud Platform. My problem was that I didn't execute
gcloud app deploy dispatch.yaml
Once executed, everything was fine. I hope it helps
Another problem that could be causing this is caching in Google's frontend, which depends on the cache header returned by your application. In my case, I opened Firefox's inspector on the Network tab, and saw that the stale file had a cache-control setting of 43200 seconds, i.e. 12 hours:
This was for a file returned from Flask, so I fixed this by explicitly specifying max-age in the Flask response from my GAE code:
return flask.send_from_directory(directory, filename, max_age=600)
This causes intermediate caches such as Google's frontend to only cache the file for a period of 600 seconds (10 minutes).
Unfortunately, once a file has been caches there is no way to flush it, so you will have to wait out the 12 hours. But it will solve the problem for the next time.
Whenever I make a change to my html file, I have to resart appengine server to see the change reflected. Why is this so? In Flask for example the server automatically reloads whenever it detects a file change. Can this be done in AppEngine?
That's not the normal behavior for local development. You should be able to update your html files, as your code, without the need to restart the server.
Can you tell us more about your setup? Might there be a caching mechanism in your code that doesn't check the filesystem to refresh appropriately?
I upload my_app to app engine app by:
appcfg.py update c:\my_app ...
If I already uploaded for my_app then done a minor changement in file,
Does it upload whole project to app engine and overwrite whole previous project?
Or does it upload only relevent change and overwrite relevent part?
And what is the case for the issue for this command:
bulkloader.py --restore --filename=my_kind.dump ...
Did you try it?
update uploads the whole application each time. There's no concept of a delta. Normally, when you upload a new version I would suggest changing the version setting - that way you can keep up to 10 previous versions of your app on the site, and only set the new one to be the default once you are sure it is working.
If you upload without changing the version, AppEngine actually creates a new version before deleting the old one, so you need a spare slot in your versions list.
I don't understand your question about the bulkloader. Are you asking if that does a delta? No, it can't, because it sends the data serially via the remote API - there's no way for it to know in advance which rows in your data file already exist in the datastore.
The company I work for has proxies/WAN accelerators between our international sites to cache Intranet web content. I have a Silverlight application being hosted on a server at one location, but being accessed by clients in another location. When the users access the web page hosting the Silverlight app, they get the stale xap file being cached by the proxy and not the latest version from the server. Local users always get the latest xap as their requests are not going through a proxy.
I've tried the various header/metadata techniques mentioned elsewhere to prevent caching, and the containing web page itself is being served up fresh, but I still get the old .xap file. Short of getting our IT admin to disable proxy caching for my site, is there anything I can do make sure the latest xap file get retrieved from the server instead of the proxy? The containing page is ASP.NET.
What I do is just add a querystring at the end of the path to the xap file. Then when you change the querystring variable, the proxies etc. should see it as a request to a new file. So far this has worked fine for me.
So basically, when embedding an .xap in a straight-up HTML file, you would do this:
<param name="source" value="ClientBin/SilverlightApplication1.xap?cachepreventer=whatevervalue"/>
And then when you deploy a new version, just change "whatevervalue" to something else.
EDIT
If you need to use this technique in many places in your app I would read the querystring value from config and just write it to the page using asp.net. That way you only need to update it in one place when you deploy.
If you want to make sure every time the xap file is retrieved and you don't want to worry about it - just use
<param name="source" value="ClientBin/YourSilverlightapp.xap?<%=Guid.NewGuid().ToString() %>"/>
of course - this lends itself to a heavier cache load. I do like the helper method above though if you only want changes to be propagated to the client.