Can I cache dynamic responses from Google App Engine?
Here's an old question for Python, but the official docs are pretty light.
I see that I can define static files to cache, but I'm trying to figure out if I can cache dynamic requests. For example, fetching data that doesn't change very often from Cloud Datastore.
Related
I am using Google Cloud Storage as my CDN backend to save images for user content. However when loading those images I did not yet implemented a way to scaled them by a specific width. That makes my thumbnail renderings very heavy regarding to the image size and that's not nice.
So while I was trying to implement "Scrimage" as a library to manipulate my images before I pass them to the requesting frontend I stumpled over this: https://cloud.google.com/appengine/docs/standard/java/images/#transforming-images
It shows how you can create a "functional URL" that takes parameters and manipulates images when requested.
I wonder if this feature is usable for my Java backend as well without the need of AppEngine? Or does Google Cloud Storage already has that implemented so I can call my storage image URLs with the mentioned parameters to resize the images?
I couldn't find anything therefore I am asking here.
GCS doesn't have that implemented.
You're referring to the images API which is one of Google App Engine's bundled Services API. My understanding is that they can't be used outside of a GAE App
I have been using Google Cloud Storage to save photos that users upload from a mobile app (built with flutter and firebase), recently I had the need to resize/transform images and I wanted to explore if it's possible to do it directly from google cloud storage!
I found this project https://github.com/albertcht/python-gcs-image that you have to deploy on google app engine and if you call it with a bucket and an image it returns a URL to a Google CDN I think (something like this http://lh3.googleusercontent.com/*).
I looked at the code in the repository and the only thing it does is to return the result of google.appengine.api.images.get_serving_url and I don't understand why I cannot get this serving_url directly from my dart code?
What is the difference between Google Cloud Storage and lh3.googleusercontent.com?? Can I make the same image processing directly from cloud storage?
It seems odd that I have to run an app engine app that just returns a URL?
What am I missing?
The lib use this api. It's in Python 2.7 which have a end of life the 01/01/2020.
Moreover, the image api is available only with AppEngine 1st generation (python 2.7) and not available for the 2nd generation (python 3).
All of this for not recommended you to use this.
The best design today is to perform the resize/crop when the file is uploaded and to store the result in Cloud Storage. Example here and here
Then, you only have to serve, from Cloud Storage, the resized/cropped images.
Look # Firebase Extensions, there is already such an Extension provided there
What I found is:
The Java, Python, and go Standard environments for Google App Engine
include the GAE Images API (Java, Python, which can resize, rotate,
flip, and crop an image, as well as return an image serving URL which
allows for client side transformations, similar to Cloudinary and
Imgix.
This matches my previous understanding and experience as well. serving_url is really convenient for image manipulations.
Having said that, as you correctly pointed out, it's first and foremost an AppEngine feature and will require you to use AppEngine in one way on another.
If it's not what you want, you can create a service that will crop your images and deploy it serverless. It's a lot less burdensome that having AppEngine service running 24/7. What's more, AWS had several pre-baked templates to do just that - crop images that can be deployed in a couple of clicks.
If you are, like myself, interested in Google Cloud solution I can offer a similar function that I wrote. It can be deployed in Cloud Run as-is. See details in my other answer.
With it you can not only resize the images for mobile, but also map your own domain to the Cloud Run function and put it behind any CDN you like, which potentially can be faster that service from Google Storage. You can find plenty info on the Internet about why full-fledged CND is better than just bare Google Storage.
I have many websites and also websites made by clients which I would like to optimize. I am currently using different CDN providers but I would like to simplify my workflow and hopefully also lower the costs.
I would like to have a CDN with a Pull Zone, and that CDN would also optimize the images (while not modifying the other static resources).
Ideally, I would also have access to statistics for each Pull Zone (since I would like to charge my clients for this service instead of guessing).
What are the different ways to do this with the Google Cloud? Is there a way to do this only using Google Functions, CDN, and Google Storage? Of course, I guess that having a little NodeJS app running to optimize the images would be needed as well. I just wonder about the general architecture and if it is even possible (I know it is with Azure and AWS but I am already running a few things on the Google Cloud).
Thanks a lot :)
In GCP a pull zone can be created by associating a HTTP(S) Load Balancer to a Cloud Storage Bucket and enabling Cloud CDN.
Having a different bucket for every client will break down the logs on your project, but not the billing for it.
To be able to separate billing you can always export the logs to a BigQuery and use it to break down the billing costs per client based on their use.
Regarding the optimization of the images, Google CDN will not perform any operation, neither GCS Bucket.
The only operation available in this direction is when using and serving GZip-compressed files.
I suggest you to dedicate one Instance to be able to prepare the images before storing or to add/replace the optimized versions of the images already inside the bucket.
I'm building a static site which deploys to Google App Engine. Are there any advantages of storing the assets (js, css, imgs) in CloudStorage. All of the assets are going be under 32MB (if thats a limit).
Based on this slide (slides 24-28), it sounds like requests for static assets of a GAE app would use Google's special infrastructure designed for serving static assets. However its not clear about how its performance compares to Google CloudStorage.
Any clarifications on this would be much appreciated. Similar questions on Stackoverflow exists, but they are fairly dated (2010), and Google's Cloud products have changed since then.
Setting your cache headers will result in similar performance AFAIK.
Both GAE and GCS use Google Edge cache.
GCS is probably easier to manage your static resources, whereas it takes a redeploy to GAE to add/remove static resources.
Other important note : GCS as CDN does not work over HTTPS with custom domains. GAE does support HTTPS with custom domains.
Where do developers store their data? I heard that it can be hard to access S3 with the existing libraries and you need to update all URL calls using GAE's methods. Do developers typically find other ways to access S3 or do they use some other sort of storage (Google Storage is not public yet) such as databases? Does google provide anything for this, what is the "ideal" way that app engine would solve the problem of having users get and post data to persistent storage through app engine?
GAE provides the Datastore API, built on their Bigtable.
There's many good questions on the GAE Datastore here on stackoverflow.