I am writing an Android application that allows users to upload and share photos. The server is based on Google App Engine. App Engine's datastore does not allow to save file, so currently I just have URLs saved. Looking for a way to store files I read about Google Cloud Storage. My question is - if I'm looking for a host for user uploaded files, is Google Cloud Storage what I'm looking for?
Yes. Google Cloud Storage is the way forward. There is also the BlobStore API on App Engine that allows you to store large amounts of information but the road map seems to be clear i.e. use Google Cloud Storage moving forward.
The reason for going with GCS will also be influenced that eventually you might want various tools or utilities that people have written that work directly with GCS. With Blobstore API, you will need to write those utilities yourself or rely on Admin console's support for taking backup,etc - which is not really much.
In summary, go with GCS.
Yes thats what you want. Says the same if you read the docs about google cloud storage.
Related
I have been using Google Cloud Storage to save photos that users upload from a mobile app (built with flutter and firebase), recently I had the need to resize/transform images and I wanted to explore if it's possible to do it directly from google cloud storage!
I found this project https://github.com/albertcht/python-gcs-image that you have to deploy on google app engine and if you call it with a bucket and an image it returns a URL to a Google CDN I think (something like this http://lh3.googleusercontent.com/*).
I looked at the code in the repository and the only thing it does is to return the result of google.appengine.api.images.get_serving_url and I don't understand why I cannot get this serving_url directly from my dart code?
What is the difference between Google Cloud Storage and lh3.googleusercontent.com?? Can I make the same image processing directly from cloud storage?
It seems odd that I have to run an app engine app that just returns a URL?
What am I missing?
The lib use this api. It's in Python 2.7 which have a end of life the 01/01/2020.
Moreover, the image api is available only with AppEngine 1st generation (python 2.7) and not available for the 2nd generation (python 3).
All of this for not recommended you to use this.
The best design today is to perform the resize/crop when the file is uploaded and to store the result in Cloud Storage. Example here and here
Then, you only have to serve, from Cloud Storage, the resized/cropped images.
Look # Firebase Extensions, there is already such an Extension provided there
What I found is:
The Java, Python, and go Standard environments for Google App Engine
include the GAE Images API (Java, Python, which can resize, rotate,
flip, and crop an image, as well as return an image serving URL which
allows for client side transformations, similar to Cloudinary and
Imgix.
This matches my previous understanding and experience as well. serving_url is really convenient for image manipulations.
Having said that, as you correctly pointed out, it's first and foremost an AppEngine feature and will require you to use AppEngine in one way on another.
If it's not what you want, you can create a service that will crop your images and deploy it serverless. It's a lot less burdensome that having AppEngine service running 24/7. What's more, AWS had several pre-baked templates to do just that - crop images that can be deployed in a couple of clicks.
If you are, like myself, interested in Google Cloud solution I can offer a similar function that I wrote. It can be deployed in Cloud Run as-is. See details in my other answer.
With it you can not only resize the images for mobile, but also map your own domain to the Cloud Run function and put it behind any CDN you like, which potentially can be faster that service from Google Storage. You can find plenty info on the Internet about why full-fledged CND is better than just bare Google Storage.
I've been able to get jQuery-File-Upload from https://github.com/blueimp/jQuery-File-Upload (python version) to allow my website users to upload images to a google app engine app, however it saves the images (for a limited time) as a blob.
It would be preferable to have the images saved in a google bucket but I'm unable to figure out how to get the files to save in a bucket rather than a blob.
Can anyone recommend a similar way for my website visitors to upload images to my google bucket, or if jQuery-File-Upload to do so?
I created an Appengine Python signed url example here:
There is a cloud API to directly upload files to google cloud bucket via javascript, however implementing security could make things a bit sticky.
In general,It should be fairly easy to use google cloud services found here
use storage insert api call
Now if you like to give access to clients to upload data on you application's behalf, then you may have to build a signed url ,signed with your key and then directly upload to cloud, read more here
I find it easy to do server side operations if the upload size is fairly small by using BlobStore API. You can use the Blobstore API to store blobs in Cloud Storage instead of storing them in Blobstore. Read all about it here.
This method might be the easiest to implement in your case .
I am working on a project which needs to store profiles of people on the cloud.
Information includes multiple photos and multiple text fields. I don't need Messaging.
There is lot if confusion in the documentation provided Google Cloud Services.
I am confused about what storage services should I opt for out of the 3:
1-Google Cloud Services,
2-Google Datastore,
3-Google Cloud SQL
So the the things i need to confirm are:
0.Is there a storage limit on using Google cloud SQL?
1.Does Google Cloud Storage and Google cloud Datastore provide unlimited storage?
2.Can an Android user write data on the cloud. I heard from some where that the applications only have access to read the data and the developer needs to put the data as blob on the cloud him self when using Google Cloud Services. Is this fact true for all the 3 storage services.
3.Is the data fully 'Sharable'+'Searchable'?
In other words:
If an Android user stores data in cloud in Google Datastore(text)+Google Cloud Storage(image), can this data be accessed by another android user without any headache of permissions or authentication(after I authenticate my app/app-engine)?
4.Is it the best option to store the images in Google Cloud Storage and their URLs in Google Datastore?
5.Does all the three storage services need app-engine to work?
6.Are any limitations on each of these services?
(0) Cloud SQL has currently a limit of 250Gb.
(1) With regards to Cloud Storage, there isn't a limit you could reach.
(2) and (3) They're not created for easiness of searchability. They should be accessed through applications, that are authorized, i.e. is not a substitute to Google Drive or Dropbox.
If you're the owner of the project, you can "browse" the contents of your Cloud Storage, but it's not meant for that.
Furthermore, objects in Cloud Storage can't be modified once created. A change needs to create a new copy of the object.
(4) It's a good idea, and something is used by many developers who have their applications in Google App Engine.
(5) No, they can exist without you using Google App Engine, but as I said earlier, you'd probably need an "application" to allow your Web/Android users to interact with the data, and there's where GAE comes handy.
(6) Yes, your budget.
If you provide a more detailed use case, I could tell you what you'd need to do to get it done with the whole array of Google Cloud products.
I'm new to AppEngine and I'm building an app that accept user image uploads from Android devices.
I built it with Cloud Storage but then I realized that I have problems uploading large files (maybe because of request time limits?)
so I figured out I should use Blobstore's upload URL to properly upload multiple large files.
Blobstore also has the on-the-fly image resizing feature which is very nice.
the thing is, Cloud Storage is cheaper than the Blobstore.
should I move the uploaded files from Blobstore to Cloud Storage after uploading ?
is there a way to upload multiple large files to AppEngine without going through the Blobstore upload URL way ?
I'm using Go if it matters..
The simplest answer is probably to use a signed url to allow the user to upload directly to Cloud Storage. This lets you bypass App Engine entirely for your upload, which in turn simplifies the network usage and allows you to take full advantage of all of Cloud Storage's upload infrastructure.
Currently, blobstore is $0.0009 / GBHour, while Cloud Storage is $0.0027 / GBHour, so it seems that blobstore is now 3 times cheaper than Cloud Storage. So while there may be reasons to move to Cloud Storage, cost is not currently one of them. Note that the prices changed recently.
If you need the richer API provided by Cloud Storage, then that's another story of course.
Google Cloud Storage seems more cost effective than the App Engine Blobstore.
At the moment I am storing user-uploaded image files as Blob type fields in the Datastore (App Engine Java API, by the way).
But I'm debating whether to switch to either the Blobstore or Google Cloud Storage to permit image sizes greater than 1MB.
Google Cloud Storage seems more cost-effective than Blobstore. Which one would be better for storing large images?
The Blobstore is tightly integrated with App Engine, while Cloud Storage is offered stand-alone. Otherwise, they look like different interfaces to the same basic service.
I don't see any good technical reason for them to be priced differently; perhaps it's a marketing thing.
The Blobstore has a more mature API and does some neat stuff like making thumbnails on the fly without incurring extra storage. The Cloud Storage API is experimental and rapidly changing. They're both perfectly capable of storing large images. To choose, you'll need to decide if the Blobstore's more rubust API justifies the nominally higher cost.
Well, As of Dec 19 2014 It looks like GCS is the way to go. stated on https://cloud.google.com/appengine/docs/java/storage#alternative_storage_solutions
Google Cloud Storage is recommended over using the Blobstore API.
I try to give a different answer. I would suggest Picasa as the image storage in common situation.
I ran into the same question when Google stopped to serve Google Storage for free.
But then I noticed that Picasa is free for pictures smaller than 2048x2048 when your Google plus account is enabled. And it's still more cost-effective for paid-storage.
So if you are going to store small pictures like profile pictures, I would suggest App Engine Blobstore.
If you are dealing with some more photos and pictures, take a look at Picasa.
Note that with the latest release, there's now an API for accessing Cloud Storage from App Engine.