Where can I write critical environment variables like on heroku?
I do not want to store the keys of my database in clear in app.yaml file.
The recommended approach to store secrets in GCP is using Cloud Key Management Service and Cloud Storage.
The Storing Secrets documentation guides you through the process of storing a secret in Cloud Storage bucket encrypted at the application layer with Cloud KMS. There's also other ways to manage secrets with Cloud KMS, it's documented in the Secret management with Cloud KMS page
Related
I have an old AppEngine Java application using the AppEngine datastore. Is this what the marketing renamers at Google now (2019) call "Cloud Datastore"?
Can I create Google Cloud Functions that interact with the same datastore, and what are the steps needed to do so?
Yes, it's the same datastore. Also called/soon-to-be Cloud Firestore in Datastore mode (which all older apps will be converted to at some point).
Yes, you can access it from anywhere, even from outside Google Cloud. From Cloud Datastore (emphasis mine):
You can access Cloud Datastore from anywhere using the Cloud
Datastore API. Use the Google Cloud client libraries to store and
retrieve data from Cloud Datastore.
The same Cloud Datastore data is available regardless of if you use the App Engine libraries, the Google Cloud client libraries, or call
the API directly.
The major steps to access the datastore from a Cloud Function:
you can't use the GAE-specific client libraries like the one you likely used in your old app, you'll have to use one of the generic client libraries (or the REST or RPC APIs)
you'll have to give your CF's Identity/service account the proper access permissions, see Setting up authentication and Accessing your database from another platform.
I am interested in knowing if it's possible to do ssh to google cloud storage from an AppEngine to read some file from there, process and insert data in BigQuery. Thanks
Neither Cloud Storage nor App Engine Standard have any concept of an "instance" that you could SSH to (unless you're using App Engine Flexible, but you still can't SSH to Cloud Storage).
However you can talk to Cloud Storage from App Engine, or just directly load your GCS bucket into BigQuery.
I am developing an app where users can upload images. The app has a NodeJs Backend an Angular Frontend with a Redis and Neo4j all dockerize and run by Kubernetes. Now I would like to store images, but there are so many service that I think could do the job that I don't know what to do... Can I use my Google Drive account and the Drive Sdk to upload the images of my users ? Should I look into Google Cloud Storage ? What about the persistence storage option in Kubernetes ? Or can I use my Flickr Account ??? Could someone point me the right direction... Thanks
For uploading and storing static files such as images in the cloud using GCP should probably be using Cloud Storage.
While both Google Drive and Google Cloud Storage provide an API to upload files, Cloud storage is more suited for your use case. I took this excerpt from here
Cloud Storage is intended to be accessed primarily through its API and
provides all the functionality necessary for developers to use it as a
backing store for their own applications.
and
Cloud Storage enables developers to store their application data in
the Google cloud (and they’re responsible for the storage their app
consumes), whereas in Drive, users allow apps to interact with the
user’s private storage and content.
I am working on a project which needs to store profiles of people on the cloud.
Information includes multiple photos and multiple text fields. I don't need Messaging.
There is lot if confusion in the documentation provided Google Cloud Services.
I am confused about what storage services should I opt for out of the 3:
1-Google Cloud Services,
2-Google Datastore,
3-Google Cloud SQL
So the the things i need to confirm are:
0.Is there a storage limit on using Google cloud SQL?
1.Does Google Cloud Storage and Google cloud Datastore provide unlimited storage?
2.Can an Android user write data on the cloud. I heard from some where that the applications only have access to read the data and the developer needs to put the data as blob on the cloud him self when using Google Cloud Services. Is this fact true for all the 3 storage services.
3.Is the data fully 'Sharable'+'Searchable'?
In other words:
If an Android user stores data in cloud in Google Datastore(text)+Google Cloud Storage(image), can this data be accessed by another android user without any headache of permissions or authentication(after I authenticate my app/app-engine)?
4.Is it the best option to store the images in Google Cloud Storage and their URLs in Google Datastore?
5.Does all the three storage services need app-engine to work?
6.Are any limitations on each of these services?
(0) Cloud SQL has currently a limit of 250Gb.
(1) With regards to Cloud Storage, there isn't a limit you could reach.
(2) and (3) They're not created for easiness of searchability. They should be accessed through applications, that are authorized, i.e. is not a substitute to Google Drive or Dropbox.
If you're the owner of the project, you can "browse" the contents of your Cloud Storage, but it's not meant for that.
Furthermore, objects in Cloud Storage can't be modified once created. A change needs to create a new copy of the object.
(4) It's a good idea, and something is used by many developers who have their applications in Google App Engine.
(5) No, they can exist without you using Google App Engine, but as I said earlier, you'd probably need an "application" to allow your Web/Android users to interact with the data, and there's where GAE comes handy.
(6) Yes, your budget.
If you provide a more detailed use case, I could tell you what you'd need to do to get it done with the whole array of Google Cloud products.
In my application on Google App Engine, I have files/documents uploaded to Blob Store. The size of Blob Store has grown to more than 100 GB.
I am in search of a mechanism by which I can backup my blob store data, may be to some other server location or Google Cloud Storage or some other safe place.
AFAIK there isn't a simple tool that you can use to backup your Blobstore data. There are at least a few approaches that you could take to write your own tool:
Use cron.xml and a servlet (or some similar scheduling) to copy batches of your entity blobs to Google Cloud Storage and then download from there (see below for tools).
Use remote-API or provide some other API (REST etc) where a remote tool you make can query your entities and then over a serving url (that you probably already have) download the Blobs to your local machine.
Those approaches are not very friendly and there are however tools such as GSUtil that would be helpful if you migrated your data to Google Cloud Storage (GCS). I've seen a few people migrate their data now that upload and serving with GCS is supported with the Blobstore API.
It's important to also note that the Files API for Blobstore is now deprecated (since 1.8.1?) and that there is a new preferred way through the App Engine GCS client.
Additionally if you check Posts about 1.8.1 Pre-release SDKs Available. you'll see that there is some indication from Google that blobs might be auto-migrated over to Cloud Storage (free of charge) before the end of the year (presumably close to when Blobstore will be deprecated). Depending on the urgency of your needs it may be feasible to just wait.
You can use Backup/Restore functionality of GAE Admin console.