I am interested in knowing if it's possible to do ssh to google cloud storage from an AppEngine to read some file from there, process and insert data in BigQuery. Thanks
Neither Cloud Storage nor App Engine Standard have any concept of an "instance" that you could SSH to (unless you're using App Engine Flexible, but you still can't SSH to Cloud Storage).
However you can talk to Cloud Storage from App Engine, or just directly load your GCS bucket into BigQuery.
Related
I have an old AppEngine Java application using the AppEngine datastore. Is this what the marketing renamers at Google now (2019) call "Cloud Datastore"?
Can I create Google Cloud Functions that interact with the same datastore, and what are the steps needed to do so?
Yes, it's the same datastore. Also called/soon-to-be Cloud Firestore in Datastore mode (which all older apps will be converted to at some point).
Yes, you can access it from anywhere, even from outside Google Cloud. From Cloud Datastore (emphasis mine):
You can access Cloud Datastore from anywhere using the Cloud
Datastore API. Use the Google Cloud client libraries to store and
retrieve data from Cloud Datastore.
The same Cloud Datastore data is available regardless of if you use the App Engine libraries, the Google Cloud client libraries, or call
the API directly.
The major steps to access the datastore from a Cloud Function:
you can't use the GAE-specific client libraries like the one you likely used in your old app, you'll have to use one of the generic client libraries (or the REST or RPC APIs)
you'll have to give your CF's Identity/service account the proper access permissions, see Setting up authentication and Accessing your database from another platform.
I'm currently new to web developing and right now I started with a Spring Boot application which I converted to standard App Engine project. I'm wondering if it is possible to not convert my Spring Boot application to my App Engine project and still use Datastore as database?
Yes, it's possible to use the Cloud Datastore from an app in the GAE flexible environment, from outside GAE or even from outside the Google Cloud. From Cloud Datastore (emphasis mine):
You can access Cloud Datastore from anywhere using the Cloud
Datastore API. Use the Google Cloud client libraries to store and
retrieve data from Cloud Datastore.
The same Cloud Datastore data is available regardless of if you use
the App Engine libraries, the Google Cloud client libraries, or call
the API directly.
But you can't use the GAE Standard Environment Client Libraries, you have to use either the Cloud Datastore Client Libraries or the Cloud Datastore API v1.
Potentially of interest: the Deploying to the App Engine Flexible Environment guide happens to use a spring boot app as example.
I am developing an app where users can upload images. The app has a NodeJs Backend an Angular Frontend with a Redis and Neo4j all dockerize and run by Kubernetes. Now I would like to store images, but there are so many service that I think could do the job that I don't know what to do... Can I use my Google Drive account and the Drive Sdk to upload the images of my users ? Should I look into Google Cloud Storage ? What about the persistence storage option in Kubernetes ? Or can I use my Flickr Account ??? Could someone point me the right direction... Thanks
For uploading and storing static files such as images in the cloud using GCP should probably be using Cloud Storage.
While both Google Drive and Google Cloud Storage provide an API to upload files, Cloud storage is more suited for your use case. I took this excerpt from here
Cloud Storage is intended to be accessed primarily through its API and
provides all the functionality necessary for developers to use it as a
backing store for their own applications.
and
Cloud Storage enables developers to store their application data in
the Google cloud (and they’re responsible for the storage their app
consumes), whereas in Drive, users allow apps to interact with the
user’s private storage and content.
I have a GAE based application that pulls a file from cloud storage and then does some processing on that file. To run the application in the remote appengine environment, I first upload a file to cloud storage using the browser console, and then make requests to the application, which pulls the file I uploaded from cloud storage. I'd like to be able to do development locally, however there is not a sweet browser console for the local implementation of gcs, as discussed here: Local storage browser for Google Cloud Storage and dev_appserver.py.
I'm wondering if it's possible to use gsutil. It seems the local gcs implementation is accessible through a localhost endpoint, mentioned here: Google Cloud Storage on Appengine Dev Server.
Right now, what I want to do is just load a file into my local gcs instance. I could do this my writing a little utility, but it seems much better to use gsutil if I can get that to connect to my local instance.
Thank you,
Ben
I am looking to use Google Cloud Storage services outside of launching a google app engine application. My application is designed for running on a hosted server of our own, and I would like to still use the cloud storage service.
I've written a dev app that did this, and ran it with dev_appserver.py in the python GAE framework. It seems the dev_appserver.py is necessary as it fills in how to perform the RPC functions the cloud storage provides.
Is there a way to easily use google cloud storage service without the rest of it?
There is gsutil command line tool:
https://developers.google.com/storage/docs/gsutil
There is an xml/json restful service all at:
https://developers.google.com/storage/index
You'll need to enable the google cloud storage on the google console at:
https://code.google.com/apis/console/ and get the necessary api keys.