We are currently running a combined AppEngine / GCE app and thus far have kept all of our datastore access on the AppEngine side of things. Now we are exploring also allowing our GCE instance to make some queries into the (shared) datastore. To start, I'm trying to figure out how to run things locally. What we have so far:
A Go devappserver running
A Go standalone binary that wants to issues queries to the devappserver datastore.
We installed ('go get') google-api-go-client/datastore/v1beta2 so that we can use an API instead of issuing direct HTTP calls. However we are definitely willing to issue direct HTTP calls if this API library won't work in development.
We have service accounts set up (we already access GCS from GCE) but I doubt that's relevant for running locally...
I've seen some docs but they (a) only talk about Python & Java, and (b) discuss connecting to the (standalone) development datastore server, as opposed to the datastore embedded in AppEngine's devappserver (if those are even different?). There is also the following answer here on StackOverflow, but again it discusses connecting to the standalone development datastore server:
How to connect to the local google cloud Datastore db?
Any pointers would be much appreciated!
Ian
Currently this is not possible in the development environment for several reasons. The Google Cloud Datastore tool (gcd.sh) uses the java development server. However when developing go for App Engine you use the python development server, which has different underlying storage. There is a bug to track this issue on the github page.
You can still develop a Google Cloud Datastore application in go however there are many bugs in the current go client library. Unfortunately, the development server does not currently support the JSON API, which the go library uses (see the note at the top of the page).
Update: I wanted to make sure proppy's comment was seen as part of the answer. His suggestion does provide a way to use the protocol version of the API, which is probably more stable than the go client library above. It could also let you use the gcd.sh tool to test this in the development server. You will have to craft the HTTP requests yourself though, and you won't be able to share the data in the datastore between your application and the Cloud Datastore in development. However it is definitely a good workaround and lets you use the Cloud Datastore API, which as it develops will be easier to work with than other workarounds.
From proppy:
Note that you can still use Cloud Datastore Protobuf HTTP API with Go. The protobuf definition is available on GitHub, you can compile it to Go code using the Go protobuf compiler plugin and then send POST HTTP requests to /datastore/{version}/datasets/{datasetId}/{method}.
If the use case from your "GO" app server is straight forward enough, you may want to implement access by using an API call to your GAE service (perhaps extending the service to receive the API calls).
This has the added benefit of only having to make changes in one place if your datastore definitions or functions change.
Related
I am using Google cloud to run some services. speifically, I have code running in firebase, as well as Cloud Run and Cloud SQL.
For some reason, however, I am seeing constant traffic to a compute engine API, which I have no idea why? I don't have any VM's setup (outside of what Cloud SQL is running), but I'm assuming that's a separate service.
Why would I be seeing this traffic against this API? I could "disable" it, but the warning says any resources created with it could soon be deleted and I don't want to bring down my Cloud SQL or Cloud Run instances. There's nothing I can taht would generate 418,000+ requests this month....unless this is just internal Google stuff to support my cloud SQL and Cloud run instances?
Any ideas what this could be? Should it be safe to disable this?
You will see traffic on the Compute Engine API because you are actually using the service.
CE API is not just related for GCE instances, it handles much more of that, for example: Disk, Firewall, Snapshots etc.
You can find more information about the CE API here
So I recommend not disable the CE API because it can cause errors and other problems.
Im creating a Node.js website that probably won't have loads of traffic, and was looking into cheap solutions to host the site. Came across Google cloud services offering free usage for their services with limits. A f1-mirco is more than enough for my needs, but I will happily pay for some usage if it goes over by any chance.
I wanted to setup a linux centOS 7 on GCE (which I already did), and run my application and REST API on it. Now here comes the problem.
I tried to use Google's datastore service, but it sprung an app engine instance and without it datastore won't work.
Is datastore entirely relying on app engine to function?? In the docs, it said if you use any of the client API, it requires app engine. What can I do to not use the client api and query data then? Don't want to use the app engine at the moment or datastore is just not for me then?
Thanks for any help!
Some of the underlying infrastructure of Cloud Datastore and App Engine are still tied together for creation, etc. So while creating an Cloud Datastore database also defines an App Engine instance for the project, it doesn't require you to use it. You don't get charged for App Engine either, unless you decide to deploy an App using it.
You should be totally fine use the Google Cloud Node client library on the f1 micro instance.
I want to deploy a multi-service application on GAE using custom runtimes.
I've everything set, my service expects some environment variable to know when to connect to the database and other services.
But I can't see anywhere in the doc where it explains how to connect to other Google Cloud service from a custom runtime.
If it is explained, thanks for pointing me there. if not, can someone who has been successful on doing that explain to me how he/she achieved it.
The documentation for connecting from Flexible Environment apps is found in Cloud SQL > Documentation > MySQL > Connecting from App Engine. Some of the languages (for example, for python) make use of the Cloud SQL Proxy, while others (such as java) don't.
As for Pub/Sub, you can read about client libraries in the documentation.
I'm thinking of building a dynamic web site using Google web engine Technology, however I would like to use it in a local network without internet . Can I use the Google web engine SDK as a LOCAL SERVER.In other words, is the SDK just for development and testing purposes or it can be used as a server?
Should I worry about long-term use of SDK as a local web server, is there any scalability issues I should be aware of.
Thank you.
You probably could, but there'd be no point.
GAE makes all sorts of tradeoffs which only make sense in terms of the benefits you get by running on Google's infrastructure: for example, the limited querying you can do with the datastore, or the strict timeout on requests.
Plus, the development server is extremely limited in what it can serve, so you'd need to put a proper web server in front of it anyway.
If you're using Python, you can use webapp2 as a standalone WSGI application, although you might be better off going directly for something like Django. I don't know much about serving Java apps directly but I guess you can use Tomcat.
I have my website currently hosted on paid server, but i want to transfer it on GAE.
How can i do it? Can anyone please help me in this case.I'd appreciate your help.
Thanks:)
1) First you will have to adapt your website to the GAE framework (python with django or the new Java environment). You can test your work by downloading the SDK of GAE which offer a local server.
2) Then create an account on appengine.google.com and upload your application on something.appspot.com, test it.
3) If you have a domain name, create a google apps account on this domain, and finally bind this domain with your GAE website. Here is the Google doc.
If it is just a static website which does not need server side scripts or a database, then you might want to look into Google Sites instead of Appengine. You can find out more about Sites here: http://www.google.com/sites/help/intl/en/overview.html
If you do have some server side logic going on, you will need to convert it to either python or java and convert your relational database to Google's Data API which does not support the SQL your current database uses. You can read more about the APIs and what is supported with the Data API and tutorials at: http://code.google.com/appengine/
In response to sanorita's comment "Actually, it's generated html and not plain html. and google appengine is for static data... right?":
AppEngine can host static data, but that is far from its intent.
The purpose of AppEngine is to allow developers to easily deploy their dynamic applications on Google's infrastructure. In the end, assuming you have programmed your app in effective ways to handle scaling (basically just noting that writes to the database are expensive, and contention is the root of all evil) you can handle nearly any amount of traffic.