I've built systems on top of Google's App Engine and leveraged Google's Datastore, but for my new project I'm considering a containerized solution (using Google's Container Engine). Does anyone with experience using both technologies together know:
if this is possible to use Container Engine with Datastore?
if it's easy to set up a local containerized dev environment with gcd?
if there are some serious headaches I should consider before going down this route?
Absolutely! You can run any code you want in Container Engine, and if you add the datastore scope to your cluster when you create it, authentication to the Datastore API will be automatic if you're using Datastore's client libraries or tools.
I'm not familiar with the local gcd environment, so I can't help much here. Testing Docker containers locally before pushing them to the cloud works great, so the only question will be making sure the gcd dev environment can be exposed to your local containerized app.
The dev environment is the one issue I'm not sure of. Using Datastore from Container Engine should work fine.
What I have done is just created a service account and use the json key to access datastore when I am working locally. It seems to work pretty well.
Related
I'm looking at moving my application to google cloud and I'm having a hard time understanding how best to organize my project. It seems like you can only have one App Engine application per project with services available to support a microservices architecture and instances representing the App Engine instances created via auto scaling.
What is the correct way to build an App Engine for my API server and an App Engine for my Web Server? Do I need to have a project for each? I'm essentially trying to accomplish the following:
It is straightforward to have a single GAE project implement both your website and an API. You can even do this within a single service. You could put each in a separate service, and the advantage of that is that you can update one without updating the other. For small projects, a disadvantage is that two services are more expensive than one (though GAE is quite inexpensive overall).
For prod vs dev, you'll need to explain your requirements a little more, but here are some thoughts.
Each GAE service has multiple versions. You can deploy your production version to www.mycompany.com and deploy a dev version to dev-dot-myapp.appspot.com (that's the way GAE does URLs for versions of your app). Both of these versions will access the same datastore so you need to be careful with the dev version so that it doesn't mess up your prod implementation.
If you have a dev situation that is bleeding edge and shouldn't be able to access the datastore of your production app, then you would create a different GAE project for that.
Here is a way to visualize it:
Google Cloud Project A
GAE Project A
production www service
production API service (this could be combined with production www service)
dev www service
dev API service
Google Cloud Project B
GAE Project B
bleeding edge www service
bleeding edge API service
Best Practices:
Your Dev and Prod should be in separate projects for both security and billing purposes.
App Engine:
You can only have one App Engine per project. This will create a problem for you to use App Engine for both API Server and Web Server. In this case, I would not use App Engine at all and instead look at Containers on Compute Engine or go for Kubernetes.
Even a single node Kubernetes Cluster will shock you with its flexibility and power. Containers on Compute Engine still have a lot flexibility and power too. If you like the concepts of App Engine Flexible, then you might really like containers. The exception here is that App Engine makes some concepts brain dead simple where you have more work in configuration for Containers or Kubernetes.
In my previous App Engine projects I used the Cloud Datastore, and during development I could debug my app on the local server and it would use a local database, stored in a file I could wipe out if I wanted to start from scratch.
With Cloud Firestore, even when I'm running locally it's talking to my real cloud database. Is there still a local option? Note that I'm not talking about client-side persistence, I'm talking about a mock development DB.
Google recommends setting up multiple projects if you want dev/staging/production, and I'm guessing that's the answer, but I'd like to know before adjusting my workflow.
I think (now only a few months later) that this is supported. When I run my app, using dev_appserver.py, I see a message
INFO 2019-02-14 00:08:56,030 admin_server.py:150] Starting admin server at: http://localhost:8000
Going to that URL shows me all the instances I have been seeing. These seem to persist even when the dev_appserver is restarted. Reading this and other posts I was convinced that my development was using my actual cloud database, but going to https://console.firebase.google.com/project/myproject was showing completely different content.
Just to be sure (because google is google and everything is named the same) I'm using an appengine app and a gcloud project, storing things to Firestore using ndb.Models...
Oh, but careful. My app I was also using the cloudstorage (blobstore?) and even though the localhost:8000 showed these, THESE WERE THE REMOTE INSTANCES.
There is a local emulator for Firestore when using the Firebase CLI:
https://firebase.google.com/docs/rules/emulator-setup
Im creating a Node.js website that probably won't have loads of traffic, and was looking into cheap solutions to host the site. Came across Google cloud services offering free usage for their services with limits. A f1-mirco is more than enough for my needs, but I will happily pay for some usage if it goes over by any chance.
I wanted to setup a linux centOS 7 on GCE (which I already did), and run my application and REST API on it. Now here comes the problem.
I tried to use Google's datastore service, but it sprung an app engine instance and without it datastore won't work.
Is datastore entirely relying on app engine to function?? In the docs, it said if you use any of the client API, it requires app engine. What can I do to not use the client api and query data then? Don't want to use the app engine at the moment or datastore is just not for me then?
Thanks for any help!
Some of the underlying infrastructure of Cloud Datastore and App Engine are still tied together for creation, etc. So while creating an Cloud Datastore database also defines an App Engine instance for the project, it doesn't require you to use it. You don't get charged for App Engine either, unless you decide to deploy an App using it.
You should be totally fine use the Google Cloud Node client library on the f1 micro instance.
I was able to create the basic 'hello world' program.
When I tried to understand the difference between a cloud and a server I learned that Cloud is where you have an access to virtual instance created exclusively for you and you are free to choose and install software of your choice.Why Google App Engine(GAE) is used widely where as tomcat is not used. What are major differences between GAE and Tomcat?
Cloud is Google Cloud Platform at this case. App Engine is just one of their services.
App Engine is a platform to build your apps on top of it. A Platform As A Service or PaaS. It simplifies the process of building a scalable application, and you should use it when you understand what you really need and understand principles of scalable application.
Tomcat is a Java web container, and there're many alternatives. Google App Engine is using Jetty. You could actually use it with Tomcat by using Flexible VM, though it doesn't make much sense.
App Engine is not about web server, it's a set of services that helps you to build a scalable app. It includes Memcache, Datastore, Task Queue, Images API, deployments tools and versioning, CDN for static files, and most important automatic scale.
Actually you aren't limited to App Engine on Google Cloud Platform. There is more traditional service, like own server in the cloud, called Compute Engine. There you can run your Tomcat or anything else.
I'm trying to deploy one of my java apps in GAE and I have used MongoDB. Unfortunately GAE doesn't allow me to create databases, however it suggests using Google Datastore. Are there any plugins or add-ons to use MongoDB.
You can't run MongoDB on GAE. You need to use the Google datastore.
Matt is right. If you want to deploy to Google's servers you can't use Mongo. If you are interesting in running your own server(s), check out AppScale.
If you really want to use MongoDB anyway, you'll have to find a third-party host that provides a REST interface through which you can make your transactions. I've personally tried and recommend MongoLab on App Engine. Then, in your code, use an HttpUrlConnection object to send/receive requests if you're doing it with Java.
I think you could deploy your app on app engine and create a separate vm instance in google compute engine that would have mongo installed, then you plug your GAE app on that vm for the db connection. This would allow you to keep all your app in the google cloud. See https://cloud.google.com/solutions/mongodb/ and http://docs.mongodb.org/ecosystem/platforms/google-compute-engine/
A lot of things have changed since this question has been asked, as of June 2021, you can use MongoDB Atlas with Google App Engine Standard running on Java 8 environment. I built my App with the Ktor framework that runs on App Engine and interacts with databases on MongoDB Atlas.
if you're still looking for an answer to this you might want to check this project that did exactly the same but in reverse, it will surely help you out with what you want to do.
http://code.google.com/appengine/articles/django-nonrel.html
you can use mongo, only if have something like on mongolab then use pymongo to connect and most important thing you need to enable socket api from gae dashboard.
it worked for me thanks