Im confused about these three different constructs in google cloud and trying to understand how these fit together.
First, from what I have read "modules" are just the old name for "services" right? So anything I read about google cloud "modules" would also apply to services?
Are you using namespaces AND services/modules together or are they generally mutually exclusive?
Is this a good example of how to use these things together:
Put my shared resources (storage, DBs, etc) in a "namespace" and that way multiple modules/services can access them. "Any App Engine request can access any namespace" so namespaces resources are bound by their project container.
Create "services" that access the namespaced resources
All of this is in a single "project" which is used as an environment (so I'd have a "dev" and a "prod" project
Right, services and modules are the same thing. Note the actual url path inside this quote from App Engine Services as microservices:
In an App Engine project, you can deploy multiple microservices as
separate services, previously known as modules in App Engine.
Namespaces are supported by just a few APIs that services can invoke:
App Engine currently supports namespaces in the following APIs:
Datastore
Memcache
Task queue
Search
They are really just a way to separate/slice (not share) data served by these APIs and can help prevent accidental data leaks across the namespace boundaries. See for example Implementing Multitenancy Using Namespaces. But note that the protection is only as good as the app code is (data will leak if the code is setting the wrong namespace).
Services do not offer data isolation, they can share data regardless of it being in a namespace or not, by appropriately setting the namespace when invoking the respective API. So it's not placing data in a namespace that makes that data shareable across services.
Project bounds apply to both namespaces and services. But it is possible to configure APIs and projects to allow access even across project boundaries (or even from outside Google network - see, for example How do I use Google datastore for my web app which is NOT hosted in google app engine?)
The primary purpose of using services is to obtain code isolation. But it comes with a price - each service has its own instances. Also a bit more difficult: docs and even tools are often a bit behind, many of them assume a single-service GAE app context.
Even though using services to create environments is offered as an alternative in Naming Developer Environments I would stick, as you mention, to using separate projects for that (to also have data isolation).
Related
I deploy my Docker image to default GAE by gcloud app deploy --image-url=us.gcr.io
I have successfully mapped my custom domain to this application with custom runtime and flex env.
My dispatch.yaml sends requests to its sub-domain:
dispatch:
- url: "dev.domain.com/*"
service: default
Now I want to use different images from Container Registry for test.domain.com and domain.com
While having all these images sharing same Cloud Storage and Firebase credentials.
Being new to GCP I wanted to learn any simple approach to organize such basic structure without going into services and versions (just by assigning proper images to serve relevant domains).
Is it even possible to do within one GAE project or I should create separate projects for it?
Mapping custom domains can only be done at the service level, so if you don't want to go into services separate projects is your only choice.
Actually using separate projects instead of services (or service versions) for implementing different environments has some advantages, I'd choose separate projects, too. See Advantages of implementing CI/CD environments at GAE project/app level vs service/module level?
I'm not sure if sharing the storage and credentials between production and other environments is a good idea (what if something goes wrong?). I'd keep them separate, too (maybe with some jobs to populate non-production projects with production data, if you need to). But if you do want to share them across projects you'll probably need to make some extra steps.
I want to build a web application using a mixture of App Engine Standard and Flexible Environment as described in the Google docs (flexible as microservice where third party software is needed, standard for everything else).
I need the mentioned microservice to run latex, a few linux tools and python. What is the best way to go from here?
My guess is:
Build a docker container from a Linux OS and use either Google Pub/Sub, Google Task Queue or plain HTTP for communication with the Standard Env App.
But how is this custom runtime then managed by Google regarding security updates, scaling, loadbalancing and everything else promised in the docs?
Sorry for the rather generic question, the infos are thin IMHO and so I have to ask.
It would be your responsability to re-build the custom runtime images (done during every app deployment) to incorporate security updates. If your Dockerfile references other Google-supplied base images then the security updates for them will be automatically picked up in the process. But for any additional packages or customisations you added to your runtime you may need to incorporate the updates yourself.
Scaling depends on your app's configuration (your responsability), see Service scaling settings.
Google automatically load-balances traffic across your app's instances.
It seems like static files can only be served from the same domain as the app.
I could create a new app for hosting static files but I'm a little nervous that that would a violation of the terms of service.
You don't need to serve your static content from a different app, you just need to use a different hostname. App Engine makes it pretty easy to have many different hostnames that point to the same app.
With wildcard subdomains you don't even have to create a DNS entry. If your app lives at myapp.appspot.com, you can also reach it through any subdomain, like static.myapp.appspot.com. If you're using your own domain, you'll need to configure it manually.
"4.4. You may not develop multiple Applications to simulate or act as a single Application or otherwise access the Service in a manner intended to avoid incurring fees"
"You may not develop" but, you can "Enable Billing" for multiple application. An example:
1) mysite.appspot.com
2) mysite-static.appspot.com
3) mysite-data-service.appspot.com
Section 4.4 of the terms of service prohibit one from splitting one logical app into two pieces - so hosting your dynamic content and static content in two separate GAE apps would violate the terms.
However, you could host static files on another web hosting service - anything from a simple shared hosting solution all the way up to a big CDN. This approach enables your site to serve static content from domains other than your app's domain.
I am a fan of the extensibility of the CMSes. You can upload some code (usually PHP), authorize it from the CMS admin panel and it's running.
I wonder if it is possible in Google App Engine. I haven't checked the extensibility of existing CMSes for Google App Engine, but if there is any of them that supports plugins I would like to know how they did it, and whether they are JS plugins only, or if they support Python/Java plugins too.
Nick Johnson from Google wrote an entire blog post series on how to write a blog system for app engine. If it doesn't do what you want, I am sure that you can extend it but normally a blogging system is sufficient for a CMS for most people.
I don't have a public example to point to (sorry), but I can confirm that it is possible to create Python plugins for an App Engine project. I completed a project a few months ago that does something like this. The crux of the thing comes down to a single line of python:
exec plugincode in someDict
Above 'plugincode' is a string containing some python code to execute, and someDict is a dictionary of globals to execute it in. This is arguably cleaner than using eval(). In our case the globals dictionary contained an instance of an object that the plugincode used to communicate with the system. I can't think of any major limitations with this (or similar) approaches. e.g. plugincode could declare a class, and register an instance of that class as a callback handler etc etc.
In our case we stored the plugin code in the Data Store, and loaded it at appropriate times (e.g. when an instance of the app is started).
Actually I see no conceptual problem with supporting plugins in App Engine application. For example on Java you may fetch plugin jar to memory from data store or memcache (on application initialization phase), and then use custom class loader to load plugin classes as needed). Actually you even may load classes from request data and evaluate them on the fly if needed (how we do it in AppWrench Java console).
Regards,
Pavel.
Is that somehow possible to access one datastore? Or access one app from different domains.
App Engine recently add support for a feature called modules (Docs: go, python, java)
App Engine Modules (or just "Modules" hereafter) is a feature that lets developers factor large applications into logical components that can share stateful services and communicate in a secure fashion. An app that handles customer requests might include separate modules to handle other tasks:
API requests from mobile devices
Internal, admin-like requests
Backend processing such as billing pipelines and data analysis
When you create a new module, App Engine will create a url that corresponds to the module name. If you only have one module then the name will be default. e.g.
http://default.myapp.appspot.com
http://mobile‑frontend.myapp.appspot.com
http://my-module.myapp.appspot.com
Using Domain masking, you could then direct from:
www.myapp.com => http://default.myapp.appspot.com
www.myapp-mobile.com => http://mobile‑frontend.myapp.appspot.com
www.example.com => http://my-module.myapp.appspot.com
Every app has its own datastore and memcache (shared among all versions of that app).
It seems not possible to share datastores between applications right now (unless you provide some web service for that), but that would be a nice feature to have, so maybe you should file a feature request with Google vote for it.
As for domains, you can associate your app with domains managed by Google Apps. Multiple domains for the same application should be no problem (except for SSL certificates).
Every version of an app is backed by the same datastore. If you want to limit access for individual requests, you'll need to add a field to your model to enforce that restriction. There are low level hooks in the datastore API for this sort of thing, if you want to go that far.
And yes, you can add a single App Engine app to multiple domains - even in multiple Apps accounts.
Kyle's solution would work, but App Engine was never designed to be used in this way. So if you architect your app(s) to rely on this kind of setup and Google clamps down for whatever reason then you'd be screwed.
You can have multitenancy using the Namespace Java API