Google Cloud's structure related to "projects" has me really confused.
On the one hand all GCP services are encapsulated in a "project" right? So I think, OK I'll create something like "test", "stage", and "prod" projects. All my applications can be tested in "test" and eventually move to "prod" when they are ready to go live. Also, I can have SQL,bigquery,bigtable and whatever else in the test project that developers can hack on without having to worry about effecting production.
But I can only have one app engine app per project? How does that work? I can see how in app engine you have different versions so if I have one project per app engine app the test/staging mechanism is in that app's project, but what about the other GCP services?
If I have a bigtable or bigquery or something on storage multiple apps need to access what "project" do I put that stuff in?
Do I still have a "test","stage","prod" project for my services (where my DBs, storage, etc live), but then also create separate projects for each app engine app?
If multiple apps need to access something, it can live in one of the app's projects- that doesn't make sense.
Edit: google does have some good docs about how projects and services can be organized https://cloud.google.com/appengine/docs/python/creating-separate-dev-environments
While you can only have one App Engine app per project, an App Engine app can host multiple services, each of which has several versions of code deployed.
You can configure resources in one project to allow access to users/apps outside that project. See, for example Setting ACLs for how you can allow multiple projects to access a Cloud Storage bucket. Similar cross-projects access can be configured for most if not all Google Cloud resources/services/apps - but you need to check the respective docs for each of them to see the specific details each of them may have.
With this in mind it's really up to you to organize and map your apps and resources into projects.
Related
I'm looking at moving my application to google cloud and I'm having a hard time understanding how best to organize my project. It seems like you can only have one App Engine application per project with services available to support a microservices architecture and instances representing the App Engine instances created via auto scaling.
What is the correct way to build an App Engine for my API server and an App Engine for my Web Server? Do I need to have a project for each? I'm essentially trying to accomplish the following:
It is straightforward to have a single GAE project implement both your website and an API. You can even do this within a single service. You could put each in a separate service, and the advantage of that is that you can update one without updating the other. For small projects, a disadvantage is that two services are more expensive than one (though GAE is quite inexpensive overall).
For prod vs dev, you'll need to explain your requirements a little more, but here are some thoughts.
Each GAE service has multiple versions. You can deploy your production version to www.mycompany.com and deploy a dev version to dev-dot-myapp.appspot.com (that's the way GAE does URLs for versions of your app). Both of these versions will access the same datastore so you need to be careful with the dev version so that it doesn't mess up your prod implementation.
If you have a dev situation that is bleeding edge and shouldn't be able to access the datastore of your production app, then you would create a different GAE project for that.
Here is a way to visualize it:
Google Cloud Project A
GAE Project A
production www service
production API service (this could be combined with production www service)
dev www service
dev API service
Google Cloud Project B
GAE Project B
bleeding edge www service
bleeding edge API service
Best Practices:
Your Dev and Prod should be in separate projects for both security and billing purposes.
App Engine:
You can only have one App Engine per project. This will create a problem for you to use App Engine for both API Server and Web Server. In this case, I would not use App Engine at all and instead look at Containers on Compute Engine or go for Kubernetes.
Even a single node Kubernetes Cluster will shock you with its flexibility and power. Containers on Compute Engine still have a lot flexibility and power too. If you like the concepts of App Engine Flexible, then you might really like containers. The exception here is that App Engine makes some concepts brain dead simple where you have more work in configuration for Containers or Kubernetes.
You have a Google Cloud project Project1 with App Engine enabled and serving your legacy application.
You want to rebuild your application using a different programming language and hence need a separate project (Project2) with its own separate App Engine.
You want to install some kind of reverse proxy appliance in front of your Project1 App Engine, so that you can gradually route more and more traffic to Project2's App Engine.
What's the best way to achieve this?
Google Cloud Load Balancer doesn't appear to work with App Engine (only Google Compute Engine), so I'm thinking establishing a reverse proxy server (e.g. NginX) container would be the best bet.
Depending on how you want to organize, you can duplicate your entire application within the same project by just deploying as a different service or as new versions of your existing service. For example, say you have two services frontend and backend for a simple web app. (Let's say they're both written in Python)
For deploying as a separate service, you can create a new version of your application in a different language, say Node.js. You can deploy the new services as frontend-node and backend-node.
If you don't want to do that, you can rewrite your application in a different language and then deploy as a new version of your existing services. In the "Versions" section, you can see your versions of the same service and they can be different language runtimes.
Either way, you can then use the "split-traffic" feature to customize and test implementations of your application.
Generally speaking, you should avoid using projects to isolate different variants and/or components of your application unless you really need too. App Engine services can each be a different runtime from each other so there's almost no point in provisioning a whole new project even though you're redeploying in a different language.
I have an application deployed to an app engine service. I have many services under the same app engine. How can I make the application available to certain white-listed IPs? In other words, I want this application to accept requests from certain IPs and deny all other request?
Can we do this by writing some configurations in app.yaml file?
Note: I just want to apply the rule to one service only so that other services will not be affected.
Applying this kind of restriction at a service level is, at the moment, not possible.
The best option would be to deploy the services you want to protect on a different project, and use the App Engine firewall there.
I am considering developing a platform in GAE.
for this to work I would like to be able to have run the same app engine instance run several 'sister' sites. This would be something similar to the way that the stack exchange sites work, so each 'sister' site would have its own users, data and theme (but may potentially be configured slightly differently - think different plugins used), but essentially would have the same function & format.
I would like to know if it would be possible to have each sister site have its own (customer defined) url, which would use the same app engine instance, and the app engine instance would be configured (by setting the site id and valid plugins for that site) depending on the url which was used to access it.
Yes, this is entirely possible. As Fox32 points out, you can use namespaces to separate different datasets. For domains, App Engine supports wildcard mappings, so you can map *.yoursite.com to your app. If you want to add custom TLDs, either you or your customers will need to create a Google Apps account for them, or add the domain as an alias on an existing Apps account, and associate it with the App Engine app - this doesn't require that they be admins of your app to do so.
You can use namespaces, to separate different websites, for many api methods. Take a look at the GAE documentation about it.
The Namespaces API is integrated with Google Apps, allowing you to use your Google Apps domain as the current namespace. Because Google Apps lets you deploy your app to any domain that you own, you can easily set unique namespaces for all domains linked to your Google Apps account.
We would like to develop and sell custom commercial GAE applications. I would like information on the deployment of GAE applications on arbitrary Google Apps domains (i.e. not appspot).
Suppose our company is abc.com and we are selling app to def.com and xyz.com. What are the steps to deploy our app on our customer's domain?
When an app is deployed on many domains:
Is the code duplicated or shared?
Is the datastore definition duplicated or shared (or more precisly the kinds)?
Any domain attribute add to the entity?
You have three options at the moment, when it comes to a 'multi-tenant' app such as you describe:
You can have a single app that your customers add to their domains. Your app will have a single datastore, but you can use the Host header to determine which customer is accessing the app, and segregate the datastore entries based on that.
Easy to deploy and upgrade
Easy for customers to install
Users have to have Google accounts, not Apps accounts, to log in.
You can deploy a fresh app instance for each customer.
Harder to deploy and upgrade
More customer involvement required to install
Provides firm separation of data
Users can log in with their Apps credentials
You can work with Google to create a new Apps Marketplace app
All the benefits of point 1 and 2, above
Requires Google involvement
No certain release date yet
This is possible but to the best of my knowledge, def.com & xyz.com will have to sign up for Google Apps.
Once the domain is registered with Google Apps by your prospective customers, they can add not only Google Apps like Gmail & Docs to subdomains (mail.def.com & docs.def.com) but also any GAE apps (fooapp.def.com).
Is the code duplicated or shared?
Shared
Is the datastore definition duplicated or shared (or more precisly the kinds)?
While the schemas (definitions) are shared, what isn't shared is the actual datastore. i.e. each instance of your app will have separate data.
Any domain attribute add to the entity?
You can figure out which domain your app is being served under using one of the CGI/HTTP environment variables. I don't remember which variable exactly but I'll update the answer once I look at some old code of mine. Once you find out what is the current domain, you can choose what to do with that info, including saving it to an Entity or simply denying access.
Another cool feature about this is that let's say company def.com is using hosted GMail and have Google Authentication for their employees. Now if your app is using the GAE provided authentication hooks, then you can automatically authenticate just their employees without any code change on your part! At least, that's the theory since I haven't tried it out myself ;-)