I've been thinking lately about the pros and cons of using AppEngine.
My concern would be, when we create application for GAE, the front-end code (the UI stuff) is served from the same application instance in the GAE cloud as with the Datastore codes.
The question would be when my applications grows:
For GAE:
Do I need to create multiple instance of my application?
If so, what do I need to manually update all instances?
For Appscale:
Do I also need to create multiple instance of my application?
If so, what do I need to manually update all instances?
GAE starts new frontend instances automatically, you even can't create or update frontend instances. You just need to configure min/max latency, min/max idle instances in Application Settings. See docs for performance settings
Btw, there are also Backend Instances that can be Resident and started manually from Admin Console. But it's useful only when you need something very specific
You seem to have missed the whole point of AppEngine, which is that Google takes care of scaling your app for you automatically. You seem to be confusing 'instance' with 'version' - you have control over which version of your app is serving, but Google dynamically creates and kills instances of that app depending on load. That's the main benefit of using AppEngine in the first place.
Related
How do I move instances from one project to the next?
Problem: I have frontend production instance and backend production instance in 2 separate project. I would like to have my production instances (both frontend and backend) in the same project.
How do I move instances from one project to the next within Google App Engine?
You could have the 2 apps run as 2 separate services inside the same project. To do that you'd have to deploy the app from the project that will disappear as another service in the project that will remain. Some code changes might be required, depending on how the app is coded. Typically they're small changes, though.
IMHO it would be simpler to keep the project currently hosting the frontend and move the backend over from the project to be retired. The reason is that the frontend is typically better fitted to play the default service role and it is already setup as such, changes should be smaller. Performing the move in the other direction would mean both services would need to changed.
You might need a dispatch.yaml file to route incoming packets, unless the URL mappings of the services are really well done. Not a big deal.
If you're using custom domains, with a bit of care it is possible to make the move (almost) transparent - by having the new service up and running before retiring the old project. DNS propagation time becomes rather irrelevant - both old and new services being simultaneously ready during the transition.
Note: technically you wouldn't be "moving" instances, you'd just stop the ones serving the project to be retired and start new ones for the service added to the project to be kept.
More or less related:
Change runtime from Python to Go in App Engine standard environment
Can a default service/module in a Google App Engine app be a sibling of a non-default one in terms of folder structure?
In my app (Google App Engine Standard Python 2.7) I have some flags in global variables that are initialized (read values from memcache/Datastore) when the instance start (at the first request). That variables values doesn't change often, only once a month or in case of emergencies (i.e. when google app engine Taskqueue or Memcache service are not working well, that happened not more than twice a year as reported in GC Status but affected seriously my app and my customers: https://status.cloud.google.com/incident/appengine/15024 https://status.cloud.google.com/incident/appengine/17003).
I don't want to store these flags in memcache nor Datastore for efficiency and costs.
I'm looking for a way to send a message to all instances (see my previous post GAE send requests to all active instances ):
As stated in https://cloud.google.com/appengine/docs/standard/python/how-requests-are-routed
Note: Targeting an instance is not supported in services that are configured for auto scaling or basic scaling. The instance ID must be an integer in the range from 0, up to the total number of instances running. Regardless of your scaling type or instance class, it is not possible to send a request to a specific instance without targeting a service or version within that instance.
but another solution could be:
1) Send a shutdown message/command to all instances of my app or a service
2) Send a restart message/command to all instances of my app or service
I use only automatic scaling, so I'cant send a request targeted to a specific instance (I can get the list of active instances using GAE admin API).
it's there any way to do this programmatically in Python GAE? Manually in the GCP console it's easy when having a few instances, but for 50+ instances it's a pain...
One possible solution (actually more of a workaround), inspired by your comment on the related post, is to obtain a restart of all instances by re-deployment of the same version of the app code.
Automated deployments are also possible using the Google App Engine Admin API, see Deploying Your Apps with the Admin API:
To deploy a version of your app with the Admin API:
Upload your app's resources to Google Cloud Storage.
Create a configuration file that defines your deployment.
Create and send the HTTP request for deploying your app.
It should be noted that (re)deploying an app version which handles 100% of the traffic can cause errors and traffic loss due to:
overwriting the app files actually being in use (see note in Deploying an app)
not giving GAE enough time to spin up sufficient instances fast enough to handle high income traffic rates (more details here)
Using different app versions for the deployments and gradually migrating traffic to the newly deployed apps can completely eliminate such loss. This might not be relevant in your particular case, since the old app version is already impaired.
Automating traffic migration is also possible, see Migrating and Splitting Traffic with the Admin API.
It's possible to use the Google Cloud API to stop all the instances. They would then be automatically scaled back up to the required level. My first attempt at this would be a process where:
The config item was changed
The current list of instances was enumerated from the API
The instances were shutdown over a time period that allows new instances to be spun up and replace them, and how time sensitive the config change is. Perhaps close on instance per 60s.
In terms of using the API you can use the gcloud tool (https://cloud.google.com/sdk/gcloud/reference/app/instances/):
gcloud app instances list
Then delete the instances with:
gcloud app instances delete instanceid --service=s1 --version=v1
There is also a REST API (https://cloud.google.com/appengine/docs/admin-api/reference/rest/v1/apps.services.versions.instances/list):
GET https://appengine.googleapis.com/v1/{parent=apps/*/services/*/versions/*}/instances
DELETE https://appengine.googleapis.com/v1/{name=apps/*/services/*/versions/*/instances/*}
Let me start with a bit of background: I'm helping a non-profit organization that would like to have a browser-based application that is backed by Salesforce, but has very specific requirements.
I see Salesforce has a REST API that we can call, so we can develop a standalone application to serve the web pages they want and use the REST API to call Salesforce when needed.
I'm wondering if there is a way to host a web application directly on Salesforce; this way we don't have to have a separate application server. Any recommendations or pointers to documentation/open source products is greatly appreciated.
Yes, you can create services that will allow your app to hit Salesforce
Depending on the type of application, yes you can host it on salesforce using the Salesforce Sites feature, also you can develop and host your app on Heroku which is owned by salesforce and can sync data to and from salesforce using Heroku Connect, or you can build and host it on another service like AWS and connect via the REST API. You just need to investigate and choose the option that best fits your use-case. One thing to be aware of is that there are API limits (the number of calls you can make to salesforce in a rolling 24hr period). Depending the the needs of the app be sure to see if those limits will be an issue. Because if the app makes constant calls to salesforce that could be an issue. But there are things you can do to get around that, like caching.
Yes, both Force.com Sites and Site.com features allow you to host webpages on the Force.com Platform. The markup is stored in Visualforce Pages and can use Apex to access records in the Database. I have migrated multiple websites (including our company's www.mkpartners.com) to Force.com using Force.com Sites.
One thing to keep in mind is that you are limited to 500,000 views per month and the rendering of a page with images that are also stored on the platform will incur a single view for the page and a single view for each image. If you already have a very popular website, I wouldn't migrate. If you're a small business or nonprofit, then it should be fine.
Another thing to keep in mind is that dynamic functionality based on records in the database will not work during maintenance windows. There is the ability to upload a static version of your website to be rendered during these windows though.
I have an standard appengine app currently running. I am currently developing another flask app which will use flexible runtime. I am wondering is it possible for me to have both apps in same project?
Yes, it is possible, with each of your "applications" being implemented as separate services/modules in the same GAE app. Services/modules offer complete code isolation, see Comparison of service isolation and project isolation.
See also: Custom runtime for non-flexible environment app?
There is an even easier way to do this that doesn't require creating a separate service :)
Since you are only developing your second application, you do not need to set it as the default version to serve traffic to the public. You only need to access it yourself, (or perhaps whoever you give the URL to). The only drawback really is that you will not be able to access your project at the default url, usually .appspot.com
Therefore, you can simply deploy your flexible app engine project, but make sure you don't give it any traffic sharing or promote it to the main version!! You can then access your version simply by clicking on the version in the google cloud console, or by visiting http://-dot-.appspot.com/ (It was not possible to access individual versions like this for flexible for quite some time, but this has now been resolved.)
We have Java servlets up and running on GAE, using blobstore, datastore and other cloud services.
Currently, we're starting a migration process to cloud Endpoints and we've hit an issue: if we use a different GAE project, we would not be able to query regarding current datastore entities (to the best of my knowledge, Google doesn't want you to do this - see
this question
and the GAE terms of service - section 3.3d), so we need to use the same project for both.
I looked up whether it's possible to have one GAE instance running Java servlets and one instance running Endpoints, but I found no conclusive answer anywhere.
If we try to implement and something goes wrong, we're looking at a potentially major issue for our users, so we need to be sure beforehand.
Has anyone tried something similar, and can assure us that this works?
You have 2 options to run the old and the new code inside the same app (thus with no issues sharing access to the datastore) but as separate engine instances, so they can be developed/deployed/managed independently:
as different versions of the same app/module(s):
the old version remains the default, the new one can be accessed at a different URL during development (possibly via URL routing)
you can use traffic splitting to do live A/B testing on the new code and for gradual final migration until you make the new version the default
as different modules of the same app:
both can run (fully functional) side by side indefinitely, but you need to be more careful during development
traffic is routed to the modules in several possible ways
final migration is done by publishing the new URLs, eventually re-directing the old URLs and finally bringing down the old module code
The 2 approaches can even be combined, if needed, as the final solution described by the OP's in this somehow similar question (for the python environment, but java equivalents exist): Google App Engine upgrading part by part