It takes so long to deploy an app to App Engine by this command
gcloud app deploy
After I updated my source code, even if it's just one line of change, I had to issue this command again to upload the changes to App Engine. Again it will take a long time to deploy for just one line of code change. Is there a way to quickly update the code on App Engine instead of doing this time consuming deploy again?
I am looking for options that can speed up the code update after it's first deployment to App Engine. Since I am using flex, would using standard make it faster to update the code? Is it true that there is no way around it on App Engine and the only way to do it is to use another service such as Compute Engine?
Yes, standard env app deployment in such context is much faster, because it only involves uploading the modified files and restarting the instances.
But using standard env is often not an option for other reasons. The potential duplicate post I referenced contains itself references to both explanations and suggestions for potential speed improvements. I don't see a point in duplicating them.
The most common reasons for choosing flex env:
language support
restrictions of the standard env sandbox
See also: Google App Engine Standard or Flexible environment?
(Sep 2021): First of all, the issue you describe for App Engine (flexible) does not apply to App Engine (standard), which is almost a completely different product. (For those who are new, here is a comparison b/w both products.) This question is a few years old, and there are a few more options available to you now:
App Engine standard: If your app can run in Python 3.7+, Java 11, Go 1.12+, PHP 7, Node.js 10+, or Ruby 2.5+, then this is the obvious place to go. A deploy should take less than 1 minute.
Cloud Functions: If you don't have an entire app or your app is made up of one function/route or is otherwise short & simple, consider turning it into a microservice for Cloud Functions. Its deployment time is also less than a minute.
Cloud Run: If you're using App Engine flexible because your app is containerized, consider Cloud Run, which runs containers in a serverless way. A big bonus is that a Dockerfile is optional (due to Cloud Buildpacks). It has taken many GAE flexible use cases. Deployment is also less than a minute. If interested, I produced a pair of videos on how to migrate from App Engine standard to Cloud Run, using Docker as well as no Docker. At some point, I'll do one on transitioning from GAE flexible.
App Engine flexible: If none of the options above work, requiring you to stay on GAE flexible, then yes, you'll have to deal with its longer deployment time (due to the complexities of the system and all the resources it needs to spin-up & manage on your behalf). If you have a container and Cloud Run isn't an option, also consider GKE Autopilot or GCE MIGs (managed [VM] instance groups).
Related
I have around 10 microservices. Usually I use a script that loops through their folders (each of which contain an app.yaml for that service) and use gcloud app deploy to deploy it. However, is it possible to speed this up by using gcloud app deploy in say, 10 different terminals, one for each microservice? Will GAE processs the deploys in parallel or will this cause any issues?
Note: I am using App Engine flex, and from what I've been told, Standard environment deploys are a lot faster than deploys to Flexible environment. Regardless, switching to standard is not possible.
I tried deploying a couple of services on different tabs of the Cloud Shell Terminal and it works, so it appears to be possible but you should test that with all your microservices to see if this is not a bigger load than app engine could handle (which shoudn't be, but just in case).
However, it felt not worth the effort, since you have to lose time by opening up a new tab, waiting for it to be setup and typing the commands all over again. I did this manually, so if you are able to automate that opening of tabs into a script, it could be worth it.
Also, as you can see on this community post, another way to do a faster deploy is to deploy to another "version" of your app, then when all modules are deployed, do a very fast version switch.
NOTE: Not sure if you are doing this already, but an alternative is to stack the deployment of your services into a single command on Cloud Shell like this:
gcloud app deploy ~/my_app/app.yaml ~/my_app/another_service.yaml
Turns out it does work. I deployed 6 microservices in parallel, and it took < 10 mins for all of them to get deployed, compared to ~60 mins (average of 10 mins each for each service)!
For parallelizing it, I used Azure Devops pipelines, and since each job runs in parallel, all I had to do was create a job for each microservice, then run gcloud app deploy in each of the jobs to deploy them.
I have created a GCP Project and using App Engine standard environment for deployment purposes.
I have various developers working with me on the same project and deploying on App engine using various versions, however, we are using a default version to which all the traffic is allocated.
So are there any ways by which we can restrict the deployment on the default version. i.e we want specific people to be able to deploy on default version without removing deployment rights of other people on the same project.
And is there any alternative approach to this situation.
It depends on the runtime where you are deploying the application.
For example, in Python, if you are using the command gcloud app deploy (see the documentation), you can do the following:
gcloud app deploy --no-promote --version=<MY-VERSION-NAME>
The --no promote flag will avoid allocating all of the traffic to the version you are deploying, while the --version=<MY-VERSION-NAME> specifies the name of the version that you will create from the deployment, and replace an older one with the same name if it exists.
AFAIK there is no way to restrict deployment of a specific version. All access control method revolve around a certain identity being allowed access to deploy a certain GAE project or not. The version string used (i.e the version being deployed to in your approach) is irrelevant.
This falls into the "allows you to separate IAM roles" advantage mentioned in the accepted answer to Advantages of implementing CI/CD environments at GAE project/app level vs service/module level?.
As a note: you're attempting to implement environments at the service/module version level, which is IMHO worse than both methods compared in that post, see Continuous integration/deployment/delivery on Google App Engine, too risky?
I have a website on AppEngine that is 99% static. It is running on Python 2.7 runtime. Now the time has come to evolve this webapp, and since I have almost none Python code in it, I'd prefer to write it in Go instead.
Can I change runtime from Python 2.7 to Go, while keeping the project intact? Specifically, I want to keep the same app-ID, the same custom domain attached to it, the same SSL certificate, and so on.
What do I have to do in order to do that? I surely have to change runtime in the app.yaml. Is there anything else?
Bonus question: will such change happen without a downtime?
I'd be grateful for any links to documentation on exactly that (swapping runtime on a live app). I can't find any.
Specify a runtime as well as a new value for version. When deployed you'll have an older version that is Python and a newer version that is Go. There won't be any downtime (same as when deploying a newer version of Python).
Rather than trusting links/docs (that may be out of date or not 100% exactly what you're trying to do), why not create a new GAE-Std project for testing purposes and try it yourself. Having a GAE-Std test project is good for testing new function (especially by other testers who won't have access to the dev environ on your laptop).
The GAE services offer complete code isolation. So it should be possible to simply deploy a new version of the service, which can be written in a different language or even use a different GAE (standard/flex) environment. Personally I didn't go through a language change, but I did go through a split of a single-service app into a multi-service one, I see no reason for which the same principles wouldn't apply.
Maybe develop the new version as a separate app first, to be able to test it properly without risking an accidental impact on the old version and only after that bring the code as a new version in the old app. That'd be using the GAE project isolation. You can, in fact, test the entire version migration as a separate app if you so desire without even touching the existing app. I am using this technique - a separate app ID - to implement a staging environment for my app, completely isolated from my production app, see How to copy / clone entire Google App Engine Project
Make sure to not switch traffic to the new version at deployment time. This keeps the app working with the old version. Test first that the new version works as expected using Targeted routing. Then maybe use Splitting traffic across multiple versions to perform A/B testing with just a small percentage of the traffic going to the new version. Finally, when happy with the results, switch all traffic to the new version.
You need to pay special attention to the app-level configs (dispatch, cron, queue, datastore indexes), shared by all services/versions. They need to be functionally equivalent in the 2 versions. The service isolation doesn't apply to them, only project isolation can ensure no impact to the old version.
There should be no need to make any change to the app ID, custom domain mapping or SSL config. The above mentioned tests should confirm that.
A few potentially interesting posts related to re-working services/modules:
Converting App Engine frontend versions to modules
Google App Engine upgrading part by part
Migrating to app engine modules, test versions first?
Advantages of implementing CI/CD environments at GAE project/app level vs service/module level?
What is meant by "staging instances" of your application in this Google Cloud Debugger doc? As per this question it seems that only the default version of an application can be used with Google Cloud Debugger. Does Google simply intend that you can use this with the default version of a separate application which you dedicate to your staging environment?
Google Cloud Debugger can be used with any module or any version of your running application. The question you referred to is obsolete.
Many projects setup a module/version or simply another project to be used as their 'staging instance' to test their code before pushing to the production instance. There is no one way to implement a 'staging instance'.
The term staging in programming mostly refers to the term development.
In the app engine, you can have multiple instances of your application. One in Production environment(one instance) and other in a Staging environment(another instance).
So production means that its a version currently running or available to the public. Staging means that it is running only for testing purposes (usually new features) before its being deployed to the public
Google saying:
You can use the Cloud Debugger on both production and staging instances of your application
Simply means that you can you the Cloud Debugger on both the Production and Staging(Development) version of your application.
I have this application that will be run in a local network where a number of devices should interact with a database. I could use xampp and go for CherryPy or any other Python framework (Python is usually my choice) but it is the sum of a lot of different things: Python, Apache, MySQL... With GAE, which I have previously used in a number of applications successfully, I feel everything is neatly packed in a single box. Thay may not be true, but using the Google App Engine Launcher to create a local working copy of an app couldn't be easier.
But is it reliable? Should it be used like that? I know it's intended for development, so I'm unsure about using it as a local server in production. A few versions ago there even was this nasty bug that flushed the local datastore from time to time. But it seems that they fixed it and now data persists.
Would you recommend GAE for an application running in a local network or should I stick to LAMP (P for Python)?
Other alternative is http://code.google.com/p/appscale/.
May be you can check the the project TyphoonAE. I think it is exactly what you need.
The TyphoonAE project aims at providing a full-featured and productive
serving environment to run Google App Engine (Python) applications. It
delivers the parts for building your own scalable App Engine while
staying compatible with Google's API.