Is there a way to update selected files when using the App Engine Flexible env?
I'm facing an issue whenever I do a small change in the app.yaml file: to test it I would need to deploy the whole application which takes ~5mins.
Is there a way to update only the config file? OR is there a way to test these files locally.
Thanks!
The safe/blanket answer would be no as the flex env docker image would need to be updated regardless of how tiny the changes are, see How can I speed up Rails Docker deployments on Google Cloud Platform?
However, there might be something to try (YMMV).
From App Engine Flexible Environment:
You always have root access to Compute Engine VM instances. SSH access to VM instances in the flexible environment is disabled by
default. If you choose, you can enable root access to your app's VM
instances.
So you might be able to login as root on your GAE instance VM and try to manually modify a particular app artifact. Of course, you'd need to locate the artifact first.
Some artifacts might not even be present in the VM image itself (those used exclusively by the GAE infra, queue definitions, for example). But it should be possible to update these artifacts without updating the docker image, since they aren't part of the flex env service itself.
Other artifacts might be read-only and it might not be possible to change them to read-write.
Even if possible, such manual changes would be volatile, they would not survive an instance reload (which would be using the unmodified docker image), which might be required for some changes to take effect.
Lots of "might"s, lots of risks (manual fiddling with the app code could negatively impact its functionality), up to you to determine if a try is really worthy.
Update: it seems this is actually documented and supported, see Accessing Google App Engine Python App code in production
Related
I deploy my Docker image to default GAE by gcloud app deploy --image-url=us.gcr.io
I have successfully mapped my custom domain to this application with custom runtime and flex env.
My dispatch.yaml sends requests to its sub-domain:
dispatch:
- url: "dev.domain.com/*"
service: default
Now I want to use different images from Container Registry for test.domain.com and domain.com
While having all these images sharing same Cloud Storage and Firebase credentials.
Being new to GCP I wanted to learn any simple approach to organize such basic structure without going into services and versions (just by assigning proper images to serve relevant domains).
Is it even possible to do within one GAE project or I should create separate projects for it?
Mapping custom domains can only be done at the service level, so if you don't want to go into services separate projects is your only choice.
Actually using separate projects instead of services (or service versions) for implementing different environments has some advantages, I'd choose separate projects, too. See Advantages of implementing CI/CD environments at GAE project/app level vs service/module level?
I'm not sure if sharing the storage and credentials between production and other environments is a good idea (what if something goes wrong?). I'd keep them separate, too (maybe with some jobs to populate non-production projects with production data, if you need to). But if you do want to share them across projects you'll probably need to make some extra steps.
How do I move instances from one project to the next?
Problem: I have frontend production instance and backend production instance in 2 separate project. I would like to have my production instances (both frontend and backend) in the same project.
How do I move instances from one project to the next within Google App Engine?
You could have the 2 apps run as 2 separate services inside the same project. To do that you'd have to deploy the app from the project that will disappear as another service in the project that will remain. Some code changes might be required, depending on how the app is coded. Typically they're small changes, though.
IMHO it would be simpler to keep the project currently hosting the frontend and move the backend over from the project to be retired. The reason is that the frontend is typically better fitted to play the default service role and it is already setup as such, changes should be smaller. Performing the move in the other direction would mean both services would need to changed.
You might need a dispatch.yaml file to route incoming packets, unless the URL mappings of the services are really well done. Not a big deal.
If you're using custom domains, with a bit of care it is possible to make the move (almost) transparent - by having the new service up and running before retiring the old project. DNS propagation time becomes rather irrelevant - both old and new services being simultaneously ready during the transition.
Note: technically you wouldn't be "moving" instances, you'd just stop the ones serving the project to be retired and start new ones for the service added to the project to be kept.
More or less related:
Change runtime from Python to Go in App Engine standard environment
Can a default service/module in a Google App Engine app be a sibling of a non-default one in terms of folder structure?
Apologies for the seemingly obvious question, but I figure the answer might help others. I can't for the life of me find documentation on the filepath within the Google App Engine VM (Cloud Shell) where I can find the static files being served from. I need to pull the latest upstream changes from a private github repo.
Note that I navigated elsewhere in the VM and even restarting the session didn't put me in a default project root path within the VM as I expected it to.
There are several issues to address here:
The Cloud Shell is a virtual shell
Google Cloud Shell is an interactive shell environment for Google
Cloud Platform.
The environment where you're working is a container running in a VM in a Google-owned project inside GCP.
You can verify this by checking the metadata server (only available for GCP VMs):
curl -H 'Metadata-Flavor:Google' "http://metadata.google.internal/computeMetadata/v1/?recursive=true&alt=text"
In the metadata provided you'll see how this container is created and configured.
The Cloud Shell is tied to the user, so you'll always access the same environment if you access it with the same credentials, no matter the project. However, if you access with a different user, you'll get a different environment.
You can't access GAE standard instances
GAE is a fully managed environment, and you won't be able to access it. In this way, you won't be able to find the root of the running app engine project.
However, by the way GAE deploys your code, it uses a staging bucket to gather the code before compiling. You can find your staging bucket through the App Engine Admin API. This is usually staging.<PROJECT_ID>.appspot.com, although you can change this configuration. You can get your files from there.
You can access GAE flex apps
However, the deployment in flex gets your files, build a Docker container with them, and then deploys this container inside a VM.
As per the docs, you can connect directly to your container by running:
gcloud app instances ssh [INSTANCE-NAME] --service [SERVICE] --version [VERSION]
docker exec -it gaeapp /bin/bash
Regarding your issue
According what you say in the comments of the question, your issue could come from a myriad of places. From changing the shell you're connecting to, to resetting your shell environment (deleting all the files), to a thousand different possible problems.
The best way to think about it is regard the Cloud Shell as a temporal environment to run commands, but not as a virtual machine.
Knowing that, you could mount a persistent filesystem (GCS through GCSFuse, Cloud Filestore, ...) to persist your work, or simply use Git to have your work always synced on a repo.
GAE Flex has some nice CI integrations, so that's a plus for going the Git route.
I'm just going through the node.js tutorials with a free trial account, and i'm stuck on the second one where you add a db. I add the mongodb deployment, shows up as a VM instances, fine. And my first deploy worked, but now that i'm trying to edit stuff, my deploy's keep failing.
The error i get is that I've exceeded my CPU quota. Watching the list of VM Instances under Compute Engine, i see it keeps spawning up instances, even though the app isn't being used. Guessing it just spins up 8 instances by default?
But then i guess the build system needs its own VM's, but the CPU capacity is used up, so none available to do subsequent builds?! I feel like i'm missing something...
Also, i see i can explicitly start VM's myself, so what process is creating them form me? And can i turn it off? or set a cap on number of instances it spawns?
Can i tell my project to only use 4
Also, the deploy takes forever, is that normal? Following the tutorials, so far I've only seen this command to deploy:
gcloud preview app deploy app.yaml --set-default
Is there another command that does an incremental deploy or something?
By using gcloud preview app deploy you're actually using Managed VMs which is an App Engine runtime which in turn runs Docker containers on Google Compute Engine, which it creates on its own. In other words, you're not using Google Compute Engine directly.
To get rid of extra VMs, you need to delete old app versions: navigate to Compute > App Engine > Versions and delete the versions you don't want.
See also this answer for more details and suggestions.
When I re-deploy an app to the app engine, the app the existing data in the datastore is cleaned-up.
How do I prevent this from happening?
Update:1 I am using Java, 1.6.3. This is on the production system.
the datastore in production does not get flushed if you redeploy. if you change model definitions and structures it might be that the data you need does not belong to the new(er) model definition...
if you are speaking about the datastore on your local machine start the dev server with the
--datastore_path=/tmp/myapp_datastore myapp
if you are using python. don't know what the equivalent for the java SDK is.
you need to provide more details.
It might be that you are changing the app name just prior to deployment (so you're deploying to the correct app). But previous data in your local datastore was stored under the prior name. If you don't change the app name back before using the local datastore again, it can appear as though the data has disappeared.