Short of writing a microservice which runs the appcfg command in the shell ("appcfg -q -A my-app-engine list_versions") and returns the list of the deployed versions, is there a way from within a deployed application to query for the list of deployed versions?
In case of Java it seems that com.google.appengine.api.modules.ModuleService (javadoc) is what you are looking for. It provides the below method returning the set of versions that are available to the given module.
Set<String> ModuleService.getVersions(String module)
In case of Python look at google.appengine.api.modules.modules module (pythondoc) that has similar function.
get_versions(module=None)
Related
Apologies for the seemingly obvious question, but I figure the answer might help others. I can't for the life of me find documentation on the filepath within the Google App Engine VM (Cloud Shell) where I can find the static files being served from. I need to pull the latest upstream changes from a private github repo.
Note that I navigated elsewhere in the VM and even restarting the session didn't put me in a default project root path within the VM as I expected it to.
There are several issues to address here:
The Cloud Shell is a virtual shell
Google Cloud Shell is an interactive shell environment for Google
Cloud Platform.
The environment where you're working is a container running in a VM in a Google-owned project inside GCP.
You can verify this by checking the metadata server (only available for GCP VMs):
curl -H 'Metadata-Flavor:Google' "http://metadata.google.internal/computeMetadata/v1/?recursive=true&alt=text"
In the metadata provided you'll see how this container is created and configured.
The Cloud Shell is tied to the user, so you'll always access the same environment if you access it with the same credentials, no matter the project. However, if you access with a different user, you'll get a different environment.
You can't access GAE standard instances
GAE is a fully managed environment, and you won't be able to access it. In this way, you won't be able to find the root of the running app engine project.
However, by the way GAE deploys your code, it uses a staging bucket to gather the code before compiling. You can find your staging bucket through the App Engine Admin API. This is usually staging.<PROJECT_ID>.appspot.com, although you can change this configuration. You can get your files from there.
You can access GAE flex apps
However, the deployment in flex gets your files, build a Docker container with them, and then deploys this container inside a VM.
As per the docs, you can connect directly to your container by running:
gcloud app instances ssh [INSTANCE-NAME] --service [SERVICE] --version [VERSION]
docker exec -it gaeapp /bin/bash
Regarding your issue
According what you say in the comments of the question, your issue could come from a myriad of places. From changing the shell you're connecting to, to resetting your shell environment (deleting all the files), to a thousand different possible problems.
The best way to think about it is regard the Cloud Shell as a temporal environment to run commands, but not as a virtual machine.
Knowing that, you could mount a persistent filesystem (GCS through GCSFuse, Cloud Filestore, ...) to persist your work, or simply use Git to have your work always synced on a repo.
GAE Flex has some nice CI integrations, so that's a plus for going the Git route.
So it seems from a few SO questions I've seen that this is a problem among other users. Recently one of our head dev's left and I inherited a lot of his projects. One of which, is a website that what seems like lives on an app engine from google cloud platforms. From the App Engine documentation, to download source code you use the appcfg.py download_app command. Which I did, however the only results I get back from that call is:
Fetching file list...
Fetching files...
And then it just ends. No error message or any kind of message at all, and of course, it did not download the source code into the output dir I specified.
Scratching my head and looking at various SO posts, someone mentioned something about going into the google cloud vm directly and doing the same command, and to my surprise finding the same exact behavior that I did in my local terminal.
This made me realize it must be something else at play. I took a look at my versions tab in the App Engine dashboard on GCP. I see my instance running, it correctly says Serving and if I click the link it brings me to the website which loads fine. However, under Size it says 0 B which made me think perhaps this is why the download_app isn't downloading anything, because the version is 0 B?
What I'm trying to figure out is why it says 0 B for the version, when clearly the site runs fine and how I can get the source code for this. Here's a screenshot for reference
And screenshot of my terminal (local). Obviously I omitted the -A and -V flags, but they are correctly set and if I purposely make them incorrect I do indeed get an error message.
EDIT
Just so everyone is aware, I also made sure my user had the correct permissions. Owner, App Engine Owner... and some others. I don't think that's the problem.
When you deploy an App Engine Flexible application, the source code is uploaded to Cloud Storage on your project in a bucket named staging.<project-id>.appspot.com. You can navigate in this bucket and download the source code for a specific version as a .tar file.
Alternatively, you can find the exact Cloud Storage URL for your source code by going to Dev Console > Container Registry > Build History and select the build for your version. You'll find the link to your source code under Build Information.
One thing to note however is that the staging... bucket is created by default with a Lifecycle rule that deletes files older than 15 days automatically. You can delete this rule if you want so that all versions' source code is kept indefinitely.
In your case I believe that may not have helped since files may have been deleted already but it's worth knowing you can get the source code from there (source code isn't pushed to Source Repository by default, your developer had to configure it manually).
Posting this since none of the listed methods on the web didn't take me to the code (by June 2021)
Note: appcfg.py is deprecated by Google
You could try accessing your source code through;
Google Cloud Platform > Debugger > choosing the version of the
Application from combo at top.
This will list the files of that version on the left pane. There is no way to download code automatically but you can copy-paste the code.
Advice: Push your code to a Git repository to avoid this hassle next time.
Hope you will find this helpful.
In the developer console you can select the respective project and check:
on the Services page - which services, AKA modules - as they used to be (and still are) called in various places, you app has deployed
on the Versions page - which versions for each of the services are deployed
This information is what appcfg.py download_app expects. See also:
the various appcfg.py options using its --help flag
How do I download a specific service's source code off of AppEngine?
You can also access the deployed source code live (if everything else fails it could still be a last resort method to get the code, but tedious), see my answer to Google Cloud DataStore automatic indexing
Update:
I just now noticed in your screenshot that it's a flexible environment app. The appcfg.py docs are in the standard environment section, I suspect it's not applicable to the flexible environment, for which what's deployed is actually a docker image built during the deployment operation. From Deploying your application:
Deploy your app to App Engine using the gcloud app deploy
command. This command automatically builds a container image by using
the Container Builder service and then deploys that image to the
App Engine flexible environment. The container will include any local
modifications that you've made to the runtime image.
It might be possible to access the code on the actual GCE instance running the app, by connecting to the running instance and starting a shell in your app container, see Connecting to the instance
I am writing a web service (java servlet) in the flexible environment of Google App Engine and I would like to be able to retrieve the GCP project ID that I am running in at runtime programatically. How?
There is a SystemProperty which contains such thins as project id, app version, etc:
SystemProperty.applicationId.get()
Javadoc is here: https://cloud.google.com/appengine/docs/java/javadoc/com/google/appengine/api/utils/SystemProperty
SystemProperty is a part of com.google.appengine.api which is not available on App Engine Flex environment.
On Flex your project name is available as environment variable.
So you simply can use just this:
System.getenv("GCLOUD_PROJECT");
Please see Migrating Services from the Standard Environment to the Flexible Environment for more details.
Also please note that on local development server running Jetty this environment variable might not be set. If so the above code will return null on your development machine.
You might want to set it explicitly though for your local development server needs. For example if you use Bash you can do:
export GCLOUD_PROJECT=myproject
Also you can add it to your development server startup script or to your ~/.bashrc script, whatever works better for your needs.
Suppose I uploaded another version (say, "version: 2" in file app.yaml ) of my Google App Engine application. Version 1 is still the default and version 2 is for testing. How do I run it then?
Once you upload a version on Appengine, you can switch between them easily.
Say that your app name is myapp, currently running version 1. You also have uploaded a version called 2-testing. Your default app (with version 1) can be reached by accessing myapp.appspot.com
If you wanted to access your versions explicitly, you joust need to access <version_name>-dot-myapp.appspot.com. Following the example it would be:
1-dot-myapp.appspot.com or 2-testing-dot-myapp.appspot.com
The -dot- is equivalent to <version>.<appname> but allows you to correctly serve a secure application with SSL
You can mark any version you want as default (serving myapp.appspot.com) using the admin console
edit: this is the official documentation page talking about domains and subdomains in Appengine
Under versions in your admin console you can find the live uri of a version, if you select the version.
And you can use traffic splitting, where you can use your own client ip or a cookie to test a version.
Docs: https://developers.google.com/appengine/docs/adminconsole/trafficsplitting
is there an easy way to execute scripts on Google App Engine localhost?
For instance i have few scripts to generate pilot data, it's quite painful to copy-and-paste them to the interactive console all the time.
Currently, i'm using a simple bash script to do this, but i'm not sure this is the best solution.
curl --data-urlencode "code=`cat src/gen_pilot_data.py`" http://localhost:8079/_ah/admin/interactive/execute
thanks,
V
Check out remote_api_shell.py, included in the SDK. This lets you run code on your local machine against APIs on your App Engine app, be they in production or on the dev_appserver.
The other option you have is to make your scripts into handlers, and simply GET or POST to the relevant URLs.
There is nothing special to do in order to execute script on AppEngine, but if you want to use AppEngine APIs you have to boot AppEngine before. I don't know the builtin one-line way, but you can take look how various projects do this:
https://bitbucket.org/wkornewald/djangoappengine/src/4f5d7a223084/boot.py
http://code.google.com/p/nose-gae/source/browse/trunk/nosegae.py?r=54
You can also try to import your script in console:
import gen_pilot_data