I currently have a small Google App Engine project written in PHP. On a traditional web server, I would simply invoke this particular code via command line (e.g., php whatever.php). I'd like to set this up with cron.yaml to just run every hour or so without invoking a HTTP request if possible. How would you go about doing this?
You can do this by adding a handler url that points to your script as follows:
In your app.yaml it would look like this:
handlers:
- url: /mycron
script: cron.php
login: admin
The login parameter will only allow execution of the endpoint by Google system accounts or something like that so you enpoint won't be publicly accessible.
More information in the below link
ref: https://cloud.google.com/appengine/docs/standard/php/config/cron#securing_urls_for_cron
In order to schedule tasks using the cron.yaml file, otherwise known as cron jobs, you can use the following structure inside the file:
cron:
- description: "running my PHP code"
url: /your-app-url
target: your-service
schedule: every 60 minutes
You can edit the different cron fields to your convenience by following this syntax.
Place the cron.yaml file on the same directory as your app.yaml file (your application's root directory) and, before deploying, test it by going to http://localhost:8080/cron. If it works, you can deploy the application with the cron job by running this command:
gcloud app deploy cron.yaml
You can find additional information about cron jobs, such as how to retry failing cron jobs and securing or deleting them by following this link.
Since I upgraded my Gradle build to deploy my App Engine application with SDK 1.9.46, I get deployment conflict messages, like the one below.
Beginning interaction for module admin...
0% Created staging directory at: '/var/folders/m8/6z4h4k2x11s3whxrqmd79lym0000gn/T/appcfg2292422843793738444.tmp'
5% Scanning for jsp files.
8% Generated git repository information file.
20% Scanning files on local disk.
25% Scanned 250 files.
28% Scanned 500 files.
31% Scanned 750 files.
33% Scanned 1000 files.
34% Initiating update.
Nov 13, 2016 11:11:41 AM com.google.appengine.tools.admin.AbstractServerConnection send1
WARNING: Error posting to URL: https://appengine.google.com/api/appversion/create?module=admin&app_id=the-outdoor-game&version=dev&
409 Conflict
There is an operation pending for this application: AppLock held for "Updating engine_version_id='dev' within engine_id='default'." acquired by Peter.Fortuin at 2016-11-13 09:15:05.482616 GMT. Please wait and try again or use 'appcfg rollback' to attempt to cleanly abort the current operation.
This is try #0
A rollback if not solving the problem. The only thing that seems to work is downgrading the App Engine SDK to 1.9.44. But that gives me other problems that I don't want to solve.
Anybody has any idea how to solve this? Can it be that this is a bug in the 1.9.46 SDK? Anybody else seeing the same thing?
I have resolved this problem by changing 'service' to 'module' in appengine-web.xml of my module.
See https://github.com/GoogleCloudPlatform/gradle-appengine-plugin/issues/268 for more information.
Currently the GAE Managed VMs feature is broken for any project with a name longer than 27 characters.
The underlying issue is that Docker restricts image namespace to between 4-30 chars. This has been fixed (https://github.com/docker/docker/issues/10392) but is still awaiting a release at time of writing.
It seems when deploying a Managed VM to GAE that the namespace is automatically generated from your project name plus _m_ prefix. This leads to error when attempting to deploy the vm:
DEBUG: "POST /v1.10/images/gcr.io/_m_<my project name>/<my project name>.default.20150330t140211/push HTTP/1.1" 500 111
INFO: Exception 500 Server Error: Internal Server Error ("Invalid namespace name (_m_<my project name>). Cannot be fewer than 4 or more than 30 characters.") thrown in ProgressHandler. Retrying.
The obvious solution would be for GAE gcloud tools to respect the underlying limit via some auto-truncation or hashing scheme.
Does anyone know a way around this? Or I have to wait for Google to fix or Docker to release a new version and Google to update?
We're aware of the issue and we're working on a long-term fix. For now, you can switch to an old version of gcloud. You can do this by setting this variable to point to an old version (0.9.51):
gcloud config set --scope=installation component_manager/fixed_sdk_version 0.9.51
then run "gcloud components update"
Then run "gcloud config set app/hosted_registry false"
and you should be able to deploy. I'll update this answer when we've fixed the naming issue.
UPDATE:
The naming issue has been fixed as of this week's release (0.9.57).
Im having problems deploying my GAE. I've got two running already so I should have all the right lirbaries and whatnot. But obviously Im missing something.
I registered my applet with googe appengine developers.
i enter the app_id in the app.yaml of the project as application name.
in cmd, i go to google_appengine and run the "appcfg.py update" on the root directory of my project. "C:/Development/GAE/projectname"
type in email and pw,
and then i get this error msg
12:47 AM Application: processing#######; version: 1
12:47 AM Host: appengine.google.com
12:47 AM
Starting update of app: processing######, version: 1
12:47 AM Getting current resource limits.
Password for ############gmail.com: Invalid username or password.
2014-09-19 00:47:36,404 ERROR appcfg.py:2416 An error occurred processing file '': HTTP Error 401: Unauthorized. Aborting.
Error 401: --- begin server output ---
Must authenticate first.
--- end server output ---
Any ideas what Im missing? Ive triple-checked all these steps cause thats all i can think of.
My App.yaml contains this
application: processing######
version: 1
runtime: python27
api_version: 1
threadsafe: yes
handlers:
- url: /statics
static_dir: statics
- url: /sketches
static_dir: sketches
- url: .*
script: main.app
libraries:
- name: webapp2
version: "2.5.2"
- name: jinja2
version: latest
Ive upgraded my GAE to the latest version too, still no change. I also tried uploading it to a different application, same problem.
Ive added 2 static folder I use for to run Processing embedded in my website.
PS my app runs fine locally and there's nothing that throws errors or doesnt compile. I think the fault lies somewehre else.
Cheers
Ok so as usual, more research eventually revealed the answer. And stackoverflow.
Guess I missed this first because it specified with php.Stack Overflow Question
Basically i wasnt doing anything wrong afterall, the permission problem was that it was being blocked by my google account.
I had to go to my Account Security permissions, and enable access from less-secure devices.
This must be a new, recent setting Google changed, becuase Ive already deployed 2 apps before.
To mention here for other people with the same problem:
First go through the steps I went through to make sure everything is set up correctly.
Then you should solving it the way I did.
Otherwise you can try setting it up so you can upload without password, this might avoid the problem if it lies somewehre else --oauth without pw
Finally I read that for some people the problem was in timezones. When the time on their computer was different to GAE or server I think that lead to miscommunication, so double-check that your time is set correctly and that it agrees with the time you see in the Launcher Log.
well for me the below 02 options worked
appcfg.py update . --oauth2 --noauth_local_webserver
The authentication flow was :
the command and options generated a url in stdout, which i copied and accessed from my browser, the browser generated a code, which I gave in my command line. I could see appcfg as one of the apps under connected apps and services of my google account.
It seems like an error with your user or password.
Are you sure you can access the project in the developer console, the email configured under permissions is the one which you are using to connect and the app id in yaml file is the same ID on the developers console? If so, the only thing I can imagine is a fail in your password.
Maybe a key pressed? Try typing it in a text processor.
A last idea, deploy the app via the launcher of the sdk. Add the existing app in the file menu (search the app.yaml file) and click deploy to see if there you can authenticate.
This seems like it should be very easy but I don't see a link to it anywhere.
How do I download the source code of a google app engine project?
Windows
appengine-java-sdk\bin\appcfg.cmd -A <your_app_id> -V <your_app_version> download_app <output-dir>
Linux
./appengine-java-sdk/bin/appcfg.sh -A <your_app_id> -V <your_app_version> download_app <output-dir>
For completeness, using the Python implementation:
appcfg.py download_app -A $appID -V $appVersionNumber $downloadDirectory --oauth2
--oauth2 is of course optional, you can omit it and provide your email + app-specific password (or your password, and then go implement two-factor authentication right after), but it's easier, and frankly there's no reason not to.
Documentation.
App Engine actually recently added the ability for the developer who uploaded a given app version to download its source code.
As of October 2019 you can simply go to --> App Engine --> Services and in the tool dropdown select 'source' and the source code is there
Posting this since none of the listed methods above didn't take me to the code (by June 2021)
You could try accessing it through;
Google Cloud Platform > Debugger > choosing the version of the
Application from combo at top.
This will list the files of that version on the left pane. There is no way to download it automatically but you can copy-paste the code.
Hope you will find this helpful.
IMHO, the best option today (Aug 2018) is:
Under the main menu, under Products, go to Tools -> Cloud Build -> Build history.
There, click the ID of the build you want (for me - the last one).
Then, in the opened window (Build details), click the "source" link, the download of your compressed code begins.
As simple as that.
HTH.
Working with App engine standard using Go, the debugger isn't available yet.
How I managed to download the source code for an existing service was to use the gcloud tool.
First: Get the version id of your service using the app engine console or running: gcloud app versions list
Second: use the version and service name and run: gcloud app versions describe <versionID> --service=<service name>
the describe parameter will give you the storage locations for your source files that looks like this:
cmd/main.go:
sha1Sum: e3fe5848c2640eca7ac3591490e1debc2d3a9b09
sourceUrl: https://storage.googleapis.com/<project>/<file id>
Third: you can then use the storage console, using the file id, to download the files you are interested in.
this process based on java sdk
Its works for me...
Download Google cloud SDK
gcloud init
enter image description here
Follow through process of logging in using your credentials
Enter following command from SDK
C:\Program Files (x86)\Google\appengine-java-sdk-1.9.49\bin
enter image description here
Enter Following command to download source code
appcfg.sh -A [YOUR_APP_ID] -V [YOUR_APP_VERSION] download_app [OUTPUT_DIR]
Eg: appcfg.sh -A my-project-name-1234 -V 2 download_app C:\Users\india\Desktop\my project
Note: this progress based on java-appengine sdk so we use appcfg.sh instead of appcfg.py
check if your app is uploaded with same email id that is in your app engine. if you are not sure then in app engine > control > Clear deployment credentials and then click on any project, deploy to sign in again then use this
appcfg.py download_app -A {app id from google app engine} -V {1} "{c:\path}" --oauth2_credential_file=C:\Users\{your account name}/.appcfg_oauth2_tokens
change all {} to your needs
Things have changed since this question was asked so I'm adding an updated answer. Note that this only applies to GAE Standard Environment
Google has deprecated appcfg.py and so the previous responses appcfg.py download_app no longer works.
gcloud which is the SDK in use (it replaced appcfg) does not have the functionality to download your source code.
When you deploy your app via gcloud app deploy, it copies your source code to a bucket. The default bucket is staging.<project_name>.appspot.com. Your files will stay in this bucket for a maximum of 15 days before they are deleted. You can modify the rule so that the files are retained for longer or less time.
The file names in the bucket are encoded so you can't figure out what each file is unless you open it (i.e. download it). Google has a mapping of the encoded names to the original file names. To get this mapping, you run the gcloud app versions describe command and it will list the file names and their encoded names. To download the files, you have to manually click each url one by one. So essentially, you have to download each file manually and then use the mapping to rename them (or open the file, check the content and then rename them). Also note that downloading the files manually will not maintain the folder structure in which they were uploaded.
If you do not wish to go through all of the above hassles (imagine having to manually open each url for each file if you have a small to mid-sized project which has hundreds of files), our App - https://nocommandline.com - now supports downloading source code from the default bucket - staging.<project_name>.appspot.com (so far as your files are still there which means any deployment i.e update not older than 15 days from your current date unless you previously increased the deletion age on your staging bucket's lifecycle page).
In simple terms, you enter your project name, the version number and our App will take care of retrieving the original file name to encoded name mapping, automatically downloading the files and renaming them to the original names, while maintaining the folder structure. For more information, refer to https://nocommandline.com/help/#faq_download_source_code_from_gae.
Log in to the console.developers.google.com
Select the project you want to download the code from (Google App Engine Standard Envoronment).
Go to the App Engine Dashboard. Under Summary is Debug and Source. Click on Source.
Select each file one at a time and copy it (highlight the code, copy and paste into your local editor.)
Select the next file....
You need to use svn to checkout the files.
If you are on Windows, you can use tortoise svn for your GUI end.
Here are tutorials on how to do it, here is the related question.