I've set up a cron.yaml file for my app to run some cronjobs, i.e. every 2 mins, every 15 mins... Is it possible to specify the tasks to start from a specific date? Say I want to start running all cronjobs from 6th december 2017?
The GAE cron Schedule format doesn't have direct support for the functionality you seek.
However it's relatively easy to obtain such functionality, see How to make a cron job run during business hours on Google App Engine cron.yaml?
Related
Currently I have a function deployed in Google Functions, which is triggered by a Cron Job using Cloud Scheduler in Google Cloud Platform.
The Cron Job Frequency in unix is the following: 0 9 1-7,15-21,29-31 * 1
Which means that it should be triggered every Monday in a Month at 9:00 AM, which day falls between 1st and 7th, 15th and 21st, 29th and 31st, respectively. Or in other words, it gets triggered every 2 weeks.
The following image (cron job testers) also confirms what I am intending to do.
As expected, yesterday March 15th (Monday) it got triggered, however today March 16th (Tuesday) it got triggered as well
I am wondering if this has happen to any of you and whether it's a problem with my cron job or either Google Cloud Scheduler doesn't interpret it this way.
Any help/experience is highly appreciated!!
Crontab uses OR when both day of month and day of week are provided: please, see your screenshot, it shows that your process will run tomorrow, March 17th, at 9:00.
I honestly do not know the actual reason of this crontab exception: for curiosity, I came across this SO question, which provides a possible explanation.
In order to express similar conditions, in a general use case, you can take advantage of crontab expressions, see for example this SO question or this other in Server Fault, but I do not know whether they are or not applicable in Google Cloud Scheduler.
Perhaps, instead of a cron expression, you can use the so-called English Style Schedule, the same used on App Engine - see the tab Custom interval, to define your expressions, maybe it can be more suitable for your use case.
Over the past few weeks my gcloud app deploy has been failing regularly, today it fails even after attempting 10 times so far. According to the google cloud web interface, it is failing after build in the exporting phase:
7: exporter
/cnb/lifecycle/exporter asia.gcr.io/<project>/app-engine-tmp/app/ttl-2h:<uuid>
Is there anything we can do about this at all?
(The build itself takes just a few minutes, then this exporter phase runs until the 10m mark and is killed.)
Perhaps there is a way to make gcloud app deploy do the build in the asia region?
I deleted everything in the Google Cloud Console under the "Container Registry" menu option. Not sure why that helps, but builds now average 2 minutes instead of the historical 9-11 minutes for the past year or so.
(Although this technically doesn't answer why behind the question, this does solve the strange delays in building on appengine (standard). I am going to mark it as solved and leave this question here in case others encounter the error message above.)
Trying out new flexible app engine runtime. In this case a custom Ruby on Rails runtime based on the google provided ruby runtime.
When firing of gcloud preview app deploy the whole process takes ~8 minutes, most of which is "updating service". Is this normal? And more importantly, how can I speed it up?
Regards,
Ward
Yes, that is totally normal. Most of the deployment steps happen away from your computer and are independent of your codebase size, so there's not a lot you can do to speed up the process.
Various steps that are involved in deploying an app on App Engine can be categorized as follows:
Gather info from app.yaml to understand overall deployment
Collect code and use the docker image specified in app.yaml to build a docker image with your code
Provision Compute Instances, networking/firewall rules, install docker related tools on instance, push docker image to instance and start it
Make sure all deployments were successful, start health-checks and if required, transfer/balance out the load.
The only process which takes most of time is the last part where it does all the necessary checks to make sure deployment was successful and start ingesting traffic. Depending upon your code size (uploading code to create container) and requirements for resources (provisioning custom resources), step 2 and 3 might take a bit more time.
If you do an analysis you will find that about 70% of time is consumed in last step, where we have least visibility into, yet the essential process which gives app-engine the ability to do all the heavy lifting.
Deploying to the same version got me from 6 minutes to 3 minutes in subsequent deploys.
Example:
$ gcloud app deploy app.yaml --version=test
Make sure you check what is in the zip it's uploading (it tells you the location of this on deploy), and make sure your yaml skip_files is set to include things like your .git directory if you have one, and node_modules
Note that the subsequent deploys should be much faster than 8 mins. It's usually 1 minute or less in my tests with Node.js on App Engine Flex.
As suggested above by #ludo you could use in the meantime Google App Engine Standard instead of Flex. Which, takes approximately ~30-50 seconds after the first deployment.
You can test GAE Standard by running this tutorial, which doesn't require a billing account:
https://codelabs.developers.google.com/codelabs/cloud-app-engine-springboot/index.html#0
And agreed. this doesn't address GAE Flex but gives some options to accelerate during development.
Just fire this command from root directory of app.yaml
From shell visit directory of app.yaml then run gcloud app deploy
It will be uploaded within few seconds.
I tried to delete one kind of entity at once from GAE datastore admin page. The problem is, I fired two jobs for deleting one kind (same kind). After one job successfully finished, another just freeze, preventing other jobs from being run.
The job description is:
Job #158264924762856ED17CF
Overview
Running
Elapsed time: 00:00:00
Start time: Tue Nov 20 2012 09:58:27 GMT+0800
entity_kind: "CacheObj"
Counters
How can I clear these jobs? Deleting them from task queue won't help much, they are still inside datastore admin page.
I faced the same problem, but the frozen job didn't prevent new jobs to be executed. However, getting a frozen job is misleading. My workaround:
Go to the Datastore Viewer
Select _AE_DatastoreAdmin_Operation as kind
Find the frozen job
Delete it
You might get an error saying that the app failed
Go back to the Datastore Admin, and check that the job is no longer there
Recently I've started using limited staging on my Google App Engine project. The data is still shared between all versions, but behaviour (especially user facing behaviour) is different.
Naturally when I implement something incredibly new it only runs on the latest version of my code and I don't feel like it should be backported to the older versions.
Some of this new functionality requires cron jobs to be run periodically, but I'm hitting a problem. I have to run a cron job to call the latest code, but this is what Google's documentation has to say about the issue:
Cron requests are always sent to the default version of the application.
The default version is the oldest because the first versions of the client code that went out to users weren't future proof and don't know how to select which API version to call.
So my question is, how can I get around this limitation and make a cron job that will call the latest rather than the default version of the application?
You can now specify a version using the target tag.
<target>version-2</target>
You can't change the cron jobs to run on a different version then the default.
Depending on how much time your cron job takes to run you could change your cron job script to to do a URLFetch to "http://latest.appname.appspot.com/cron_job_endpoint".
If you're cron job takes longer then 10 minutes to run, then I would design it in a way that you can chain the different tasks using task queues.