When setting up a Google App Engine instance you can configure a cron.yaml to set up Cron jobs.
There does not seem to be any documentation on how to configure jobs that run say every 30 seconds.
I tried
schedule: every 30 seconds
and
schedule: 0/30 0 0 ? * * *
But no good. Google Cloud tells me the format is incorrect when I deploy. Can you schedule in frequencies less then 1 minute with Google App Engine Cron jobs?
You cannot configure GAE cron services with resolutions below 1 minute. FWIW, you can't do that on unix/linux systems either.
But it is possible to use an every 1 minutes cron job from which you can further trigger delayed execution of deferred/push/pull queue tasks with down to 1 second resolution, see High frequency data refresh with Google App Engine
Had the same problem, solved with setTimeout().
Using setTimeout for 30 seconds inside the appengine one-minute cron job did the trick, in this case we'll be fetching data 2 times per minute that is every 30 seconds.
saveData();
setTimeout(function () { saveData(); }, 30000);
Tested and worked fine,
Related
I want to migrate from App Engine Cron jobs to Cloud Scheduler, but in Cloud Scheduler the request deadline timeout is 60 seconds, not the 10 minutes that has the requests from Cron jobs.
Is there a way to configure Cloud Scheduler App Engine request's to have a deadline timeout of 10 minutes?
This will set the deadline for a job to 30mins. Which is the max for HTTP targets.
gcloud beta scheduler jobs update http <job> --attempt-deadline=1800s --project <project>
The allowed duration for this deadline is: For HTTP targets, between 15 seconds and 30 minutes. For App Engine HTTP targets, between 15 seconds and 24 hours.
A duration in seconds with up to nine fractional digits, terminated by 's'. Example: "3.5s".
Source: https://cloud.google.com/scheduler/docs/reference/rest/v1/projects.locations.jobs#Job
According to their scheduler.v1beta1, it is possible to set that Deadline using the attemptDeadline.
The deadline for job attempts. If the request handler does not respond by this deadline then the request is cancelled and the attempt is marked as a DEADLINE_EXCEEDED failure. The failed attempt can be viewed in execution logs. Cloud Scheduler will retry the job according to the RetryConfig.
The allowed duration for this deadline is:
For HTTP targets, between 15 seconds and 30 minutes.
For App Engine HTTP targets, between 15 seconds and 24 hours.
For PubSub targets, this field is ignored.
https://cloud.google.com/nodejs/docs/reference/scheduler/0.3.x/google.cloud.scheduler.v1beta1#.Job
When we look at Cloud Scheduler, we see that when the time is reached to fire a job the request to fire that job may fail. At this point, the request will be retried based on the configuration of that job ... see:
https://cloud.google.com/sdk/gcloud/reference/beta/scheduler/jobs/create/http
Among these settings we find:
--max-backoff
--max-doublings
--max-retry-attempts
--max-retry-duration
--min-backoff
It seems that if we want to keep trying for a solid 10 minutes we might be able to specify:
--max-backoff: 0s
--max-doublings: 0
--max-retry-attempts: 0
--max-retry-duration: 10m
--min-backoff: 0s
We have multiple App Engine Cron entries triggering our App Engine application, but recently we detected a decrease on the number of the processed events handled by one of the endpoints of our application. By looking at the App Engine Cron logs for this specific Cron entry on StackDriver, we found out that, during the days we invesgated (March 11-15), that are missing entries. Most of the missing triggers coincide through the days (12:15, 14:15, 16:15, 18:15, 20:15, 22:15, 00:15).
The screenshot below displays one specific day, and the red lines indicate the missing entries:
There are no requests with HTTP status code different than 200.
This is the configuration of the specific Cron entry (replaced some words with XXX due to business restrictions):
- description: 'Hourly job for XXX'
url: /schedule/bigquery/XXX
schedule: every 1 hours from 00:15 to 23:15
timezone: UTC
target: XXX
retry_parameters:
min_backoff_seconds: 2.5
max_doublings: 5
Could someone # GCP side take a look? The task name is 53751dd6a70fb9af38f49993b122b79f.
it seems like if the request takes longer than an hour, then the next one gets skipped (i.e. cron doesn't launch the next iteration if the current iteration is still running)
maybe do the actual work in a separate task and then the only thing the cron task does is launch this separate task
I need create a schedule job, i am using Google App engine.
The requirement is the cron job will be execute each 10 minutes like this
0,10,20,30,40,50,60 in each hour.
I read the documentation from Google site at : [https://cloud.google.com/appengine/docs/standard/php/config/cronref#schedule_format][1]
This is my config :
schedule: every 10 minutes from 00:00 to 23:50
Is it correct with the requirment ?
Yes, correct. But, if you are covering the entire day, just:
schedule: every 10 minutes
I want to run a cron for every 2 minutes interval, 0,2,4,6,8 .... each cron execution runs for 2 minutes.
I configured cron schedule with synchronized as below. But I still see scheduler is behaving as if synchronized not given.
Crons are scheduled at
0-2 First cron
4-6 Second cron
8-10 third cron
Cron scheduler is waiting for 2 minutes after last cron execution.
If I understand synchronized correctly, it is added to avoid this behavior.
Why this happening.
<cron>
<url>/cron/syncPrices</url>
<description>Fetch data from source and cache it in data store.</description>
<schedule>every 2 minutes synchronized</schedule>
</cron>
You can check the actual cron configuration for your application on the old GAE console in the Cron Jobs menu on the left. You're looking to confirm if synchronized (or its from 00:00 to 23:59 equivalent) is present for the respective job:
if synchronized is missing it's possible that the cron.yaml file wasn't uploaded/updated during the regular app upload - I noticed this to be the case with my multi-module (python) application. You have to specifically update the cron configuration using the update_cron option of your AppCfg utility.
if synchronized is indeed present and the unexpected behaviour continues you should open a support case with Google.
FYI It looks like 'synchronized' is no longer part of the cron.yaml documentation. I've reached out to Google via the documentation feedback link.
I have AppEngine Java application which use cron to activate a servlet.
Because I'm doing very short operation I do it inside a loop every few seconds.
I want to know how much time I have left to finish the work before I get the deadline exception so I use getCurrentEnvironment().getRemainingMillis() but for some reason it return 10 minutes instead of 1 minute.
Thank you,
Ido.
This is expected behavior. The cron docs page clearly states that requests invoked by cron can run for 10 minutes.