I have this cron job in GAE
cron:
- description: interactive
url: /interactive
schedule: every 1 minutes
everytime I want to change schedule to 5 minutes or 10 minutes, I have to open cron.yaml file, change the parameter and update the app. I think it is too annoying.
Is there anyway I can change the parameter without doing those steps.
For example, store schedule parameter in an entity and update the entity. Is it possible?
The only way to change these params is to update the cron.yaml and redeploy the app.
To make your life easier though you could enable the push to deploy for your app and edit this file (or any other file) either on GitHub or directly from Google Console.
Related
We have multiple App Engine Cron entries triggering our App Engine application, but recently we detected a decrease on the number of the processed events handled by one of the endpoints of our application. By looking at the App Engine Cron logs for this specific Cron entry on StackDriver, we found out that, during the days we invesgated (March 11-15), that are missing entries. Most of the missing triggers coincide through the days (12:15, 14:15, 16:15, 18:15, 20:15, 22:15, 00:15).
The screenshot below displays one specific day, and the red lines indicate the missing entries:
There are no requests with HTTP status code different than 200.
This is the configuration of the specific Cron entry (replaced some words with XXX due to business restrictions):
- description: 'Hourly job for XXX'
url: /schedule/bigquery/XXX
schedule: every 1 hours from 00:15 to 23:15
timezone: UTC
target: XXX
retry_parameters:
min_backoff_seconds: 2.5
max_doublings: 5
Could someone # GCP side take a look? The task name is 53751dd6a70fb9af38f49993b122b79f.
it seems like if the request takes longer than an hour, then the next one gets skipped (i.e. cron doesn't launch the next iteration if the current iteration is still running)
maybe do the actual work in a separate task and then the only thing the cron task does is launch this separate task
Is there a way to schedule a cron job using the cron.yaml to trigger a HTTP cloud function. I tried to implement it but passing the entire URL is throwing an error.
cron:
- description: "Test Call"
url: https://us-central1-***.cloudfunctions.net/helloGET
schedule: every 1 mins
I see this error in the console when I try to deploy the cron job
Unable to assign value 'https://us-central1-***.cloudfunctions.net/helloGET' to attribute 'url':
Value 'https://us-central1-***.cloudfunctions.net/helloGET' for url does not match expression '^(?:^/.*$)$'
in "/Users/xyz/Desktop/cron.yaml", line 3, column 8
I know that error is being thrown because I have the full URL path but instead of the full path if I just pass the following
cron:
- description: "Test Call"
url: /helloGET
schedule: every 1 mins
then it is able to deploy the cron job but when the job is run it throws a 404 error because by just passing the path and not the full URL I believe it is looking for the URL in the app engine and since I dont have any code in the app engine and my service call is in the cloud function it is not able to find it.
Also is there a way to set the schedule to be run every 1 seconds instead of 1 mins.
The url in the cron.yaml needs to be a URl handled by your app, not an arbitrary one - which is why only the relative path works. From Syntax (emphasis mine):
url
Required. The url field specifies a URL in your application that
will be invoked by the Cron Service.
What you can do is have your application cron handler reach out to the arbitrary URL you need to trigger your Cloud Function. See Issuing HTTP(S) Requests
As for going below 1 minute intervals - that's not supported by cron itself. But there are ways to achieve something almost equivalent, see, for example High frequency data refresh with Google App Engine
I am using google app engine and have 2 applications that use cron jobs to schedule events. I am able to deploy both applications by using gcloud app deploy app.yaml cron.yaml. Even though both apps are deployed and working, only one of the cron jobs actually runs. This is what the files look like.
First cron.yaml
cron:
- description: "GET first group"
url: /
schedule: every 5 minutes
target: pubsubone
Second cron.yaml
cron:
- description: "GET second group"
url: /
schedule: every 5 minutes
target: pubsubtwo
These files are in different folders and associated with different applications.
When you deploy the second service with a new cron.yaml file, the first cron job gets overwritten since only one cron.yaml is expected for deployments. To deploy both cron jobs contemporarily join them in a single file, as shown in the example here and then deploy the resulting cron.yaml file as shown here. The cron.yaml should look like this:
cron:
- description: "GET first group"
url: /
schedule: every 5 minutes
target: pubsubone
- description: "GET second group"
url: /
schedule: every 5 minutes
target: pubsubtwo
And the command line to deploy it is this one:
$ gcloud app deploy cron.yaml
There are several reasons this can be failing for you, and sorting out which will be easier if you use something other than / as the url. /cron, perhaps. That'll make it a lot easier to determine, when looking at the logs, that the url is being called as intended.
Next, there the target. If there aren't versions of your app active that have the specified value as their version (or service, though I don't have experience with that), AFAIK, the request generated by cron will get dropped on the floor.
I would like to create a cron job in GAE but I do not want it to be scheduled.
I would like to call it manually whenever I need it.
What should I put in the schedule element?
From Scheduling Tasks With Cron for Python:
A cron job makes an HTTP GET request to a URL as scheduled. The
handler for that URL executes the logic when it is called.
So, as #snakecharmerb mentioned, you have to create that handler doing whatever you want the cron job to do and map it to the desired cron url.
In order to avoid scheduling you simply don't upload the cron configuration, instead you manually trigger the job by making that same GET request that the scheduler would otherwise do, for example using curl:
curl https://your_app.appspot.com/cron_handler_url
You can also do something like
schedule: every 99999 hours
That way, its almost like 11 years so you know that cron won't be triggered till then.
Before reading, this is slightly hacky although it does work.
I needed something similar to this, primarily due to the need for a slightly longer processing time than 60 seconds.
In my cron.yaml I specified:
- description: My example cron
url: /cron/my/example/cron
schedule: 25 of dec 12:00
Then within my handler I check if the date is the 25th december and abandon the request.
This seems to be the only way to have a 'manual' cron.
In Python this is simple to check the date.
from datetime import datetime
import logging
now = datetime.utcnow()
if now.day == 25 and now.month == 12:
logging.info("Skipping as it's the 25th December")
return
I want to run a cron for every 2 minutes interval, 0,2,4,6,8 .... each cron execution runs for 2 minutes.
I configured cron schedule with synchronized as below. But I still see scheduler is behaving as if synchronized not given.
Crons are scheduled at
0-2 First cron
4-6 Second cron
8-10 third cron
Cron scheduler is waiting for 2 minutes after last cron execution.
If I understand synchronized correctly, it is added to avoid this behavior.
Why this happening.
<cron>
<url>/cron/syncPrices</url>
<description>Fetch data from source and cache it in data store.</description>
<schedule>every 2 minutes synchronized</schedule>
</cron>
You can check the actual cron configuration for your application on the old GAE console in the Cron Jobs menu on the left. You're looking to confirm if synchronized (or its from 00:00 to 23:59 equivalent) is present for the respective job:
if synchronized is missing it's possible that the cron.yaml file wasn't uploaded/updated during the regular app upload - I noticed this to be the case with my multi-module (python) application. You have to specifically update the cron configuration using the update_cron option of your AppCfg utility.
if synchronized is indeed present and the unexpected behaviour continues you should open a support case with Google.
FYI It looks like 'synchronized' is no longer part of the cron.yaml documentation. I've reached out to Google via the documentation feedback link.