Is there a way to schedule a cron job using the cron.yaml to trigger a HTTP cloud function. I tried to implement it but passing the entire URL is throwing an error.
cron:
- description: "Test Call"
url: https://us-central1-***.cloudfunctions.net/helloGET
schedule: every 1 mins
I see this error in the console when I try to deploy the cron job
Unable to assign value 'https://us-central1-***.cloudfunctions.net/helloGET' to attribute 'url':
Value 'https://us-central1-***.cloudfunctions.net/helloGET' for url does not match expression '^(?:^/.*$)$'
in "/Users/xyz/Desktop/cron.yaml", line 3, column 8
I know that error is being thrown because I have the full URL path but instead of the full path if I just pass the following
cron:
- description: "Test Call"
url: /helloGET
schedule: every 1 mins
then it is able to deploy the cron job but when the job is run it throws a 404 error because by just passing the path and not the full URL I believe it is looking for the URL in the app engine and since I dont have any code in the app engine and my service call is in the cloud function it is not able to find it.
Also is there a way to set the schedule to be run every 1 seconds instead of 1 mins.
The url in the cron.yaml needs to be a URl handled by your app, not an arbitrary one - which is why only the relative path works. From Syntax (emphasis mine):
url
Required. The url field specifies a URL in your application that
will be invoked by the Cron Service.
What you can do is have your application cron handler reach out to the arbitrary URL you need to trigger your Cloud Function. See Issuing HTTP(S) Requests
As for going below 1 minute intervals - that's not supported by cron itself. But there are ways to achieve something almost equivalent, see, for example High frequency data refresh with Google App Engine
Related
I recently updated my cron.yaml file and now my cron tasks fail with no entries in the logs.
It is acting like the java servlet at the url is not being run.
I can paste the url into a browser and the servlet runs fine.
My cron.yaml file:
cron:
- description: Daily revenues report
url: /revenues
schedule: every day 07:35
timezone: America/Denver
Using below deploycron.sh
PROJECT_ID='my-project-id'
gcloud config set project ${PROJECT_ID}
gcloud info
gcloud app deploy cron.yaml
Is there an error in my .yaml?
Is there a special task queue set up required?
Is some other configuration or permissions piece missing?
It was running fine last week. I have tried deleting and starting over to no avail.
https://console.cloud.google.com/cloudscheduler?project=project-id
Shows the job. Result column 'Failed'.
Logs 'View' link shows:
protoPayload.taskName="01661931846119241031" protoPayload.taskQueueName="__cron"
with no log entries.
Is __cron not automatic?
I am at a loss.
App Engine Standard. Java 8.
After installing the latest update of GCloud locally and re-running the deploy cron script. The cron jobs now run as before. 02/02/2021.
'Failed' means that the endpoint /revenues is not returning a success http status code.
Logs 'View' link shows: protoPayload.taskName="01661931846119241031" protoPayload.taskQueueName="__cron" with no log entries
Maybe don't use the premade filter, and just try filtering for /revenues or viewing all the logs at 07:35 am (when it was supposed to have run)
Is there an error in my .yaml?
if there was then gcloud app deploy cron.yaml would fail
Is there a special task queue set up required?
you shouldn't need to do anything, i didn't
I can paste the url into a browser and the servlet runs fine.
When you paste the url into the browser, is there any redirecting (like from /revenues to /revenues/) or anything that your browser is handling for you. Maybe /revenues is expecting there to be cookies present now.
What are there any special app.yaml or dispatch.yaml rules that /revenues would be hitting?
Is /revenues being handled by a service other than the default service?
I had a similar problem: CRON tasks fail without any logs.
The root cause was that the IP address of App Engine was blocked by the App Engine Firewall. Thus I had to update the allow-list, as described here: https://cloud.google.com/appengine/docs/standard/nodejs/scheduling-jobs-with-cron-yaml#validating_cron_requests
I started having the same problem a few days ago on my existing CRON schedules. I've tried everything including tearing my code down to the bare minimum and creating a new GAE project with the Hello World quick start. It still fails. Nothing useful in the logs and the UI just says 'Failed'. I'm pulling my hair out.
Sorry I don't have an answer to contribute but your post makes me think it's on Google's side. I know they're moving CRON jobs to Cloud Scheduler->App Engine Cron Jobs. My gut tells me it's a permissions issue related to this move and IAM. I'm really at a loss.
We have multiple App Engine Cron entries triggering our App Engine application, but recently we detected a decrease on the number of the processed events handled by one of the endpoints of our application. By looking at the App Engine Cron logs for this specific Cron entry on StackDriver, we found out that, during the days we invesgated (March 11-15), that are missing entries. Most of the missing triggers coincide through the days (12:15, 14:15, 16:15, 18:15, 20:15, 22:15, 00:15).
The screenshot below displays one specific day, and the red lines indicate the missing entries:
There are no requests with HTTP status code different than 200.
This is the configuration of the specific Cron entry (replaced some words with XXX due to business restrictions):
- description: 'Hourly job for XXX'
url: /schedule/bigquery/XXX
schedule: every 1 hours from 00:15 to 23:15
timezone: UTC
target: XXX
retry_parameters:
min_backoff_seconds: 2.5
max_doublings: 5
Could someone # GCP side take a look? The task name is 53751dd6a70fb9af38f49993b122b79f.
it seems like if the request takes longer than an hour, then the next one gets skipped (i.e. cron doesn't launch the next iteration if the current iteration is still running)
maybe do the actual work in a separate task and then the only thing the cron task does is launch this separate task
I am using google app engine and have 2 applications that use cron jobs to schedule events. I am able to deploy both applications by using gcloud app deploy app.yaml cron.yaml. Even though both apps are deployed and working, only one of the cron jobs actually runs. This is what the files look like.
First cron.yaml
cron:
- description: "GET first group"
url: /
schedule: every 5 minutes
target: pubsubone
Second cron.yaml
cron:
- description: "GET second group"
url: /
schedule: every 5 minutes
target: pubsubtwo
These files are in different folders and associated with different applications.
When you deploy the second service with a new cron.yaml file, the first cron job gets overwritten since only one cron.yaml is expected for deployments. To deploy both cron jobs contemporarily join them in a single file, as shown in the example here and then deploy the resulting cron.yaml file as shown here. The cron.yaml should look like this:
cron:
- description: "GET first group"
url: /
schedule: every 5 minutes
target: pubsubone
- description: "GET second group"
url: /
schedule: every 5 minutes
target: pubsubtwo
And the command line to deploy it is this one:
$ gcloud app deploy cron.yaml
There are several reasons this can be failing for you, and sorting out which will be easier if you use something other than / as the url. /cron, perhaps. That'll make it a lot easier to determine, when looking at the logs, that the url is being called as intended.
Next, there the target. If there aren't versions of your app active that have the specified value as their version (or service, though I don't have experience with that), AFAIK, the request generated by cron will get dropped on the floor.
I have this cron job in GAE
cron:
- description: interactive
url: /interactive
schedule: every 1 minutes
everytime I want to change schedule to 5 minutes or 10 minutes, I have to open cron.yaml file, change the parameter and update the app. I think it is too annoying.
Is there anyway I can change the parameter without doing those steps.
For example, store schedule parameter in an entity and update the entity. Is it possible?
The only way to change these params is to update the cron.yaml and redeploy the app.
To make your life easier though you could enable the push to deploy for your app and edit this file (or any other file) either on GitHub or directly from Google Console.
I have configured Google App Engine to record exception with ereporter.
The cron job is configured to run every 59 minutes. The cron.yaml is as follows
cron:
- description: Daily exception report
url: /_ereporter?sender=xxx.xxx#gmail.com # The sender must be an app admin.
schedule: every 59 minutes
How to do I run this immediately.
What I am trying to do here is simulate a 500 HTTP error and see the stack trace delivered immediately via the cron job.
Just go to the URL from your browser.
You can't using cron. Cron is a scheduling system, you could get it to run every minute.
Alternately you could wrap your entire handler in a try/except block and try to catch everything. (You can do this for some DeadlineExceededErrors for instance) then fire off a task which invokes ereporter handler, and then re-raise the Exception.
However in many cases Google infrastructure can be the cause of the Error 500 and you won't be able to catch the error. To be honest you are only likely to be able to cause an email sent for a subset of all possible Error 500's. The most reliable way probably be to have a process continuously monitor the logs, and email from there.
Mind you email isn't consider reliable or fast so a 1 min cron cycle is probably fast enough.
I came across this thread as I was trying to do this as well. A (hacky) solution I found was to add a curl command at the end of my cloudbuild.yaml file that triggers the file immediately per this thread. Hope this helps!
Make a curl request in Cloud Build CI/CD pipeline