My cron jobs on Google App Engine stopped to work few days ago.
It responds a 404 code.
It's strange, because I didn't changed yaml files, and it was working properly.
Any help is appreciated. Thanks!
cron.yaml
cron:
- description: Push a "tick" onto pubsub every 5 minutes
url: /publish/minutes-5-tick
schedule: every 5 minutes
- description: Push a "tick" onto pubsub every hour
url: /publish/hourly-tick
schedule: every 1 hours
- description: Push a "tick" onto pubsub every hour
url: /publish/hourly-tick-2
schedule: every 1 hours
- description: Push a "tick" onto pubsub every day
url: /publish/daily-tick
schedule: every 24 hours
- description: Push a "tick" onto pubsub every week
url: /publish/weekly-tick
schedule: every saturday 00:00
app.yaml
runtime: python27
api_version: 1
threadsafe: true
handlers:
# Handler for the pubsub cron.
- url: /publish/.*
script: main.app
login: admin
secure: always
- url: /.*
script: main.app
libraries:
- name: webapp2
version: latest
- name: pycrypto
version: latest
- name: ssl
version: latest
instance_class: F1
main.py
class PushToPubSub(webapp2.RequestHandler):
def get(self, topic):
pubsub_utils.publish_to_topic(topic, str(time.time()))
self.response.headers['Content-Type'] = 'application/json'
self.response.write(json.dumps({"status": "200"}))
app = webapp2.WSGIApplication([
webapp2.Route(r'/publish/', handler=PushToPubSub)
], debug=True)
You don't have a url handler for /publish/hourly-tick. Try:
webapp2.Route(r'/publish/<topic:\w+>', handler=PushToPubSub),
That will send "hourly-tick" as the topic to be handled in PushToPubSub
The solution was redo steps described here
https://github.com/FirebaseExtended/functions-cron
Thanks.
Related
using push queues and flexible environment on Google AppEngine I get 403 (Forbidden) error when a task (to be executed on backend service), created with default service, is executed. The task is successfully pushed to queue, confirmed locally, but the execution of the task(s) fails with log:
INFO 2020-12-24 13:42:39,897 module.py:865] default: "POST /tasks/test-handler HTTP/1.1" 403 31
WARNING 2020-12-24 13:42:39,897 taskqueue_stub.py:2158] Task task2 failed to execute. The task has no remaining retries. Failing permanently after 1 retries and 0 seconds
The same happens both locally and on production. However, if a task is created with a cron job then the execution works just fine. I am using dev_appserver.py with Go 1.11 with the following .yaml definitions:
# backend service
service: backend
runtime: go111
instance_class: F2
inbound_services:
- warmup
- default
handlers:
- url: /tasks/.*
login: admin
redirect_http_response_code: 301
# default app service
service: default
runtime: go111
instance_class: F2
inbound_services:
- warmup
handlers:
- url: /api/.*
script: auto
secure: always
redirect_http_response_code: 301
Initial API request comes to an /api endpoint which then succesfully pushes a queue message using:
t := taskqueue.NewPOSTTask(taskURL, url.Values{
"testParam": {strconv.Itoa(testParam)},
})
if _, err := taskqueue.Add(ctx, t, "test-queue"); err != nil {
return ErrPublishingTaskToQueue
}
My queue.yaml definition (in reality I have many more):
total_storage_limit: 120M
queue:
- name: test-queue
rate: 1/s
bucket_size: 100
max_concurrent_requests: 10
retry_parameters:
task_retry_limit: 1
Any ideas why I'd be getting 403 (Forbidden) statuses on task execution if a task is not created via a cron job? The documentation and existing resources on this matter do not help much :/
Managed to make it work. If anyone struggles with getting 403 responses on task execution for push queues on Google AppEngine make sure that you set the right target service. In my example above I was missing target: backend in queue.yaml:
total_storage_limit: 120M
queue:
- name: test-queue
rate: 1/s
bucket_size: 100
max_concurrent_requests: 10
target: backend
retry_parameters:
task_retry_limit: 1
The problem was that the tasks were created with default service which by default means they hit the default service, but should hit backend service. Unfortunately default service had the required endpoint deployed as well, so I got 403 instead of 404.
More details on the target field:
https://cloud.google.com/appengine/docs/standard/python/config/queueref#target
I have an App Engine Python Application which has an endpoint that puts a task in the Task Queue. - This is working fine.
I have a Task Handler Python Application which will be execute the task in the queue.
When the task handler is invoked, the below error accurs
Request failed because URL requires user login. For requests invoked within App Engine (offline requests like Task Queue, or webhooks like XMPP and Incoming Mail), the URL must require admin login (or no login).
My App Engine Python Application app.yml is below
service: dataload-test
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: main.app
- url: /_ah/queue/deferred
script: google.appengine.ext.deferred.deferred.application
login: admin
libraries:
- name: ssl
version: latest
builtins:
- deferred: on
- appstats: on
env_variables:
GAE_USE_SOCKETS_HTTPLIB : 'true'
My Task Handler Application app.yml is below
service: adobe-dataload-worker
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: load_data_worker.app
login: admin
Any help would be appreciated
Your wildcard - url: /.* handler is handling EVERY URL.
Put that last, or else the deferred handler will never be seen:
handlers:
- url: /_ah/queue/deferred
script: google.appengine.ext.deferred.deferred.application
login: admin
- url: /.*
script: main.app
I have the following in my app.yaml file
builtins:
- deferred: on
handlers:
- url: /_ah/queue/deferred.*
script: google.appengine.ext.deferred.deferred.application
login: admin
It is possible to schedule some of the tasks exposed using google.appengine.ext.deferred.deferred.application as cron jobs? What would the URL in cron.yaml file?
Yes it is possible.
cron.yaml
cron:
- description: daily summary job
url: /cron-job-url
schedule: every 24 hours
For example (not tested) In the request handler of /cron-job-url call the deferred task
class CronJob(webapp2.RequestHandler):
def get(self):
deferred.defer(do_something_expensive, "Hello, world!", 42, True)
self.response.set_status(200)
app = webapp2.WSGIApplication([
('/cron-job-url', CronJob),
], debug=True)
Problem fixed via Google Cloud support:
I removed the /_ah/spi/* handler from my endpoints yaml file, and the
- url: /.*
script: api.app
did not instantiate the endpoint.
It used to work before since the API was deployed for the previous version, but with the new version, there was nothing explicit to deploy the endopoints. So handlers need to be
handlers:
- url: /_ah/spi/.*
script: api.app
- url: /.*
script: api.app
Keeping the issue below for reference purposes
I've just deployed a new version of my application, and calls to http://app.appspot.com/_ah/api/app/version/method now return a 404. It worked perfectly before the update.
However, there's no trace at all in the logs, and no instance launched when I call/ping those URIs.
While if I call /_ah/whatever/app/version/method, I still have a 404, but it appears in my module logs, and it has the following message
{"state": "APPLICATION_ERROR", "error_message": "Not Found"}
The app is configured using modules, my app.yaml is defined with
application: appname
version: 2015-04-07
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /_ah/spi/.*
script: api.api.app
- url: /.*
script: www.www.app
secure: always
libraries:
- name: pycrypto
version: latest
- name: endpoints
version: 1.0
- name: webapp2
version: latest
- name: jinja2
version: latest
And, in api/api.yaml
application: appname
module: default
version: 2015-04-07
runtime: python27
api_version: 1
threadsafe: true
inbound_services:
- warmup
handlers:
- url: /.*
script: api.app
libraries:
- name: pycrypto
version: latest
- name: endpoints
version: 1.0
- name: ssl
version: latest
I've updated the app to serve this new version in the admin console, and all other modules work perfectly.
Also, I can't see my API in the API explorer, https://appname.appspot.com/_ah/api/explorer returns an empty list (while I see it when running the dev server on localhost).
Update: I've just noticed, looking at #bossylobster reply in GAE cloud endpoints - Api not updating after deploy, that I do not have "Successfully updated API configuration" in my logs after the "Completed update of a new alternate version". However, I have "API deletion serving" at about the time everything started to be 404. Yet, I have no idea why there's this API deletion query in my logs. Any idea of what can be wrong?
That's an app in production and so the mobile version is down at the moment. I'm happy to send the app ID to a devrel in PM if that helps.
I have an app.yaml as below:
application: myapp
module: mymodule
version: 1
runtime: python27
api_version: 1
threadsafe: true
...
The app also has a cron.yaml as below:
cron:
- description: increase a value every hour
url: /test/inc
schedule: every 60 minutes synchronized
How do I make my cron job target mymodule instead of the default module?
In newer versions of GAE (tested at least with 1.8.6), you can just set the target for your cron task:
cron:
- description: increase a value every hour
url: /test/inc
schedule: every 60 minutes synchronized
target: mymodule
Thanks to #voscausa you can use a dispatch.yaml file https://developers.google.com/appengine/docs/python/modules/routing to route a cron job to the right module.
Make sure you run appcfg.py update_dispatch after you create the dispatch.yaml file.