I noticed that the com.google.appengine.api.taskqueue.DeferredTask in the Google Cloud retries if it throws an exception. Is there a way to have it not retry if it gets an error?
Note: using JAVA
With first gen GAE and Python, you can specify options to the deferred call like this:
retry = taskqueue.TaskRetryOptions(task_retry_limit=0)
deferred.defer(..., _retry_options=retry)
This will prevent the task from being retried.
If you are using second gen GAE, please update your question to indicate that.
From the documentation, I see this in Interface DeferredTask which I think you may be able to use:
Normal return from this method is considered success and will not
retry unless DeferredTaskContext.markForRetry() is called.
Exceptions thrown from this method will indicate a failure and will be
processed as a retry attempt unless
DeferredTaskContext.setDoNotRetry(boolean) was set to true.
Related
Basically what the title says. The API and client docs state that a retry can be passed to create_task:
retry (Optional[google.api_core.retry.Retry]): A retry object used
to retry requests. If ``None`` is specified, requests will
be retried using a default configuration.
But this simply doesn't work. Passing a Retry instance does nothing and the queue-level settings are still used. For example:
from google.api_core.retry import Retry
from google.cloud.tasks_v2 import CloudTasksClient
client = CloudTasksClient()
retry = Retry(predicate=lambda _: False)
client.create_task('/foo', retry=retry)
This should create a task that is not retry. I've tried all sorts of different configurations and every time it just uses whatever settings are set on the queue.
You can pass a custom predicate to retry on different exceptions. There is no formal indication that this parameter prevents retrying. You may check the Retry page for details.
Google Cloud Support has confirmed that task-level retries are not currently supported. The documentation for this client library is incorrect. A feature request exists here https://issuetracker.google.com/issues/141314105.
Task-level retry parameters are available in the Google App Engine bundled service for task queuing, Task Queues. If your app is on GAE, which I'm guessing it is since your question is tagged with google-app-engine, you could switch from Cloud Tasks to GAE Task Queues.
Of course, if your app relies on something that is exclusive to Cloud Tasks like the beta HTTP endpoints, the bundled service won't work (see the list of new features, and don't worry about the "List Queues command" since you can always see that in the configuration you would use in the bundled service). Barring that, here are some things to consider before switching to Task Queues.
Considerations
Supplier preference - Google seems to be preferring Cloud Tasks. From the push queues migration guide intro: "Cloud Tasks is now the preferred way of working with App Engine push queues"
Lock in - even if your app is on GAE, moving your queue solution to the GAE bundled one increases your "lock in" to GAE hosting (i.e. it makes it even harder for you to leave GAE if you ever want to change where you run your app, because you'll lose your task queue solution and have to deal with that in addition to dealing with new hosting)
Queues by retry - the GAE Task Queues to Cloud Tasks migration guide section Retrying failed tasks suggests creating a dedicated queue for each set of retry parameters, and then enqueuing tasks accordingly. This might be a suitable way to continue using Cloud Tasks
I need to make sure some code is initialized on a GAE instance when it is spun up. Is there a way to detect in the code that the request is a "loading request"?
I tried adding the initialization code to my warmup handler but it does not seem that warmup is guaranteed to be called. I also tried giving each service in my application a single idle instance to increase the chance of warmup being called.
Note: this answer is written with the python 2.7 standard environment in mind, I'm not sure if something equivalent would be possible for all other languages/runtimes and how it'd work.
Indeed, the warm-up requests, if configured, are only effective if there is already at least one instance running for the service, which isn't always the case.
You could place the desired init code in the main service file which declares your service's app - that code is only executed when the service app is initialized, which happens exactly once per instance lifetime, regardless of the request being a loading or a warm-up one.
Something along these lines:
import ...
def my_desired_init_function():
pass # replace with what you need to do
my_desired_init_function()
app = webapp2.WSGIApplication(...)
I am still beginner with golang in Google Cloud Appengine (standard).
I want to use a function that is automatically call for the instance shutting down.
There is a function init called during startup.
Now I am looking for the opposite part like a destroy function.
Seems there is something like that for python, but could not find
anything for golang.
How could you realise such a destroy fuction in google appengine instances ?
This is documented at Go - How Instances are Managed.
Unfortunately the Go doc is incomplete, here's the link to the Pyton version: Python - How Instances are Managed. The way it is implemented / supported is language-agnostic.
When an instance is spin up, an HTTP GET request is sent to the /_ah/start path.
Before an instance is taken down, an HTTP GET request is sent to the /_ah/stop path.
You should use package init() functions for initialization purposes as that always runs, and only once. If a request is required for your init functions, then register a handler to the _/ah/start path.
And you may register a handler to /_ah/stop and implement "shutdown" functionality like this:
func init() {
http.HandleFunc("/_ah/stop", shutdownHandler)
}
func shutdownHandler(w http.ResponseWriter, r *http.Request) {
doSomeWork()
saveState()
}
But you can't rely on this 100%:
Note: It's important to recognize that the shutdown hook is not always able to run before an instance terminates. In rare cases, an outage can occur that prevents App Engine from providing 30 seconds of shutdown time. Thus, we recommend periodically checkpointing the state of your instance and using it primarily as an in-memory cache rather than a reliable data store.
One of my instances in GAE Standard (Java) is somehow in a strange state. Trying to delete it results in "There is an operation pending for this application. Please wait and try again" for a long time now. Is there any resolution for this short of redeploying a new version?
interesting:
Error mapping custom domain on Appengine: This guy has the same error with a different task, but also just now. Google status says everything is ok, but its an interesting coincidence.
I have a Google App Engine servlet that is cron configured to run once a week. Since it will take more than 1 minute of execution time, it launches a task (i.e. another servlet task/clear) on the application's default push task queue.
Now what I'm observing is this: if the task causes an exception (e.g. NullPointerException inside its second servlet), this gets translated into HTTP status 500 (i.e. HttpURLConnection.HTTP_INTERNAL_ERROR) and Google App Engine apparently reacts by immediately relaunching the same task again. It announces this by printing:
Web hook at http://127.0.0.1:8888/task/clear returned status code 500. Rescheduling..
I can see how this can sometimes be a feature, but in my scenario it's inappropriate. Can I request that Google App Engine should not do such automatic rescheduling, or am I expected to use other status codes to indicate error conditions that would not cause rescheduling by its rules? Or is this something that happens only on the dev. server?
BTW, I am currently also running other tasks (with different frequencies) on the same task queue, so throttling reschedules on the level of task queue configuration would be inconvenient (so I hope there is another/better option too.)
As per https://developers.google.com/appengine/docs/java/taskqueue/overview-push#Java_Task_execution - the task must return a response code between 200 and 299.
You can either return the correct value, set the taskRetryLimit in RetryOptions or check the header X-AppEngine-TaskExecutionCount when task launches to check how many times it has been launched and act accordingly.
I think I've found a solution: in the Java API, there is a method RetryOptions#taskRetryLimit, which serves my case.