I have a nested logic app which takes some time for 4 retries in case of a failure. According to the documentation, the default HTTP timeout is 100 seconds. I'm able to increase the HTTP client timeout value in my code which triggers the parent logic app, but in case of a failure in the child logic app, it is retried 4 times and takes much longer. Meanwhile, the parent logic app responds with a 504 (gateway timeout). There are some more actions to take care of after the child logic app returns a response, so I can't make it asynchronous and return 202 to the code trigger. Is there a way to increase the timeout in the nested logic app without making it async?
E.g. - My nested logic app retried 4 times and failed after 4 minutes
However, my code already receives a response of 504 after 2 minutes 9 seconds of triggering the parent logic app
The HTTP client which triggers the parent logic app has a timeout of 20 minutes. I verified that this timeout value is working, because without it, we were receiving the timeout reponse in 1 minute 40 seconds (100 seconds), which is default HTTP trigger timeout. I'm under the impression that if the nested logic app also doesn't respond within 100 seconds of being triggered, the parent throws a timeout because it didn't receive a response.
Is there a way to work around this?
It might not be possible without asynchronous patterns or async calls but One of the workarounds can be using HTTP+Webhooks where you can set the timeout value.
You can research this more from HERE.
I figured out that Azure has a timeout value on incoming HTTP requests and the connection cannot be kept open for more than 2 minutes. This is different from the HTTP client timeout of 100 seconds. So, even though I increased the HTTP client timeout, the Azure logic app couldn't keep the incoming connection open for more than 2 minutes and hence was throwing a 504 Gateway Timeout.
Since my logic app takes more than 2 minutes to finish running, to fix this issue, I started using the asynchronous polling mechanism, by responding to the incoming HTTP request with a 202 (Accepted), and then polling the location sent in this response from my code every 3 minutes till the response status != 202.
From what I understand, the above solution of using HTTP webhooks might also work, because the client timer is paused when we use webhooks. However, that wasn't possible for me, so I had to change my design and use the asynchronous polling pattern.
Related
Google describes basic scaling like this:
I don't really have any other choice, as I'm using a B1 instance, so automatic scaling is not allowed.
That raises the question though, if I have an endpoint that takes a variable amount of time (could be minutes, could be hours), and I basically have to set an idle_timeout, will App Engine calculate idle_timeout from the time the request was made in the first place or when the app is done handling the request?
If the former is right, then it feels a bit unfair to have to guess how long requests will take when thread activity is a usable indicator of whether or not to initiate shutdown of an app.
Here you are mixing up two different terms.
idle_timeout is the time that the instance will wait before shutting down after receiving its last request
Request timeout is the amount of time that App Engine will wait for the return of a request from your app
As per the documentation:
Requests can run for up to 24 hours. A manually-scaled instance can
choose to handle /_ah/start and execute a program or script for many
hours without returning an HTTP response code. Task queue tasks can
run up to 24 hours.
I have an application where a user can upload a PDF using angular-file-upload.js
This library does not support file chunking: https://github.com/nervgh/angular-file-upload/issues/41
My elastic load balancer is configured to have an idle timeout of 10 seconds and other parts of the application depend on keeping this parameter.
The issue is if the file upload takes longer than 10 seconds the user receives a 504 Gateway Timeout in the browser and an error message. However, the file still reaches the server after some time.
How can I ignore or not show the user this 504 Gateway Timeout that comes from the ELB? Is there another way around this issue?
The issue you have is that an ELB is always going to close the connection unless it gets some traffic back from your server. See below from AWS docs. It's the same behaviour for an ALB or a Classic load balancer.
By default, Elastic Load Balancing sets the idle timeout to 60 seconds
for both connections. Therefore, if the instance doesn't send some
data at least every 60 seconds while the request is in flight, the
load balancer can close the connection. To ensure that lengthy
operations such as file uploads have time to complete, send at least 1
byte of data before each idle timeout period elapses, and increase the
length of the idle timeout period as needed.
So to get around this, you have two options:
Change the server processing to start sending some data back as soon as the connection is established, on an interval of less than 10 seconds.
Use another library for doing your uploads, or use vanilla javascript. There are plenty of examples out there, e.g. this one.
Edit: Third option
Thanks to #colde for making the valid point that you can simply work around your load balancer altogether. This has the added benefit of freeing up your server resources which get tied up with lengthy uploads. In our implementation of this we used pre-signed urls to securely achieve this.
We need our Camel app to raise an exception and reject incoming request if the msg count exceeds specified threshold (i.e., allow 4 requests in 10 sec time span).
Below is the configuration we have in our camel context file right after jetty front side http listener.
<throttle timePeriodMillis="10000" rejectExecution="true">
<constant>4</constant>
<to uri="bean:someEndPoint"/>
</throttle>
When we invoke Camel app via jmeter, throttle happens for 5th request and subsequent requests too... however, throttle keeps happening forever even after the expiry of 10 sec time span. While, some other times throttle doesn't happen at all during the newer 10 sec time span windows.
Please help here.
Thx
Ramesh
Everytime I ping amazon to get _GET_FBA_MYI_UNSUPPRESSED_INVENTORY_DATA_ I can make the request and it SUBMITTED for about 10 seconds then when I ping again it says cancel I have tried making the request 3 times and all of the times it seems to get cancelled.
You are querying too much. For that particular report I request once an hour (the most you can) and if you get a cancel message have it wait an hour and then query again. It is initially hard to debug so I would keep around the exports to test your other portions of your code while keep the report download code separate.
I am connecting to a 3rd party server on the following wsdl
http://webservices.ticketvala.com/axis2/services/WSTicketvala?wsdl
I am using JAX-WS to generate client code and call relevent method on 3rd party server. 3rd party server may take time between 15-25 seconds to send response.
It works fine on tomcat.
Now when i deploy this to GAE 1.5.3, often i get ScocketTimeoutException in less than 10 seconds. Sometimes it is succesfull taking even 20 seconds. I want to know why it fails many times. And any workaround to increase this response deadline time / to avoid this ScoketTimeOutException forever.
Similarly,
I have another RESTfull service at http://ticketgoose.com/bookbustickets/TGWSStationNameResponseAction.do?" +
"event=getStationDetails&password=123456&userId=ctshubws
I am connecting it through java.net.URL and many times i get TimeoutException. How can i raise this timeout limit to more than 30 seconds?
Thanks
Deepak
No user-initiated request can take more than 30 seconds to complete in Google App Engine:
http://code.google.com/intl/en/appengine/docs/java/runtime.html#The_Request_Timer
And a HTTP request to a external URL in a user-initiated request can't take more than 10 seconds to complete:
http://code.google.com/intl/en/appengine/docs/java/urlfetch/overview.html#Requests
If you need to do the overall work in more than 30 seconds and can do it in background (not needing to return the response directly via HTTP), use Task Queues. Note that the simplest way to do background work with tasks is to use the DeferredTask. And the best part: HTTP requests to external URLs on tasks can take up to 10 minutes to complete.