We need our Camel app to raise an exception and reject incoming request if the msg count exceeds specified threshold (i.e., allow 4 requests in 10 sec time span).
Below is the configuration we have in our camel context file right after jetty front side http listener.
<throttle timePeriodMillis="10000" rejectExecution="true">
<constant>4</constant>
<to uri="bean:someEndPoint"/>
</throttle>
When we invoke Camel app via jmeter, throttle happens for 5th request and subsequent requests too... however, throttle keeps happening forever even after the expiry of 10 sec time span. While, some other times throttle doesn't happen at all during the newer 10 sec time span windows.
Please help here.
Thx
Ramesh
Related
I have a nested logic app which takes some time for 4 retries in case of a failure. According to the documentation, the default HTTP timeout is 100 seconds. I'm able to increase the HTTP client timeout value in my code which triggers the parent logic app, but in case of a failure in the child logic app, it is retried 4 times and takes much longer. Meanwhile, the parent logic app responds with a 504 (gateway timeout). There are some more actions to take care of after the child logic app returns a response, so I can't make it asynchronous and return 202 to the code trigger. Is there a way to increase the timeout in the nested logic app without making it async?
E.g. - My nested logic app retried 4 times and failed after 4 minutes
However, my code already receives a response of 504 after 2 minutes 9 seconds of triggering the parent logic app
The HTTP client which triggers the parent logic app has a timeout of 20 minutes. I verified that this timeout value is working, because without it, we were receiving the timeout reponse in 1 minute 40 seconds (100 seconds), which is default HTTP trigger timeout. I'm under the impression that if the nested logic app also doesn't respond within 100 seconds of being triggered, the parent throws a timeout because it didn't receive a response.
Is there a way to work around this?
It might not be possible without asynchronous patterns or async calls but One of the workarounds can be using HTTP+Webhooks where you can set the timeout value.
You can research this more from HERE.
I figured out that Azure has a timeout value on incoming HTTP requests and the connection cannot be kept open for more than 2 minutes. This is different from the HTTP client timeout of 100 seconds. So, even though I increased the HTTP client timeout, the Azure logic app couldn't keep the incoming connection open for more than 2 minutes and hence was throwing a 504 Gateway Timeout.
Since my logic app takes more than 2 minutes to finish running, to fix this issue, I started using the asynchronous polling mechanism, by responding to the incoming HTTP request with a 202 (Accepted), and then polling the location sent in this response from my code every 3 minutes till the response status != 202.
From what I understand, the above solution of using HTTP webhooks might also work, because the client timer is paused when we use webhooks. However, that wasn't possible for me, so I had to change my design and use the asynchronous polling pattern.
I have a requirement where I need to throttle by shaping (queuing) inbound traffic when client app sends more than 1000 requests in a 5 sec time span.
The solution I followed is:
I have a camel:throttle setting max requests to 1000 and timespan to 5 sec. When threshold is exceeded I am catching throttle exception and within the onException block, I am sending the throttled messages to an ActiveMQ request queue for further processing later as Camel is overloaded based on 1000 req/ 5 sec config.
I am successful in implementing the above, however I would like to have Camel consumer to further process later not all messages from ActiveMQ request queue at one shot instead process each message with a delay of 10 sec for e.g.
I am not able to set a parameter in ActiveMQ to say delay the message to consumer nor delay Camel consumer pulling off the message from request queue.
How do I cater to my above requirement
Please help
Thanks
Ramesh.
In another SO thread the winning answers promotes the following solution:
from("activemq:queueA").throttle(10).to("activemq:queueB")
To me this solution only makes sense, if you define a prefetch limit, without which the consumer would not care about any downstream throttling. This route should work:
from("activemq:queueA?&destination.consumer.prefetchSize=10").throttle(10).to("activemq:queueB")
This is the threory behind it, right from http://activemq.apache.org/what-is-the-prefetch-limit-for.html
So ActiveMQ uses a prefetch limit on how many messages can be streamed to a consumer at any point in time. Once the prefetch limit is reached, no more messages are dispatched to the consumer until the consumer starts sending back acknowledgements of messages (to indicate that the message has been processed). The actual prefetch limit value can be specified on a per consumer basis.
You can enable scheduled delivery of ActiveMQ and then set in your Camel Route AMQ_SCHEDULED_DELAY header and then send your exchange to a queue. This will result to introducing a delay of AMQ_SCHEDULED_DELAY millis before the message appears in the queue (i.e. be available for consumption).
Check this: http://activemq.apache.org/delay-and-schedule-message-delivery.html
I am using Apache camel to implement dispatcher EIP. There are thousands of messages in a queue which needs to be delivered at different URLs. Each message has its own delivery URL and delivery protocol (ftp,email,http etc).
The way it is been implemented:
Boot a single camel context, the context is disabled for JMX and the
loadStatisticsEnabled is set to false on the ManagementStrategy. As
mentioned in a jira issue, addressed in 2.11.0 version, for disabling
the background management thread creation.
For each message a route is being constructed , the message is being
pushed to the route for delivery.
After the message is processed route is shutdown and removed from
context.
Did a small perf test by having 200 threads of dispatcher component, each sharing the same context.
Observed that the time to start a route increases upto a maximum of 60 seconds while the time to process is in milliseconds.
Issue CAMEL-5675 mentions that this has been fixed but still observing significant time being taken in starting up routes.
https://issues.apache.org/jira/browse/CAMEL-5675
The route that is being creating for http is
from("direct:"+dispatchItem.getID())
.toF("%s?httpClient.soTimeout=%s&disableStreamCache=true", dispatchItem.getEndPointURL(),timeOutInMillis);
Each dispatchItem has a unique ID.
This is being active discussed elsewhere, where the user posted this question first: http://camel.465427.n5.nabble.com/Slow-startup-of-routes-tp5732356.html
I am observing strange behavior of my tomcat server, it seems like tomcat is not writing response to the client fast enough. Here is what I am seeing:
When firing aound 200 requests at the same time at my tomcat server, my application logs shows that my servlet's doGet() finishes process the request in about 500ms. However, at the client side the average response time is about 30 seconds (which means client start seeing response from tomcat after 30 seconds)!
Does anyone have any idea about how come there are such long delay between the end of my servlet's process time and the time when client receives response?
My server is hosted on Rackspace VM.
Found the culprit. I observed that the hosting server was using abnormally high CPU usage for even for only few requests, so I attached JConsole to Tomcat and found that all my worker thread has high blocking count... and are constantly in blocking state. Looking at the stack trace the locking happened during JAXBContext instantiation. Digg further, the application creating JAXBContext, which is relatively expensive, for each request.
So in summary, the problem was caused by JAXBContext instantiation per thread. Solution was to ensure JAXBContext is created once per application.
I am connecting to a 3rd party server on the following wsdl
http://webservices.ticketvala.com/axis2/services/WSTicketvala?wsdl
I am using JAX-WS to generate client code and call relevent method on 3rd party server. 3rd party server may take time between 15-25 seconds to send response.
It works fine on tomcat.
Now when i deploy this to GAE 1.5.3, often i get ScocketTimeoutException in less than 10 seconds. Sometimes it is succesfull taking even 20 seconds. I want to know why it fails many times. And any workaround to increase this response deadline time / to avoid this ScoketTimeOutException forever.
Similarly,
I have another RESTfull service at http://ticketgoose.com/bookbustickets/TGWSStationNameResponseAction.do?" +
"event=getStationDetails&password=123456&userId=ctshubws
I am connecting it through java.net.URL and many times i get TimeoutException. How can i raise this timeout limit to more than 30 seconds?
Thanks
Deepak
No user-initiated request can take more than 30 seconds to complete in Google App Engine:
http://code.google.com/intl/en/appengine/docs/java/runtime.html#The_Request_Timer
And a HTTP request to a external URL in a user-initiated request can't take more than 10 seconds to complete:
http://code.google.com/intl/en/appengine/docs/java/urlfetch/overview.html#Requests
If you need to do the overall work in more than 30 seconds and can do it in background (not needing to return the response directly via HTTP), use Task Queues. Note that the simplest way to do background work with tasks is to use the DeferredTask. And the best part: HTTP requests to external URLs on tasks can take up to 10 minutes to complete.