I am trying to make an asynchronous call to a webservice using apache camel API.
If I am making normal call, it is working fine. But when I am trying to make asynchronous call then it is not hitting the API. Can you please help me what is the issue here.
#Test
public void testSendAsync() {
Map<String, Object> headers = new HashMap<>();
headers.put("content-type", "application/json");
System.out.println("start = "+headers);
ProducerTemplate template = context.createProducerTemplate();
/*template.sendBodyAndHeaders("direct:invokeAPI","{\n" +
"\"source\":\"PDF\"\n" +
"}", headers);
*/
Future fut = template.asyncRequestBodyAndHeaders("direct:invokeAPI","{\n" +
"\"source\":\"PDF\"\n" +
"}", headers);
System.out.println("end = "+headers + " , " + fut.isDone());
}
In above code sendBodyAndHeaders is working fine, I can see that it is hitting the API successfully.
In the case of asynchronous call asyncRequestBodyAndHeaders, it is not hitting the API, I am seeing the system.out messages for start and end. What is the issue here?
It is possible the async request is creating another thread and the test is stopping completion before that other thread has a chance to complete the API operation.
Can you try to "wait" or "yield" the thread?
Multi-threading can sometimes be difficult in tests.
Related
I am currently on Flink version 1.6 and am facing an issue with AsyncIO wherein the performance is not up to my expectation.
I am sure I am doing something wrong in my implementation, so any advice/suggestions would be appreciated.
Issue Synopsis -
I am consuming a stream of ids.
For each id, I need to call a REST service.
I've implemented a RichAsyncFunction, which performs the async REST call.
Here's the relevant code method and the asyncInvoke method
// these are initialized in the open method
ExecutorService executorService =
ExecutorService.newFixedThreadPool(n);
CloseableHttpAsyncClient client = ...
Gson gson = ...
public void asyncInvoke(String key, final ResultFuture<Item> resultFuture) throws Exception {
executorService.submit(new Runnable() {
client.execute(new HttpGet(new URI("http://myservice/" + key)), new FutureCallback<HttpResponse>() {
#Override
public void completed(final HttpResponse response) {
System.out.println("completed successfully");
Item item = gson.fromJson(EntityUtils.toString(response.getEntity), Item.class);
resultFuture.complete(Collections.singleton(item));
}
});
});
}
With the above implementation, I've tried :-
Increasing the parallelism of the enrichment operation
Increasing the number of threads in the executor service
Using apache http async client, I've tried tweaking the connection manager settings - setDefaultMaxPerRoute and setMaxTotal.
I am consistently getting a throughput of about 100 requests/sec. The service is able to handle more than 5k per sec.
What am I doing wrong, and how can I improve this ?
I am having a scenario where I am using consumer template to receive file from a endpoint. The Endpoint could be either File System or FTP site. Currently I am using only File System with following endpoint URL:
file://D:/metamodel/Seach.json?noop=true&idempotent=false
On every hit to following code:
Exchange exchange = consumerTemplate.receive(endPointURI, timeout);
if (exchange != null) {
String body = exchange.getIn().getBody(String.class);
consumerTemplate.doneUoW(exchange);
return body;
}
It creating a new Camel context thread and after some hits it giving error as
java.util.concurrent.RejectedExecutionException: PollingConsumer on Endpoint[file://D:/metamodel/Seach.json?noop=true&idempotent=false] is not started, but in state:Stopped
I am not sure why this is happening and its sporadic in nature.
Any suggestion on this would do great help.
I want time delay before function call in apex code. I already created one delay method but it is not working as per expectation. So, is there any way to get this working.
Thanks in advance.
Probably a better way to do this would be to break up your Apex code such that the part you want to execute later is in a separate method. You can then call this method from another method that has an #future annotation, or use the Apex Scheduler to schedule that code for a future time. Either of these methods will cause the code to be executed asynchronously after your original method has completed (the #future method is easier to implement but the scheduler method has the advantage of running at a predictable time).
If you need something like the sleep() function, one way to do it is to make a call to a http service which will sleep a requested amount of time. This is fairly simple to do, and there are existing publicly available services for it, for example the one at http://1.cuzillion.com/bin/resource.cgi.
First you have to Configure a new Remote Site in SalesForce (Security Controls -> Remote Site Settings), name it however you want but make sure the Remote Site URL matches the above URL, and that the "Active" checkbox is checked.
After that, you can define your method in code like so:
public static void sleep(Integer sleepSeconds) {
Long startTS = System.currentTimeMillis();
HttpRequest req = new HttpRequest();
req.setEndpoint('http://1.cuzillion.com/bin/resource.cgi?sleep=' + sleepSeconds);
req.setMethod('GET');
Http http = new Http();
HTTPResponse res = http.send(req);
Long duration = System.currentTimeMillis() - startTS;
System.debug('Duration: ' + duration + 'ms');
}
Running it yields:
sleep(1);
-> 08:46:57.242 (1242588000)|USER_DEBUG|[10]|DEBUG|Duration: 1202ms
You can easily do this in Visualforce. Either use apex:actionPoller and set the timeout property to whatever you want the interval to be. Or use window.setTimeout(function, 1000) in JavaScript. Then from the function in JavaScript you can either use JavaScript Remoting or apex:actionFunction to call back into Apex.
I'm trying to make multiple consecutive HttpWebRequest in WP7. Each one uses info (an ID from a cookie, an URL location for a redirect, some HTML...) from the previous one, so we need to "wait" until the previous one finishes before continue.
In Android, we make a new thread and then you go sequential with multiple Synchronous HttpPost or HttpGet, but here it's different because of the async nature of Silverlight.
What I have right now is this in my helper class (instantiated on the UI Thread):
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(theUrl);
request.Method = "POST";
try
{
IAsyncResult ias = request.BeginGetResponse(new AsyncCallback(getCookiesStatus), request);
}
catch (Exception e)
{
Console.Write("ERROR" + e.Message);
}
The getCookieStatus:
private void getCookiesStatus(IAsyncResult result)
{
HttpWebRequest request = (HttpWebRequest)result.AsyncState;
HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(result);
ccCookies = response.Cookies;
HttpStatusCode sc = response.StatusCode;
if (sc == HttpStatusCode.Found)
{
location = response.Headers["Location"];
}
}
What would be the best way to use that "location" string we got to make another consecutive http request without chaining calls from function to function? Is there any way to raise a "waker" (an event?) in the UI thread after the IAsyncResult?
The async ctp is a new visual studio feature that provides some great syntactic sugar for exactly this type of asynchronous sequential workflow. It is unfortunately not suitable for production code yet as it's still a technology preview.
Until then, you have to either chain callbacks (not awful for 2 or 3 operations but beyond that: spaghetti) or use something like a coroutine pattern implemented with iterators and generators.
Edit: #AnthonyWJones linked to another coroutine implementation using a generator in the comments above.
Reading the Google App Engine documentation on asynchronous URL Fetch:
The app can have up to 10 simultaneous
asynchronous URL Fetch calls
What happens if an application calls more than 10 async fetch at a time?
Does Google App Engine raise an exception or simply queue the remain calls waiting to serve them?
Umm, Swizzec is incorrect. Easy enough to test:
rpc = []
for i in range(1,20):
rpc.append(urlfetch.createrpc())
urlfetch.make_fetch_call(rpc[-1],"http://stackoverflow.com/questions/3639855/what-happens-if-i-call-more-than-10-asynchronous-url-fetch")
for r in rpc:
response = r.get_result().status_code
This does not return any exceptions. In fact, this works just fine! Note that your results may vary for non-billable applications.
What Swizec is reporting is a different problem, related to maximum simultaneous connections INTO your application. For billable apps there is no practical limit here btw, it just scales out (subject to the 1000ms rule).
GAE has no way of knowing that your request handler will issue a blocking URL fetch, so the connection 500's he is seeing are not related to what his app is actually doing (that's an oversimplification btw, if your average request response time is > 1000ms your likelyhood of 500's increases).
This is an old question, but I believe the accepted answer is incorrect or outdated and may confuse people. It's been a couple of months that I actually tested this, but in my experience Swizec is quite right that GAE will not queue but rather fail most asynchronous URL fetches exceeding the limit of around 10 simultaneous ones per request.
See https://developers.google.com/appengine/docs/python/urlfetch/#Python_Making_requests and https://groups.google.com/forum/#!topic/google-appengine/EoYTmnDvg8U for a description of the limit.
David Underhill has come up with a URL Fetch Manager for Python, which queues asynchronous URL fetches that exceed the limit in application code.
I have implemented something similar for Java, which synchronously blocks (due to the lack of a callback function or ListenableFutures) additional requests:
/**
* A URLFetchService wrapper that ensures that only 10 simultaneous asynchronous fetch requests are scheduled. If the
* limit is reached, the fetchAsync operations will block until another request completes.
*/
public class BlockingURLFetchService implements URLFetchService {
private final static int MAX_SIMULTANEOUS_ASYNC_REQUESTS = 10;
private final URLFetchService urlFetchService = URLFetchServiceFactory.getURLFetchService();
private final Queue<Future<HTTPResponse>> activeFetches = new LinkedList<>();
#Override
public HTTPResponse fetch(URL url) throws IOException {
return urlFetchService.fetch(url);
}
#Override
public HTTPResponse fetch(HTTPRequest request) throws IOException {
return urlFetchService.fetch(request);
}
#Override
public Future<HTTPResponse> fetchAsync(URL url) {
block();
Future<HTTPResponse> future = urlFetchService.fetchAsync(url);
activeFetches.add(future);
return future;
}
#Override
public Future<HTTPResponse> fetchAsync(HTTPRequest request) {
block();
Future<HTTPResponse> future = urlFetchService.fetchAsync(request);
activeFetches.add(future);
return future;
}
private void block() {
while (activeFetches.size() >= MAX_SIMULTANEOUS_ASYNC_REQUESTS) {
// Max. simultaneous async requests reached; wait for one to complete
Iterator<Future<HTTPResponse>> it = activeFetches.iterator();
while (it.hasNext()) {
if (it.next().isDone()) {
it.remove();
break;
}
}
}
}
}
500 errors start happening. Silently.
You only find out about these when you look at your log under all requests (don't get listed as errors). It simply says "The request was aborted because you reached your simultaneous requests limit".
So when you're making lots of asynchronous calls, make sure you can handle some of them spazzing out.
See if this answers your question:
http://groups.google.com/group/google-appengine/browse_thread/thread/1286139a70ef83c5?fwc=1