I'm having difficulty understanding some aspects of Google AppEngine Push Task Queue for java, namely, when a task is executed, where does the response go?
If I add something to the queue like this:
Queue queue = QueueFactory.getDefaultQueue();
queue.add(withUrl("/worker").param("key", key));
Then sometime later the task is executed, with something like this:
public class SomeServlet extends HttpServlet {
#Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, IOException {
...
resp.getWriter().println("something"); //-- where does this response go???
}
}
Sorry for the newbie question, but where does this response go? My app didn't set up a normal http request so it's not waiting for a normal http response. I must be missing something fundamental about the task queue.
The response doesn't go anywhere. In a task you would typically write to the datastore, add new tasks to task queue, send xmpp, call external URLs.
If the response code is 200 OK, the task is removed from the queue, if it's an error code, it is retried (depending on retry settings).
There is no document indicated where GAE stored these responses. I would not surprise GAE just
don't care the response after it trigger the request url successfully.
By the way, I don't see a point to care about the response.
If you want to log what happen during the task execution, you should use Log instead.
https://developers.google.com/appengine/docs/java/runtime#Logging
Related
I have an app engine task queue that tries to call a cloud endpoint, but when I see the task queue fire off it gets a 404. I verified the endpoint is configured for post:
#ApiMethod(name = "sendemail", path = "sendemail", httpMethod = HttpMethod.POST)
and I am queueing like this:
TaskOptions lOptions = TaskOptions.Builder.withUrl("/_ah/api/email/v1/sendemail");
I can hit the endpoint using the endpoint explorer, what am I missing? Thanks!
It's probably defaulting to get method try adding method POST or making endspoints method get.
withUrl("....").method(Method.POST)
I have 1,000,000 IDs, and need to call a REST API once per day for each user. How can I use Camel to send batch HTTP requests?
My current solution is to create an HTTP component for every ID, using Quartz. But It will create 1,000,000 Quartz jobs, the performance is bad.
I have two ideas:
If Camel can provide a batch API, I could use that to send 1,000,000
requests to the same URL but with different parameters. That is one
Quartz job. Does Camel support that API?
Every HTTP request only visit one time, and use spring schedule to
control. But for each request Camel will poll the server. How can I
make Camel only call the REST API once?
The codes here:
// ids is a List, contains 1,000,000 id
for (final String id : ids) {
context.addRoutes(new RouteBuilder() {
#Override
public void configure() throws Exception {
from(String.format("quartz://%s/?cron=0+0+12+*+*+?", id))
.to(String.format("http:xxx.com?id=%s", id))
.to("file:to");
}
});
}
It could more heave to create million routes for sending the request with different id.
with the help of bean, you can define the loop of sending the request in the bean method.
BTW, you can change the id with message header "CamelHttpQuery".
I created an App Engine backend to serve http requests for a long running process. The backend process works as expected when the query references an input of small size, but times out when the input size is large. The query parameter is the url of an App Engine BlobStore blob, which is the input data for the backend process. I thought the whole point of using App Engine backends was to avoid the timeout restricts that App Engine frontends possess. How can I avoid getting a timeout?
I call the backend like this, setting the connection timeout length to infinite:
HttpURLConnection connection = (HttpURLConnection)(new URL(url + "?" + query).openConnection());
connection.setRequestProperty("Accept-Charset", charset);
connection.setRequestMethod("GET");
connection.setConnectTimeout(0);
connection.connect();
InputStream in = connection.getInputStream();
int ch;
while ((ch = in.read()) != -1)
json = json + String.valueOf((char) ch);
System.out.println("Response Message is: " + json);
connection.disconnect();
The traceback (edited for anonymity) is:
Uncaught exception from servlet
java.net.SocketTimeoutException: Timeout while fetching URL: http://my-backend.myapp.appspot.com/somemethod?someparameter=AMIfv97IBE43y1pFaLNSKO1hAH1U4cpB45dc756FzVAyifPner8_TCJbg1pPMwMulsGnObJTgiC2I6G6CdWpSrH8TrRBO9x8BG_No26AM9LmGSkcbQZiilhC_-KGLx17mrS6QOLsUm3JFY88h8TnFNer5N6-cl0iKA
at com.google.appengine.api.urlfetch.URLFetchServiceImpl.convertApplicationException(URLFetchServiceImpl.java:142)
at com.google.appengine.api.urlfetch.URLFetchServiceImpl.fetch(URLFetchServiceImpl.java:43)
at com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler$Connection.fetchResponse(URLFetchServiceStreamHandler.java:417)
at com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler$Connection.getInputStream(URLFetchServiceStreamHandler.java:296)
at org.someorg.server.HUDXML3UploadService.doPost(SomeService.java:70)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
As you can see, I'm not getting the DeadlineExceededException, so I think something other than Google's limits is causing the timeout, and also making this a different issue from similar stackoverflow posts on the topic.
I humbly thank you for any insights.
Update 2/19/2012: I see what's going on, I think. I should be able to have the client wait indefinitely using GWT [or any other type of client-side async framework] async handler for an any client request to complete, so I don't think that is the problem. The problem is that the file upload is calling the _ah/upload App Engine system endpoint which then, once the blob is stored in the Blobstore) calls the upload service's doPost backend to process the blob. The client request to _ah/upload is what is timing out, because the backend doesn't return in a timely fashion. To make this timeout problem go away, I attempted to make the _ah_upload service itself a public backend accessible via http://backend_name.project_name.appspot.com/_ah/upload, but I don't think that google allows a system service (like _ah/upload) to be run as a backend. Now my next approach is to just have ah_upload immediately return after triggering the backend processing, and then call another service to get the original response I wanted, after processing is finished.
The solution was to start a backend process as a tasks and add that to the task queue, then returning a response to client before it waits to process the backend task (which can take a long time). If I could have assigned ah_upload to a backend, this would have also solved the problem, since the clien't async handler could wait forever for the backend to finish, but I do not think Google permits assigning System Servlets to backends. The client will now have to poll persisted backend process response data, as Paul C mentioned, since tasks can not respond like a normal servlet.
I'd like to route a webservice request to an InOnly endpoint of a jms queue. Then route a response jms message received from a separate InOnly endpoint back to the webservice client as the response. The webservice request/response is a synchronous InOut pattern, and the sub-routes are asynchronous. What options do I have to achieve this with Camel?
The Camel route here is used to explain my question:
String uri={webserice uri}
from(uri)
.process(new Processor() {
public void process(Exchange exchange) throws Exception {
ServcieRequest req =
exchange.getIn().getBody(ServcieRequest.class);
// One option to me is to spawn another route here to route to jms queue...
ProducerTemplate template = exchange.getContext().createProducerTemplate();
template.sendBodyAndHeaders("jms:queue:INQueue", req.getPayload(), headers);
// then need to wait ...until received jms response from the route below
}});
from("jms:queue:OUTQueue")
.process(new Processor() {
public void process(Exchange exchange) throws Exception {
// received jms response message
// need to update the exchange data in the above route based on jms message
// so the final response to the webservice cilent can receive the data ...
}});
I think you should rely on the request reply mechanism in Camel for this task.
Camel Doc, Exclusive Fixed Reply Queue
So I guess the following DSL route does pretty much what you want (if there is no other reason why you should use a InOnly pattern for the JMS part?). Make sure you tune the requestTimeout if you need (as it defaults to 20Sec timeout).
from(webserviceURI)
.inOut().to("jms:queue:INQueue?replyTo=OUTQueue?replyToType=Exclusive");
Yes, one other thing. If you run this on several nodes, then you need one exclusive queue per node.
I'm trying to consume a webservice from camel using the cxf component, like this:
<cxf:cxfEndpoint id="webservice"
address="http://webservice.url.com/webservice"
serviceClass="com.url.webservice.MyWebService"/>
<camel:camelContext>
<camel:route>
<camel:from uri="direct:a"/>
<camel:inOnly uri="cxf:bean:webservice?defaultOperationName=sendMessage"/>
</camel:route>
</camel:camelContext>
The sendMessage method has no response, hence the inOnly rather than to (although I have the same problem when I try to instead). The problem is that apparently camel still expects a response, and the route hangs while waiting for one. I suppose if I let it try long enough, it would eventually time out.
To be clear, I'm running a test method:
/* ... */
#Produce(uri = "direct:a")
protected ProducerTemplate directA;
#Test
public void sendMessage() throws Exception {
directA.sendBody(new String[] {"client id", "message"});
directB.sendBody(new String[] {"client id", "message 2"});
}
And I'm seeing the effect of the first call (that is, the message arrives at the server), but not the second, and the method doesn't finish running (again, I'm assuming it'll timeout at some point... if so, the timeout's pretty long: I ran the test as I started writing this, and it's still running).
Am I missing something? Is it a bug? Is it just bad practice to have webservice methods with no response?
By the way, when testing methods which have a response, it works fine.
I think Willem Jiang recently fixed some issue with one-way CXF in Camel recently. Maybe try Camel 2.6-SNAPSHOT from trunk.
I personally prefer two-way with web services, just returning some ACK back in case there is no data to return back. The client most often want some confirmation the server has received and acknowledged the data.
camel-cxf producer decides if it will handle the response message by checking if the operation is oneway. Even the sendMessage operation is oneway invocation, your server should send Http states 202 message to the client according the HTTP protocol, otherwise camel-cxf producer will still wait for the response.
BTW, I think the issue[1] that Claus said was related to async invocation of camel-cxf producer with one way message, not sure if it's the same issue as you mentioned(I didn't know which version of camel are you using).
[1]https://issues.apache.org/jira/browse/CAMEL-3426
Willem