A route which polls for files has an interceptor written as below. We wanted to know the total time processed. So we logged the startTime in header.
We observed that the startTime never changes. This code is inside a RouteBuilder::configure method. I then moved the code to a bean and call the bean from the interceptor. Wanted a bit more clarity, reading the camel documenation on the interceptor, I am still not clear, why this does not work
interceptFrom()
.setHeader("fileName",
regexReplaceAll(simple("${file:onlyname.noext.single}"),
"[^a-zA-z\\d]", ""))
.setHeader("startTime", constant(System.currentTimeMillis()))
Because this header is a constant that is set when the route is "constructed" and it never changes during runtime. So it stays the same constant value for every message processed.
See the Camel docs for constant. You can't set dynamic values with it.
But #claus-ibsen added a comment that there is already a message header with the creation timestamp of the exchange. You can just use it.
simple("${in.header.CamelCreatedTimestamp}")
Related
I am using the latest #stomp/stompjs package in react and it appears to be not working as described so I was wondering if anyone know what was going on.
When I do stompClient.subscribe() I expect only 1 message from the backend. But I normally get an empty object first "{}" and then then the callback goes off again I then get the expect json string. Sometimes I get a 3rd callback with the same json string. I know the page is being redrawn with the new data by react, but I would not think this would cause the callback to go off a second time. Also, this behavior seems to be only for large json string responses. Small response status json objects never returned multiple times in the callback (just the {} object the first time). My workaround was to have a flag that I sett when I do the send command and then is set to false when I get a valid response back. Any extract responses are skipped not saved in the useState hook so react doesn't redraw again.
The second thing I see is sometimes the wrong response is received by the callback. The bookmark paths would be like /location/read and /location/readBL but the response for /location/read would be read by the readBL subscription callback sometimes. This happens 1 out of 10, but I don't understand why it would doing that. The workaround I did is to have the main object key have different words like {camera: {}) and {cameraBL: {}} and see if the object key matches in the expected key in the callback, otherwise skip it.
I assume react is somehow responsible for all this. So, has anyone seen this in their stomp code and know what's going on?
I have inserted flags or filtering to workaround the problem. I have not seen any description of these problems on the web and the stomp home page doesn't talk about any of this.
I'm using Camel to implement the saga pattern using its Saga EIP, one of my requirements is having the route body in the compensation routes, to achieve this the documentation advises to use the option method:
from("direct:example")
.routeId("example")
.enrich("direct:getData", dataAggregationStrategy)
.process(somethingProcessor)
.setHeader("Body", body())
.saga()
.option("OptionBody", body())
.completion("direct:exampleDone")
.compensation("direct:exampleCompensated")
.to("direct:doSomething");
In my code I'm doing some enrich and process steps to manipulate the body and all is well in the routes called by to which receive the correct processed body.
In the compensation routes though I'm receiving an header OptionBody containing the original body passed to the direct:example route and not the processed one.
I tried to set an header with setHeader and the body is the correct version (the processed one).
I'm not sure why this is happening, is it because of lazy evaluation of the expressions?
I have this simple route in my RouteBuilder.
from("amq:MyQueue").routeId(routeId).log(LoggingLevel.DEBUG, "Log: ${in.headers} - ${in.body}")
As stated in the doc for HTTP-component:
Camel will store the HTTP response from the external server on the OUT body. All headers from the IN message will be copied to the OUT message, ...
I would like to know if this concept also applies to amq-component, routeId, and log? Is it the default behaviour, that IN always gets copied to OUT?
Thank you,
Hadi
First of all: The concept of IN and OUT messages is deprecated in Camel 3.x.
This is mentioned in the Camel 3 migration guide and also annotated on the getOut method of the Camel Exchange.
However, it is not (yet) removed, but what you can take from it: don't care about the OUT message. Use the getMessage method and don't use getIn and getOut anymore.
To answer your question:
Yes, most components behave like this
Every step in the route takes the (IN) message and processes it
The body is typically overwritten with the new processing result
The headers typically stay, new headers can be added
So while the Camel Exchange traverses the route, typically the body is continuously updated and the header list grows.
However, some components like aggregator create new messages based on an AggregationStrategy. In such cases nothing is copied automatically and you have to implement the strategy to your needs.
Far as I understand, PUT request is not supposed to return any content.
Consider the client wants to run this pseudo code:
x = resource.get({id: 1});
x.field1 = "some update";
resource.put(x);
x.field2 = "another update";
resource.put(x);
(Imagine I have an input control and a button "Save", this allows me to change a part of object "x" shown in an input control, then on button click PUT changes to server, then continue editing and maybe "save" another change to "x")
Following different proposals on how to implement optimistic locking in REST APIs, the above code MUST fail, because version mark (however implemented) for "x" as returned by get() will become stale after put().
Then how do you people usually make it work?
Or do you just re-GET objects after every PUT?
You can use "conditional" actions with HTTP, for example the If-Match header described here:
https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.24
In short: You deliver an ETag with the GET request, and supply this ETag back to the server in the If-Match header. The server will respond with a failure if the resource you are trying to PUT has another ETag. You can also use simple timestamps with the If-Unmodified-Since header.
Of course you will have to make your server code understand conditional requests.
For multiple steps, the PUT can indeed return the new representation, it can therefore include the new ETag or timestamp too. Even if the server does not return the new representation for a PUT, you could still use the timestamp from the response with an If-Unmodified-Since conditional PUT.
Here is probably what I was looking for: https://www.rfc-editor.org/rfc/rfc7231#section-4.3.4
They implicitly say that we CAN return ETag from PUT. Though only in the case server applied the changes as they were given, without any corrections.
However this raises yet another question. In real world app PUT caller will run asynchronously in JS gui, like in my example in the question. So, Save button might be pressed several times with or without entering any changes. If we don't use optimistic locking, then supposed PUT idempotency makes it safe to send another PUT query with each button click, as long as the last one wins (but actually if there were changes then it's not guaranteed, so the question remains).
But with optimistic locking, when first PUT succeeds, it returns updatred ETag, ok? And if there is another PUT request running, still with outdated tag version, that latter request will get 412 and the user will see a message "someone else changed the resource" - but actually it was our former changes.
What do you usually do to prevent that? Disable the Save button until its request is fully completed? What if it times out? Or do you think it's acceptable to see concurrent-change error message if it was a timeout, because the stability is already compromised anyway?
I have 2 routes. The first route uses poll enrich to check if a file is present. The second route uses a poll enrich on the same uri to read and process the file. The first route invokes the second via a SEDA queue, like so:
public void configure() throws Exception {
String myFile = "file://myDir?fileName=MyFile.zip&delete=false&readLock=none";
from("direct:test")
.pollEnrich(myFile, 10000)
.to("seda:myQueue")
;
from("seda:myQueue")
.pollEnrich(myFile, 10000)
.log("Do something with the body")
;
}
As it stands, if I execute the first route, the poll enrich finds a file, but when the poll enrich in the second route executes, it returns a body of null. If I just execute the second route on its own, it retrieves the file correctly.
Why does the second poll enrich return null, is the file locked? (I was hoping using a combination of noop,readLock, and delete=false would prevent any locking)
Does camel consider the second poll enrich as a duplicate, therefore filtering it out? (I have tried implementing my own IdempotentRepository to return false on contains(), but the second pollEnrich still returns null)
You may wonder why I'm trying to enrich from 2 routes, the first route has to check if a number of files exist, only when all files are present (i.e., pollEnrich doesn't return null) can the second route start processing them.
Is there an alternative to pollEnrich that I can use? I'm thinking that perhaps I'll need to create a bean that retrieves a file by URI and returns it as the body.
I'm using camel 2.11.0
I realize this is now an old topic, but I just had a similar problem.
I suggest you try the options:
noop=true
which you already have, and
idempotent=false
To tell Camel it is OK to process the same file twice.
Update after testing:
I actually tested this with both settings as suggested above, it works some times, but under moderate load, it fails, i.e. returns null body for some exchanges, although not all.
The documentation indicates that setting noop=true automatically sets idempotent=true, so I am not sure the idempotent setting is being honoured in this case.
Is there any specific reason why you are not using just one route?
I don't understand why you are using two routes for this. File component can check if the file is there and if it is, pull it. If you are worried about remembering the files so you don't get duplicates, you can use an idempotent repository. At least, based on your question, I don't think you need to complicate the logic using two routes and the content enricher EIP.
the second route returns NULL because the file was already consumed in the first route...if you are just looking for a signal message when all files are present, then use a file consumer along with an aggregator and possibly a claim check to avoid carrying around large payloads in memory, etc...
As you've probably learned this does not work as one might expect
noop=true&idempotent=false
my guess is that Camel ignores idempotent=false and as documented uses instance of MemoryMessageIdRepository. To work around this, one can configure file endpoint to use custom idempotent repo:
noop=true&idempotentRepository=#myRepo
and register custom repository in the registry or spring context:
#Component("myRepo")
public class MyRepo implements IdempotentRepository {
#Override
public boolean contains(Object o) {
return false;
}
...
}
Try pollEnrich with strategyMethodAllowNull="true". By default , this value is false. When it is false, the aggregation strategy looks for the existing Exchange body, to aggregate the content returned from file.
When we make strategyMethodAllowNull="true", the existing body is considered as null. So every time , the content of the file is set into the current exchange body