An existing application uses Camel logging (bog the "log()" DSL, and also the Log component.
We would like to either intercept or override so that every log message also logs out a specific Header value (e.g. x-correlation-id=ABC-123)
What is a good, idiomatic way to achieve this?
Apache Camel supports pluggable LogListener since version 2.19.0. This is pretty powerful, because its method onLog, which is invoked right before logging, have instances of Exchange, CamelLogger and message. You can customize the message there with almost no limitations.
Implementation of LogListener:
public class MyLogListener implements LogListener {
#Override
public String onLog(Exchange exchange, CamelLogger camelLogger, String message) {
return String.format("%s: %s", exchange.getIn().getHeader(Exchange.CORRELATION_ID), message);
}
}
LogListener registration:
getContext().addLogListener(new MyLogListener());
If you are using Apache Camel version 2.21.0 and newer, you dont need register it to context, because it is looked up in Registry, so annotating MyLogListener as #Bean is enough.
Related
I understand that this is because of the way proxies are created for handling caching, transaction related functionality in Spring. And the way to fix it is use AspectJ but I donot want to take that route cause it has its own problems. Can I detect self-invocation using any static analyis tools?
#Cacheable(value = "defaultCache", key = "#id")
public Person findPerson(int id) {
return getSession().getPerson(id);
}
public List<Person> findPersons(int[] ids) {
List<Person> list = new ArrayList<Person>();
for (int id : ids) {
list.add(findPerson(id));
}
return list;
}
If it would be sufficient for you to detect internal calls, you could use native AspectJ instead of Spring AOP for that and then throw runtime exceptions or log warnings every time this happens. That is not static analysis, but better than nothing. On the other hand, if you use native AspectJ, you are not limited to Spring proxies anyway and the aspects would work for self-invocation too.
Anyway, here is what an aspect would look like, including an MCVE showing how it works. I did it outside of Spring, which is why I am using a surrogate #Component annotation for demo purposes.
Update: Sorry for targeting #Component classes instead of #Cacheable classes/methods, but basically the same general approach I am showing here would work in your specific case, too, if you simply adjust the pointcut a bit.
Component annotation:
package de.scrum_master.app;
import static java.lang.annotation.ElementType.TYPE;
import static java.lang.annotation.RetentionPolicy.RUNTIME;
import java.lang.annotation.Retention;
import java.lang.annotation.Target;
#Retention(RUNTIME)
#Target(TYPE)
public #interface Component {}
Sample classes (components and non-components):
This component is to be called by other components should not lead to exceptions/warnings:
package de.scrum_master.app;
#Component
public class AnotherComponent {
public void doSomething() {
System.out.println("Doing something in another component");
}
}
This class is not a #Component, so the aspect should ignore self-invocation inside it:
package de.scrum_master.app;
public class NotAComponent {
public void doSomething() {
System.out.println("Doing something in non-component");
new AnotherComponent().doSomething();
internallyCalled("foo");
}
public int internallyCalled(String text ) {
return 11;
}
}
This class is a #Component. The aspect should flag internallyCalled("foo"), but not new AnotherComponent().doSomething().
package de.scrum_master.app;
#Component
public class AComponent {
public void doSomething() {
System.out.println("Doing something in component");
new AnotherComponent().doSomething();
internallyCalled("foo");
}
public int internallyCalled(String text ) {
return 11;
}
}
Driver application:
Please note that I am creating component instances throughout this sample code with new instead of requesting beans from the application context, like I would do in Spring. But you can ignore that, it is just an example.
package de.scrum_master.app;
public class Application {
public static void main(String[] args) {
new NotAComponent().doSomething();
new AComponent().doSomething();
}
}
Console log when running without aspect:
Doing something in non-component
Doing something in another component
Doing something in component
Doing something in another component
Now with the aspect, instead of the last message we would expect an exception or a logged warning. Here is how to do that:
Aspect:
Sorry for using native AspectJ syntax here. Of course, you could also use annotation-based syntax.
package de.scrum_master.aspect;
import de.scrum_master.app.*;
public aspect SelfInvocationInterceptor {
Object around(Object caller, Object callee) :
#within(Component) &&
call(* (#Component *).*(..)) &&
this(caller) &&
target(callee)
{
if (caller == callee)
throw new RuntimeException(
"Self-invocation in component detected from " + thisEnclosingJoinPointStaticPart.getSignature() +
" to "+ thisJoinPointStaticPart.getSignature()
);
return proceed(caller, callee);
}
}
Console log when running with aspect:
Doing something in non-component
Doing something in another component
Doing something in component
Doing something in another component
Exception in thread "main" java.lang.RuntimeException: Self-invocation in component detected from void de.scrum_master.app.AComponent.doSomething() to int de.scrum_master.app.AComponent.internallyCalled(String)
at de.scrum_master.app.AComponent.internallyCalled_aroundBody3$advice(AComponent.java:8)
at de.scrum_master.app.AComponent.doSomething(AComponent.java:8)
at de.scrum_master.app.Application.main(Application.java:6)
I think, you can use this solution and maybe rather log warnings instead of throwing exceptions in order to softly guide your co-workers to inspect and improve their AOP-dependent Spring components. Sometimes maybe they do not wish self-invocation to trigger an aspect anyway, it depends on the situation. You could run the Spring application in full AspectJ mode and then, after evaluating the logs, switch back to Spring AOP. But maybe it would be simpler to just use native AspectJ to begin with and avoid the self-invocation problem altogether.
Update: In AspectJ you can also make the compiler throw warnings or errors if certain conditions are met. In this case you could only statically determine calls from components to other components, but without differentiating between self-invocation and calls on other methods from other components. So this does not help you here.
Please also notice that this solution is limited to classes annotated by #Component. If your Spring bean is instantiated in other ways, e.g. via XML configuration or #Bean factory method, this simple aspect does not work. But it could easily be extended by checking if the intercepted class is a proxy instance and only then decide to flag self-invocations. Then unfortunately, you would have to weave the aspect code into all of your application classes because the check can only happen during runtime.
I could explain many more things, such as using self-injection and call internal methods on the injected proxy instance instead of via this.internallyCalled(..). Then the self-invocation problem would be solved too and this approach also works in Spring AOP.
Can I detect self-invocation using any static analysis tools?
In theory you can, but be aware of Rice's theorem. Any such tool would sometimes give false alarms.
You could develop such a tool using abstract interpretation techniques. You may need more than a year of work.
You could subcontract the development of such tools to e.g. the Frama-C team. Then email me to basile.starynkevitch#cea.fr
Requisite disclaimer about being new to Camel--and, frankly, new to developing generally. I'd like to have a string generated as the output of some function be the source of my camel route which then gets written to some file. It's the first part that seems challenging: I have a string, how do I turn it into a message? I can't write it into a file nor can I use JMS. I feel like it should be easy and obvious, but I'm having a hard time finding a simple guide to help.
Some pseudo-code using the Java DSL:
def DesiredString() {return "MyString";}
// A camel route to be implemented elsewhere; I want something like:
class MyRoute() extends RouteBuilder {
source(DesiredString())
.to("file://C:/out/?fileName=MyFileFromString.txt");
}
I vaguely understand using the bean component, but I'm not sure that solves the problem: I can execute my method that generates the string, but how do I turn that into a message? The "vague" is doing a lot of work there: I could be missing something there.
Thanks!
Not sure if I understand your problem. There is a bit of confusion about what the String should be become: the route source or the message body.
However, I guess that you want to write the String returned by your method into a File through a Camel route.
If this is correct, I have to clarify first the route source. A Camel Route normally starts with
from(component:address)
So if you want to receive requests from remote via HTTP it could be
from("http4:localhost:8080")
This creates an HTTP server that listens on port 8080 for messages.
In your case I don't know if the method that returns the String is in the same application as the Camel route. If it is, you can use the Direct component for "method-like" calls in the same process.
from(direct:input)
.to("file:...");
input is a name you can freely choose. You can then route messages to this route from another Camel route or with a ProducerTemplate
ProducerTemplate template = camelContext.createProducerTemplate();
template.sendBody("direct:input", "This is my string");
The sendBody method takes the endpoint where to send the message and the message body. But there are much more variants of sendBody with different signatures depending on what you want to send it (headers etc).
If you want to dive into Camel get a copy of Camel in Action 2nd edition. It contains everything you need to know about Camel.
Example:Sending String(as a body content)to store in file using camel Java DSL:
CamelContext context = new DefaultCamelContext();
context.addRoutes(new RouteBuilder() {
public void configure() {
from("timer:StringSentToFile?period=2000")
.setBody(simple(DesiredString()))
.to("file:file://C:/out/?fileName=MyFileFromString.txt&noop=true")
.log("completed route");
}
});
ProducerTemplate template = context.createProducerTemplate();
context.start();
I'm using Camel Rest (with restlet component) and I have the following APIs:
rest("/order")
.get("/{orderId}").produces("application/json")
.param().dataType("int").type(RestParamType.path).name("orderId").endParam()
.route()
.bean(OrderService.class, "findOrderById")
.endRest()
.get("/customer").produces("application/json").route()
.bean(OrderService.class, "findOrderByCustomerId")
.endRest()
The problem is that the /order/customer doesn't works (see Exception below). The parameters for /customer comes from JWT...
java.lang.String to the required type: java.lang.Long with value
customer due Illegal characters: customer
I think that camel is confusing the ../{orderId} parameter with .../customer.
If I change the /customer for /customer/orders it's works.
The same idea in Spring Boot could have done with:
#RequestMapping("/order/{orderId}")
public Order getOrder(#PathVariable Long orderId) {
return orderRepo.findOne(orderId);
}
#RequestMapping("/order/customer")
public List<Order> getOrder() {
return orderRepo.listOrderByCustomer(1l);
}
Any idea about what's happening?
Try changing the order of your GET operations in the Camel Rest DSL. The restlet component has some issues in matching the best possible methods.
There is a couple of JIRA tickets related to this:
https://issues.apache.org/jira/browse/CAMEL-12320
https://issues.apache.org/jira/browse/CAMEL-7906
I'm using Camel with Spring Boot. I want to set "connectionTimeToLive" option for http component at global scope so that every use of the component will have the option. How can I do that?
After reading Camel test cases, I found out this solution using Custom Camel context configuration:
#Bean
CamelContextConfiguration contextConfiguration() {
return new CamelContextConfiguration() {
#Override
public void beforeApplicationStart(CamelContext context) {
HttpComponent http = context.getComponent("http4", HttpComponent.class);
http.setConnectionTimeToLive(5000);
}
#Override
public void afterApplicationStart(CamelContext camelContext) {
}
};
}
You have several options.
Add it to the camel registry and fetch it from there.
Add it as a Camel Exchange property.
Fetch it from a property file.
The way Camel works, you will have to configure this value in a property placeholder.
Also you can define endpoints in camel, instead of defining them straight away in the routes. (Eg: <endpoint id="bla" uri="foo" .. />). This way you can refer them in multiple places.
Also if you want to use this endpoint for multiple hosts, then consider passing things like host name, etc as a header. Eg: Exchange.HTTP_PATH
I am not sure whether Camel has any other Global config approach.
Cheers.
Here is the route:
from("aws-sqs://myQueue?accessKey=RAW(xxx)&secretKey=RAW(yyy)&deleteAfterRead=false")
.log("Attributes: ${header.CamelAwsSqsAttributes}")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
Map<String, String> messageAttributes = (Map<String, String>) exchange.getIn().getHeader("CamelAwsSqsAttributes");
...
}
});
The .log() shows an empty map as well as if I print messageAttributes from the processor.
I also tried with the header "CamelAwsSqsMessageAttributes" instead of "CamelAwsSqsAttributes" but still nothing.
I see the attributes from the AWS console though.
By the way I get the message body, and I use Camel 2.15
I figured it out, here is an example to get queue attributes and message attributes:
main.bind("sqsAttributeNames", Collections.singletonList("All"));
main.bind("sqsMessageAttributeNames", Collections.singletonList("All"));
Or add those objects to the registry if you don't use org.apache.camel.main.Main
Then:
from("aws-sqs://myQueue?accessKey=RAW(xxx)&secretKey=RAW(yyy)&deleteAfterRead=false&attributeNames=#sqsAttributeNames&messageAttributeNames=#sqsMessageAttributeNames")
Of course you can replace Collections.singletonList("All") with the list of attributes you need if you don't want all of them.
I faced the same issue. When I am using camel-aws 2.16.x and I have my endpoint configured as follow
from("aws-sqs://myQueue?...&messageAttributeNames=#sqsMsgAttributeNames")
.to(...)
Then I have defined a Collection of String in my spring configuration file
#Bean
public Collection<String> sqsMsgAttributeNames() {
return Arrays.asList("Attr1", "Attr2");
}
Above settings work fine but ever since I upgraded to camel-aws 2.17.3. It no longer works. As mentioned in Camel SQS Component, collection of string no longer will be supported for messageAttributeNames and it should be a String with attributes separated by comma.
Note: The string containing attributes should not contain any white
spaces otherwise camel-aws component will only read the first
attribute. I went through the pain to debug on this. Besides, setting the
attribute value to be "All" does not work for me, none of the message
attributes will be read.
Below is the changes I made that allowed camel-aws's SqsConsumer to work again:
#Bean
public String sqsMsgAttributeNames() {
return String.format("%s,%s", "Attr1", "Attr2");
}
It is not an issue of Camel. It can be the default behavior of SQS or aws-java-sdk-core library.
As a quick solution this aws-sqs URL can be used
aws-sqs://myQueue?<other attributes here>&attributeNames=All
Keep in mind that localstack can work well without attributeNames parameter, unlike SQS.