I'd like to know if it is possible to use Project Reactor with Apache Camel, so applications be fully reactive and non-blocking IO. I'd like to know how does the Project Reactor support works when integrating other Apache Camel's components.
Can I read for example from S3 reactively (therefore I'll need to use the Async S3 client behind the scenes)? Or will I block when reading from S3 and then just create a Flux out of what has been returned?
Where reactiveness is needed, you should use the relevant spring and reactor libraries. there are pseudo camel code also u can db call in camel bean or processors etc
#RestController
#RequestMapping(value = "/api/books")
#RequiredArgsContructor
public class HomeController {
private final BookRepository bookRepository;
private final ProducerTemplate template
#GetMapping("")
public Flux<Book> getHome() {
List <Book> books=bookRepository.findAll();
X ret = template.requestBody("direct:something", books, X.class);
}
}
#Component
public class SomeRoute extends RouteBuilder {
#Override
public void configure() throws Exception {
from("direct:something")
.process(e-> {
List<Book> books = e.getIn.getBody(List.class)
// some logic
e.getIn.setBody(new X( ))
})
}
Related
I have just started working around the Apache Camel. I have a requirement to implement an FTP/FTPS/SFTP client, which would be used to fetch the files from the respective servers. I was looking into the possibility of using Apache Camel to do this but I am still confused after going through the examples and the tutorials.
The requirement is to fetch the files from the FTP/SFTP servers when the request is received from the scheduler.
Following is the route created using EndPoint-DSL
#Component
public class FtpReceiveRoute extends EndpointRouteBuilder {
#Override
public void configure() throws Exception {
from(
ftp("localhost:2001/home/admin")
.account("admin")
.password("admin12345")
.recursive(true)
)
.routeId("ftpReceive")
.log("From done!")
.to("log:ftp-log")
.log("To done!!");
}
}
I am trying to use the above route by invoking it when the request is made to fetch the file like below.
#Override
protected FtpResponse doMessage(String param, FtpRequest req) {
FtpResponse response = new FtpResponse ();
CamelContext ctx = new DefaultCamelContext();
ctx.addRoutes(##route); //FtpReceiveRoute, add the Routebuilder instance as EndpointRouteBuilder is acceptable.
ctx.start();
//Might need to induce sleep so that all the files are downloaded
ctx.stop();
return response;
}
The confusion is around how to invoke the Camel process with the route. I have used EndpointRouteBuilder to create the route because of the type-safe creation of the endpoint URI. I am not getting an option to add this route to the CamelContext as it expects the RouteBuilder instance which is not type-safe.
Further, the CamelContext is the engine and to invoke the route I would need to start and stop this engine. This I am not able to digest if I need to start and stop the engine to execute a route then I would need to induce some sleep in between so that all files are downloaded. Just to add there are more routes that I need to add with the implementation. Once the engine is started it would load and execute all the added routes which is not the requirement.
Maybe I am not getting how to use this properly. Any resources aiding my situation are welcome. Thanks.
You should not create and start new camel context every time you want to fetch file from server. What you should do instead is start one when your application starts and use that for all your exchanges.
You can use Spring-boot to initialize CamelContext and add annotated RouteBuilders to it automatically. Check the maven archetype camel-archetype-spring-boot for example.
If you want to call camel routes from Java you can Inject CamelContext to your bean and use it to create ProducerTemplate. This can be used to invoke Routes defined in the RouteBuilder.
Using ProducerTemplate.send you can get the resulting exchange.
Using producer template
Using File-component which works very similary to ftp-component.
package com.example;
import org.apache.camel.builder.endpoint.EndpointRouteBuilder;
import org.springframework.stereotype.Component;
#Component
public class MySpringBootRouter extends EndpointRouteBuilder {
#Override
public void configure() {
from(direct("fileFromFTP"))
.routeId("fileFromFTP")
// reads files from <project>/input using file consumer endpoint
.pollEnrich(file("input"), 1000)
// If file is found, convert body to string.
// Which in this case will read contents of the file to string.
.filter(body().isNotNull())
.convertBodyTo(String.class)
.end()
;
}
}
package com.example;
import org.apache.camel.CamelContext;
import org.apache.camel.Exchange;
import org.apache.camel.ProducerTemplate;
import org.apache.camel.support.DefaultExchange;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableScheduling;
import org.springframework.scheduling.annotation.Scheduled;
import static org.apache.camel.builder.endpoint.StaticEndpointBuilders.direct;
#Configuration
#EnableScheduling
public class MySpringBean {
#Autowired
CamelContext camelContext;
#Scheduled(fixedRate = 1000)
public void scheduledTask() {
System.out.println("Scheduled Task!");
if(camelContext.isStopped()) {
System.out.println("Camel context not ready yet!");
return;
}
useProducerTemplate();
}
public void useProducerTemplate(){
ProducerTemplate producerTemplate = camelContext.createProducerTemplate();
Exchange inExchange = new DefaultExchange(camelContext);
//synchronous call!
Exchange result = producerTemplate.send(direct("fileFromFTP").toString(), inExchange);
String resultBody = result.getMessage().getBody(String.class);
String fileName = result.getMessage().getHeader(Exchange.FILE_NAME, String.class);
if(resultBody != null){
System.out.println("Consumed file: "+ fileName + " contents: " + resultBody.toString());
}
else{
System.out.println("No file to consume!");
}
}
}
Depending on what you need to do with the files you could probably do that inside camel route. Then you would only need to call the producerTemplate.sendBody.
public void useProducerTemplate(){
ProducerTemplate producerTemplate = camelContext.createProducerTemplate();
Exchange inExchange = new DefaultExchange(camelContext);
producerTemplate.sendBody(direct("fileFromFTP").toString(), inExchange);
}
Starting stopping camel route
If you want to start polling file consumer only for a short while you can do start the route and use for example aggregation timeout to shutdown the route when no new files have been received in any given duration.
#Component
public class MySpringBootRouter extends EndpointRouteBuilder {
#Override
public void configure() {
AggregationStrategy aggregateFileNamesStrategy = AggregationStrategies
.flexible(String.class)
.accumulateInCollection(ArrayList.class)
.pick(header(Exchange.FILE_NAME))
;
from(file("input"))
.routeId("moveFilesRoute")
.autoStartup(false)
.to(file("output"))
.to(seda("moveFilesRouteTimeout"));
;
from(seda("moveFilesRouteTimeout"))
.routeId("moveFilesRouteTimeout")
.aggregate(constant(true), aggregateFileNamesStrategy)
.completionTimeout(3000)
.log("Consumed files: ${body.toString()}")
.process(exchange -> {
exchange.getContext().getRouteController().stopRoute("moveFilesRoute");
})
.end()
;
}
}
public void startMoveFilesRoute() {
try {
System.out.println("Starting moveFilesRoute!");
camelContext.getRouteController().startRoute("moveFilesRoute");
//Sending null body moveFilesRouteTimeout to trigger timeout if there are no files to transfer
camelContext.createProducerTemplate().sendBody(seda("moveFilesRouteTimeout").toString(), null);
} catch(Exception e) {
System.out.println("failed to stop route. " + e);
}
}
What is the proper way to use endpoint DSL and then reference the endpoint with ProducerTemplate? When creating a route and using endpoint DSL, it seems that Camel is creating a different uri for the endpoint. My EndpointRouteBuilder class:
#Component
public class MyRoutes extends EndpointRouteBuilder {
#Override
public void configure() throws Exception {
from(seda("STATUS_ENDPOINT"))
.routeId("stateChangeRoute")
.to(activemq("topic:statusTopic"))
}
}
and then injecting the endpoint to ProducerTemplate
#Component
public class StateChangePublisher {
#EndpointInject(value="seda:STATUS_ENDPOINT")
private ProducerTemplate producer;
public void publish(String str) {
try {
producer.sendBody(str);
} catch(CamelExecutionException e) {
e.printStackTrace();
}
}
}
When camel starts, I see two entries in the log:
o.a.camel.component.seda.SedaEndpoint : Endpoint seda:STATUS_ENDPOINT is using shared queue: seda:STATUS_ENDPOINT with size: 1000
o.a.camel.component.seda.SedaEndpoint : Endpoint seda://STATUS_ENDPOINT is using shared queue: seda://STATUS_ENDPOINT with size: 1000
The queue eventually fills up and nothing gets delivered to the "to" endpoint.
If I define the route without using the endpoint DSL method "seda()"
from("seda:STATUS_ENDPOINT")
then it works.
Is this a bug or am I doing something wrong?
I'm using camel 3.2.0 and
This was a bug in the endpoint dsl. Try upgrading to camel 3.3.0. I think it was fixed in the new release.
https://issues.apache.org/jira/browse/CAMEL-14859
The goal is to produce a report every hour by comparing two CSV files with
use of Camel 3.0.0. One is located on a FTP server, the other on disk. How to use poll enrich pattern in combination with unmarshalling the CSV on disk with Bindy Dataformat?
Example code (for simplicity the FTP endpoint is replaced by a file endpoint):
#Component
public class EnricherRoute extends RouteBuilder {
#Override
public void configure() {
from("file://data?fileName=part_1.csv&scheduler=quartz2&scheduler.cron=0+0+0/1+*+*+?")
.unmarshal().bindy(BindyType.Csv, Record.class)
.pollEnrich("file://data?fileName=part_2.csv", new ReportAggregationStrategy())
.marshal().bindy(BindyType.Csv, Record.class)
.to("file://reports?fileName=report_${date:now:yyyyMMdd}.csv");
}
}
The problem in this example is that in the ReportAggregationStrategy the resource (coming from data/part_2.csv, see below) is not unmarshalled. How to unmarshal data/part_2.csv as well?
public class ReportAggregationStrategy implements AggregationStrategy {
#Override
public Exchange aggregate(Exchange original, Exchange resource) {
final List<Record> originalRecords = original.getIn().getBody(List.class);
final List<Record> resourceRecords = resource.getIn().getBody(List.class); // Results in errors!
...
}
}
You can wrap enrichment with direct endpoint and do unmarshaling there.
from("file://data?fileName=part_1.csv&scheduler=quartz2&scheduler.cron=0+0+0/1+*+*+?")
.unmarshal().bindy(BindyType.Csv, Record.class)
.enrich("direct:enrich_record", new ReportAggregationStrategy())
.marshal().bindy(BindyType.Csv, Record.class)
.to("file://reports?fileName=report_${date:now:yyyyMMdd}.csv");
from("direct:enrich_record")
.pollEnrich("file://data?fileName=part_2.csv")
.unmarshal().bindy(BindyType.Csv, Record.class);
I am using camel-cdi and it is injecting the CamelContext, detecting all the routes in project.
But I want a CamelContext with a registry because I have some components that I use in camel routes like shown below.
SimpleRegistry registry = new SimpleRegistry();
registry.put("actionProcessor", actionProcessor);
registry.put("jpa", jpaComponent);
registry.put("jtaTransactionManager", platformTransactionManager);
CamelContext camelContext = new DefaultCamelContext(registry);
When I inject CamelContext the components like actionProcess, jpa are not recognized. when in my Route I have
.to("bean:actionProcessor?method=myMethod(${body})")
but my bean does not get executed.
I documentation I read use # before components name which are in registry but still it is not working.
Please suggest how can I achieve this using camel-cdi.
Did you already try with creating a CdiCamelContext (a subclass of DefaultCamelContext) ?
Otherwise, a more elegant would be to annotate your various classes, eg:
#Named("actionProcessor")
public class MyActionProcessor{
...
}
We have been using this for years without any problem
public class ContextFactory {
#Produces
#ApplicationScoped
#ContextName("Demo")
static final CamelContext createContext() {
CdiCamelContext context = new CdiCamelContext();
context.setStreamCaching(true);
context.disableJMX();
return context;
}
}
#ContextName("Demo")
public class MyRouteBuilder extends RouteBuilder {
from("...")
.to("bean:actionProcessor?method=myMethod")
}
#Named("actionProcessor")
public class MyActionProcessor{
public void myMethod(#Body String body) {}
}
Of course, in order to work, you need to activate the JEE bean discovery (=add a "beans.xml" file in META-INF or WEB-INF) !
I extended WebMvcConfigurationSupport to implement an api versioning scheme - i.e.
#Configuration
public class ApiVersionConfiguration extends WebMvcConfigurationSupport {
#Override
public RequestMappingHandlerMapping requestMappingHandlerMapping() {
return new ApiVersionRequestMappingHandlerMapping(readDateToVersionMap());
}}
This uses a custom handler mapping to version the api and works quite nicely.
However it also seems to disable the #EnableAutoConfiguration bean so that now static resources aren't served (as mentioned in this question Is it possible to extend WebMvcConfigurationSupport and use WebMvcAutoConfiguration?).
Ok, I thought, let's just add a resource handler to the class above - i.e.
#Configuration
public class ApiVersionConfiguration extends WebMvcConfigurationSupport {
#Override
public void addResourceHandlers(ResourceHandlerRegistry registry) {
registry.addResourceHandler("classpath:/public/").addResourceLocations("/");
}
#Override
public RequestMappingHandlerMapping requestMappingHandlerMapping() {
return new ApiVersionRequestMappingHandlerMapping(readDateToVersionMap());
}}
However.. this isn't working..? I get this error when I browse to /index.html:
No mapping found for HTTP request with URI [/index.html] in DispatcherServlet with name 'dispatcherServlet'
..If I disable this class then these resources are served just fine by #EnableAutoConfiguration magic.
I've been playing with various options to serve static content having extended the WebMvcConfigurationSupport and thus far no success.
Any ideas?
I was facing the same problem and came up with a solution that just works for me. If you just want to get the resources working without worrying of repetition you can do:
#Configuration
public class StaticResourcesConfig extends WebMvcAutoConfigurationAdapter {
}
and then
#Configuration
#EnableWebMvc
#Import(StaticResourcesConfig.class)
public class WebConfig extends WebMvcConfigurationSupport {
...
}
This successfully uses the Spring Boot defaults for serving static resources, as long as you don't map /** in your controllers.