I have java object with XML annotations
#XmlRootElement(name = "ROOT")
public class Root {
#XmlAttribute(name = "ATTR")
private long site;
#XmlElement(name = "LIST")
List<MyObject> transaction;
}
and I have route to folder
from("firstPoint")
.process(new RootToXMLConvertor())
.to(FTP_FOLDER)
.end();
I want to marshall my Root object (I have it in exchange.getIn().getBody()) to XML file and send to FTP_FOLDER.
You could use the JacksonXML Dataformat from Camel
JacksonXMLDataFormat formatPojo = new JacksonXMLDataFormat(Root.class);
from("firstPoint")
.marshall(formatPojo)
.to("file..")
.to(FTP_FOLDER);
For doing this you'll need camel-jacksonxml, camel-ftp and the camel-file components.
Related
I have just started working around the Apache Camel. I have a requirement to implement an FTP/FTPS/SFTP client, which would be used to fetch the files from the respective servers. I was looking into the possibility of using Apache Camel to do this but I am still confused after going through the examples and the tutorials.
The requirement is to fetch the files from the FTP/SFTP servers when the request is received from the scheduler.
Following is the route created using EndPoint-DSL
#Component
public class FtpReceiveRoute extends EndpointRouteBuilder {
#Override
public void configure() throws Exception {
from(
ftp("localhost:2001/home/admin")
.account("admin")
.password("admin12345")
.recursive(true)
)
.routeId("ftpReceive")
.log("From done!")
.to("log:ftp-log")
.log("To done!!");
}
}
I am trying to use the above route by invoking it when the request is made to fetch the file like below.
#Override
protected FtpResponse doMessage(String param, FtpRequest req) {
FtpResponse response = new FtpResponse ();
CamelContext ctx = new DefaultCamelContext();
ctx.addRoutes(##route); //FtpReceiveRoute, add the Routebuilder instance as EndpointRouteBuilder is acceptable.
ctx.start();
//Might need to induce sleep so that all the files are downloaded
ctx.stop();
return response;
}
The confusion is around how to invoke the Camel process with the route. I have used EndpointRouteBuilder to create the route because of the type-safe creation of the endpoint URI. I am not getting an option to add this route to the CamelContext as it expects the RouteBuilder instance which is not type-safe.
Further, the CamelContext is the engine and to invoke the route I would need to start and stop this engine. This I am not able to digest if I need to start and stop the engine to execute a route then I would need to induce some sleep in between so that all files are downloaded. Just to add there are more routes that I need to add with the implementation. Once the engine is started it would load and execute all the added routes which is not the requirement.
Maybe I am not getting how to use this properly. Any resources aiding my situation are welcome. Thanks.
You should not create and start new camel context every time you want to fetch file from server. What you should do instead is start one when your application starts and use that for all your exchanges.
You can use Spring-boot to initialize CamelContext and add annotated RouteBuilders to it automatically. Check the maven archetype camel-archetype-spring-boot for example.
If you want to call camel routes from Java you can Inject CamelContext to your bean and use it to create ProducerTemplate. This can be used to invoke Routes defined in the RouteBuilder.
Using ProducerTemplate.send you can get the resulting exchange.
Using producer template
Using File-component which works very similary to ftp-component.
package com.example;
import org.apache.camel.builder.endpoint.EndpointRouteBuilder;
import org.springframework.stereotype.Component;
#Component
public class MySpringBootRouter extends EndpointRouteBuilder {
#Override
public void configure() {
from(direct("fileFromFTP"))
.routeId("fileFromFTP")
// reads files from <project>/input using file consumer endpoint
.pollEnrich(file("input"), 1000)
// If file is found, convert body to string.
// Which in this case will read contents of the file to string.
.filter(body().isNotNull())
.convertBodyTo(String.class)
.end()
;
}
}
package com.example;
import org.apache.camel.CamelContext;
import org.apache.camel.Exchange;
import org.apache.camel.ProducerTemplate;
import org.apache.camel.support.DefaultExchange;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.annotation.EnableScheduling;
import org.springframework.scheduling.annotation.Scheduled;
import static org.apache.camel.builder.endpoint.StaticEndpointBuilders.direct;
#Configuration
#EnableScheduling
public class MySpringBean {
#Autowired
CamelContext camelContext;
#Scheduled(fixedRate = 1000)
public void scheduledTask() {
System.out.println("Scheduled Task!");
if(camelContext.isStopped()) {
System.out.println("Camel context not ready yet!");
return;
}
useProducerTemplate();
}
public void useProducerTemplate(){
ProducerTemplate producerTemplate = camelContext.createProducerTemplate();
Exchange inExchange = new DefaultExchange(camelContext);
//synchronous call!
Exchange result = producerTemplate.send(direct("fileFromFTP").toString(), inExchange);
String resultBody = result.getMessage().getBody(String.class);
String fileName = result.getMessage().getHeader(Exchange.FILE_NAME, String.class);
if(resultBody != null){
System.out.println("Consumed file: "+ fileName + " contents: " + resultBody.toString());
}
else{
System.out.println("No file to consume!");
}
}
}
Depending on what you need to do with the files you could probably do that inside camel route. Then you would only need to call the producerTemplate.sendBody.
public void useProducerTemplate(){
ProducerTemplate producerTemplate = camelContext.createProducerTemplate();
Exchange inExchange = new DefaultExchange(camelContext);
producerTemplate.sendBody(direct("fileFromFTP").toString(), inExchange);
}
Starting stopping camel route
If you want to start polling file consumer only for a short while you can do start the route and use for example aggregation timeout to shutdown the route when no new files have been received in any given duration.
#Component
public class MySpringBootRouter extends EndpointRouteBuilder {
#Override
public void configure() {
AggregationStrategy aggregateFileNamesStrategy = AggregationStrategies
.flexible(String.class)
.accumulateInCollection(ArrayList.class)
.pick(header(Exchange.FILE_NAME))
;
from(file("input"))
.routeId("moveFilesRoute")
.autoStartup(false)
.to(file("output"))
.to(seda("moveFilesRouteTimeout"));
;
from(seda("moveFilesRouteTimeout"))
.routeId("moveFilesRouteTimeout")
.aggregate(constant(true), aggregateFileNamesStrategy)
.completionTimeout(3000)
.log("Consumed files: ${body.toString()}")
.process(exchange -> {
exchange.getContext().getRouteController().stopRoute("moveFilesRoute");
})
.end()
;
}
}
public void startMoveFilesRoute() {
try {
System.out.println("Starting moveFilesRoute!");
camelContext.getRouteController().startRoute("moveFilesRoute");
//Sending null body moveFilesRouteTimeout to trigger timeout if there are no files to transfer
camelContext.createProducerTemplate().sendBody(seda("moveFilesRouteTimeout").toString(), null);
} catch(Exception e) {
System.out.println("failed to stop route. " + e);
}
}
I'd like to know if it is possible to use Project Reactor with Apache Camel, so applications be fully reactive and non-blocking IO. I'd like to know how does the Project Reactor support works when integrating other Apache Camel's components.
Can I read for example from S3 reactively (therefore I'll need to use the Async S3 client behind the scenes)? Or will I block when reading from S3 and then just create a Flux out of what has been returned?
Where reactiveness is needed, you should use the relevant spring and reactor libraries. there are pseudo camel code also u can db call in camel bean or processors etc
#RestController
#RequestMapping(value = "/api/books")
#RequiredArgsContructor
public class HomeController {
private final BookRepository bookRepository;
private final ProducerTemplate template
#GetMapping("")
public Flux<Book> getHome() {
List <Book> books=bookRepository.findAll();
X ret = template.requestBody("direct:something", books, X.class);
}
}
#Component
public class SomeRoute extends RouteBuilder {
#Override
public void configure() throws Exception {
from("direct:something")
.process(e-> {
List<Book> books = e.getIn.getBody(List.class)
// some logic
e.getIn.setBody(new X( ))
})
}
The goal is to produce a report every hour by comparing two CSV files with
use of Camel 3.0.0. One is located on a FTP server, the other on disk. How to use poll enrich pattern in combination with unmarshalling the CSV on disk with Bindy Dataformat?
Example code (for simplicity the FTP endpoint is replaced by a file endpoint):
#Component
public class EnricherRoute extends RouteBuilder {
#Override
public void configure() {
from("file://data?fileName=part_1.csv&scheduler=quartz2&scheduler.cron=0+0+0/1+*+*+?")
.unmarshal().bindy(BindyType.Csv, Record.class)
.pollEnrich("file://data?fileName=part_2.csv", new ReportAggregationStrategy())
.marshal().bindy(BindyType.Csv, Record.class)
.to("file://reports?fileName=report_${date:now:yyyyMMdd}.csv");
}
}
The problem in this example is that in the ReportAggregationStrategy the resource (coming from data/part_2.csv, see below) is not unmarshalled. How to unmarshal data/part_2.csv as well?
public class ReportAggregationStrategy implements AggregationStrategy {
#Override
public Exchange aggregate(Exchange original, Exchange resource) {
final List<Record> originalRecords = original.getIn().getBody(List.class);
final List<Record> resourceRecords = resource.getIn().getBody(List.class); // Results in errors!
...
}
}
You can wrap enrichment with direct endpoint and do unmarshaling there.
from("file://data?fileName=part_1.csv&scheduler=quartz2&scheduler.cron=0+0+0/1+*+*+?")
.unmarshal().bindy(BindyType.Csv, Record.class)
.enrich("direct:enrich_record", new ReportAggregationStrategy())
.marshal().bindy(BindyType.Csv, Record.class)
.to("file://reports?fileName=report_${date:now:yyyyMMdd}.csv");
from("direct:enrich_record")
.pollEnrich("file://data?fileName=part_2.csv")
.unmarshal().bindy(BindyType.Csv, Record.class);
I have a Camel Route which uses CDI to load a properties file from the JBoss configuration directory ...works perfect.
What I need to do is load one of the properties that are loaded in
an Arquillian integration test I am writing.
Example:
Content of Fiddler.properties file in the JBoss configuration directory
silly.value = Laughing
serious.value = politics
Example Producer class to load properties
/**
* Create the Camel properties component using CDI #Produces
*/
#Produces
#Named("properties")
PropertiesComponent propertiesComponent() {
final PropertiesComponent component = new PropertiesComponent();
// load JBoss properties file
component.setLocation(
"file:${jboss.server.config.dir}/fiddler.properties"
);
return component;
}
A given property from the Fiddler.properties file is now available in the main Camel route as {{silly.value}} or {{serious.value}}
Problem:
What I would like to do is load/reference one of these property values from my Arquillian Integration Test … probably in the #BeforeClass method …something like below:
#RunWith(Arquillian.class)
public class MainRouteIT {
.
.
Boolean allOK = false;
#BeforeClass
public static void setupTest() throws Exception {
allOK = new testCheck(
{{silly.value}}, {{serious.value}}
);
.
.
Any idea if something like this is possible in Camel within an Arquillian test ?
Here is the solution we are using (but without Arquillian):
First define a CDI alternative for the Camel "properties" component, which will use testing property values.
Then annotate your unit test in order to use the alternate producers of Camel components.
#Alternative
public class CamelAlternatives {
#Produces
#ApplicationScoped
#Named("properties")
public PropertiesComponent propertiesComponent() {
PropertiesComponent component = new PropertiesComponent();
component.setLocations( Arrays.asList("classpath:common.properties", "classpath:testing.properties") );
return component;
}
#RunWith(CamelCdiRunner.class)
#Beans(alternatives = {CamelAlternatives.class})
public class MyUnitTest {
...
}
i use the throttle in apache camel blueprint : throttle timePeriodMillis="1000"
with constant = 4
i want to put the value of timePeriodMillis dynamically using placeholder.Is this possible?
See the documentation: http://camel.apache.org/using-propertyplaceholder.html
And the section Using Property Placeholders for Any Kind of Attribute in the XML DSL
Properties component can be used here
ex:
Properties properties = new Properties();
InputStream input = null;
public void configure() throws IOException {
CamelContext context = new DefaultCamelContext();
input = new FileInputStream("sample.properties");
properties.load(input);
System.out.println("CITY IS" +properties.getProperty("wl.city"));
String city = properties.getProperty("wl.city");