SimpleScheduledRoutePolicy does not work on the specific time - apache-camel

I'm developing a web application where user adds issue specifying the date and time on which he/she should get a notification mail. I'm new to apache camel and quartz scheduler.
I have written a sample code as below. IssueDTO is nothing but a POJO. If the issue is repetitive, I have configured a cron scheduler which works properly i.e. if I specify frequency as 5, I get the expected output which is a println statement to the console. But if the issue is not repetitive, I have used SimpleScheduledRoutePolicy and hardcoded the date and time at which process() method of the Processor should run. I simply change the date time to 2 min later of the current system time to check whether code is working. But it never enters the process method and does print this statement => System.out.println("*****************" + issueDTO.getIssueId() + " running at " + gc.getTime().toString());
#Override
public void configure() throws Exception
{
System.out.println("in ReminderRouteBuilder configure()");
System.out.println("Issue ID : " + issueDTO.getIssueId());
System.out.println("Issue Frequency : " + issueDTO.getFrequency());
System.out.println("Is Repetative : " + issueDTO.getIsRepetitive());
// if Repetitive
if (StringUtil.getBoolean(issueDTO.getIsRepetitive()))
{
String fromString = "quartz2://" + issueDTO.getIssueId() + "?cron=0/" + issueDTO.getFrequency() + "+*+*+*+*+?";
from(fromString).process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception
{
System.out.println(issueDTO.getIssueId() + " running every " + issueDTO.getFrequency() + " sec...");
}
});
}
// if not Repetitive
else
{
SimpleScheduledRoutePolicy policy = new SimpleScheduledRoutePolicy();
GregorianCalendar gc = new GregorianCalendar(2019, Calendar.AUGUST, 31, 13, 45);
policy.setRouteStartDate(gc.getTime());
from("direct:start").routeId(issueDTO.getIssueId()).routePolicy(policy).process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception
{
System.out.println("*****************" + issueDTO.getIssueId() + " running at " + gc.getTime().toString());
}
});
}
}
Am I missing something?

Direct endpoint needs to be triggered manualy with some event. If you need something, what is triggered automatically after start of route, you can use Timer endpoint with repeatCount=1 or Quartz endpoint with fireNow=true.
E.g. this will trigger Exchange only once, after route startup:
from("timer:start?repeatCount=1").routeId(issueDTO.getIssueId()).routePolicy(policy)

Ok.. I got the solution :).
I used the cron expression specifying exact date and time and it worked.
#Override
public void configure() throws Exception
{
System.out.println("in ReminderRouteBuilder configure()");
System.out.println("Issue ID : " + issueDTO.getIssueId());
System.out.println("Issue Frequency : " + issueDTO.getFrequency());
System.out.println("Is Repetative : " + issueDTO.getIsRepetitive());
// if Repetitive
if (StringUtil.getBoolean(issueDTO.getIsRepetitive()))
{
String fromString = "quartz2://" + issueDTO.getIssueId() + "?cron=0/" + issueDTO.getFrequency() + "+*+*+*+*+?";
from(fromString).process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception
{
System.out.println(issueDTO.getIssueId() + " running every " + issueDTO.getFrequency() + " sec...");
}
});
}
// if not Repetitive
else
{
String fromString = "quartz2://" + issueDTO.getIssueId() + "?cron=0 40 12 4 SEP ? 2019";
from(fromString).process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception
{
System.out.println(issueDTO.getIssueId() + " running now");
}
});
}
}

Related

Spring boot: Database connection not closing properly

I'm executing queries periodically (by a scheduler) using my Spring Boot application
application.properties
src_mssqlserver_url=jdbc:sqlserver://192.168.0.1;databaseName=Test;
src_mssqlserver_username=tester
src_mssqlserver_password=tester1
src_mssqlserver_driverClassName=com.microsoft.sqlserver.jdbc.SQLServerDriver
Datasource and JdbcTemplate Bean
#Primary
#Bean(name = "src_mssqlserver")
#ConfigurationProperties(prefix = "spring.ds_mssqlserver")
public DataSource srcDataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName(env.getProperty("src_mssqlserver_driverClassName"));
dataSource.setUrl(env.getProperty("src_mssqlserver_url"));
dataSource.setUsername(env.getProperty("src_mssqlserver_username"));
dataSource.setPassword(env.getProperty("src_mssqlserver_password"));
return dataSource;
}
#Bean(name = "srcJdbcTemplate")
public JdbcTemplate srcJdbcTemplate(#Qualifier("src_mssqlserver") DataSource dsSrcSqlServer) {
return new JdbcTemplate(dsSrcSqlServer);
}
Usage: This method is called from a scheduler with list of items to process (normally 1000 records), this process runs in an hour once.
#Autowired
#Qualifier("srcJdbcTemplate")
private JdbcTemplate srcJdbcTemplate;
public void batchInsertUsers(final List<User> users) {
String queryInsert = "INSERT INTO [User] ([Name]"
+ " , [Created_Date]"
+ " , [Notes])"
+ " VALUES (?, SYSDATETIMEOFFSET(), ?)";
srcJdbcTemplate.batchUpdate(queryInsert, new BatchPreparedStatementSetter() {
#Override
public void setValues(PreparedStatement ps, int i) throws SQLException {
User user = users.get(i);
ps.setString(1, user.getName());
ps.setString(2, user.getNotes());
}
#Override
public int getBatchSize() {
return sites.size();
}
});
I'm getting warnings from database administrator that my code keeping too much connections open. Please share some standard and workable way to handle such situation.
Thanks.
DriverManagerDataSource is NOT meant for production, it opens and closes a connection each time it needs one.
Use a connection pool like c3p0DataSource.

Apache Camel: route using custom processor, splitter and aggregator doesn't output anything

I'm cutting my teeth on Camel using the following use case:
Given a GitHub username, I want to fetch a certain number of public
repos in descending order of activity, then for each repo I want to
fetch a certain number of commits, and finally, for each commit, I
want to print some information.
To achieve this, I wrote a Producer and the following route. The Producer works (I've tests), and so does the route without the aggregator. When using the aggregator, nothing comes out (my tests fail).
public void configure() throws Exception {
from("direct:start")
.id("gitHubRoute")
.filter(and(
isNotNull(simple("${header." + ENDPOINT + "}")),
isNotNull(simple("${body}")))
)
.setHeader(USERNAME, simple("${body}"))
.toD("github:repos?username=${body}")
.process(e -> {
// some processing
})
.split(body())
.parallelProcessing()
.setHeader(REPO, simple("${body.name}"))
.toD("github:commits" +
"?repo=${body.name}" +
"&username=${header." + USERNAME + "}"
)
.process(e -> {
// some processing
})
.split(body())
.toD("github:commit" +
"?repo=${header." + REPO + "}" +
"&username=${header." + USERNAME + "}" +
"&sha=${body.sha}"
)
.process(e -> {
// some processing
})
.aggregate(header(REPO), new GroupedExchangeAggregationStrategy()).completionTimeout(10000l)
.toD("${header." + ENDPOINT + "}");
from("direct:end")
.process().exchange(this::print);
}
During testing, I set the header ENDPOINT to mock:result. In reality, it's set to direct:end.
What am I doing wrong? There are no errors but the print method, or the mock during testing, is never invoked.
I solved it myself. Couple of things that I'd to change:
Completion check: I used a completionPredicate as shown below.
eagerCheckCompletion(): Without this, the exchange passed into the completionPredicate is the aggregated exchange, not the incoming exchange.
I also took the opportunity to do little refactoring to improve readability.
public void configure() throws Exception {
from("direct:start")
.id("usersRoute")
.filter(isNotNull(simple("${header." + ENDPOINT + "}")))
.setHeader(USERNAME, simple("${body}"))
.toD("github:users/${body}/repos")
.process(e -> this.<GitHub.Repository>limitList(e))
.to("direct:reposRoute1");
from("direct:reposRoute1")
.id("reposRoute1")
.split(body())
.parallelProcessing()
.setHeader(REPO, simple("${body.name}"))
.toD("github:repos/${header." + USERNAME + "}" + "/${body.name}/commits")
.process(e -> this.<GitHub.Commit>limitList(e))
.to("direct:reposRoute2");
from("direct:reposRoute2")
.id("reposRoute2")
.split(body())
.toD("github:repos/${header." + USERNAME + "}" + "/${header." + REPO + "}" + "/commits/${body.sha}")
.process(e -> {
GitHub.Commit commit = e.getIn().getBody(GitHub.Commit.class);
List<GitHub.Commit.File> files = commit.getFiles();
if (!CollectionUtils.isEmpty(files) && files.size() > LIMIT) {
commit.setFiles(files.subList(0, LIMIT));
e.getIn().setBody(commit);
}
})
// http://camel.apache.org/aggregator2.html
.aggregate(header(REPO), new AggregateByRepoStrategy())
.forceCompletionOnStop()
.eagerCheckCompletion()
.completionPredicate(header("CamelSplitComplete").convertTo(Boolean.class).isEqualTo(TRUE))
.toD("${header." + ENDPOINT + "}");
from("direct:end")
.process().exchange(this::print);
}
The AggregationStrategy I used as follows:
private static final class AggregateByRepoStrategy extends AbstractListAggregationStrategy<GitHub.Commit> {
#Override
public GitHub.Commit getValue(Exchange exchange) {
return exchange.getIn().getBody(GitHub.Commit.class);
}
}

Read file locations from table and copy to specific folder using pollEnrich()

I am trying to write a camel route that reads a database table to get the list of absolute file paths and then copy those files to another folder. However only the file path is created as content instead of the original content.
from("timer://testDataGen?repeatCount=1")
.to("sql:" + positionSql + "?dataSource=dataSource")
.split(body())
.to("file://" + positionlistDir )
.log("Finished copying the list of Files.")
Please let me know what am i missing here to convert an absolute file path to an actual file.
Update #1.
Below snippet is invoking the pollEnrich(). But instead the pollEnrich() is copying the no of files which is equal to the no of rows returned by the sql and not according to the file name from the previous exchange.
String positionListSqlOptions = "?dataSource=dataSource";
// String positionSrcDirOptions = "?noop=true&delay=500&readLockMarkerFile=false&fileName=${header.positionFileToBeCopied}";
String positionSrcDirOptions = "?noop=true&delay=500&readLockMarkerFile=false&fileName=${body}";
String positionStagingDirOptionsForWriting = "?doneFileName=${file:name}.DONE";
from("timer://testDataGen?repeatCount=1")
.to("sql:" + positionListSql + positionListSqlOptions)
.split(body())
\\ Getting the column value from the resultset which is a LinkedCaseInsensitiveMap and storing in the body
.process(new positionFeederProcessor())
.setHeader("positionFileToBeCopied", body())
.pollEnrich("file://" + positionSrcDir + positionSrcDirOptions)
// .pollEnrich().simple("file://" + positionSrcDir + positionSrcDirOptions)
.to("file://" + positionStagingDir + positionStagingDirOptionsForWriting)
.log("Finished copying the list of Files.");
I am still unable to get the actual file name passed to the pollingEnrich() endpoint. I tried extracting it from body as well as through a header too. What could have gone wrong.
Well, Finally I was able to do this without using pollEnrich() at all.
String positionListSqlOptions = "?dataSource=dataSource";
String positionSrcDirOptions = "?noop=true&delay=500&readLockMarkerFile=false&fileName=${header.CamelFileName}";
String positionStagingDirOptionsForWriting = "?fileName=${header.position.file.name}&doneFileName=${file:name}.DONE";
from("timer://testDataGen?repeatCount=1")
.to("sql:" + positionListSql + positionListSqlOptions)
.routeId("Copier:")
.setHeader("positionFileList", body())
.log("Creating the list of position Files ...")
.split(body())
.process(new PositionListProcessor())
.setHeader("position.file.name", body())
.setHeader("position.dir.name", constant(positionSrcDir))
.process(new PositionFileProcessor())
.choice()
.when(body().isNull())
.log("Position File not found. ${header.position.file.name}")
.otherwise()
.to("file://" + positionStagingDir + positionStagingDirOptionsForWriting)
.log("Position File Copied from Src to : " + "${header.CamelFileNameProduced} ... ${headers} ...");
And here are the processors.
public class PositionListProcessor implements Processor {
public void process(Exchange exchange) throws Exception {
LinkedCaseInsensitiveMap positionFilesResultSet = (LinkedCaseInsensitiveMap) exchange.getIn().getBody();
try {
String positionFileStr = positionFilesResultSet.get("PF_LOCATION_NEW").toString();
}
exchange.getOut().setBody(positionFileStr.trim());
} catch (Exception e) { }
} }
public class PositionFileProcessor implements Processor {
public void process(Exchange exchange) throws Exception {
String filename = exchange.getIn().getBody(String.class);
String filePath = exchange.getIn().getHeader("position.dir.name", String.class);
URI uri = new URI("file:///".concat(filePath.concat(filename)));
File file = new File(uri);
if (!file.exists()) {
logger.debug((String.format("File %s not found on %s", filename, filePath)));
exchange.getIn().setBody(null);
} else {
exchange.getIn().setBody(file);
}
} }
the file component, when used on a to definition, produce a file with the content of the exchange, it doesn't read a file. you can use for example a pollEnrich processor :
from("timer://testDataGen?repeatCount=1")
.to("sql:" + positionSql + "?dataSource=dataSource")
.split(body())
.pollEnrich().simple("file:folder?fileName=${body}")
.to("file://" + positionlistDir )
.log("Finished copying the list of Files.")

SimpleBuilder usage in camel

I have following processor, when I run it from my route I get the following error. I know exchange body is not null and you can see it in logs below. What is wrong with my usage of SimpleBuilder here ?
public class UpdateCustomerProcessor implements Processor {
public static final Logger log = LoggerFactory.getLogger(UpdateCustomerProcessor.class);
public void process(Exchange exchng) throws Exception {
Customer c = (Customer) exchng.getIn().getBody(Object[].class)[0];
System.out.println("Updating customer " + c.getFirstName() + " " + c.getLastName());
System.out.println(SimpleBuilder.simple("Hello ${body.getFirstName()}").evaluate(exchng, String.class));
exchng.getOut().setBody(new Object[] {});
}
}
log->Updating customer aaa bbb
error-> org.apache.cxf.interceptor.Fault: Failed to invoke method: .getFirstName() on null due to: org.apache.camel.language.bean.RuntimeBeanExpressionException: Failed to invoke method: getFirstName() on null
I can't make sense of the "null"-exception, since exchange in appears to be filled. Nevertheless, your expression looks incorrect - since your exchange.in seems to hold an array, it should be:
SimpleBuilder.simple("Hello ${body[0].firstname}").evaluate(exchng, String.class))

Setting a header based on an XQuery filter

I have a route that's set to run in batched mode, polling several thousand XML files. Each is timestamped inside the XML structure and this dateTime element is used to determine whether the XML should be included in the batch's further processing (an XQuery transform). As this is a batch route it self-terminates after execution.
Because the route needs to close itself I have to ensure that it also closes if every message is filtered out, which is why I don't use a filter but a .choice() statement instead and set a custom header on the exchange which is later used in a bean that groups matches and prepares a single source document for the XQuery.
However, my current approach requires a second route that both branches of the .choice() forward to. This is necessary because I can't seem to force both paths to simply continue. So my question is: how can get rid of this second route? One approach is setting the filter header in a bean instead but I'm worried about the overhead involved. I assume the XQuery filter inside Camel would greatly outperform a POJO that builds an XML document from a string and runs an XQuery against it.
from(sourcePath + "?noop=true" + "&include=.*.xml")
.choice()
.when()
.xquery("[XQuery Filter]")
.setHeader("Filtered", constant(false))
.to("direct:continue")
.otherwise()
.setHeader("Filtered", constant(true))
.to("direct:continue")
.end();
from("direct:continue")
.routeId(forwarderRouteID)
.aggregate(aggregationExpression)
.completionFromBatchConsumer()
.completionTimeout(DEF_COMPLETION_TIMEOUT)
.groupExchanges()
.bean(new FastQueryMerger(), "group")
.to("xquery:" + xqueryPath)
.bean(new FileModifier(interval), "setFileName")
.to(targetPath)
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
new RouteTerminator(routeID, exchange.getContext()).start();
new RouteTerminator(forwarderRouteID, exchange.getContext()).start();
}
})
.end();
Wouldn't .end() help here?
I mean the following:
from(sourcePath + "?noop=true" + "&include=.*.xml")
.choice()
.when()
.xquery("[XQuery Filter]")
.setHeader("Filtered", constant(false)).end()
.otherwise()
.setHeader("Filtered", constant(true)).end()
.aggregate(aggregationExpression)
.completionFromBatchConsumer()
.completionTimeout(DEF_COMPLETION_TIMEOUT)
.groupExchanges()
.bean(new FastQueryMerger(), "group")
.to("xquery:" + xqueryPath)
.bean(new FileModifier(interval), "setFileName")
.to(targetPath)
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
new RouteTerminator(routeID, exchange.getContext()).start();
new RouteTerminator(forwarderRouteID, exchange.getContext()).start();
}
});
just quickly tested the following one and it worked:
#Produce(uri = "direct:test")
protected ProducerTemplate testProducer;
#EndpointInject(uri = "mock:test-first")
protected MockEndpoint testFirst;
#EndpointInject(uri = "mock:test-therest")
protected MockEndpoint testTheRest;
#EndpointInject(uri = "mock:test-check")
protected MockEndpoint testCheck;
#Test
public void test() {
final String first = "first";
final String second = "second";
testFirst.setExpectedMessageCount(1);
testTheRest.setExpectedMessageCount(1);
testCheck.setExpectedMessageCount(2);
testProducer.sendBody(first);
testProducer.sendBody(second);
try {
testFirst.assertIsSatisfied();
testTheRest.assertIsSatisfied();
testCheck.assertIsSatisfied();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
#Override
protected RouteBuilder createRouteBuilder() {
return new RouteBuilder() {
public void configure() {
from("direct:test")
.choice()
.when(body().isEqualTo("first")).to("mock:test-first")
.otherwise().to("mock:test-therest").end()
.to("mock:test-check");
}
};
}

Resources