I'm trying to use Apache Camel (version 2.20.0) with mybatis component.
More specifically, I have to export a large set or record from database to file.
I'd like to prevent memory issues so I want to use the option consumer.useIterator. My route is:
from("mybatis:selectItemsByDate?statementType=SelectList&consumer.useIterator=true")
.split()
.body()
.process(doSomething)
to(file:my-path-file);
but my query has in input a parameter (the starting date to get data). How should I set this parameter?
In many example on internet I saw the parameter in the body or in the header of the Exchange message but I think is possibile only if the mybatis endpoint is in a "to" method. But the option "consumer.useIterator" is working only when the enpdoint is in a "from" method.
Please help me to understand how I can set the input for my query or if this is not supported yet (in this case if you can give some hint how to implement would be great)
thank you.
Then you need to start your route from something else, like a timer or direct endpoint, and then call the mybatis endpoint in a to, where you have set that information in the message body/header you use in the mybatis query so its dynamic.
Also you should set the splitter to be in streaming mode so it walks the iterator it gets from mybatis on-demand.
Related
While monitoring an Apache Camel application with hawt.io, I noticed that the nodes of a camel route have some properties that I cannot influence with the Java DSL, but which are displayed in hawt.io. It would be pretty awesome if you could define them anyway.
I am particularly interested in the id and the description of a node in the route. My route currently looks something like this (screenshot below):
My route
rabbitmq://tso11
log4
process4
to3
process5
to4
The displayed names (log4, process4, process5, ...) are automatically generated "id" properties. There is also an always empty property "description".
It would be awesome if I could change this somehow for a much better readable route:
My route
rabbitmq://tso11
log incoming message
process for header extraction
to xslt processor additional-mappint.xslt
process for conversion to nms42 format
to nms42 endpoint
Maybe there is a way? Maybe only with XML based DSL?
Here is a screenshot:
In Java DSL, you can set the id of a node thanks to the method id(String id).
In the next example, the id of the endpoint mock:bar has been set to bar:
from("direct:start")
.to("mock:foo")
.to("mock:bar").id("bar")
.to("mock:result");
I have a camel route that splits and aggregates according to some ids. When an id is retrieved, a call is made to another endpoint to retrieve the project information according to this id. After retrieving the project information i had to enrich it by calling multiple enrich() methods on it. In the first enrich method i have to do some xpath processing wherein ill be able to retrieve a primaryOrgId value that i will set as a property in the exchange, dont worry about the xpath processing, i had that sorted out but my problem is when I set the property (primaryOrgId) inside the 1st enrich. The property value doesn't get persisted when the route goes to the 2nd enrich part. When I log the primaryOrgId value, the original value of "testValue" (this was set in the direct:createSomeIds route) is the one getting displayed instead of "changeTheValueHere" which was set in the 1st enrich part.
I am using Camel 2.15 based on Fuse 6.2.1.
I went to the camel site and read this part from http://camel.apache.org/content-enricher.html . I'm not sure I understood how to implement... "For that you must set the filename in the
endpoint URI" .. this text was talking about the header, i'm thinking its also applicable to the properties in the exchange.
pollEnrich or enrich does not access any data from the current
Exchange which means when polling it cannot use any of the existing
headers you may have set on the Exchange. For example you cannot set a
filename in the Exchange.FILE_NAME header and use pollEnrich to
consume only that file. For that you must set the filename in the
endpoint URI.
Here is my code:
from("direct:createSomeIds")
.routeId("createSomeIds")
.process(new IdCreatorProcessor()
.setProperty("primaryOrgId").constant("testValue")
.split(xpath("/TempProjects/TempProject/Code/text()").namespaces(ns) , new IdStrategy())
.to("direct:splitRouteById")
.end();
from("direct:splitRouteById")
.routeId("splitRouteById")
.to("direct:getProjectByID")
.to("xquery:template/AllProjectToSingleProject.xq") //xquery template
.convertBodyTo(Project.class)
.enrich("direct:getAdditionalInfo", new ProjectStrategy(ProjectStrategy.AggregatorType.AdditionalInfo))
.enrich("direct:getSecondaryInfo", new ProjectStrategy(ProjectStrategy.AggregatorType.SecondaryInfo))
.end();
from("direct:getAdditionalInfo")
//some xpath stuff here
.setProperty("primaryOrgId").constant("changeTheValueHere")
.end();
from("direct:getSecondaryInfo")
.log("Value of primaryOrgId = " + "${exchangeProperty.primaryOrgId}")
.end();
If you can provide some code example, that would be helpful.
If you read a bit further down you will see that it's recommended that you instead use RecipientList with an AggregationStrategy.
.recipientList("direct:getAdditionalInfo", "direct:getSecondaryInfo")
.aggregationStrategy(new ProjectStrategy())
The setting of filename in your endpoint URI would only be applicable if you were to access some file on an FTP or some other file area.
Edit:
I now see that you need the property from the first enrichment in your second enrichment. However, if you're not modifying the message body in the first enrich then I don't actually see the need for it at all.
If you are in fact modifying the body then you can still use the RecipientList but instead you use two separate ones calling only one endpoint in each.
Looking through Camel docs I couldn't find any way that allow me to bind the query parameters within the headers. For example :
Let's say I have an endpoint like that
http://localhost:8080/services/resource?filter=xxx
End I want to get that parameter from the header
exchange.getIn.().getHeaders().get('filter')
The query parameter 'filter' is not returned in the header. Anyone of you knows if this feature is coming by default in camel? I know I can build the binding by myself, but i am just looking for choose among camel-servlet (apparently that binding is implemented by default) and camel-restlet.
If you choose camel-restlet you can use the
restletBinding=#refName
to convert the query string parameters to headers.
The #refName is the bean ID of a RestletBinding object in the Camel Registry.
I could not find an answer to this. I found the previous similar question unanswered. I'd like to use Spring data solr for queries. But #Query is insufficient for my needs. As I understood, whatever you give here becomes a q parameter to `select' handler of solr.
In my case I need to add more parameters for example sfield for a spatial search. If #Query wont cut it, I am ready to write a custom repository implementation by autowiring SolrTemplate, But then the Criteria API does not seem to let me add a raw query parameter either.
Any help/points will be greatly appreciated.
I worked around it by creating a QueryParser decorator that adds the required parameters to a parsed solr query. The QueryParser was registered using solrTemplate.registerQueryParser().
Note however that I had to do a really nasty hack to get this working, since all queries that are sent to solrTemplate.queryForPage are wrapped by a static package protected inner class in QueryBase. So my registration code above had to be in a package org.springframework.data.solr.core
The toQueryString method inside Solrj's ClientUtils class is called when the http request is formed internally. But in this process, it also encodes the commas (,) that need to be sent in the facet.pivot field.
eg . facet.pivot=A1,A2 gets sent as facet.pivot=A1%2CA2
Because of this the query returns no result.
Please suggest a mechanism to report this or any work around for the same.
Your question is about escaping/encoding the query for a solr request.
In current version of solr the toQueryString method is moved to SolrParams. But never the less "%2C" ist correct for "," in utf-8.
So most likely you have a problem on server side with unencoding the params.
Try solr in current version, because in this case you don't need to config the servlet container properly: it is now part of solr.
btw: take a look to sub-facets instead of pivot-faceting: http://yonik.com/solr-subfacets/