I serialize the files to a byte arrays and send it to the ActiveMQ queue (I using the code from this example: Apache ActiveMQ File Transfer Example):
private void sendFileAsBytesMessage(File file) throws JMSException, IOException {
BytesMessage bytesMessage = session.createBytesMessage();
// Set file name here
bytesMessage.setStringProperty("fileName", file.getName());
// Set file body here
bytesMessage.writeBytes(fileManager.readfileAsBytes(file));
MessageProduce msgProducer =
session.createProducer(session.createQueue(queueName));
MessageProducer msgProducer.send(bytesMessage);
}
Method readfileAsBytes is presented below:
public byte[] readfileAsBytes(File file) throws IOException {
try (RandomAccessFile accessFile = new RandomAccessFile(file, "r")) {
byte[] bytes = new byte[(int) accessFile.length()];
accessFile.readFully(bytes);
return bytes;
}
}
In my OSGi bundle I have a custom processor:
public class Deserializer implements Processor {
#Override
public void process(Exchange exchange) throws Exception {
// Get file body here
byte[] bytes = exchange.getIn().getBody(byte[].class);
}
}
I use it in my Spring DSL route as follows:
<?xml version="1.0"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd
http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd">
<bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="brokerURL" value="tcp://localhost:61616" />
<property name="userName" value="admin" />
<property name="password" value="admin" />
</bean>
<bean id="deserializer" class="org.fusesource.example.Deserializer"/>
<camelContext id="blueprintContext" trace="false" xmlns="http://camel.apache.org/schema/blueprint">
<route id="testRoute">
<from uri="activemq:source-queue"></from>
<process ref="deserializer"/>
<to uri="activemq:sink-queue"></to>
</route>
</camelContext>
</blueprint>
I need to get the file name in my processor. How can I do that?
The problem can be solved as follows. For example, I send the README.html file to the alfresco-queue. Then the body of the file and the file name, as well as some additional information can be obtained as follows:
public class MyRouteBuilder extends RouteBuilder {
#Override
public void configure() throws Exception {
from("activemq:alfresco-queue?username=admin&password=admin")
.process(new Processor() {
public void process(Exchange exchange) throws Exception {
// Get file body
byte[] bytes = exchange.getIn().getBody(byte[].class);
for(int i = 0; i < bytes.length; i++) {
System.out.print((char) bytes[i]);
}
// Get headers
Map<String, Object> headers = exchange.getIn().getHeaders();
Iterator iterator = headers.entrySet().iterator();
while (iterator.hasNext()) {
Map.Entry pair = (Map.Entry)iterator.next();
System.out.println(pair.getKey() + " == " + pair.getValue());
}
}
})
// SKIPPED
It gives the following output:
<head>
root<> title>README
</title>
</head>
<body>
Please refer to http://java.com/licensereadme
</body>
</html>
breadcrumbId == ID:63-DP-TAV-59000-1531813754416-1:1:1:1:1
fileName == README.html
JMSCorrelationID == null
JMSCorrelationIDAsBytes == null
JMSDeliveryMode == 2
JMSDestination == queue://alfresco-queue
JMSExpiration == 0
JMSMessageID == ID:63-DP-TAV-59000-1531813754416-1:1:1:1:1
JMSPriority == 4
JMSRedelivered == false
JMSReplyTo == null
JMSTimestamp == 1531813754610
JMSType == null
JMSXGroupID == null
JMSXUserID == null
Related
Available examples of the usage of the Camel Test component show how to test the expectations of a route:
However what I need to do is mock the body (manually setting it) of an intermediate route, e.g.:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd">
<bean id="exampleBean" class="xxx.ExampleBean"/>
<routeContext id="routesTest" xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="direct:route1" />
<to uri="direct:route2" />
<log message="${body}"/>
</route>
<route>
<from uri="direct:route2"/>
<to uri="bean:exampleBean"/>
<to uri="direct:route3" />
</route>
<route>
<from uri="direct:route3"/>
<log message="${body}"/>
</route>
</routeContext>
</beans>
In this scenario I want to completely avoid the actual execution of bean:exampleBean, mocking the result of its execution.
My test class:
public class MyTests extends CamelSpringTestSupport {
#Produce(uri = "direct:route1")
protected ProducerTemplate inputProducerTemplate;
#EndpointInject(uri = "mock:bean:exampleBean")
protected MockEndpoint mockBeanExampleBean;
#Test
public void testRoute() throws Exception {
CompletableFuture<Object> future = inputProducerTemplate.asyncSendBody("direct:route1", "Some message");
Object o = future.get();
}
#Override
public String isMockEndpoints() {
return "bean:exampleBean";
}
#Override
protected AbstractApplicationContext createApplicationContext() {
return new ClassPathXmlApplicationContext("spring/gesti-test-application-context.xml");
}
}
public class ExampleBean {
public String enhance(String message) {
System.out.println(message);
//Here I would call a REST API
return "MY API RESULT";
}
}
When using mockBeanExampleBean.whenAnyExchangeReceived(exchange -> exchange.getMessage().setBody("My message")); it allows to override the input to exampleBean, but doesn't avoid its execution.
In the context of your unit test, route2 might be a "mock" component instead. A clean way of achieving that is to declare the route(s) in the properties file. The legibility of the routes gets harder, though.
Then, you could:
#EndpointInject("mock://route2")
MockEndpoint mockSecondStep;
mockSecondStep.whenExchangeReceived(1, e -> {
List whatever = new ArrayList<>();
e.getMessage().setBody(whatever);
});
I solved it using an InterceptStrategy:
public class MyTests extends CamelSpringTestSupport {
#Test
public void testRoute() throws Exception {
CompletableFuture<Object> future = template.asyncSendBody("direct:route1", "Some message");
Object o = future.get();
assertEquals("INTERCEPTED!", o);
}
#Override
protected AbstractApplicationContext createApplicationContext() {
return new ClassPathXmlApplicationContext("spring/gesti-test-application-context.xml");
}
#Override
protected RouteBuilder createRouteBuilder() {
return new RouteBuilder() {
#Override
public void configure() {
context.getProcessorDefinition("bean:exampleBean").addInterceptStrategy(
(context, definition, target, nextTarget) -> exchange -> exchange.getOut().setBody("INTERCEPTED!"));
}
};
}
}
public class ExampleBean {
public String enhance(String message) {
System.out.println(message);
//Here I would call a REST API
return "MY API RESULT";
}
}
I need to implement db connection and query in process step.
So, I defined datasource in bean property.
And I tried to use jdbctemplate.
But result is returned with java.lang.NullPointException.
Do camel ignore autowired annotation in process statement?
If there is another solution, let me know it please.
Thank you.
CamelContext.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:camel="http://camel.apache.org/schema/spring"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
...
http://camel.apache.org/schema/spring/camel-spring.xsd">
<bean class="com.ktds.openmzn.common.bean.ProcFormat" id="procFormat"/>
<bean class="com.ktds.openmzn.common.bean.ProcessDistributor" id="splitChannel"/>
<bean class="org.springframework.jdbc.datasource.DriverManagerDataSource" id="dataSource">
<property name="driverClassName" value="${spring.datasource.driver-class-name}"/>
<property name="url" value="${spring.datasource.url}"/>
<property name="username" value="${spring.datasource.username}"/>
<property name="password" value="${spring.datasource.password}"/>
</bean>
<bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
<property name="dataSource" ref="dataSource" />
</bean>
<bean class="com.ktds.openmzn.common.bean.FilePathProcessor" id="filePathProcessor"/>
...
<camelContext id="camelContext-f611cb6c-d516-4346-9adc-5512d327a88d"
trace="true" xmlns="http://camel.apache.org/schema/spring">
<camel:route id="fixed_processor">
<camel:from id="_from1" uri="timer:fromPollTimer?period=20000"/>
<camel:process id="_sourceDirectory" ref="filePathProcessor"/>
...
FilePathProcessor.java
public class FilePathProcessor implements Processor {
...
#Override
public void process(Exchange exchange) throws Exception {
List<Map<String, Object>> rows = SetFilePath.getInstance().getPathList("aaaa");
SetFilePath.java
#ManagedResource
public class SetFilePath {
private static SetFilePath instance = null;
private String sourceDirectory;
private String targetDirectory;
#Autowired
private JdbcTemplate jdbcTemplate;
public static SetFilePath getInstance() {
if(instance == null) {
instance = new SetFilePath();
}
return instance;
}
Result
Message History
---------------------------------------------------------------------------------------------------------------------------------------
RouteId ProcessorId Processor
Elapsed (ms)
[fixed_processor ] [fixed_processor ] [timer://fromPollTimer?period=20000 ] [ 0]
[fixed_processor ] [_sourceDirectory ] [ref:filePathProcessor ] [ 0]
Stacktrace
---------------------------------------------------------------------------------------------------------------------------------------
java.lang.NullPointerException: null
at com.ktds.openmzn.common.bean.FilePathProcessor.process(FilePathProcessor.java:20) ~[classes/:na]
at org.apache.camel.processor.DelegateSyncProcessor.process(DelegateSyncProcessor.java:63) ~[camel-core-2.23.1.jar:2.23.1]
Well of course its null, because you are creating the instance yourself via the new constructor:
instance = new SetFilePath();
The #Autowired is from spring framework, and you would essentially need to use spring to create this bean for you.
There are different ways of doing this such as creating a <bean> in the XML file and then configure that on your processor via a setter <property>.
You can also let spring/camel create the bean instance instead of the new constructor, but this requires a bit of Camel API to do so
public static SetFilePath getInstance(CamelContext camel)
if (instance == null) {
instance = camel.getInjector().newInstance(SetFilePath.class);
}
Which will then via the injector create the bean instance via spring framework that does its auto-wiring.
I have a simple Apache Camel route in JBoss FUSE:
<?xml version="1.0"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd
http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd">
<bean id="startPolicy" class="org.apache.camel.routepolicy.quartz.CronScheduledRoutePolicy">
<property name="routeStartTime" value="*/3 * * * * ?"/>
</bean>
<bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="brokerURL" value="tcp://localhost:61616" />
<property name="userName" value="admin" />
<property name="password" value="admin" />
</bean>
<camelContext id="blueprintContext" trace="false" xmlns="http://camel.apache.org/schema/blueprint">
<route id="testRoute" routePolicyRef="startPolicy" autoStartup="false">
<from uri="activemq:source-queue?username=admin&password=admin"></from>
<log message="${body}" loggingLevel="INFO"></log>
<to uri="activemq:sink-queue?username=admin&password=admin"></to>
</route>
</camelContext>
</blueprint>
I can connect to the ActiveMQ broker and send a message to the queue, by using this standalone client:
public class MessageSender {
public static void main(String[] args) throws Exception {
ActiveMQConnectionFactory factory =
new ActiveMQConnectionFactory("tcp://localhost:61616");
factory.setUserName("admin");
factory.setPassword("admin");
Connection connection = factory.createConnection();
try {
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
Queue queue = session.createQueue("source-queue");
MessageProducer producer = session.createProducer(queue);
Message message = session.createTextMessage("some message to queue...");
producer.send(message);
} finally {
connection.close();
}
}
}
From the logs I see, that messages is consumed from the queue and message bodies are displays in the log:
How to send a file to the ActiveMQ Queue? For example, I have a simple form with <input type="file"> encoded in multipart/form-data. By using this form I need to send a payload of POST request to the ActiveMQ Queue.
How can I do that?
I would be very grateful for the information.
Thanks to all.
#Mary Zheng provided an excellent example, how it may be done:
ActiveMQ File Transfer Example
Method of class QueueMessageProducer, that sends the file message to ActiveMQ Broker:
private void sendFileAsBytesMessage(File file) throws JMSException, IOException {
BytesMessage bytesMessage = session.createBytesMessage();
bytesMessage.setStringProperty("fileName", file.getName());
bytesMessage.writeBytes(fileManager.readfileAsBytes(file));
msgProducer.send(bytesMessage);
}
, where:
ConnectionFactory connFactory =
new ActiveMQConnectionFactory(username, password, activeMqBrokerUri);
Connection connection = connFactory.createConnection();
ActiveMQSession session =
(ActiveMQSession) connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
FileAsByteArrayManager class that performs a low-level operations with files:
public class FileAsByteArrayManager {
public byte[] readfileAsBytes(File file) throws IOException {
try (RandomAccessFile accessFile = new RandomAccessFile(file, "r")) {
byte[] bytes = new byte[(int) accessFile.length()];
accessFile.readFully(bytes);
return bytes;
}
}
public void writeFile(byte[] bytes, String fileName) throws IOException {
File file = new File(fileName);
try (RandomAccessFile accessFile = new RandomAccessFile(file, "rw")) {
accessFile.write(bytes);
}
}
}
The file process strategy checks databases for specific preprocess conditions/steps were executed before processing a file XYZ.
The files XYZ comes from different sub directories of a root directory that camel scans.
If one of the file is not eligible camel should move to the next one. Not sure what flag should I use if any so it won't bother checking the current file again and again instead of moving to the next one.
My File process Strategy:
public class MigrFileProcessStrategy<T> extends GenericFileProcessStrategySupport<T> {
private static final Logger log = LoggerFactory.getLogger(MigrFileProcessStrategy.class);
#Autowired
DefaultMigrationProcessor defaultMigrationProcessor;
#Override
public boolean begin(GenericFileOperations<T> operations, GenericFileEndpoint<T> endpoint, Exchange exchange, GenericFile<T> file) throws Exception {
//check if HIST table has entry for this filename, if yes all preprocessing done, file is ready.
boolean readyForProcessing = false;
String fileAbsPath = file.getAbsoluteFilePath();
File inMigrFile = new File(fileAbsPath);
readyForProcessing = defaultMigrationProcessor.isFileReadyForProcessing(inMigrFile);
if (!readyForProcessing) {
String msg = String.format("\n####Process?:%b File:%s", readyForProcessing, fileAbsPath);
log.info(msg);
}
return readyForProcessing;
}
}
My Config:
<bean id="migrFilesToCopy" class="org.apache.camel.component.file.MigrFileFilter">
<!-- Filter for migr files that need to be copied-->
<property name="caseSensitive" value="true" />
<property name="excludes" value="**/**/*.migratedLnk, **/_inProgress/**, **/_done/**, **/_failed/**" />
</bean>
<endpoint id="endpoint_migrFilesToCopy"
uri="file://#{migrationProcessor.migrRootDir.toFile()}?processStrategy=#migrFileProcessStrategy&directoryMustExist=true&idempotent=true&recursive=true&delete=true&initialDelay=1000&delay=5000&readLock=changed&readLockTimeout=100000&readLockCheckInterval=1000&moveFailed=_failed&maxMessagesPerPoll=10&filter=#migrFilesToCopy" />
<route id="chMigrate" autoStartup="true">
<from uri="ref:endpoint_migrFilesToCopy" />
<pipeline>
<log message="Procesing file: ${header.CamelFileName}" />
<!--threads executorServiceRef="migrThreadPool" -->
<bean ref="migrationProcessor" method="createMetadata" /><!-- MetaDataVo -->
<bean ref="migrationProcessor" method="createCopyObj" /><!-- CacheCopyObj -->
<bean ref="migrationProcessor" method="migrate" />
<!--/threads -->
</pipeline>
</route>
Resolved by extending ant filter:
public class MigrFileFilter extends AntPathMatcherGenericFileFilter implements GenericFileFilter {
private static final Logger log = LoggerFactory.getLogger(MigrFileFilter.class);
#Autowired
DefaultMigrationProcessor defaultMigrationProcessor;
public MigrFileFilter() {
super();
}
#Override
public boolean accept(GenericFile<T> file) {
String fileAbsPath = file.getAbsoluteFilePath();
File inMigrFile = new File(fileAbsPath);
boolean readyForProcessing = false;
if (Files.isDirectory(inMigrFile.toPath())) {
readyForProcessing = true; //To recursivly process directories.
} else {
boolean validFilePatten = super.accept(file);
boolean preprocessDataExist = false;
if (validFilePatten) {
preprocessDataExist = defaultMigrationProcessor.isFileReadyForProcessing(inMigrFile);
}
readyForProcessing = (validFilePatten && preprocessDataExist);
}
return readyForProcessing;
}
Use the filter to filter out unwanted files
You can implement a custom implementation, and just return true|false if you want to include the file or not.
See the section Filter using org.apache.camel.component.file.GenericFileFilter at
http://camel.apache.org/file2
We are using RestEasy Client version 3.0*.
We are trying to post very large content in the request body.
When the content length is less than 1 MB the following code is running correctly.
When the content length is very large (about 500 MB) the request is getting stuck.
The code is looking like this:
InputStream inputStream = new FileInputStream(tempFile);
ResteasyClient client = new ResteasyClientBuilder().build();
ResteasyWebTarget target = client.target("<SomeUrl>");
response = target.request(MediaType.WILDCARD).post(Entity.entity(inputStream, MediaType.WILDCARD));
How should I change the code to support large content?
After a week...
Since nobody answered me - I used the good old Spring solution (see below).
Is Spring better than RestEasy client?
InputStream inputStream = new FileInputStream(tempFile);
SimpleClientHttpRequestFactory factory = new SimpleClientHttpRequestFactory();
factory.setBufferRequestBody(false);
RestTemplate r = new RestTemplate(factory);
RequestCallback requestCallback = new RequestCallback() {
#Override
public void doWithRequest(ClientHttpRequest clientHttpRequest) throws IOException {
List<org.springframework.http.MediaType> MEDIA_TYPES = Collections.unmodifiableList(Arrays.asList(org.springframework.http.MediaType.ALL));
clientHttpRequest.getHeaders().setAccept(MEDIA_TYPES);
OutputStream requestOutputStream = clientHttpRequest.getBody();
try {
int copiedBytes = IOUtils.copy(inputStream, requestOutputStream);
} finally {
IOUtils.closeQuietly(requestOutputStream);
}
}
};
ResponseExtractor<Response> responseExtractor = new ResponseExtractor<Response>() {
#Override
public Response extractData(ClientHttpResponse response) throws IOException {
return Response.status(response.getRawStatusCode()).build();
}
};
response = r.execute(url, HttpMethod.POST, requestCallback, responseExtractor);
I guess in Spring 4.0.1 you can use Apache HTTP client and HttpComponentsClientHttpRequestFactory instead of SimpleClientHttpRequestFactory. Here is the original note: SPR-10728
<bean id="restTemplateStreaming" class="org.springframework.web.client.RestTemplate">
<constructor-arg>
<bean class="org.springframework.http.client.HttpComponentsClientHttpRequestFactory">
<property name="bufferRequestBody" value="false" />
<property name="httpClient">
<bean class="com.spotfire.server.http.spring.HttpClientFactory">
<property name="clientBuilder">
<bean class="org.apache.http.impl.client.HttpClientBuilder">
<property name="connectionManager" ref="poolingHttpClientConnectionManager" />
<property name="redirectStrategy">
<bean class="org.apache.http.impl.client.LaxRedirectStrategy" />
</property>
</bean>
</property>
</bean>
</property>
</bean>
</constructor-arg>
</bean>
Note the
However callback implementation changes as well. You no longer can call clientHttpRequest.getBody();
Instead you have to use a bit convoluted structure as shown below:
class StreamingRequestCallback implements RequestCallback {
private final InputStream inputStream;
StreamingRequestCallback(InputStream inputStream) {
this.inputStream = inputStream;
}
#Override
public void doWithRequest(final ClientHttpRequest request) throws IOException {
request.getHeaders().add("Content-type", MediaType.APPLICATION_OCTET_STREAM_VALUE);
((StreamingHttpOutputMessage) request).setBody(new StreamingHttpOutputMessage.Body() {
#Override
public void writeTo(final OutputStream outputStream) throws IOException {
IOUtils.copy(inputStream, outputStream);
}
});
}
};