How to send a file to the ActiveMQ Queue? - apache-camel

I have a simple Apache Camel route in JBoss FUSE:
<?xml version="1.0"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd
http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd">
<bean id="startPolicy" class="org.apache.camel.routepolicy.quartz.CronScheduledRoutePolicy">
<property name="routeStartTime" value="*/3 * * * * ?"/>
</bean>
<bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="brokerURL" value="tcp://localhost:61616" />
<property name="userName" value="admin" />
<property name="password" value="admin" />
</bean>
<camelContext id="blueprintContext" trace="false" xmlns="http://camel.apache.org/schema/blueprint">
<route id="testRoute" routePolicyRef="startPolicy" autoStartup="false">
<from uri="activemq:source-queue?username=admin&password=admin"></from>
<log message="${body}" loggingLevel="INFO"></log>
<to uri="activemq:sink-queue?username=admin&password=admin"></to>
</route>
</camelContext>
</blueprint>
I can connect to the ActiveMQ broker and send a message to the queue, by using this standalone client:
public class MessageSender {
public static void main(String[] args) throws Exception {
ActiveMQConnectionFactory factory =
new ActiveMQConnectionFactory("tcp://localhost:61616");
factory.setUserName("admin");
factory.setPassword("admin");
Connection connection = factory.createConnection();
try {
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
Queue queue = session.createQueue("source-queue");
MessageProducer producer = session.createProducer(queue);
Message message = session.createTextMessage("some message to queue...");
producer.send(message);
} finally {
connection.close();
}
}
}
From the logs I see, that messages is consumed from the queue and message bodies are displays in the log:
How to send a file to the ActiveMQ Queue? For example, I have a simple form with <input type="file"> encoded in multipart/form-data. By using this form I need to send a payload of POST request to the ActiveMQ Queue.
How can I do that?
I would be very grateful for the information.
Thanks to all.

#Mary Zheng provided an excellent example, how it may be done:
ActiveMQ File Transfer Example
Method of class QueueMessageProducer, that sends the file message to ActiveMQ Broker:
private void sendFileAsBytesMessage(File file) throws JMSException, IOException {
BytesMessage bytesMessage = session.createBytesMessage();
bytesMessage.setStringProperty("fileName", file.getName());
bytesMessage.writeBytes(fileManager.readfileAsBytes(file));
msgProducer.send(bytesMessage);
}
, where:
ConnectionFactory connFactory =
new ActiveMQConnectionFactory(username, password, activeMqBrokerUri);
Connection connection = connFactory.createConnection();
ActiveMQSession session =
(ActiveMQSession) connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
FileAsByteArrayManager class that performs a low-level operations with files:
public class FileAsByteArrayManager {
public byte[] readfileAsBytes(File file) throws IOException {
try (RandomAccessFile accessFile = new RandomAccessFile(file, "r")) {
byte[] bytes = new byte[(int) accessFile.length()];
accessFile.readFully(bytes);
return bytes;
}
}
public void writeFile(byte[] bytes, String fileName) throws IOException {
File file = new File(fileName);
try (RandomAccessFile accessFile = new RandomAccessFile(file, "rw")) {
accessFile.write(bytes);
}
}
}

Related

Autowired annotation is ignored in camel process statement

I need to implement db connection and query in process step.
So, I defined datasource in bean property.
And I tried to use jdbctemplate.
But result is returned with java.lang.NullPointException.
Do camel ignore autowired annotation in process statement?
If there is another solution, let me know it please.
Thank you.
CamelContext.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:camel="http://camel.apache.org/schema/spring"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
...
http://camel.apache.org/schema/spring/camel-spring.xsd">
<bean class="com.ktds.openmzn.common.bean.ProcFormat" id="procFormat"/>
<bean class="com.ktds.openmzn.common.bean.ProcessDistributor" id="splitChannel"/>
<bean class="org.springframework.jdbc.datasource.DriverManagerDataSource" id="dataSource">
<property name="driverClassName" value="${spring.datasource.driver-class-name}"/>
<property name="url" value="${spring.datasource.url}"/>
<property name="username" value="${spring.datasource.username}"/>
<property name="password" value="${spring.datasource.password}"/>
</bean>
<bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
<property name="dataSource" ref="dataSource" />
</bean>
<bean class="com.ktds.openmzn.common.bean.FilePathProcessor" id="filePathProcessor"/>
...
<camelContext id="camelContext-f611cb6c-d516-4346-9adc-5512d327a88d"
trace="true" xmlns="http://camel.apache.org/schema/spring">
<camel:route id="fixed_processor">
<camel:from id="_from1" uri="timer:fromPollTimer?period=20000"/>
<camel:process id="_sourceDirectory" ref="filePathProcessor"/>
...
FilePathProcessor.java
public class FilePathProcessor implements Processor {
...
#Override
public void process(Exchange exchange) throws Exception {
List<Map<String, Object>> rows = SetFilePath.getInstance().getPathList("aaaa");
SetFilePath.java
#ManagedResource
public class SetFilePath {
private static SetFilePath instance = null;
private String sourceDirectory;
private String targetDirectory;
#Autowired
private JdbcTemplate jdbcTemplate;
public static SetFilePath getInstance() {
if(instance == null) {
instance = new SetFilePath();
}
return instance;
}
Result
Message History
---------------------------------------------------------------------------------------------------------------------------------------
RouteId ProcessorId Processor
Elapsed (ms)
[fixed_processor ] [fixed_processor ] [timer://fromPollTimer?period=20000 ] [ 0]
[fixed_processor ] [_sourceDirectory ] [ref:filePathProcessor ] [ 0]
Stacktrace
---------------------------------------------------------------------------------------------------------------------------------------
java.lang.NullPointerException: null
at com.ktds.openmzn.common.bean.FilePathProcessor.process(FilePathProcessor.java:20) ~[classes/:na]
at org.apache.camel.processor.DelegateSyncProcessor.process(DelegateSyncProcessor.java:63) ~[camel-core-2.23.1.jar:2.23.1]
Well of course its null, because you are creating the instance yourself via the new constructor:
instance = new SetFilePath();
The #Autowired is from spring framework, and you would essentially need to use spring to create this bean for you.
There are different ways of doing this such as creating a <bean> in the XML file and then configure that on your processor via a setter <property>.
You can also let spring/camel create the bean instance instead of the new constructor, but this requires a bit of Camel API to do so
public static SetFilePath getInstance(CamelContext camel)
if (instance == null) {
instance = camel.getInjector().newInstance(SetFilePath.class);
}
Which will then via the injector create the bean instance via spring framework that does its auto-wiring.

How to get the file name in a custom processor?

I serialize the files to a byte arrays and send it to the ActiveMQ queue (I using the code from this example: Apache ActiveMQ File Transfer Example):
private void sendFileAsBytesMessage(File file) throws JMSException, IOException {
BytesMessage bytesMessage = session.createBytesMessage();
// Set file name here
bytesMessage.setStringProperty("fileName", file.getName());
// Set file body here
bytesMessage.writeBytes(fileManager.readfileAsBytes(file));
MessageProduce msgProducer =
session.createProducer(session.createQueue(queueName));
MessageProducer msgProducer.send(bytesMessage);
}
Method readfileAsBytes is presented below:
public byte[] readfileAsBytes(File file) throws IOException {
try (RandomAccessFile accessFile = new RandomAccessFile(file, "r")) {
byte[] bytes = new byte[(int) accessFile.length()];
accessFile.readFully(bytes);
return bytes;
}
}
In my OSGi bundle I have a custom processor:
public class Deserializer implements Processor {
#Override
public void process(Exchange exchange) throws Exception {
// Get file body here
byte[] bytes = exchange.getIn().getBody(byte[].class);
}
}
I use it in my Spring DSL route as follows:
<?xml version="1.0"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd
http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd">
<bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="brokerURL" value="tcp://localhost:61616" />
<property name="userName" value="admin" />
<property name="password" value="admin" />
</bean>
<bean id="deserializer" class="org.fusesource.example.Deserializer"/>
<camelContext id="blueprintContext" trace="false" xmlns="http://camel.apache.org/schema/blueprint">
<route id="testRoute">
<from uri="activemq:source-queue"></from>
<process ref="deserializer"/>
<to uri="activemq:sink-queue"></to>
</route>
</camelContext>
</blueprint>
I need to get the file name in my processor. How can I do that?
The problem can be solved as follows. For example, I send the README.html file to the alfresco-queue. Then the body of the file and the file name, as well as some additional information can be obtained as follows:
public class MyRouteBuilder extends RouteBuilder {
#Override
public void configure() throws Exception {
from("activemq:alfresco-queue?username=admin&password=admin")
.process(new Processor() {
public void process(Exchange exchange) throws Exception {
// Get file body
byte[] bytes = exchange.getIn().getBody(byte[].class);
for(int i = 0; i < bytes.length; i++) {
System.out.print((char) bytes[i]);
}
// Get headers
Map<String, Object> headers = exchange.getIn().getHeaders();
Iterator iterator = headers.entrySet().iterator();
while (iterator.hasNext()) {
Map.Entry pair = (Map.Entry)iterator.next();
System.out.println(pair.getKey() + " == " + pair.getValue());
}
}
})
// SKIPPED
It gives the following output:
<head>
root<> title>README
</title>
</head>
<body>
Please refer to http://java.com/licensereadme
</body>
</html>
breadcrumbId == ID:63-DP-TAV-59000-1531813754416-1:1:1:1:1
fileName == README.html
JMSCorrelationID == null
JMSCorrelationIDAsBytes == null
JMSDeliveryMode == 2
JMSDestination == queue://alfresco-queue
JMSExpiration == 0
JMSMessageID == ID:63-DP-TAV-59000-1531813754416-1:1:1:1:1
JMSPriority == 4
JMSRedelivered == false
JMSReplyTo == null
JMSTimestamp == 1531813754610
JMSType == null
JMSXGroupID == null
JMSXUserID == null

camel: Check if file is "eligible" for processing. If a file is not eligible for procssing move to next

The file process strategy checks databases for specific preprocess conditions/steps were executed before processing a file XYZ.
The files XYZ comes from different sub directories of a root directory that camel scans.
If one of the file is not eligible camel should move to the next one. Not sure what flag should I use if any so it won't bother checking the current file again and again instead of moving to the next one.
My File process Strategy:
public class MigrFileProcessStrategy<T> extends GenericFileProcessStrategySupport<T> {
private static final Logger log = LoggerFactory.getLogger(MigrFileProcessStrategy.class);
#Autowired
DefaultMigrationProcessor defaultMigrationProcessor;
#Override
public boolean begin(GenericFileOperations<T> operations, GenericFileEndpoint<T> endpoint, Exchange exchange, GenericFile<T> file) throws Exception {
//check if HIST table has entry for this filename, if yes all preprocessing done, file is ready.
boolean readyForProcessing = false;
String fileAbsPath = file.getAbsoluteFilePath();
File inMigrFile = new File(fileAbsPath);
readyForProcessing = defaultMigrationProcessor.isFileReadyForProcessing(inMigrFile);
if (!readyForProcessing) {
String msg = String.format("\n####Process?:%b File:%s", readyForProcessing, fileAbsPath);
log.info(msg);
}
return readyForProcessing;
}
}
My Config:
<bean id="migrFilesToCopy" class="org.apache.camel.component.file.MigrFileFilter">
<!-- Filter for migr files that need to be copied-->
<property name="caseSensitive" value="true" />
<property name="excludes" value="**/**/*.migratedLnk, **/_inProgress/**, **/_done/**, **/_failed/**" />
</bean>
<endpoint id="endpoint_migrFilesToCopy"
uri="file://#{migrationProcessor.migrRootDir.toFile()}?processStrategy=#migrFileProcessStrategy&directoryMustExist=true&idempotent=true&recursive=true&delete=true&initialDelay=1000&delay=5000&readLock=changed&readLockTimeout=100000&readLockCheckInterval=1000&moveFailed=_failed&maxMessagesPerPoll=10&filter=#migrFilesToCopy" />
<route id="chMigrate" autoStartup="true">
<from uri="ref:endpoint_migrFilesToCopy" />
<pipeline>
<log message="Procesing file: ${header.CamelFileName}" />
<!--threads executorServiceRef="migrThreadPool" -->
<bean ref="migrationProcessor" method="createMetadata" /><!-- MetaDataVo -->
<bean ref="migrationProcessor" method="createCopyObj" /><!-- CacheCopyObj -->
<bean ref="migrationProcessor" method="migrate" />
<!--/threads -->
</pipeline>
</route>
Resolved by extending ant filter:
public class MigrFileFilter extends AntPathMatcherGenericFileFilter implements GenericFileFilter {
private static final Logger log = LoggerFactory.getLogger(MigrFileFilter.class);
#Autowired
DefaultMigrationProcessor defaultMigrationProcessor;
public MigrFileFilter() {
super();
}
#Override
public boolean accept(GenericFile<T> file) {
String fileAbsPath = file.getAbsoluteFilePath();
File inMigrFile = new File(fileAbsPath);
boolean readyForProcessing = false;
if (Files.isDirectory(inMigrFile.toPath())) {
readyForProcessing = true; //To recursivly process directories.
} else {
boolean validFilePatten = super.accept(file);
boolean preprocessDataExist = false;
if (validFilePatten) {
preprocessDataExist = defaultMigrationProcessor.isFileReadyForProcessing(inMigrFile);
}
readyForProcessing = (validFilePatten && preprocessDataExist);
}
return readyForProcessing;
}
Use the filter to filter out unwanted files
You can implement a custom implementation, and just return true|false if you want to include the file or not.
See the section Filter using org.apache.camel.component.file.GenericFileFilter at
http://camel.apache.org/file2

RestEasy Client 3.0.* handling large content

We are using RestEasy Client version 3.0*.
We are trying to post very large content in the request body.
When the content length is less than 1 MB the following code is running correctly.
When the content length is very large (about 500 MB) the request is getting stuck.
The code is looking like this:
InputStream inputStream = new FileInputStream(tempFile);
ResteasyClient client = new ResteasyClientBuilder().build();
ResteasyWebTarget target = client.target("<SomeUrl>");
response = target.request(MediaType.WILDCARD).post(Entity.entity(inputStream, MediaType.WILDCARD));
How should I change the code to support large content?
After a week...
Since nobody answered me - I used the good old Spring solution (see below).
Is Spring better than RestEasy client?
InputStream inputStream = new FileInputStream(tempFile);
SimpleClientHttpRequestFactory factory = new SimpleClientHttpRequestFactory();
factory.setBufferRequestBody(false);
RestTemplate r = new RestTemplate(factory);
RequestCallback requestCallback = new RequestCallback() {
#Override
public void doWithRequest(ClientHttpRequest clientHttpRequest) throws IOException {
List<org.springframework.http.MediaType> MEDIA_TYPES = Collections.unmodifiableList(Arrays.asList(org.springframework.http.MediaType.ALL));
clientHttpRequest.getHeaders().setAccept(MEDIA_TYPES);
OutputStream requestOutputStream = clientHttpRequest.getBody();
try {
int copiedBytes = IOUtils.copy(inputStream, requestOutputStream);
} finally {
IOUtils.closeQuietly(requestOutputStream);
}
}
};
ResponseExtractor<Response> responseExtractor = new ResponseExtractor<Response>() {
#Override
public Response extractData(ClientHttpResponse response) throws IOException {
return Response.status(response.getRawStatusCode()).build();
}
};
response = r.execute(url, HttpMethod.POST, requestCallback, responseExtractor);
I guess in Spring 4.0.1 you can use Apache HTTP client and HttpComponentsClientHttpRequestFactory instead of SimpleClientHttpRequestFactory. Here is the original note: SPR-10728
<bean id="restTemplateStreaming" class="org.springframework.web.client.RestTemplate">
<constructor-arg>
<bean class="org.springframework.http.client.HttpComponentsClientHttpRequestFactory">
<property name="bufferRequestBody" value="false" />
<property name="httpClient">
<bean class="com.spotfire.server.http.spring.HttpClientFactory">
<property name="clientBuilder">
<bean class="org.apache.http.impl.client.HttpClientBuilder">
<property name="connectionManager" ref="poolingHttpClientConnectionManager" />
<property name="redirectStrategy">
<bean class="org.apache.http.impl.client.LaxRedirectStrategy" />
</property>
</bean>
</property>
</bean>
</property>
</bean>
</constructor-arg>
</bean>
Note the
However callback implementation changes as well. You no longer can call clientHttpRequest.getBody();
Instead you have to use a bit convoluted structure as shown below:
class StreamingRequestCallback implements RequestCallback {
private final InputStream inputStream;
StreamingRequestCallback(InputStream inputStream) {
this.inputStream = inputStream;
}
#Override
public void doWithRequest(final ClientHttpRequest request) throws IOException {
request.getHeaders().add("Content-type", MediaType.APPLICATION_OCTET_STREAM_VALUE);
((StreamingHttpOutputMessage) request).setBody(new StreamingHttpOutputMessage.Body() {
#Override
public void writeTo(final OutputStream outputStream) throws IOException {
IOUtils.copy(inputStream, outputStream);
}
});
}
};

mandatory request param in cxf

I developed a web service with CXF and It work fine.
I have a service with two input parameters and both of them should be mandatory.
but when I call my service just the first parameter is mandatory.
please let me know what should I do?
my SEI
#WebService(
endpointInterface = "com.myCompany.product.webService",
targetNamespace = "http://product.myCompany.com",
portName = "product",
serviceName = "ProductService")
#DataBinding(org.apache.cxf.aegis.databinding.AegisDatabinding.class)
public interface ProductService {
#WebMethod(operationName = "authentication")
#WebResult(name = "authenticationResponseParam")
public AuthenticationResponseParam authentication(#WebParam(name = "user", header = true) String user,
#WebParam(name = "authenticationRequestParam") AuthenticationRequestParam authenticationRequestParam);
}
and my AuthenticationResponseParam class
#XmlAccessorType(XmlAccessType.FIELD
)
#XmlType(name = "authenticationRequestParam", propOrder = {
"account", "password"
})
public class AuthenticationRequestParam implements Serializable {
#XmlElement(name = "account", required = true)
private BigDecimal account;
#XmlElement(name = "password", required = true)
private String password;
public BigDecimal getAccount() {
return account;
}
public void setAccount(BigDecimal account) {
this.account = account;
}
public String getPassword() {
return password;
}
public void setPassword(String password) {
this.password = password;
}
#Override
public String toString() {
return "AuthenticationRequestParam{" +
"account=" + account +
", password='" + password + '\'' +
'}';
}
}
and my CXF servlet xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:jaxws="http://cxf.apache.org/jaxws"
xmlns:cxf="http://cxf.apache.org/core"
xmlns:soap="http://cxf.apache.org/bindings/soap"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://cxf.apache.org/jaxws http://cxf.apache.org/schemas/jaxws.xsd
http://cxf.apache.org/core http://cxf.apache.org/schemas/core.xsd
http://cxf.apache.org/bindings/soap
http://cxf.apache.org/schemas/configuration/soap.xsd">
<import resource="classpath:META-INF/cxf/cxf.xml"/>
<import resource="classpath:META-INF/cxf/cxf-extension-soap.xml"/>
<import resource="classpath:META-INF/cxf/cxf-servlet.xml"/>
<cxf:bus>
<cxf:features>
<cxf:logging/>
</cxf:features>
</cxf:bus>
<!--Data binding-->
<bean id="aegisBean" class="org.apache.cxf.aegis.databinding.AegisDatabinding" scope="prototype"/>
<bean id="jaxws-and-aegis-service-factory"
class="org.apache.cxf.jaxws.support.JaxWsServiceFactoryBean"
scope="prototype">
<property name="dataBinding" ref="aegisBean"/>
</bean>
<jaxws:endpoint id="telBank" implementor="#myService" address="/telBank">
<jaxws:binding>
<soap:soapBinding mtomEnabled="false" version="1.2"/>
</jaxws:binding>
</jaxws:endpoint>
<bean id="myService" class="com.myCompany.product.webService.impl.ProductServiceImpl"/>
</beans>
thank you
Hey guys
I added a new service in my web service
public BigDecimal sample(#WebParam(name = "sam1") BigDecimal a1,#WebParam(name = "sam2") BigDecimal a2);
and none of both parameters are mandatory
what should I do?please help me
I found what my problem.
I use org.apache.cxf.aegis.databinding.AegisDatabinding az data binder and it just recognize primitive type az mandatory.when I commend that my input param become mandatory.
what kind of data binder should I use?
If you want to use AegisDatabinding class as data binder,set this property it bean definition.
<bean id="aegisBean" class="org.apache.cxf.aegis.databinding.AegisDatabinding" scope="prototype">
<property name="configuration">
<bean class="org.apache.cxf.aegis.type.TypeCreationOptions">
<property name="defaultMinOccurs" value="1"/>
<property name="defaultNillable" value="false"/>
</bean>
</property>
</bean>

Resources