Apache Camel - protocol buffer endpoint - apache-camel

I am trying to create a camel endpoint that listens on a tcp port to receive a message encoded using protocol buffers. [https://code.google.com/p/protobuf/]
I am trying to use netty to open the tcp port but I cannot get it to work.
My camel route builder is:
from("netty:tcp://localhost:9000?sync=false").to("direct:start");
from("direct:start").unmarshal(format)
.to("log:protocolbuffers?level=DEBUG")
.to("mock:result");
I have tried the textline code, but this just causes the error com.google.protobuf.InvalidProtocolBufferException: While parsing a protocol message, the input ended unexpectedly in the middle of a field. This could mean either than the input has been truncated or that an embedded message misreported its own length.
I think I need to use a byte array codec rather than a String, but I can't see a way to do it. I think I could write a custom endpoint to do it, but I'd rather not. Any pointers would be much appreciated.
I sent the message to the camel endpoint using the code below:
public static void main(String[] args) {
try {
TestProtos.Person me = TestProtos.Person.newBuilder().setId(2).setName("Alan").build();
//set up socket
SocketChannel serverSocket;
serverSocket = SocketChannel.open();
serverSocket.socket()
.setReuseAddress(true);
serverSocket.connect(new InetSocketAddress("127.0.0.1", 9000));
serverSocket.configureBlocking(true);
//create BAOS for protobuf
ByteArrayOutputStream baos = new ByteArrayOutputStream();
//mClientDetails is a protobuf message object, dump it to the BAOS
me.writeDelimitedTo(baos);
//copy the message to a bytebuffer
ByteBuffer socketBuffer = ByteBuffer.wrap(baos.toByteArray());
//keep sending until the buffer is empty
while (socketBuffer.hasRemaining()) {
serverSocket.write(socketBuffer);
}
serverSocket.close();
} catch (Exception e) {
System.out.println("error....");
}
}
}
I also ran a test using a file endpoint which worked as expected. I created the file
with:
#Test
public void fileTest() throws Exception {
TestProtos.Person me = TestProtos.Person.newBuilder().setId(2).setName("Chris").build();
File file = new File("/tmp/test.txt");
FileOutputStream out = new FileOutputStream(file);
me.writeTo(out);
out.close();
};

Related

Changing apache camel message type to InOut

From what I understand, an InOut message is one where a response can be received from the destination.
However, I have not been able to find any example of how to convert a message to InOut type, and how to access the response from the destination
For example, given a route like:
from("direct:start").to("smtps://smtp.gmail.com:465?username=user#gmail.com&password=usrpw&to=address#gmail.com")
How to convert the message routed to smtps component into InOut type?
Can I expect a response from the smtp component, e.g. indicating that the message was sent successfully?
how to access this response?
By default, each ".to(uri)" are in InOut. The body in the next step will be replaced by the response of the InOut destination. For example, in HTTP component, if you have the following route :
from(direct:start)
.to(http://...)
.log(INFO, "${body}")
The response to the http call will be logged.
If you don't find good informations in the document, I highly recommend you to check the code of the related producer to know what's returned or can be used.
https://github.com/apache/camel
For example, for SMTP, I haven't found the doc saying what's happening to the body, but the code is pretty much clear :
public void process(final Exchange exchange) {
ClassLoader tccl = Thread.currentThread().getContextClassLoader();
try {
ClassLoader applicationClassLoader = getEndpoint().getCamelContext().getApplicationContextClassLoader();
if (applicationClassLoader != null) {
Thread.currentThread().setContextClassLoader(applicationClassLoader);
}
MimeMessage mimeMessage;
final Object body = exchange.getIn().getBody();
if (body instanceof MimeMessage) {
// Body is directly a MimeMessage
mimeMessage = (MimeMessage) body;
} else {
// Create a message with exchange data
mimeMessage = new MimeMessage(sender.getSession());
getEndpoint().getBinding().populateMailMessage(getEndpoint(), mimeMessage, exchange);
}
if (LOG.isDebugEnabled()) {
LOG.debug("Sending MimeMessage: {}", MailUtils.dumpMessage(mimeMessage));
}
sender.send(mimeMessage);
// set the message ID for further processing
exchange.getIn().setHeader(MailConstants.MAIL_MESSAGE_ID, mimeMessage.getMessageID());
} catch (MessagingException e) {
exchange.setException(e);
} catch (IOException e) {
exchange.setException(e);
} finally {
Thread.currentThread().setContextClassLoader(tccl);
}
}
Your exchange will have an header with the MAIL_ID ("CamelMailMessageId"), and in case of any messaging exception, the exception will be propagated. The body seems to be left untouched, even if it's InOut.

Windows IoT TcpClient

I want to send a command to my Sonos speaker. With my Windows application that is easy. I just use the TcpClient example provided on Microsoft website (shown below).
public void Connect(String server, String message)
{
try
{
TcpClient client = new TcpClient(server, 1400);
// Translate the passed message into ASCII and store it as a Byte array.
Byte[] data = System.Text.Encoding.ASCII.GetBytes(message);
// Get a client stream for reading and writing.
// Stream stream = client.GetStream();
NetworkStream stream = client.GetStream();
// Send the message to the connected TcpServer.
stream.Write(data, 0, data.Length);
//Console.WriteLine("Sent: {0}", message);
// Receive the TcpServer.response.
// Buffer to store the response bytes.
data = new Byte[256];
// String to store the response ASCII representation.
String responseData = String.Empty;
// Read the first batch of the TcpServer response bytes.
Int32 bytes = stream.Read(data, 0, data.Length);
responseData = System.Text.Encoding.ASCII.GetString(data, 0, bytes);
//Console.WriteLine("Received: {0}", responseData);
// Close everything.
stream.Close();
client.Close();
}
catch (ArgumentNullException e)
{
//Console.WriteLine("ArgumentNullException: {0}", e);
}
catch (SocketException e)
{
// Console.WriteLine("SocketException: {0}", e);
}
//Console.WriteLine("\n Press Enter to continue...");
//Console.Read();
}
Now, how would I go about doing this with Windows 10 IoT on a Raspberry Pi 3?
With UWP, you may need to reference the "System.Net.Sockets" Nuget package in order to use TcpClient. You probably end up with something like below snippet,
async void FunctionName()
{
try
{
using (var client = new TcpClient())
{
await client.ConnectAsync(server, 1400);
// Translate the passed message into ASCII and store it as a Byte array.
Byte[] data = System.Text.Encoding.ASCII.GetBytes(message);
// Get a client stream for reading and writing.
// Stream stream = client.GetStream();
NetworkStream stream = client.GetStream();
// Send the message to the connected TcpServer.
stream.Write(data, 0, data.Length);
//Console.WriteLine("Sent: {0}", message);
// Receive the TcpServer.response.
// Buffer to store the response bytes.
data = new Byte[256];
// String to store the response ASCII representation.
String responseData = String.Empty;
// Read the first batch of the TcpServer response bytes.
Int32 bytes = stream.Read(data, 0, data.Length);
responseData = System.Text.Encoding.ASCII.GetString(data, 0, bytes);
//Console.WriteLine("Received: {0}", responseData);
}
}
catch (ArgumentNullException e)
{
//Console.WriteLine("ArgumentNullException: {0}", e);
}
catch (SocketException e)
{
// Console.WriteLine("SocketException: {0}", e);
}
//Console.WriteLine("\n Press Enter to continue...");
//Console.Read();
}
Note that you need to declare the Internet client capability in your project manifest file.
PS: There's also an alternative an alternative called StreamSocket.
Refer to an complete code sample from Microsoft github repository.
Also, if you're new to UWP programming, you should get yourself familar with the async/await pattern.

Appengine not encoding request body in UTF-8

Appengine is not respecting req.setCharacterEncoding('UTF-8') when reading the request body.
This is how I read the request body
StringBuilder sb = new StringBuilder();
BufferedReader reader;
req.setCharacterEncoding("UTF-8");
reader = req.getReader();
String line;
while ((line = reader.readLine()) != null) {
sb.append(line).append('\n');
}
reader.close();
// parse body as JSON
data = new JSONObject(sb.toString());
Request with non-english character are parsed properly when running local test server (mvn appengine:devserver) but the version pushed to production does not parse non-english characters (mvn appengine:update); they are read as ?. This discrepancy is what I'm really confused about.
I also tried setting environment variables like
<env-variables>
<env-var name="DEFAULT_ENCODING" value="UTF-8" />
</env-variables>
in appengine-web.xml, but that doesn't change anything.
What could be causing the prod server to not parse non-english characters?
I don't really know why it wouldn't parse the body properly. I needed to parse the body to validate it before passing it onto my backend to do further processing. So, instead of parsing it in GAE, I relayed the body as a byte array to the backend, and let my backend handle the validation. This was the only working solution I can find.
Make sure you set the content-type header on your request correctly - on the client side, as in:
requestBuilder.setHeader("Content-type", "application/json; charset=utf-8");
I had a similar problem and this is the solution that worked for me. What I learned was that by the time the string is completely built (or appended to the string builder), it's too late because you need to specify the charset while reading the bytes and building the string.
The request.setCharacterEncoding doesn't work well in this regard, for reasons I'm unsure of.
The alternative I used for this was:
StringBuilder stringBuilder = new StringBuilder();
BufferedReader bufferedReader = null;
try {
InputStream inputStream = request.getInputStream();
if (inputStream != null) {
bufferedReader = new BufferedReader(new InputStreamReader(inputStream, "UTF-8"));
char[] charBuffer = new char[128];
int bytesRead = -1;
while ((bytesRead = bufferedReader.read(charBuffer)) > 0) {
stringBuilder.append(charBuffer, 0, bytesRead);
}
} else {
stringBuilder.append("");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
if (bufferedReader != null) {
try {
bufferedReader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
String body = stringBuilder.toString();
I got the input stream of bytes directly from the request and used a BufferedReader to read characters from this stream. I specified the charset here and this allowed me to build the string, while decoding in the respective charset.

save UDP socket data into file

I have a code to send and receive UDP socket
Send UDP code:
public static void main(String args[]) throws Exception
{
try
{
FileOutputStream fo = new FileOutputStream("OUTFILE.txt");
PrintStream ps = new PrintStream(fo);
DatagramSocket Socket = new DatagramSocket(4555);
byte[] receiveData = new byte[1000000];
DatagramPacket receivePacket = new DatagramPacket(receiveData, receiveData.length);
while(true)
{
receivePacket.setLength(receiveData.length);
Socket.receive(receivePacket);
String sentence = new String( receivePacket.getData());
System.out.printf("RECEIVED: %s " , new String(receivePacket.getData()));
ps.println(sentence);
ps.println();
ps.close();
fo.close();
}
File file = new File("OUTFILE.txt");
FileInputStream fis = new FileInputStream(file);
byte[] fsize = new byte[(int) file.length()];
int size = fis.read(fsize);
System.out.println("Received Size = " + size);
}
catch (Exception e) {
System.err.println(e);
}
}
}
I want to write the value of each received data packet into file then get the size of the whole file.
In my code I just got the first received value written in the file.
Could you please tell me how can I write the whole received value in the file.
At the end of the first loop, you're closing the streams so additional lines can not be written. You need to move the ps.close(); and fs.close() calls to be outside of the loop. In addition, you have an indefinite loop which guarantees that the code reading from the file can not be called--you need to have some mechanism to determine when to stop looping.

Get stream from java.sql.Blob in Hibernate

I'm trying to use hibernate #Entity with java.sql.Blob to store some binary data. Storing doesn't throw any exceptions (however, I'm not sure if it really stores the bytes), but reading does. Here is my test:
#Test
public void shouldStoreBlob() {
InputStream readFile = getClass().getResourceAsStream("myfile");
Blob blob = dao.createBlob(readFile, readFile.available());
Ent ent = new Ent();
ent.setBlob(blob);
em.persist(ent);
long id = ent.getId();
Ent fromDb = em.find(Ent.class, id);
//Exception is thrown from getBinaryStream()
byte[] fromDbBytes = IOUtils.toByteArray(fromDb.getBlob().getBinaryStream());
}
So it throws an exception:
java.sql.SQLException: could not reset reader
at org.hibernate.engine.jdbc.BlobProxy.getStream(BlobProxy.java:86)
at org.hibernate.engine.jdbc.BlobProxy.invoke(BlobProxy.java:108)
at $Proxy81.getBinaryStream(Unknown Source)
...
Why? Shouldn't it read bytes form DB here? And what can I do for it to work?
Try to refresh entity:
em.refresh(fromDb);
Stream will be reopened. I suspect that find(...) is closing the blob stream.
It is not at all clear how you are using JPA here, but certainly you do not need to deal with Blob data type directly if you are using JPA.
You just need to declare a field in the entity in question of #Lob somewhat like this:
#Lob
#Basic(fetch = LAZY)
#Column(name = "image")
private byte[] image;
Then, when you retrieve your entity, the bytes will be read back again in the field and you will be able to put them in a stream and do whatever you want with them.
Of course you will need a getter and setter methods in your entity to do the byte conversion. In the example above it would be somewhat like:
private Image getImage() {
Image result = null;
if (this.image != null && this.image.length > 0) {
result = new ImageIcon(this.image).getImage();
}
return result;
}
And the setter somewhat like this
private void setImage(Image source) {
BufferedImage buffered = new BufferedImage(source.getWidth(null), source.getHeight(null), BufferedImage.TYPE_INT_RGB);
Graphics2D g = buffered.createGraphics();
g.drawImage(source, 0, 0, null);
g.dispose();
ByteArrayOutputStream stream = new ByteArrayOutputStream();
try {
ImageIO.write(buffered, "JPEG", stream);
this.image = stream.toByteArray();
}
catch (IOException e) {
assert (false); // should never happen
}
}
}
You need to set a breakpoint on method org.hibernate.engine.jdbc.BlobProxy#getStream on line stream.reset() and examine a reason of IOException:
private InputStream getStream() throws SQLException {
try {
if (needsReset) {
stream.reset(); // <---- Set breakpoint here
}
}
catch ( IOException ioe) {
throw new SQLException("could not reset reader");
}
needsReset = true;
return stream;
}
In my case the reason of IOException was in usage of org.apache.commons.io.input.AutoCloseInputStream as a source for Blob:
InputStream content = new AutoCloseInputStream(stream);
...
Ent ent = new Ent();
...
Blob blob = Hibernate.getLobCreator(getSession()).createBlob(content, file.getFileSize())
ent.setBlob(blob);
em.persist(ent);
While flushing a Session hibernate closes Inpustream content (or rather org.postgresql.jdbc2.AbstractJdbc2Statement#setBlob closes Inpustream in my case). And when AutoCloseInputStream is closed - it rases an IOException in method reset()
update
In your case you use a FileInputStream - this stream also throws an exception on reset method.
There is a problem in test case. You create blob and read it from database inside one transaction. When you create Ent, Postgres jdbc driver closes InputStream while flushing a session. When you load Ent (em.find(Ent.class, id)) - you get the same BlobProxy object, that stores already closed InputStream.
Try this:
TransactionTemplate tt;
#Test
public void shouldStoreBlob() {
final long id = tt.execute(new TransactionCallback<long>()
{
#Override
public long doInTransaction(TransactionStatus status)
{
try
{
InputStream readFile = getClass().getResourceAsStream("myfile");
Blob blob = dao.createBlob(readFile, readFile.available());
Ent ent = new Ent();
ent.setBlob(blob);
em.persist(ent);
return ent.getId();
}
catch (Exception e)
{
return 0;
}
}
});
byte[] fromStorage = tt.execute(new TransactionCallback<byte[]>()
{
#Override
public byte[] doInTransaction(TransactionStatus status)
{
Ent fromDb = em.find(Ent.class, id);
try
{
return IOUtils.toByteArray(fromDb.getBlob().getBinaryStream());
}
catch (IOException e)
{
return new byte[] {};
}
}
});
}
My current and only solution is closing the write session and opening new Hibernate session to get back the streamed data. It works. However I do not know what is the difference. I called inputStream.close(), but that was not enough.
Another way:
I tried to call free() method of blob after session.save(attachment) call too, but it throws another exception:
Exception in thread "main" java.lang.AbstractMethodError: org.hibernate.lob.SerializableBlob.free()V
at my.hibernatetest.HibernateTestBLOB.storeStreamInDatabase(HibernateTestBLOB.java:142)
at my.hibernatetest.HibernateTestBLOB.main(HibernateTestBLOB.java:60)
I am using PostgreSQL 8.4 + postgresql-8.4-702.jdbc4.jar, Hibernate 3.3.1.GA
Is the method IOUtils.toByteArray closing the input stream?

Resources