I have a golang based client that must to speak with a C daemon, using a binary protocol. I cannot change the server, to support json, xml or other protocol.
in the C code, i have to fill in the following struct and send it via network:
typedef struct pkt_struct{
int16_t pkt_version;
int16_t pkt_type;
u_int32_t crc32_v;
int16_t ret_code;
char buffer[1024];
}pkt;
to have an idea, how do I need the data, it should look like the output from:
$ irb
2.0.0-p353 :002 > [2, 1, 0, 0, 'version', 3].pack("nnNna1024n")
is gob the answer? reading through the documentation, looks that it is not. maybe ProtoBuf?
what I did until now is:
import "encoding/binary"
....
type NPacket struct {
packet_version int16
packet_type int16
crc32_value uint32
ret_code int16
buffer string
}
....
var pkt_send NPacket
pkt_send = NPacket{2,1,0,0,"version"}
buf := new(bytes.Buffer)
if err := binary.Write(buf, binary.BigEndian, &pkt_send); err != nil {
fmt.Println(err)
os.Exit(1)
}
error that I'm getting:
binary.Write: invalid type string
binary.Write only works on fixed sized objects. A string may be any size. Instead, you probably want to copy your C code and use a fixed size byte array:
type NPacket struct {
packet_version int16
packet_type int16
crc32_value uint32
ret_code int16
buffer [1024]byte
}
You can set the buffer with copy(pkt.buffer[:], "string").
Neither gob nor protobufs are the answer. They are both encoding formats and give you no control over how the data is marshalled.
Related
I would like to write a serializer application that will encode any type of C data structure into MessagePack format. All examples I have seen show the encoding of known structures such as this using MPack:
// encode to memory buffer
char* data;
size_t size;
mpack_writer_t writer;
mpack_writer_init_growable(&writer, &data, &size);
// write the example on the msgpack homepage
mpack_start_map(&writer, 2);
mpack_write_cstr(&writer, "compact");
mpack_write_bool(&writer, true);
mpack_write_cstr(&writer, "schema");
mpack_write_uint(&writer, 0);
mpack_finish_map(&writer);
// finish writing
if (mpack_writer_destroy(&writer) != mpack_ok) {
fprintf(stderr, "An error occurred encoding the data!\n");
return;
}
// use the data
do_something_with_data(data, size);
free(data);
But what I would like is to be able to encode any C data structure. For example if I had the following:
struct My_Struct{
int number1;
float number2;
char array[6];
};
struct My_Struct ms = {10, 4.44, "Hello"};
programatically, how would I know that the first 4 bytes represents an int so that I could call an mpack_write_int function to start packing the int into a messagepack format?
I'm using Nanopb to try and send protobuf messages from a VxWorks based National Instruments Compact RIO (9025). My cross compilation works great, and I can even send a complete message with data types that don't require extra encoding. What's getting me is the callbacks. My code is cross compiled and called from LabVIEW and the callback based structure of Nanopb seems to break (error out, crash, target reboots, whatever) on the target machine. If I run it without any callbacks it works great.
Here is the code in question:
bool encode_string(pb_ostream_t *stream, const pb_field_t *field, void * const *arg)
{
char *str = "Woo hoo!";
if (!pb_encode_tag_for_field(stream, field))
return false;
return pb_encode_string(stream, (uint8_t*)str, strlen(str));
}
extern "C" uint16_t getPacket(uint8_t* packet)
{
uint8_t buffer[256];
uint16_t packetSize;
ExampleMsg msg = {};
pb_ostream_t stream = pb_ostream_from_buffer(buffer, sizeof(buffer));
msg.name.funcs.encode = &encode_string;
msg.value = 17;
msg.number = 18;
pb_encode(&stream, ExampleMsg_fields, &msg);
packetSize = stream.bytes_written;
memcpy(packet, buffer, 256);
return packetSize;
}
And here's the proto file:
syntax = "proto2"
message ExampleMsg {
required int32 value = 1;
required int32 number = 2;
required string name = 3;
}
I have tried making the callback an extern "C" as well and it didn't change anything. I've also tried adding a nanopb options file with a max length and either didn't understand it correctly or it didn't work either.
If I remove the string from the proto message and remove the callback, it works great. It seems like the callback structure is not going to work in this LabVIEW -> C library environment. Is there another way I can encode the message without the callback structure? Or somehow embed the callback into the getPacket() function?
Updated code:
extern "C" uint16_t getPacket(uint8_t* packet)
{
uint8_t buffer[256];
for (unsigned int i = 0; i < 256; ++i)
buffer[i] = 0;
uint16_t packetSize;
ExampleMsg msg = {};
pb_ostream_t stream = pb_ostream_from_buffer(buffer, sizeof(buffer));
msg.name.funcs.encode = &encode_string;
msg.value = 17;
msg.number = 18;
char name[] = "Woo hoo!";
strncpy(msg.name, name, strlen(name));
pb_encode(&stream, ExampleMsg_fields, &msg);
packetSize = stream.bytes_written;
memcpy(packet, buffer, sizeof(buffer));
return packetSize;
}
Updated proto file:
syntax = "proto2"
import "nanopb.proto";
message ExampleMsg {
required int32 value = 1;
required int32 number = 2;
required string name = 3 [(nanopb).max_size = 40];
}
You can avoid callbacks by giving a maximum size for the string field using the option (nanopb).max_size = 123 in the .proto file. Then nanopb can generate a simple char array in the structure (relevant part of documentation).
Regarding why callbacks don't work: just a guess, but try adding extern "C" also to the callback function. I assume you are using C++ there, so perhaps on that platform the C and C++ calling conventions differ and that causes the crash.
Does the VxWorks serial console give any more information about the crash? I don't remember if it does that for functions called from LabView, so running some test code directly from the VxWorks shell may be worth a try also.
Perhaps the first hurdle is how the code handles strings.
LabVIEW's native string representation is not null-terminated like C, but you can configure LabVIEW to use a different representation or update your code to handle LabVIEW's native format.
LabVIEW stores a string in a special format in which the first four bytes of the array of characters form a 32-bit signed integer that stores how many characters appear in the string. Thus, a string with n characters requires n + 4 bytes to store in memory.
LabVIEW Help: Using Arrays and Strings in the Call Library Function Node
http://zone.ni.com/reference/en-XX/help/371361L-01/lvexcodeconcepts/array_and_string_options/
This has to be one of the most common things to do when working with IP Sockets. You get the data as a array and then you probably need to convert it to a struct.
The code looks something like this:
array<Byte> ^ buffer = gcnew array<Byte>(2048);
while( nDataLen < 16 )
{
nDataLen += m_Socket->Receive( buffer, 16-nDataLen, SocketFlags::None );
m_Socket->Poll( 100000, SelectMode::SelectRead );
}
Now I need to take the 16 Bytes I just received and convert them to the following struct;
[StructLayout(LayoutKind::Explicit, Size=16, Pack=1,CharSet=CharSet::Ansi)]
public ref struct ResponseHeader
{
public:
[FieldOffset(0)]
UInt32 m_StartMessage; // STRP
[FieldOffset(4)]
Int32 m_MessageLength; // Total message length
[FieldOffset(8),MarshalAs(UnmanagedType::ByValTStr,SizeConst=6)]
String ^ m_Status; // Message Status
[FieldOffset(14),MarshalAs(UnmanagedType::ByValTStr,SizeConst=2)]
String ^ m_Reason; // Message Reason
};
I would like to see a cast from the array directly to the struct but I am not sure that is doable in managed code, simple in native cpp.
If there isn't a way to cast this, then what is the best way to proceed?
I am developing the client server application in c in linux environment.
I used some communication protocols.
the client can send any of two structures below.
1)
typedef struct
{
String userName;
} user;
typedef struct
{
msgId = 20;
user u1;
} msg1;
2)
typedef struct
{
String mailId;
} mail;
typedef struct
{
msgId = 30;
mail m1;
} msg2;
now suppose client send the msg2 to the server , after filling the data to the structure.
write (socket_fd, (struct sockaddr*)&msg2, sizeof(msg2));
at the server side , when I receive the structure
read (client_socket_fd, &a, sizeof(a)); // here I have written &a because I do not know which structure I received.
so , how would I know what the structure is received at server side ?
please assist me , and if the question looks so dumb then please , suggest me to the article or name of the topic to read.
thank you.
This is where you should do serialization of message. Whatever be the structure you send from the client, it should be serialized to a stream of characters while sending over the network. Also, let the first element of the message be an identifier which lets the server know what structure it is.
Example to convert a structure into char buffer:
memcpy(data, ®n, sizeof(regn));
regn - a struct
data - allocated char pointer
Beware that if your client and server exists in different platforms, you need to take care of endianness.
Check this link for more details
Send struct over socket in C
The server needs to get the ID first before reading the rest of the message:-
read (socket_fd, &message_id, sizeof message_id);
switch (message_id)
{
case 20:
read (socket_fd, &user, sizeof user);
break;
case 30:
read (socket_fd, &mail, sizeof mail);
break;
}
Be careful to make sure the second read uses the size of the message without the prefixed ID field.
since you already have a id at first member. you may separately read the int? and then the payload dependent of that id. (you have to read the msgcontent typ since the id was already read.
make the whole thing a struct containing a union:
union rcv
{
struct msg1 m1;
struct msg2 m2;
};
struct msg
{
int msg_id;
union rcv msg;
};
I'm need to implement a few functions that read messages from different devices that have different interface possibilities and different message structure. (but the messages have pretty much the same data)
Eg
Device_A {
message type: A
iface 1: tcp
}
Device_B {
message type: B
iface 1: serial
iface 2: tcp
}
... and so on
In my main...
struct msg_data;
while(user_wants_to_read) {
read_msg(); // reads and sets data in msg_data
do_work(msg_data);
}
In an OO Language I would use the strategy pattern. I think I could do this with a void* read_func;?
I'm inexperienced in C and I want to learn to program this like a good C programmer would do. What sort of design pattern/functions should I implement?
It sounds like you got two or more different abstractions to solve for:
Different stream sources (TCP vs. Serial). Is the the TCP protocol the same for device A and device B?
Different message types that are structurally different but semantically the same.
Different device classes (device A vs Device B)
I would focus on a strategy pattern with factories for reading from a stream. And then perhaps an adapter or strategy pattern for getting more data into message objects. But I wouldn't get held up on "which design pattern". More likely, just think in terms of interfaces.
So to start, perhaps abstracting out the serial and TCP streaming into different implementations with the same interface. One implementation that knows how connect and read bytes from a TCP socket without regard to the message contents. Another that knows how to read from a serial port. They should have the same "interface". Here's a lightweight example of a a "byte stream interface" with some hacked up socket code thrown. Forgive me if this doesn't compile. I might have a typo valid in C++ by wrong in C. In any case, it's just an example demonstrating interfaces through function table pointers.
My thinking on suggesting this is, "how would I implement this in C++?" And then I'm transposing my answer to pure "C". (Note: I'm likely making some declaration mistakes below.)
struct ByteStreamer;
typedef int (*ReadFunc)(ByteStreamer*, char* buffer, int count);
typedef int (*OpenFunc)(ByteStreamer*, char* url); // maybe 'open' isn't needed if it's handled by the factory
typedef int (*CloseFunc)(ByteStreamer*);
typedef void (*DisposeFunc)(ByteStreamer*);
typedef struct _ByteStreamer
{
ReadFunc readfunc;
OpenFunc openfunc;
CloseFunc closefunc;
DisposeFunc dispose;
// private data meant for the "class"
void* instancedata;
} ByteStreamer;
struct _tcpconnection
{
int socket;
sockaddr_in addrRemote;
} TCPConnection;
struct _serialconnection
{
int filehandle;
int baud;
} SerialConnection;
// ---------------------------------------
ByteStream* CreateStreamForTCP(const sockaddr_in *pAddr) // pass additional parameter as needed
{
ByteStreamer* pStream = (ByteStreamre*)malloc(sizeof(ByteStreamer));
TCPConnection* pTCPConnection = (TCPConnection*)malloc(sizeof(TCPConnection*));
pTCPConnection->socket = -1;
pTCPConnection->addrRemote = *pAddr;
pStream->instancedata = pTCPConnection;
pStream->ReadFunc = TCPRead;
pStream->OpenFunc = TCPOpen;
pStream->CloseFunc = TCPClose;
pStream->DisposeFunc = TCPDispose;
pStream->type = STREAM_TYPE_TCP;
return pStream;
}
int TCPRead(ByteStream* pStream, char* buffer, int count)
{
return recv(((TCPConnection*)pStream->instancedata)->socket, buffer, count, 0);
}
int TCPOpen(ByteStream* pStream, char* url)
{
// it's up to you if you want to encapsulate the socket address in url or in the instance data
TCPConnection* pConn = (TCPConnection*)(pStream->instancedata);
int sock = socket(AF_INET, SOCK_STREAM, 0);
connect(&pConn->addrRemote, sizeof(pConn->addrRemote));
return (pConn->sock >= 0); // true/false return;
}
void TCPClose(ByteStream* pStream)
{
TCPConnection* pConn = (TCPConnection*)(pStream->instancedata);
close(pConn->sock);
}
void TCPDispose(ByteStream* pStream)
{
free(pStream->instancedata);
free(pStream);
}
Now replace all the TCP code above with an equivalent serial port implementation. It would also be a good idea to implement a "file stream" (or "in memory stream") version of the ByteStream struct. Because it will be very useful in unit tests for higher level code.
So after you get all the byte stream implementations worked out, then move onto parsing device specific messages.
typedef struct _Message_A
{
// A specific data fields
} Message_A;
struct _Message_B
{
// B specific data fields
} Message_B;
struct Message
{
// commonality between Message_A and Message_B
};
typedef (*ReadMessageFromStream)(MessageReader* pReader, Message* pMsg); // pStream is an in-param, pMSg is an out-param.
typedef (*MessageReaderDispose)();
struct MessageReader
{
ReadMessageFromStream reader;
MessageReaderDispose dispose;
// -----------------------------
ByteStream* pStream;
void *instancedata;
};
// function to read a "Message_A" from a stream - and then transpose it to the generic Message type
int ReadMessage_A(ByteStream* pStream, Message* pMsg);
// function to read a "Message_B" from a stream - and then transpose it to the generic Message type
int ReadMessage_B(ByteStream* pStream, Message* pMsg);
So what's really cool about implementing ReadMessage_A and ReadMessage_B is that you can pass that "file stream" implementation of ByteStream and make some really good unit tests. So when you plug in the TCP or serial version, it has a high chance of just working (assuming your TCP and serial code are tested seperately).
And then perhaps a factory method off each class for creating the uber ReadMessageFromStream:
MessageReader* CreateTCPReaderForDeviceA(DeviceA* pA, sockaddr_in* pAddr)
{
MessageReader *pMR = (vMessageReader*)malloc(sizeof(MessageReader));
pMR->pStream = CreateStreamForTCP(pAddr);
pMR->pStream->Open();
pMR->reader = ReadMessage_A;
return pMR;
}
MessageReader* CreateSerialReaderForDeviceB(DeviceB* pB, int comport)
{
MessageReader *pMR = (vMessageReader*)malloc(sizeof(MessageReader));
pMR->pStream = CreateStreamForSerial(comport);
pMR->pStream->Open();
pMR->reader = ReadMessage_B;
return pMR;
}
And then your main loop looks something like the following:
if ((type == DEVICE_A) && (source == TCP))
pReader = CreateTCPReaderForDeviceA(pDevice, &addr)
else if ((type == DEVICE_B) && (source == SERIAL))
pReader = CreateSerialReaderForDeviceB(pDeviceB, 1);
// read the message
Message msg;
pReader->reader(pReader, &msg);
pReader->Dispose(); // free all the data allocated and close connections/files
Wooh.... I'm tired of typing this point. hope this helps.
I would agree with #rsaxvc. Function pointers are probably the best way to go about this. A google search turned up this: Strategy pattern in C
And for your message struct, you could use nested struct to emulate OO class inheritance
struct base {
// common members
}
struct child1 {
struct base;
// other data members
}
or simplely:
struct child2 {
// same data members as base
// other data members
}
use a base* parameter