I am trying to pass whole structure from client to server or vice-versa. Let us assume my structure as follows
struct temp {
int a;
char b;
}
I am using sendto and sending the address of the structure variable and receiving it on the other side using the recvfrom function. But I am not able to get the original data sent on the receiving end. In sendto function I am saving the received data into variable of type struct temp.
n = sendto(sock, &pkt, sizeof(struct temp), 0, &server, length);
n = recvfrom(sock, &pkt, sizeof(struct temp), 0, (struct sockaddr *)&from,&fromlen);
Where pkt is the variable of type struct temp.
Eventhough I am receiving 8bytes of data but if I try to print it is simply showing garbage values. Any help for a fix on it ?
NOTE: No third party Libraries have to be used.
EDIT1: I am really new to this serialization concept .. But without doing serialization cant I send a structure via sockets ?
EDIT2: When I try to send a string or an integer variable using the sendto and recvfrom functions I am receiving the data properly at receiver end. Why not in the case of a structure? If I don't have to use serializing function then should I send each and every member of the structure individually? This really is not a suitable solution since if there are 'n' number of members then there are 'n' number of lines of code added just to send or receive data.
This is a very bad idea. Binary data should always be sent in a way that:
Handles different endianness
Handles different padding
Handles differences in the byte-sizes of intrinsic types
Don't ever write a whole struct in a binary way, not to a file, not to a socket.
Always write each field separately, and read them the same way.
You need to have functions like
unsigned char * serialize_int(unsigned char *buffer, int value)
{
/* Write big-endian int value into buffer; assumes 32-bit int and 8-bit char. */
buffer[0] = value >> 24;
buffer[1] = value >> 16;
buffer[2] = value >> 8;
buffer[3] = value;
return buffer + 4;
}
unsigned char * serialize_char(unsigned char *buffer, char value)
{
buffer[0] = value;
return buffer + 1;
}
unsigned char * serialize_temp(unsigned char *buffer, struct temp *value)
{
buffer = serialize_int(buffer, value->a);
buffer = serialize_char(buffer, value->b);
return buffer;
}
unsigned char * deserialize_int(unsigned char *buffer, int *value);
Or the equivalent, there are of course several ways to set this up with regards to buffer management and so on. Then you need to do the higher-level functions that serialize/deserialize entire structs.
This assumes serializing is done to/from buffers, which means the serialization doesn't need to know if the final destination is a file or a socket. It also means you pay some memory overhead, but it's generally a good design for performance reasons (you don't want to do a write() of each value to the socket).
Once you have the above, here's how you could serialize and transmit a structure instance:
int send_temp(int socket, const struct sockaddr *dest, socklen_t dlen,
const struct temp *temp)
{
unsigned char buffer[32], *ptr;
ptr = serialize_temp(buffer, temp);
return sendto(socket, buffer, ptr - buffer, 0, dest, dlen) == ptr - buffer;
}
A few points to note about the above:
The struct to send is first serialized, field by field, into buffer.
The serialization routine returns a pointer to the next free byte in the buffer, which we use to compute how many bytes it serialized to
Obviously my example serialization routines don't protect against buffer overflow.
Return value is 1 if the sendto() call succeeded, else it will be 0.
Using the 'pragma' pack option did solved my problem but I am not sure if it has any dependencies ??
#pragma pack(1) // this helps to pack the struct to 5-bytes
struct packet {
int i;
char j;
};
#pragma pack(0) // turn packing off
Then the following lines of code worked out fine without any problem
n = sendto(sock,&pkt,sizeof(struct packet),0,&server,length);
n = recvfrom(sock, &pkt, sizeof(struct packet), 0, (struct sockaddr *)&from, &fromlen);
There is no need to write own serialisation routines for short and long integer types - use htons()/htonl() POSIX functions.
If you don't want to write the serialisation code yourself, find a proper serialisation framework, and use that.
Maybe Google's protocol buffers would be possible?
Serialization is a good idea. You can also use Wireshark to monitor the traffic and understand what is actually passed in the packets.
Instead of serialising and depending on 3rd party libraries its easy to come up with a primitive protocol using tag, length and value.
Tag: 32 bit value identifying the field
Length: 32 bit value specifying the length in bytes of the field
Value: the field
Concatenate as required. Use enums for the tags. And use network byte order...
Easy to encode, easy to decode.
Also if you use TCP remember it is a stream of data so if you send e.g. 3 packets you will not necessarily receive 3 packets. They maybe be "merged" into a stream depending on nodelay/nagel algorithm amongst other things and you may get them all in one recv... You need to delimit the data for example using RFC1006.
UDP is easier, you'll receive a distinct packet for each packet sent, but its a lot less secure.
If the format of the data you want to transfer is very simple then converting to and from an ANSI string is simple and portable.
Related
I'm writing an application which reads data from a UART interface. The data is sent in packets. Each packet has a channel associated with it. My application multiplexes received packets into virtual channels (threads) so that every channel can work independently of one another. When I receive a packet I have to do something depending on it's contents and produce a response. The response is sent back using the same UART interface.
The data sent is mostly binary. When I'm reading from the UART interface, I know the size of the packet beforehand, so I can preallocate memory with no problem.
The problem for me is producing a response. I know the maximum size of a packet, so I can create a static buffer when I'm constructing a response. If I we're to work with ASCII characters, instead of binary data, I could rely on NULL terminator to determine how long the data stored in the buffer is. However, I'm working with binary data, so using a NULL byte does not work. Instead, I have to keep a variable storing how many bytes of the buffer is used up already. I was thinking of using a custom data type for storing binary data:
typedef struct {
unsigned char buff[2048];
size_t buff_used;
} binary_data_t;
What would be a standart way of handling this?
Since you know the number of bytes you need to hold a packet, just use a flexible array member:
typedef struct
{
size_t bytes;
unsigned char data[];
} binary_data_t;
(Note that identifiers ending in _t are reserved by POSIX, and you really shouldn't be using them.)
Allocation and reading data (assumes you read() from a file descriptor):
binary_data_t *p = malloc( sizeof( *p ) + numDataBytes );
p->bytes = numDataBytes;
ssize_t bytes_read = read( uartFD, p->data, numDataBytes );
one way of doing it could be to store a pointer to where in your array next byte should be placed.
typedef struct {
unsigned char buff[2048];
char* pData;
} binary_data_t;
// at init
binary_data_t rspMsg;
rspMsg.pData = &rspMsg.buff[0];
// at entering data
*(rspMsg.pData) = data;
rspMsg.pData++;
// at sending data you know the length via
length = rspMsg.pData - &rspMsg.buff[0];
This is one way of solving this.
Can be done in many ways.
Hi I'm writing a program that sends a set of bytes through a message queue like so ...
#include <sys/msg.h>
#include <stddef.h>
key_t key;
int msqid;
struct pirate_msgbuf pmb = {2, { "L'Olonais", 'S', 80, 10, 12035 } };
key = ftok("/home/beej/somefile", 'b');
msqid = msgget(key, 0666 | IPC_CREAT);
/* stick him on the queue */
msgsnd(msqid, &pmb, sizeof(struct pirate_msgbuf) - sizeof(long), 0);
The above example is a simple program from beejs website that resembles mine.
What I'm doing however is sending a message with a struct like so ...
struct msg_queue{
long message_type;
char * buffer;
}
Now before I send my msg_queue, I created some alternative buffer that contains all sorts of information including null characters and such. Now when I do something like this ...
struct msg_queue my_queue;
my_queue.message_type = 1;
my_queue.buffer = "My message";
msgsnd(mysqid, &pmb, sizeof(struct msg_queue) - sizeof(long), 0);
I have no problems receiving the pointer and reading the values stored at that string. However if I were to do something similar like ...
struct msg_queue my_queue;
my_queue.message_type = 1;
my_queue.buffer = sum_buffer_with_lots_of_weird_values; // of type char *
msgsnd(mysqid, &pmb, sizeof(struct msg_queue) - sizeof(long), 0);
The pointer I pass through my queue to my other process will read garbage and not the values stored. I tried making my arbitrary array as a static char *, but that doesn't help either. How do I properly pass in my buffer through the queue? Thanks.
You shouldn't be sending a pointers to another process, they have no meaning (or point to something very different) in another process' address space.
Message queues aren't great for unbounded data like variable length strings. Change your pointer to a fixed length char array sufficiently big to hold the largest string and copy your string into the array before writing the queue. Or use another type of IPC such as domain socket.
Message Queue is used for inter-process communication.
When you malloc some memory in one process, it only exist in that process memory space not accessible by other process.
when you send that pointer over, you are sending a address space which is not accessible. It may even result in segmentation fault.
One way is to limit your buffer size, if applicable.
struct msg_queue{
long message_type;
char buffer[MAX_LEN];
}
Another way is to send it 2 times. The first msgsnd, sends the size of buffer to expect.
The next send, you send the char array over, using the size of the first send. :)
On receiving end, you first get the size, then receive the buffer.
Other way is to use pipes or socket.
"msgsend()" will only read the bytes in your buffer.
If one of those bytes happens to be a pointer (to some string or object somewhere else) ... guess what - the receiver will just get the binary pointer. Not the data being pointed to.
What you need to do is pack the entire contents of your message into a buffer, then send that linear buffer.
I am trying to implemante a client- server program communication using TCP socket programming in C.
It is between between two 64bit machines with linux OS installed.
I want to transfer a c-struct between the two processes.
For this I try used a pack - unpack() functioanlity.
please consider the following code snipt
/*---------------------------------------------------------
on the sending side I have:
---------------------------------------------------------*/
struct packet {
int64_t x;
int64_t y;
int64_t q[maxSize];
} __attribute__((packed));
int main(void)
{
// build packet
struct packet pkt;
pkt.x = htonl(324);
pkt.y = htonl(654);
int i;
for(i = 0; i< maxSize; i++){
pkt.q[i] = i; **// I also try pkt.q[i] = htonl(i);**
}
// and then do the send
}
/*-----------------------------------------------------------------------------
in the receiving side:
-----------------------------------------------------------------------------*/
struct packet {
int64_t x;
int64_t y;
int64_t q[maxSize];
} __attribute__((packed));
static void decodePacket (uint8_t *recv_data, size_t recv_len)
{
// checking size
if (recv_len < sizeof(struct packet)) {
fprintf(stderr, "received too little!");
return;
}
struct packet *recv_packet = (struct packet *)recv_data;
int64_t x = ntohl(recv_packet->x);
int64_t y = ntohl(recv_packet->y);
int i;
printf("Decoded: x=%"PRIu8" y=%"PRIu32"\n", x, y);
for(i=0;i<maxSize;i++){
**//int64_t res = ntohl(recv_packet->q[i]); I also try to print res**
printf("%"PRIu32"\n" , recv_packet->q[i]);
}
}
int main(int argc, char *argv[]){
// receive the data and try to call decodePacket()
int8_t *recv_data = (int8_t *)&buf; //buf is the data received
size_t recv_len = sizeof(buf);
**decode_packet(recv_data, recv_len);**
}
//-----------------------------------------------------------------------------
Now the problem is that I am receiving the value of x and y in the struct correctly,
but for the array q in the struct I am receiving a strange number, possible a memory grabage value, (I try to use memset() filling the array by zeros before receiving a data from the other side in which case the value of all zeros is received )
I don't understand why I am not receiving the correct value for the array in struct.
Please Note that I try with and with out htonl() while filling the array before putting in struct,
and on the other side: with and with out ntohl() while decoding the array from struct
Any help will be appreciated,
size_t recv_len = sizeof(buf);
decode_packet(recv_data, recv_len);
This piece of code ensures the wrong size is passed to decode_packet. So when decode_packet goes on to check recv_len < sizeof(struct packet), that test is meaningless - it will always pass, no matter how many bytes were received.
You need to fetch the size from the value returned by the recv call. My best guess is that indeed you're receiving fewer bytes than you're expecting.
While sending and receiving structs is quite convenient, it's often an exercise in futility. Manually serializing data or using some explicit mechanisms is probably the way to go.
You didn't show us the send and recv part, which is more likely to be wrong. My guess is you're receiving first items in the array correctly and they "become" garbage at some point, is it right?
Well, #cnicutar is correct, but let me extend it a little bit...
First of all, when you call send you have to examine the return value and see if all bytes have been transmitted. If your structure is large (for example larger than underlying socket buffer) you'll need more than one call to transmit the whole structure. Same with recv, don't expect you will get the whole message in one recv call, don't expect every recv will receive the same amount of data that was sent by corresponding send call. Always check how many bytes have been received and call recv again if necessary (pointing to the right place in incoming buffer and reducing number of bytes to receive).
So what is probably happening, you don't receive enough data (maybe you don't even transmit all of it) and only beginning of your incoming buffer is being filled. Therefore, the rest of the structure is garbage or (when you call memset) stays initialised with zeros.
Also note both send and recv return ssize_t rather than size_t as negative values are possible (to indicate errors).
I am writing a code for modbus protocol, that runs on a MSP430 controller. The response buffer(global) is an array of 8bit data, through which the response to the processed request is to be sent on serial UART.
Now, my problem is that the response generated is having a combination of different data types. i.e uint8, uint32, float.
How to send this data by using the global response buffer?
For float, i have tried using memcpy and this seems to be working fine. Is there any better and efficient way? Cuz the frame size is large (say 20-30 bytes). Here is a demo of what i've tried to do
int main()
{ unsigned char buff[8]; //Global buffer
float temp[2]; //temp buffer to store response values
temp[0] = 0x00ef2212;
memcpy(buff, temp, 8);
printf("buff[0]= %u \n buff[1]= %u\n buff[2] = %u\n buff[3]= %u\n", buff[0],buff[1],buff
[2],buff[3]);
return 0;
}
With casting and assignment. E.g.
uint8 *globalbuf;
uint32 myval;
*(uint32*)globalbuf = myval;
copies myval into the first 4 bytes of globalbuf.
Beware of alignment issues: it may be illegal or expensive on your platform to assign or read values to/from addresses that aren't whole multiples of that type size. For example address 0,4,8 etc. are OK places to put a uint32.
This assumes your globalbuf starts on a nice round address..
a simple implementation would be to create a response structure and memcpy the response structure to the buffer
struct response {
uint8 bytedata;
uint32 intdata;
float floatdata;
};
memcpy((void *)uartbuffer,(void *)&response,sizeof(response));
Note: since it looks like your working on a protocol the endianess may be already specified and you may need to pack items according to the particular endian type and simple memcpy would not work and you may need to pack the datatypes.
How about considering using a union to represent a same range of memory into different data types?
typedef union {
unsigned char buff[8];
float temp[2];
} un;
un global_buffer;
// give global_buffer.buff some data.
un local_buffer;
local_buffer = global_buffer;
// handle local_buffer.temp
I have a struct that I am sending to a UDP socket:
typedef struct
{
char field_id;
short field_length;
char* field;
} field_t, *field_p;
I am able to read the field_id and field_length once received on the UDP server-side, however the pointer to field is invalid as expected.
What is the best method to properly send and receive a dynamic char*?
I have a basic solution using memcpy on the client side:
char* data =
(char*)malloc(sizeof(field_t) + (sizeof(char) * strlen(my_field->field)));
memcpy(data, my_field, sizeof(field_t));
memcpy(data+sizeof(field_t), my_field->field, strlen(my_field->field) + 1);
And on the server side:
field_p data = (field_p)buffer;
field_string = (char*)buffer+sizeof(field_t);
Is there a cleaner way of doing this or is this the only way?
Thanks.
You of course cannot send a pointer over a socket - get rid of the char* field; member. Instead, just append id and size pair with the data itself. Use writev(2) or sendmsg(2) to avoid moving data around from buffer to buffer.
Watch out for structure member alignment and padding and number endianness.
Serialization is your friend.
Related Links:
SO-1
SO-2
Define your structure as:
typedef struct
{
uint8_t field_id;
uint16_t field_length;
char field[0]; // note: in C99 you could use char field[];
} field_t, *field_p;
Then, text buffer will immediately follow your structure. Just remember a few tricks:
// initialize structure
field_t *
field_init (uint8_t id, uint16_t len, const char *txt)
{
field_t *f = malloc (sizeof (field_t + len)); // note "+ len";
f->field_id = id;
f->field_length = len;
memcpy (f->field, txt, len);
return f;
}
// send structure
int
field_send (field_t *f, int fd)
{
return write (fd, f, sizeof (*f) + f->field_length); // note "+ f->field_length"
}
I don't think it's standard, though. However, most compilers (GCC && MSVC) should support this. If your compiler does not support zero-sized array, you can use one-element char array - just remember to subtract extra one byte when calculating packet size.