Hi I'm writing a program that sends a set of bytes through a message queue like so ...
#include <sys/msg.h>
#include <stddef.h>
key_t key;
int msqid;
struct pirate_msgbuf pmb = {2, { "L'Olonais", 'S', 80, 10, 12035 } };
key = ftok("/home/beej/somefile", 'b');
msqid = msgget(key, 0666 | IPC_CREAT);
/* stick him on the queue */
msgsnd(msqid, &pmb, sizeof(struct pirate_msgbuf) - sizeof(long), 0);
The above example is a simple program from beejs website that resembles mine.
What I'm doing however is sending a message with a struct like so ...
struct msg_queue{
long message_type;
char * buffer;
}
Now before I send my msg_queue, I created some alternative buffer that contains all sorts of information including null characters and such. Now when I do something like this ...
struct msg_queue my_queue;
my_queue.message_type = 1;
my_queue.buffer = "My message";
msgsnd(mysqid, &pmb, sizeof(struct msg_queue) - sizeof(long), 0);
I have no problems receiving the pointer and reading the values stored at that string. However if I were to do something similar like ...
struct msg_queue my_queue;
my_queue.message_type = 1;
my_queue.buffer = sum_buffer_with_lots_of_weird_values; // of type char *
msgsnd(mysqid, &pmb, sizeof(struct msg_queue) - sizeof(long), 0);
The pointer I pass through my queue to my other process will read garbage and not the values stored. I tried making my arbitrary array as a static char *, but that doesn't help either. How do I properly pass in my buffer through the queue? Thanks.
You shouldn't be sending a pointers to another process, they have no meaning (or point to something very different) in another process' address space.
Message queues aren't great for unbounded data like variable length strings. Change your pointer to a fixed length char array sufficiently big to hold the largest string and copy your string into the array before writing the queue. Or use another type of IPC such as domain socket.
Message Queue is used for inter-process communication.
When you malloc some memory in one process, it only exist in that process memory space not accessible by other process.
when you send that pointer over, you are sending a address space which is not accessible. It may even result in segmentation fault.
One way is to limit your buffer size, if applicable.
struct msg_queue{
long message_type;
char buffer[MAX_LEN];
}
Another way is to send it 2 times. The first msgsnd, sends the size of buffer to expect.
The next send, you send the char array over, using the size of the first send. :)
On receiving end, you first get the size, then receive the buffer.
Other way is to use pipes or socket.
"msgsend()" will only read the bytes in your buffer.
If one of those bytes happens to be a pointer (to some string or object somewhere else) ... guess what - the receiver will just get the binary pointer. Not the data being pointed to.
What you need to do is pack the entire contents of your message into a buffer, then send that linear buffer.
Related
I'm writing an application which reads data from a UART interface. The data is sent in packets. Each packet has a channel associated with it. My application multiplexes received packets into virtual channels (threads) so that every channel can work independently of one another. When I receive a packet I have to do something depending on it's contents and produce a response. The response is sent back using the same UART interface.
The data sent is mostly binary. When I'm reading from the UART interface, I know the size of the packet beforehand, so I can preallocate memory with no problem.
The problem for me is producing a response. I know the maximum size of a packet, so I can create a static buffer when I'm constructing a response. If I we're to work with ASCII characters, instead of binary data, I could rely on NULL terminator to determine how long the data stored in the buffer is. However, I'm working with binary data, so using a NULL byte does not work. Instead, I have to keep a variable storing how many bytes of the buffer is used up already. I was thinking of using a custom data type for storing binary data:
typedef struct {
unsigned char buff[2048];
size_t buff_used;
} binary_data_t;
What would be a standart way of handling this?
Since you know the number of bytes you need to hold a packet, just use a flexible array member:
typedef struct
{
size_t bytes;
unsigned char data[];
} binary_data_t;
(Note that identifiers ending in _t are reserved by POSIX, and you really shouldn't be using them.)
Allocation and reading data (assumes you read() from a file descriptor):
binary_data_t *p = malloc( sizeof( *p ) + numDataBytes );
p->bytes = numDataBytes;
ssize_t bytes_read = read( uartFD, p->data, numDataBytes );
one way of doing it could be to store a pointer to where in your array next byte should be placed.
typedef struct {
unsigned char buff[2048];
char* pData;
} binary_data_t;
// at init
binary_data_t rspMsg;
rspMsg.pData = &rspMsg.buff[0];
// at entering data
*(rspMsg.pData) = data;
rspMsg.pData++;
// at sending data you know the length via
length = rspMsg.pData - &rspMsg.buff[0];
This is one way of solving this.
Can be done in many ways.
I'm working a sort of "restaurant" implementation in C with client-server.
I am trying to send the following structure through a FIFO:
typedef struct {
int numtable; //table number to send answer
char timestamp[20]; //simple timestamp
int order[MENUSZ]; //array of int with dish IDs
} request;
About this struct, I basically send to the server the table number, to "build" the client FIFO name through a template, a timestamp, and order is a simple array filled with randomly chosen integers to "create" a sort of random menu request.
With this setup I didn't have problems, using
write(server_fd, &request, sizeof(request))
I had problems when I wanted to transform the array order[MENUSZ] in a pointer, to make a dynamic array, like this:
typedef struct {
int numtable;
char timestamp[20];
int *order;
} request;
After changing the struct, I used the malloc function to allocate enough space for the array:
request->order = malloc(sizeof(int)*numclients+1);
The array is fullfilled correctly, but for some reason the server can't read from the FIFO after I added this pointer, by doing
read(server_fd, &request, sizeof(request));
I can't figure out why it doesn't work with this pointer. Am I doing something wrong?
The array is fullfilled correctly, but for some reason the server can't read from the FIFO after I added this pointer, by doing
read(server_fd, &request, sizeof(request));
You are transferring your structure, which includes a pointer, and the value of the pointer will be transferred correctly, but it will not point to a valid address in the destination process, neither will there be memory allocated where the pointer points to.
Hence, you need to transfer the array separately and recreate the pointer in the destination process, something like:
read(server_fd, &request, sizeof(request));
/* allocate memory for request->order in the reader process */
request->order = malloc(sizeof(int)*numclients+1);
read(server_fd, request->order, sizeof(int)*numclients+1);
A yet better solution would be to also transfer the size of the array inside your structure.
On the sending side, you then need to send both, the structure and the array contents, something like
write(server_fd, &request, sizeof(request))
write(server_fd, request->order, sizeof(int)*numclients+1));
It is because sizeof(request) no longer tells you the size of the combined structure. Try this
typedef struct {
int numtable;
char timestamp[20];
int order[1];
} request;
When you have a new request
int reqsize = sizeof(request) + sizeof(int) * numclients;
request* req = malloc(reqsize);
This allows you to use req->order[1] to req->order[numclients - 1]. When you send it, use
write(server_fd, reqsize, sizeof(int))
write(server_fd, req, reqsize)
When reading
read(server_fd, &reqsize, sizeof(int))
Then allocate the request before reading
request* req = malloc(reqsize)
read(server_fd, req, reqsize)
This technique uses "the chumminess of C" http://c-faq.com/struct/structhack.html, which, as far as I know, works on all implementations of C.
I am trying to implemante a client- server program communication using TCP socket programming in C.
It is between between two 64bit machines with linux OS installed.
I want to transfer a c-struct between the two processes.
For this I try used a pack - unpack() functioanlity.
please consider the following code snipt
/*---------------------------------------------------------
on the sending side I have:
---------------------------------------------------------*/
struct packet {
int64_t x;
int64_t y;
int64_t q[maxSize];
} __attribute__((packed));
int main(void)
{
// build packet
struct packet pkt;
pkt.x = htonl(324);
pkt.y = htonl(654);
int i;
for(i = 0; i< maxSize; i++){
pkt.q[i] = i; **// I also try pkt.q[i] = htonl(i);**
}
// and then do the send
}
/*-----------------------------------------------------------------------------
in the receiving side:
-----------------------------------------------------------------------------*/
struct packet {
int64_t x;
int64_t y;
int64_t q[maxSize];
} __attribute__((packed));
static void decodePacket (uint8_t *recv_data, size_t recv_len)
{
// checking size
if (recv_len < sizeof(struct packet)) {
fprintf(stderr, "received too little!");
return;
}
struct packet *recv_packet = (struct packet *)recv_data;
int64_t x = ntohl(recv_packet->x);
int64_t y = ntohl(recv_packet->y);
int i;
printf("Decoded: x=%"PRIu8" y=%"PRIu32"\n", x, y);
for(i=0;i<maxSize;i++){
**//int64_t res = ntohl(recv_packet->q[i]); I also try to print res**
printf("%"PRIu32"\n" , recv_packet->q[i]);
}
}
int main(int argc, char *argv[]){
// receive the data and try to call decodePacket()
int8_t *recv_data = (int8_t *)&buf; //buf is the data received
size_t recv_len = sizeof(buf);
**decode_packet(recv_data, recv_len);**
}
//-----------------------------------------------------------------------------
Now the problem is that I am receiving the value of x and y in the struct correctly,
but for the array q in the struct I am receiving a strange number, possible a memory grabage value, (I try to use memset() filling the array by zeros before receiving a data from the other side in which case the value of all zeros is received )
I don't understand why I am not receiving the correct value for the array in struct.
Please Note that I try with and with out htonl() while filling the array before putting in struct,
and on the other side: with and with out ntohl() while decoding the array from struct
Any help will be appreciated,
size_t recv_len = sizeof(buf);
decode_packet(recv_data, recv_len);
This piece of code ensures the wrong size is passed to decode_packet. So when decode_packet goes on to check recv_len < sizeof(struct packet), that test is meaningless - it will always pass, no matter how many bytes were received.
You need to fetch the size from the value returned by the recv call. My best guess is that indeed you're receiving fewer bytes than you're expecting.
While sending and receiving structs is quite convenient, it's often an exercise in futility. Manually serializing data or using some explicit mechanisms is probably the way to go.
You didn't show us the send and recv part, which is more likely to be wrong. My guess is you're receiving first items in the array correctly and they "become" garbage at some point, is it right?
Well, #cnicutar is correct, but let me extend it a little bit...
First of all, when you call send you have to examine the return value and see if all bytes have been transmitted. If your structure is large (for example larger than underlying socket buffer) you'll need more than one call to transmit the whole structure. Same with recv, don't expect you will get the whole message in one recv call, don't expect every recv will receive the same amount of data that was sent by corresponding send call. Always check how many bytes have been received and call recv again if necessary (pointing to the right place in incoming buffer and reducing number of bytes to receive).
So what is probably happening, you don't receive enough data (maybe you don't even transmit all of it) and only beginning of your incoming buffer is being filled. Therefore, the rest of the structure is garbage or (when you call memset) stays initialised with zeros.
Also note both send and recv return ssize_t rather than size_t as negative values are possible (to indicate errors).
I have a simple TCP connection with a server and a client program.
I made a simple struct in both the server and the client to pass as the message:
struct {int c; char** v;} msg;
I am just trying to send the argc and argv (input from terminal) from the client:
int main(int argc, char **argv){
...
msg.c = argc;
msg.v = argv;
sendto(Socket, &msg, sizeof(msg), 0, (struct sockaddr *)&input, sizeof(input));
but when sent to the server I can call msg.c to get the number and I can use that
but if I try to use the array of strings I get a seg fault:
recvfrom(Socket, &msg, sizeof(msg), 0, (struct sockaddr *)&input, &sizep);
printf("%d\n", msg.c);
printf("%s\n", msg.v[2]);
I have tried this with just one char * and I wasn't able to send the string across either.
What am I doing wrong?
The sendto() function doesn't follow pointers, at all. So you're sending your message which consists of an integer, and one pointer, to the other side. The receiver gets a pointer value that points to some random place in memory that doesn't mean anything.
What you need to do is serialize your data into something that can be sent across a socket. That means, no pointers. For example, for a single string you could send the length, followed by the actual bytes of the string. For multiple strings, you could send a count, followed by a number of strings in the same format as a single string.
Once you receive the data you will need to unserialize the data into an array of char * strings, if that's what you need in the receiver.
You cannot send an Array. What you seem to do is to send a pointer to an array. The pointer is a value that points to the beginning of the array in the sender's RAM. However this address is local to one client. If you need to send complex data like an array or classes / structs you usually need to serialize them. There are libraries around to support you. I used Google's Protocol Buffers and liked it; you may want to take a look at it.
I am trying to pass whole structure from client to server or vice-versa. Let us assume my structure as follows
struct temp {
int a;
char b;
}
I am using sendto and sending the address of the structure variable and receiving it on the other side using the recvfrom function. But I am not able to get the original data sent on the receiving end. In sendto function I am saving the received data into variable of type struct temp.
n = sendto(sock, &pkt, sizeof(struct temp), 0, &server, length);
n = recvfrom(sock, &pkt, sizeof(struct temp), 0, (struct sockaddr *)&from,&fromlen);
Where pkt is the variable of type struct temp.
Eventhough I am receiving 8bytes of data but if I try to print it is simply showing garbage values. Any help for a fix on it ?
NOTE: No third party Libraries have to be used.
EDIT1: I am really new to this serialization concept .. But without doing serialization cant I send a structure via sockets ?
EDIT2: When I try to send a string or an integer variable using the sendto and recvfrom functions I am receiving the data properly at receiver end. Why not in the case of a structure? If I don't have to use serializing function then should I send each and every member of the structure individually? This really is not a suitable solution since if there are 'n' number of members then there are 'n' number of lines of code added just to send or receive data.
This is a very bad idea. Binary data should always be sent in a way that:
Handles different endianness
Handles different padding
Handles differences in the byte-sizes of intrinsic types
Don't ever write a whole struct in a binary way, not to a file, not to a socket.
Always write each field separately, and read them the same way.
You need to have functions like
unsigned char * serialize_int(unsigned char *buffer, int value)
{
/* Write big-endian int value into buffer; assumes 32-bit int and 8-bit char. */
buffer[0] = value >> 24;
buffer[1] = value >> 16;
buffer[2] = value >> 8;
buffer[3] = value;
return buffer + 4;
}
unsigned char * serialize_char(unsigned char *buffer, char value)
{
buffer[0] = value;
return buffer + 1;
}
unsigned char * serialize_temp(unsigned char *buffer, struct temp *value)
{
buffer = serialize_int(buffer, value->a);
buffer = serialize_char(buffer, value->b);
return buffer;
}
unsigned char * deserialize_int(unsigned char *buffer, int *value);
Or the equivalent, there are of course several ways to set this up with regards to buffer management and so on. Then you need to do the higher-level functions that serialize/deserialize entire structs.
This assumes serializing is done to/from buffers, which means the serialization doesn't need to know if the final destination is a file or a socket. It also means you pay some memory overhead, but it's generally a good design for performance reasons (you don't want to do a write() of each value to the socket).
Once you have the above, here's how you could serialize and transmit a structure instance:
int send_temp(int socket, const struct sockaddr *dest, socklen_t dlen,
const struct temp *temp)
{
unsigned char buffer[32], *ptr;
ptr = serialize_temp(buffer, temp);
return sendto(socket, buffer, ptr - buffer, 0, dest, dlen) == ptr - buffer;
}
A few points to note about the above:
The struct to send is first serialized, field by field, into buffer.
The serialization routine returns a pointer to the next free byte in the buffer, which we use to compute how many bytes it serialized to
Obviously my example serialization routines don't protect against buffer overflow.
Return value is 1 if the sendto() call succeeded, else it will be 0.
Using the 'pragma' pack option did solved my problem but I am not sure if it has any dependencies ??
#pragma pack(1) // this helps to pack the struct to 5-bytes
struct packet {
int i;
char j;
};
#pragma pack(0) // turn packing off
Then the following lines of code worked out fine without any problem
n = sendto(sock,&pkt,sizeof(struct packet),0,&server,length);
n = recvfrom(sock, &pkt, sizeof(struct packet), 0, (struct sockaddr *)&from, &fromlen);
There is no need to write own serialisation routines for short and long integer types - use htons()/htonl() POSIX functions.
If you don't want to write the serialisation code yourself, find a proper serialisation framework, and use that.
Maybe Google's protocol buffers would be possible?
Serialization is a good idea. You can also use Wireshark to monitor the traffic and understand what is actually passed in the packets.
Instead of serialising and depending on 3rd party libraries its easy to come up with a primitive protocol using tag, length and value.
Tag: 32 bit value identifying the field
Length: 32 bit value specifying the length in bytes of the field
Value: the field
Concatenate as required. Use enums for the tags. And use network byte order...
Easy to encode, easy to decode.
Also if you use TCP remember it is a stream of data so if you send e.g. 3 packets you will not necessarily receive 3 packets. They maybe be "merged" into a stream depending on nodelay/nagel algorithm amongst other things and you may get them all in one recv... You need to delimit the data for example using RFC1006.
UDP is easier, you'll receive a distinct packet for each packet sent, but its a lot less secure.
If the format of the data you want to transfer is very simple then converting to and from an ANSI string is simple and portable.