I am trying to implemante a client- server program communication using TCP socket programming in C.
It is between between two 64bit machines with linux OS installed.
I want to transfer a c-struct between the two processes.
For this I try used a pack - unpack() functioanlity.
please consider the following code snipt
/*---------------------------------------------------------
on the sending side I have:
---------------------------------------------------------*/
struct packet {
int64_t x;
int64_t y;
int64_t q[maxSize];
} __attribute__((packed));
int main(void)
{
// build packet
struct packet pkt;
pkt.x = htonl(324);
pkt.y = htonl(654);
int i;
for(i = 0; i< maxSize; i++){
pkt.q[i] = i; **// I also try pkt.q[i] = htonl(i);**
}
// and then do the send
}
/*-----------------------------------------------------------------------------
in the receiving side:
-----------------------------------------------------------------------------*/
struct packet {
int64_t x;
int64_t y;
int64_t q[maxSize];
} __attribute__((packed));
static void decodePacket (uint8_t *recv_data, size_t recv_len)
{
// checking size
if (recv_len < sizeof(struct packet)) {
fprintf(stderr, "received too little!");
return;
}
struct packet *recv_packet = (struct packet *)recv_data;
int64_t x = ntohl(recv_packet->x);
int64_t y = ntohl(recv_packet->y);
int i;
printf("Decoded: x=%"PRIu8" y=%"PRIu32"\n", x, y);
for(i=0;i<maxSize;i++){
**//int64_t res = ntohl(recv_packet->q[i]); I also try to print res**
printf("%"PRIu32"\n" , recv_packet->q[i]);
}
}
int main(int argc, char *argv[]){
// receive the data and try to call decodePacket()
int8_t *recv_data = (int8_t *)&buf; //buf is the data received
size_t recv_len = sizeof(buf);
**decode_packet(recv_data, recv_len);**
}
//-----------------------------------------------------------------------------
Now the problem is that I am receiving the value of x and y in the struct correctly,
but for the array q in the struct I am receiving a strange number, possible a memory grabage value, (I try to use memset() filling the array by zeros before receiving a data from the other side in which case the value of all zeros is received )
I don't understand why I am not receiving the correct value for the array in struct.
Please Note that I try with and with out htonl() while filling the array before putting in struct,
and on the other side: with and with out ntohl() while decoding the array from struct
Any help will be appreciated,
size_t recv_len = sizeof(buf);
decode_packet(recv_data, recv_len);
This piece of code ensures the wrong size is passed to decode_packet. So when decode_packet goes on to check recv_len < sizeof(struct packet), that test is meaningless - it will always pass, no matter how many bytes were received.
You need to fetch the size from the value returned by the recv call. My best guess is that indeed you're receiving fewer bytes than you're expecting.
While sending and receiving structs is quite convenient, it's often an exercise in futility. Manually serializing data or using some explicit mechanisms is probably the way to go.
You didn't show us the send and recv part, which is more likely to be wrong. My guess is you're receiving first items in the array correctly and they "become" garbage at some point, is it right?
Well, #cnicutar is correct, but let me extend it a little bit...
First of all, when you call send you have to examine the return value and see if all bytes have been transmitted. If your structure is large (for example larger than underlying socket buffer) you'll need more than one call to transmit the whole structure. Same with recv, don't expect you will get the whole message in one recv call, don't expect every recv will receive the same amount of data that was sent by corresponding send call. Always check how many bytes have been received and call recv again if necessary (pointing to the right place in incoming buffer and reducing number of bytes to receive).
So what is probably happening, you don't receive enough data (maybe you don't even transmit all of it) and only beginning of your incoming buffer is being filled. Therefore, the rest of the structure is garbage or (when you call memset) stays initialised with zeros.
Also note both send and recv return ssize_t rather than size_t as negative values are possible (to indicate errors).
Related
What if I had a an array of struct. Is it possible for me to send an array of structs over a socket? The struct size will be updated continuously and at any time I would be able to print out the content of the structs. I hope this makes sense, my explanation might not be cleared. My syntax is definitely not correct for some areas, it's just a snippet of what I think it would look like. I just need some guidance.
This will send the array of structs acorss the socket.
void sendOpenMessage(int num0, int num1, int num2){
struct openMessage{
int num0;
int num1;
int num2
};
struct openMessage open[100];
int i = 0;
open[i].num0 = 1;
open[i].num1 = 2;
open[i].num2 = 3;
int length = sizeof(open);
if(send(socket, &open[i], length, 0) == -1){
fprintf(stderr, "Send() failed");
}else{
printf("Open message is being sent\n");
}
i++;
}
This will receive the struct and display the contents in a message
struct openMessage open[100];
if(recv(clnSocket, &open, sizeof(open), 0) < 0){
fprintf(stderr,"Recv() failed\n");
printf("Error code: %d\n", errno);
}
//Get size of the current struct
//Print out the messages from the structs that have messages?
void printStruct(struct openMessage open){
for(int i = 0; i < sizeof(the struct); i++){
printf("%d\n",open[i].num0);
printf("%d\n", open[i].num1);
printf("%d\n", open[i].num2);
}
}
First of all, you need to read up about byte ordering and serialisation.
Secondly, length is the size of the whole array, so when i is zero you send the whole array, when i is 1 you send all but the first element plus some garbage at the end, and so on.
Thirdly, when receiving you iterate across sizeof(the struct) members of the array. Don't you want to iterate across the whole array?
When you say the struct size will be updated continuously, do you mean the number of elements in the array? Your struct has a fixed size.
If you are sending variable quantities of data, i.e. if you want to send a piece of data that has one length at one time and another length at another time, or even if you want the number of elements in the array to update dynamically, then you MUST send that size across the socket, and send it before the data. Send the size first, read it first on the receiving end, and then the receiving end will know how much more data to read, and more importantly, where the data ends so that if there is more data later it doesn't all get mixed up.
I'm writing an application which reads data from a UART interface. The data is sent in packets. Each packet has a channel associated with it. My application multiplexes received packets into virtual channels (threads) so that every channel can work independently of one another. When I receive a packet I have to do something depending on it's contents and produce a response. The response is sent back using the same UART interface.
The data sent is mostly binary. When I'm reading from the UART interface, I know the size of the packet beforehand, so I can preallocate memory with no problem.
The problem for me is producing a response. I know the maximum size of a packet, so I can create a static buffer when I'm constructing a response. If I we're to work with ASCII characters, instead of binary data, I could rely on NULL terminator to determine how long the data stored in the buffer is. However, I'm working with binary data, so using a NULL byte does not work. Instead, I have to keep a variable storing how many bytes of the buffer is used up already. I was thinking of using a custom data type for storing binary data:
typedef struct {
unsigned char buff[2048];
size_t buff_used;
} binary_data_t;
What would be a standart way of handling this?
Since you know the number of bytes you need to hold a packet, just use a flexible array member:
typedef struct
{
size_t bytes;
unsigned char data[];
} binary_data_t;
(Note that identifiers ending in _t are reserved by POSIX, and you really shouldn't be using them.)
Allocation and reading data (assumes you read() from a file descriptor):
binary_data_t *p = malloc( sizeof( *p ) + numDataBytes );
p->bytes = numDataBytes;
ssize_t bytes_read = read( uartFD, p->data, numDataBytes );
one way of doing it could be to store a pointer to where in your array next byte should be placed.
typedef struct {
unsigned char buff[2048];
char* pData;
} binary_data_t;
// at init
binary_data_t rspMsg;
rspMsg.pData = &rspMsg.buff[0];
// at entering data
*(rspMsg.pData) = data;
rspMsg.pData++;
// at sending data you know the length via
length = rspMsg.pData - &rspMsg.buff[0];
This is one way of solving this.
Can be done in many ways.
Hi I'm writing a program that sends a set of bytes through a message queue like so ...
#include <sys/msg.h>
#include <stddef.h>
key_t key;
int msqid;
struct pirate_msgbuf pmb = {2, { "L'Olonais", 'S', 80, 10, 12035 } };
key = ftok("/home/beej/somefile", 'b');
msqid = msgget(key, 0666 | IPC_CREAT);
/* stick him on the queue */
msgsnd(msqid, &pmb, sizeof(struct pirate_msgbuf) - sizeof(long), 0);
The above example is a simple program from beejs website that resembles mine.
What I'm doing however is sending a message with a struct like so ...
struct msg_queue{
long message_type;
char * buffer;
}
Now before I send my msg_queue, I created some alternative buffer that contains all sorts of information including null characters and such. Now when I do something like this ...
struct msg_queue my_queue;
my_queue.message_type = 1;
my_queue.buffer = "My message";
msgsnd(mysqid, &pmb, sizeof(struct msg_queue) - sizeof(long), 0);
I have no problems receiving the pointer and reading the values stored at that string. However if I were to do something similar like ...
struct msg_queue my_queue;
my_queue.message_type = 1;
my_queue.buffer = sum_buffer_with_lots_of_weird_values; // of type char *
msgsnd(mysqid, &pmb, sizeof(struct msg_queue) - sizeof(long), 0);
The pointer I pass through my queue to my other process will read garbage and not the values stored. I tried making my arbitrary array as a static char *, but that doesn't help either. How do I properly pass in my buffer through the queue? Thanks.
You shouldn't be sending a pointers to another process, they have no meaning (or point to something very different) in another process' address space.
Message queues aren't great for unbounded data like variable length strings. Change your pointer to a fixed length char array sufficiently big to hold the largest string and copy your string into the array before writing the queue. Or use another type of IPC such as domain socket.
Message Queue is used for inter-process communication.
When you malloc some memory in one process, it only exist in that process memory space not accessible by other process.
when you send that pointer over, you are sending a address space which is not accessible. It may even result in segmentation fault.
One way is to limit your buffer size, if applicable.
struct msg_queue{
long message_type;
char buffer[MAX_LEN];
}
Another way is to send it 2 times. The first msgsnd, sends the size of buffer to expect.
The next send, you send the char array over, using the size of the first send. :)
On receiving end, you first get the size, then receive the buffer.
Other way is to use pipes or socket.
"msgsend()" will only read the bytes in your buffer.
If one of those bytes happens to be a pointer (to some string or object somewhere else) ... guess what - the receiver will just get the binary pointer. Not the data being pointed to.
What you need to do is pack the entire contents of your message into a buffer, then send that linear buffer.
I am currently trying to port a C program that deals with datagram (UDP) packets to some higher-level language. As the packets can be of variable size, they start with an integer stating their size. In c, I call recv with the MSG_PEEK flag to first receive this value only, then allocate a fitting buffer and read the rest of the packet. The code (simplified) goes like this:
// Simplified message format.
struct message {
int length;
char[] text;
}
struct message *m = malloc (sizeof(int));
// Read out in just length.
recv (sock, m, sizeof(int), MSG_WAITALL | MSG_PEEK);
int txtlen = ntohl (m->length) * sizeof(char);
int msglen = sizeof(int) + txtlen;
// Read complete packet.
m = realloc (m, msglen);
read (sock, m, msglen);
m->text[txtlen] = '\0';
// Show result.
printf("%s\n", &m->text);
I want to avoid the seemingly common practice to allocate an enormous buffer and hope that no bigger packets will arrive. So is something like peeking at the datagram or determining its complete length beforehand possible in higher-level languages like python or java?
I want to avoid the seemingly common practice to allocate an enormous buffer and hope that the packet wont get any bigger.
Not sure what you mean by this. A UDP packet arrives all at once, so the initial integer tells you exactly how big your buffer should be; it won't "grow" after it arrives.
Since you're appending a null character, you need to account for that in your length calculation:
int msglen = sizeof(int) + txtlen + 1;
Be careful when you use realloc():
m = realloc (m, msglen);
If the realloc fails it will set m to null. That means you'll lose your only reference to the memory that was originally allocated to it, so you'll never be able to free() it. Try something like this:
void *tmp = realloc(m, msglen)
if (tmp == null) {
// handle the error
}
m = tmp;
And when you print the data, m->text evaluates to the address of the first character, so you can use
printf("%s\n", m->text);
Alternatively, you could define your structure with a fixed size, as
struct message {
int length;
char *text;
}
Then you can use malloc() to allocate (only) your text buffer:
struct message m;
recv(sock, &m.length, sizeof(int), MSG_WAITALL | MSG_PEEK);
m.text = malloc(m.length + 1); // +1 for the null that you'll append
read(sock, m.text, m.length);
m.text(m.length) = '\0';
printf("%s\n", m.text);
free(m.text);
Good luck with your project--network programming is always a learning experience!
Why not do this?
message = (struct message *)malloc(sizeof(struct message));
read(sock, &message->length, sizeof(int);
message->length = ntohl(message->length);
message->text = (char *)malloc(message->length + 1);
read(sock, message->text, message->length);
message->text[message->length] = 0;
UDP datagrams are limited to 64K, then ethernet frames are 1500 bytes (unless your network is using jumbo frames, which could be up to 9000 bytes). Protocol designers usually try to avoid IP fragmentation, so most likely your incoming packets are small, i.e. less then 1500 bytes.
I would just start with static buffer of 1472 (1500 ethernet frame length - 20 bytes of IP header - 8 bytes of UDP header). If you have to deal with some arbitrary protocols - bump that up to 64K. If you cannot afford that - collect actual sizes with MSG_PEEK, find some convenient average, and setup a fall-back plan with malloc(3).
I am trying to pass whole structure from client to server or vice-versa. Let us assume my structure as follows
struct temp {
int a;
char b;
}
I am using sendto and sending the address of the structure variable and receiving it on the other side using the recvfrom function. But I am not able to get the original data sent on the receiving end. In sendto function I am saving the received data into variable of type struct temp.
n = sendto(sock, &pkt, sizeof(struct temp), 0, &server, length);
n = recvfrom(sock, &pkt, sizeof(struct temp), 0, (struct sockaddr *)&from,&fromlen);
Where pkt is the variable of type struct temp.
Eventhough I am receiving 8bytes of data but if I try to print it is simply showing garbage values. Any help for a fix on it ?
NOTE: No third party Libraries have to be used.
EDIT1: I am really new to this serialization concept .. But without doing serialization cant I send a structure via sockets ?
EDIT2: When I try to send a string or an integer variable using the sendto and recvfrom functions I am receiving the data properly at receiver end. Why not in the case of a structure? If I don't have to use serializing function then should I send each and every member of the structure individually? This really is not a suitable solution since if there are 'n' number of members then there are 'n' number of lines of code added just to send or receive data.
This is a very bad idea. Binary data should always be sent in a way that:
Handles different endianness
Handles different padding
Handles differences in the byte-sizes of intrinsic types
Don't ever write a whole struct in a binary way, not to a file, not to a socket.
Always write each field separately, and read them the same way.
You need to have functions like
unsigned char * serialize_int(unsigned char *buffer, int value)
{
/* Write big-endian int value into buffer; assumes 32-bit int and 8-bit char. */
buffer[0] = value >> 24;
buffer[1] = value >> 16;
buffer[2] = value >> 8;
buffer[3] = value;
return buffer + 4;
}
unsigned char * serialize_char(unsigned char *buffer, char value)
{
buffer[0] = value;
return buffer + 1;
}
unsigned char * serialize_temp(unsigned char *buffer, struct temp *value)
{
buffer = serialize_int(buffer, value->a);
buffer = serialize_char(buffer, value->b);
return buffer;
}
unsigned char * deserialize_int(unsigned char *buffer, int *value);
Or the equivalent, there are of course several ways to set this up with regards to buffer management and so on. Then you need to do the higher-level functions that serialize/deserialize entire structs.
This assumes serializing is done to/from buffers, which means the serialization doesn't need to know if the final destination is a file or a socket. It also means you pay some memory overhead, but it's generally a good design for performance reasons (you don't want to do a write() of each value to the socket).
Once you have the above, here's how you could serialize and transmit a structure instance:
int send_temp(int socket, const struct sockaddr *dest, socklen_t dlen,
const struct temp *temp)
{
unsigned char buffer[32], *ptr;
ptr = serialize_temp(buffer, temp);
return sendto(socket, buffer, ptr - buffer, 0, dest, dlen) == ptr - buffer;
}
A few points to note about the above:
The struct to send is first serialized, field by field, into buffer.
The serialization routine returns a pointer to the next free byte in the buffer, which we use to compute how many bytes it serialized to
Obviously my example serialization routines don't protect against buffer overflow.
Return value is 1 if the sendto() call succeeded, else it will be 0.
Using the 'pragma' pack option did solved my problem but I am not sure if it has any dependencies ??
#pragma pack(1) // this helps to pack the struct to 5-bytes
struct packet {
int i;
char j;
};
#pragma pack(0) // turn packing off
Then the following lines of code worked out fine without any problem
n = sendto(sock,&pkt,sizeof(struct packet),0,&server,length);
n = recvfrom(sock, &pkt, sizeof(struct packet), 0, (struct sockaddr *)&from, &fromlen);
There is no need to write own serialisation routines for short and long integer types - use htons()/htonl() POSIX functions.
If you don't want to write the serialisation code yourself, find a proper serialisation framework, and use that.
Maybe Google's protocol buffers would be possible?
Serialization is a good idea. You can also use Wireshark to monitor the traffic and understand what is actually passed in the packets.
Instead of serialising and depending on 3rd party libraries its easy to come up with a primitive protocol using tag, length and value.
Tag: 32 bit value identifying the field
Length: 32 bit value specifying the length in bytes of the field
Value: the field
Concatenate as required. Use enums for the tags. And use network byte order...
Easy to encode, easy to decode.
Also if you use TCP remember it is a stream of data so if you send e.g. 3 packets you will not necessarily receive 3 packets. They maybe be "merged" into a stream depending on nodelay/nagel algorithm amongst other things and you may get them all in one recv... You need to delimit the data for example using RFC1006.
UDP is easier, you'll receive a distinct packet for each packet sent, but its a lot less secure.
If the format of the data you want to transfer is very simple then converting to and from an ANSI string is simple and portable.