Prepending designed header to UDP msg - c

I'm trying to build a reliable UDP file transfer from server to client using c sockets. I know how to use UDP socket to send a file.
However, now I need to concatenate my reliable header along with the data to sent it.
My designed header is defined as follows:
struct RudpHeader
{
int seqNo;
int ackNo;
int ackFlag;
int advWin;
int finFlag;
}
Here is the part the sends the file
int remBytes = (int) myFileSize;
char msg[1024];
while (remBytes > 0)
{
bytesRead = fread(msg, 1, sizeof(msg), file);
remBytes = remBytes - bytesRead;
// Here I want to append the header to msg
n = sendto(socketFD, msg, sizeof(msg), 0,(struct sockaddr *)&clientAdd, cAddLen);
if (n < 0)
printf("Failed to send to client. \n");
bzero(msg, sizeof(msg));
}
So my questions are:
1) What is the best to prepend the header to the message? msg is an array of strings and the header is struct.
2) at the receiver part what is the best way to separate the header from the content.

well i'm no expert in C, but here is how I would do it:
#define HEADER_SIZE 13
struct RudpHeader
{
int seqNo;
int ackNo;
int advWin;
unsigned char ackFlag:1;
unsigned char finFlag:1;
};
int main()
{
struct RudpHeader header;
char* packet_data = calloc(1, HEADER_SIZE + 1024);
char* buffer = &packet_data[HEADER_SIZE]; //we can put data in here
header.seqNo = 100;
header.ackNo = 3542;
header.ackFlag = 1;
header.finFlag = 1;
//convert everything to network order before sending
header.seqNo = htonl(header.seqNo);
header.ackNo = htonl(header.ackNo);
header.advWin = htonl(header.advWin);
memcpy(&packet_data[0], &header.seqNo, 4);
memcpy(&packet_data[4], &header.ackNo, 4);
memcpy(&packet_data[8], &header.advWin, 4);
//no need to waste an int to send a bit, lets convert it
unsigned char flags = 0;
flags |= header.ackFlag;
flags |= header.finFlag << 1;
memcpy(&packet_data[12], &flags, 1);
//put some data in the buffer
char* msg = "Hello World!\0";
strcpy(buffer, msg);
//pretend you just received the packet and convert everything
//back to the hosts byte order
printf("Sequence Number: %d\nAcknowlegement Number: %d\nAdvWindow: %d\nFlags: %d\n",
ntohl(header.seqNo), ntohl(header.ackNo), ntohl(header.advWin), flags);
printf("Ack set: %d\nFin set %d\n", (flags&1), (flags&2));
printf("data: \"%s\"\n", buffer);
//now you can send the entire packet using variable 'packet_data'
//sendto(socket, packet_data, HEADER_SIZE + data_length, 0, sockaddr);
//either reuse the packet or destroy it(or just put it on the stack)
free(packet_data);
return 0;
}
note that I changed how the struct looks, a flag is not a integer so no need to waste data on the header. HEADER_SIZE == 4+4+4+1 which is 3 ints, 1 byte.
you should never ever send a struct across the network, and never assume the two machines have the same byte order. htonl converts a 32 bit number to network byte order and ntohl converts it to host order. htons and ntohs is the same thing, it just converts 16 bits instead. also when you receive a packet the buffer size will be (length of packet) - header. I hope I commented enough to make it obvious what is going on here.

The proper way to do this is to incorporate your header into the message. Good practice is to put the header length and version as the very first piece of data, this way you will know if the client and server are on the same version of protocol.
Once you've read the message, you can extract header from it, compare versions and perform actions based on the header contents.

Take a look at this discussion. It provides a very elegant and portable way to send and receiver a datagram with a structured header and data.

Related

How to fix a segmentation fault for Ansi C Tcp client?

I'm trying to expand an example of a Tcp client developed using Ansi C, following the book "TCP/IP Sockets in C". The client connects to a Tcp Server providing strings of different lengths depending on the request provided by the client (I developed my own simple protocol). When the returned strings are short in length, everything works fine. When they're over a certain length (it happens for example with 4KB), the client crashes with a Segmentation Fault error.
The socket is handled using a wrapper to stream the i/o:
FILE *str = fdopen(sock, "r+"); // Wrap for stream I/O
And the transmission and reception are handled using fwrite() and fread().
This is the call that generates the error in my project (the caller):
uint8_t inbuf[MAX_WIRE_SIZE];
size_t respSize = GetNextMsg(str, inbuf, MAX_WIRE_SIZE); // Get the message
And this is the implementation of the GetNextMsg() function, that use to receive the data and unframe it:
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <netinet/in.h>
#include "Practical.h"
/* Read 4-byte length and place in big-endian order.
* Then read the indicated number of bytes.
* If the input buffer is too small for the data, truncate to fit and
* return the negation of the *indicated* length. Thus a negative return
* other than -1 indicates that the message was truncated.
* (Ambiguity is possible only if the caller passes an empty buffer.)
* Input stream is always left empty.
*/
uint32_t GetNextMsg(FILE *in, uint8_t *buf, size_t bufSize)
{
uint32_t mSize = 0;
uint32_t extra = 0;
if (fread(&mSize, sizeof(uint32_t), 1, in) != 1)
return -1;
mSize = ntohl(mSize);
if (mSize > bufSize)
{
extra = mSize - bufSize;
mSize = bufSize; // Truncate
}
if (fread(buf, sizeof(uint8_t), mSize, in) != mSize)
{
fprintf(stderr, "Framing error: expected %d, read less\n", mSize);
return -1;
}
if (extra > 0)
{ // Message was truncated
uint32_t waste[BUFSIZE];
fread(waste, sizeof(uint8_t), extra, in); // Try to flush the channel
return -(mSize + extra); // Negation of indicated size
}
else
return mSize;
}
I suspect that this could be related to the fact that with Tcp, sender and receiver are handling data with a streaming behavior, therefore it's not granted that the receiver
gets all of the data at once, as the simple example from which I started probably assumed. In fact, with short strings everything works. With longer strings, it doesn't.
I've done a simplified debug inserting a printf as a first thing inside of the function, but when I have the crash this doesn't even get printed.
It seems like an issue with the FILE *str passed as an argument to the function, when
via the socket a message longer than usual is received.
The buffers are sized far bigger than the length of the message causing the issue (1MB vs 4KB).
I've even tried to increase the size of the socket buffer via the setsockopt:
int rcvBufferSize;
// Retrieve and print the default buffer size
int sockOptSize = sizeof(rcvBufferSize);
if (getsockopt(sock, SOL_SOCKET, SO_RCVBUF, &rcvBufferSize, (socklen_t*)&sockOptSize) < 0)
DieWithSystemMessage("getsockopt() failed");
printf("Initial Receive Buffer Size: %d\n", rcvBufferSize);
// Double the buffer size
rcvBufferSize *= 10;
if (setsockopt(sock, SOL_SOCKET, SO_RCVBUF, &rcvBufferSize,
sizeof(rcvBufferSize)) < 0)
DieWithSystemMessage("setsockopt() failed");
but this didn't help.
Any ideas about the reason and how could I fix it?
This code:
{ // Message was truncated
uint32_t waste[BUFSIZE];
fread(waste, sizeof(uint8_t), extra, in); // Try to flush the channel
reads extra bytes into a buffer of size 4*BUFSIZE (4 because you intended to make the buffer unit8_t, but accidentally made it uint32_t instead).
If extra is larger than 4*BUFSIZE, then you will have a local buffer overflow and stack corruption, possibly resulting in a crash.
To do this correctly, something like this is needed:
int remaining = extra;
while (remaining > 0) {
char waste[BUFSIZE];
int to_read = min(BUFSIZE, remaining);
int got = fread(waste, 1, to_read, in);
if (got <= 0) break;
remaining -= got;
}

How to send a size_t variable with a TCP socket in C?

I'm working on something that sends data to a TCP server, but first it is supposed to send the size of the data in 8 bytes.
That is, the server will read the first 8 bytes sent to it and cast them back into a size_t variable. My problem is, when there is a file size that doesn't use any of the top bits (i.e. 83 = 0000000S <- char's, not hex), it only sends the non-zero bytes.
This is how I do it:
void send_file_to_server(char *filename){
struct stat buf;
if (stat(filename, &buf)==-1){ exit(1); }
size_t file_size = buf.st_size;
char *filesize_string = calloc(1, 8);
filesize_string = (char*)&file_size;
//this function actually writes to the server
write_to_server((char*) filesize_string);
// will be code later on that sends the actual file using write_to_server()
}
The char* passed into my write_to_server() function has some weird behavior: it only recognizes it as a string of size 6, and it gets distorted from before it was passed in. Any advice on how to make this work is appreciated.
Note: I do not have to worry about endianness (htonl, etc.) or a differing size of size_t since this is for a project that will only ever be run on a specific VM.
Edits:
here is the other function:
void write_to_server(char *message){
ssize_t bytes_sent = 0;
ssize_t message_size = strlen(message);
while ( bytes_sent < message_size ){
ssize_t ret = write(server_socket, message+bytes_sent, message_size-bytes_sent);
if (ret==0){
print_connection_closed();
exit(1);
}
if (ret==-1 && (errno!=EINTR || errno!=EAGAIN)){
printf("write failed: sent %zd bytes out of %zd\n", bytes_sent, message_size);
exit(1);
}
if (ret!=-1){ bytes_sent+=ret; }
}
}
You can't use strlen() to determine the length of binary data. It'll miscount the data as soon as it sees a zero (NUL) byte in the binary encoding of the length field.
Write a more "primitive" function that takes the address of the data and its length as parameters, e.g.
void write_to_server_raw(const void *message, size_t message_size) {
...
}
If you still need the ability to send NUL terminated strings you can then rewrite your existing write_to_server() function so that it calls the new function to do the real work.
void write_to_server_string(const char *message) {
size_t message_size = strlen(message);
write_to_server_raw(message, message_size);
}

Sending fragmented datagram with UDP header on every fragment

I am working with an embedded box that must be able to communicate with traditional computers using UDP. When the box sends large UDP messages (that need to be fragmented), a UDP header is included for each fragment. Thus if I want to a send a large datagram, it will be fragmented like this:
[eth hdr][ip hdr][udp hdr][ data 1 ] /* first fragment */
[eth hdr][ip hdr][udp hdr][ data 2 ] /* second fragment */
[eth hdr][ip hdr][udp hdr][ data 3 ] /* last fragment */
I understand that this is not customary, as usually the udp header would only be included in only the first ip packet of the fragmented message. However, this works perfectly for communicating with the other machines I need to talk to (ex. using recvfrom), so I have no reason to dig in and try to change it.
My issue, however, is in reading messages. The box seems to expect fragmented udp datagrams to be sent to it in the same manner. By this I mean that it expects every ipv4 fragment to have a udp header. Before trying to change this (it's a rather specialized and complicated platform) I would like to know if there is any way to configure sendto() or any other such function for sending udp messages in this format. I see when monitoring the traffic that those udp headers aren't present.
Thank you very much for the help.
No. Socket's don't work this way. Just write your own sendto wrapper to manually fragment the frames across multiple UDP packets on whatever buffer size boundary you choose. This will achieve the desired effect that you want.
Sample code as follows:
ssize_t fragmented_sendto(int sockfd, const void *buf, size_t len, int flags,
const struct sockaddr *dest_addr, socklen_t addrlen, size_t MAX_PACKET_SIZE)
{
unsigned char* ptr = (unsigned char*) buf;
size_t total = 0;
while (total <= len)
{
size_t newsize = len - total;
if (newsize > MAX_PACKET_SIZE)
{
newsize = MAX_PACKET_SIZE;
}
ssize_t result = sendto(sockfd, ptr, newsize, flags, dest_addr, addrlen);
if (result < 0)
{
// handle error
return -1;
}
else
{
total += result;
ptr += result;
}
}
return (ssize_t)total;
}

some EOF mechanism for send(),recv() in C

I wonder, If i do 4 send() with my client, written in C, will my server need to read() 4 times, or could it be that the first read will read all 4 send()'s all together?
To help you out a little, you've basically answered your own question already.
You need to do it 'in tiers', so to speak.
Client:
int len = strlen(Filename) + 1; //Mind the terminating 0.
send(sock, (const char *)&len, sizeof(int), 0);
send(sock, Filename, len, 0); //Sending the filename
send(sock, &FileSize, sizeof(int), 0);
send(sock, FileBuf, FileSize, 0);
This code will send the entire data on the way (assuming that the entire file is in the 'FileBuf'-Variable).
Server:
int len;
char *FileBuf, FileName[20];
recv(sock, &len, sizeof(int), 0); //Receives the filename length. (4 Bytes)
recv(sock, FileName, len, 0); //Receives the filename (x bytes)
recv(sock, &len, sizeof(int), 0); //Receives the file length (again, 4 bytes)
FileBuf = new char[len]; //Creates sufficient space in memory.
recv(sock, FileBuf, len, 0); //Receives the file into the appropriate variable.
This is the absolute barebones variant, not very solid, but you should get the idea.
A more robust approach requires you to check the return values of recv() and send(). Both will return the amount of bytes, which have been processed with this call. If this amount equals '0', it means that the connection has been closed by the other end. (Mainly for recv()). If it equals -1, it means that something went wrong and you should check the errno variable.
If all goes well, it equals the exact amount of bytes you sent/tried to receive.
However in case it isn't 'len' (or 0 or -1), you could write a little wrapper like this.
unsigned char Recv(int sock, void *target, int Len) {
unsigned char *ptr = (unsigned char *)target, Ret = 0;
int RecvBytes = 1;
while(Len && !Ret) {
RecvBytes = recv(sock, ptr, Len, 0);
if(!RecvBytes) Ret = 1;
else if(RecvBytes == -1) Ret = errno;
else {
Len -= RecvBytes;
ptr += RecvBytes;
}
}
return Ret;
}
What this code does: It keeps receiving, until you either have received all the data you were expecting (the Len parameter) or an error occurred. If all goes well, it returns '0', which you can check with if(!Recv()).
Another useful wrapper function (a shortcut, so to speak) is this one:
uint32_t RecvInt(int sock) {
uint32_t Ret;
Recv(sock, &Ret, sizeof(Ret));
return ntohl(Ret);
}
This function receives exactly one unsigned int and corrects the endianess from network byte order to host byte order. (Network byte order is always big endian, host byte order is often: Little endian)
Using these wrapper functions, the code may be changed like this:
uint32_t len;
char *FileBuf, FileName[20];
len = RecvInt(sock); //Receives the filename length. (4 Bytes)
Recv(sock, FileName, len); //Receives the filename (x bytes)
len = RecvInt(sock); //Receives the file length (again, 4 bytes)
FileBuf = new char[len]; //Creates sufficient space in memory.
Recv(sock, FileBuf, len); //Receives the file into the appropriate variable.
For stream sockets (e.g. TCP): it makes no difference how many send() or write() calls were made on the sending end. The data could be returned in as few as one chunk, as many as n chunks of 1 byte each (where n is the number of bytes that were sent), or anything in between.
For datagram sockets (e.g. UDP): each recv() or recvmsg() call will return one complete datagram that was sent from the other end. The number of recv() or recvmsg() calls should be the same as the number of datagrams that were sent. recv() is preferred when reading from datagram sockets, but I believe read() should behave the same.
The number of write()s and read() need not be the same - it's possible that the write() writes all the data in one piece but on the other computer read() only manages to receive it in several chunks, and vice versa. That's why you should always check the return value of these functions and if only partial data transfer occurred, then continue with sending/receiving the rest.

How would I go about sending bit-fields through a socket?

I need to send a set of bit fields along with a string of characters from a client to a server.
So given I have:
#define YES 1
#define FLAG 2
int main(int argc, char* argv[])
{
return sendToServer("The Message", YES | FLAG);
}
int sendToServer(char* msg, int bitfields)
{
/* create socket and connect to server */
/* Assume sock is set */
send(sock, msg, strlen(msg), 0);
return 0;
}
What would be the best way to send the bitfields? Is there anyway to send the bitfields along with the string?
EDIT: Ok I'm trying to implement Vlad's method. My client is pretty much identical to what he wrote. I have put the flag at the beginning data[0] and I used htonl instead of bswap. My server:
int main(int argc, char* argv[])
{
/* create socket and wait for connection */
char buffer[BUFFERSIZE];
size_t rcvdB = recv(clntSock, buffer, sizeof(int),0);
int flags = ntohl((int) buffer);
rcvdB = recv(clntSock, buffer, sizeof(size_t),0);
size_t msgSize = ntohl((size_t) buffer);
rcvdB = recv(clntSock,buffer,msgSize,0);
/* Then I send back to the client */
ssize_t sntB = send(clntSock,buffer,msgSize,0);
}
When the client prints the message there are multiple ascii characters at the end of the message.
EDIT2:
The issue seems to occur when I read more than 8 bytes of data
As others have pointed out, it depends on a protocol. I'd use something like this:
int sendToServer(char* msg, int bitfields)
{
unsigned int bits = bswap32 (bitfields); // Convert host to network byte order.
size_t len = strlen (msg); // Calculate string length.
size_t nlen = bswap64 (len); // Convert length's byte order from host to network.
iovec data[3]; // Prepare 3 I/O buffers to send data by performing a single system call (they are expensive).
// Send length of the string first.
data[0].iov_base = &nlen;
data[0].iov_len = sizeof (nlen);
// Send string...
data[1].iov_base = msg;
data[1].iov_len = len;
// And, of course, send bits.
data[2].iov_base = &bits;
data[2].iov_len = sizeof (bits);
// Write all of those to the socket.
writev (fd, &data, sizeof (data) / sizeof (data[0]));
}
On the receiving side, you can read the first sizeof (size_t) bytes, convert from network to host byte order, cast to size_t. That will tell you length of string. Then read buffer of that length - that will be your string. Finally, read another sizeof (int) bytes - that will be your bitfield.
See also:
Endianess
writev
This depends on your network protocol, of course. If you're designing it, and you have a maximum of 8 flags to send, put them in the first byte of the message. If you have at most 16, but them in the first two bytes, etc.
int sendToServer(int sock, char* msg, int flags)
{
size_t siz = strlen(msg) + 1; // +1 for NUL
unsigned char *buf = malloc_or_die(siz + 1);
buf[0] = flags;
memcpy(buf + 1, msg, size);
send(sock, msg, strlen(msg), 0);
free(buf);
return 0;
}
(I changed bitfields to flags since that's what they are. A bitfield is a collection of flags.)
It's probably best to send the bitfields as the bytes for a raw int (e.g., send(sock, &bitfields, sizeof(bitfields), 0)), making sure that the size & endianness is the same for both the client & server. It'd probably be easiest to send the bits before the string, as the server could then easily extract the string after a fixed number of bytes.
The bitfield forms a number, for instance 1|2 gives 3 since %01 | %10 = %11.
So all you have to send is that number. Kind of
snprintf(buffer, buffer_size, "%s-%d", message, number);
send(sockfd, buffer, buffer_size, 0);

Resources