I'm working on a client-server app. My app is working with variable size packets, each packet has a header and a payload of variable length.
My dilemma is what is the best approach for handing the packets when doing recv.
Most of the tutorials I've came across suggest using a ring buffer but as far as I can tell it's more efficient to use a buffer whose size is twice the size of the biggest packet you can handle.
If I use a ring buffer I need an additional buffer for recv and then I need to copy the buffer in the ring buffer which means I need to do one or two memcpys to insert the buffer in the ring buffer
If I use the single buffer approach I only need one buffer which I can pass to recv call and a memmove call to move data to the start of the buffer when I got a full packet and there still is data belonging to another packet in the buffer.
Am I getting something wrong ?
PS. If you can point me to any source code/example where variable length packets are handled that would be helpful.
If I use a ring buffer I need an additional buffer for recv and then I need to copy the buffer in the ring buffer which means I need to do one or two memcpys to insert the buffer in the ring buffer
Yeah, two reads and writes, no big deal (*). But you don't need an additional buffer. For read just maximize read to the space left till the end of the ring buffer.
(*): If you are worried about the cost of additional syscall, for scatter/gather read/write there is recvmsg and sendmsg.
Related
I'm doing an assignment where the goal is to create a basic FTP server in C capable of handling multiple clients at once.
The subject tells us to "wisely use circular buffers" but I don't really understand why or how ?
I'm already using select to know when I can read or write into my socket without blocking as I'm not allowed to use recv, send or O_NONBLOCKING.
Each connection has a structure where I store everything related to this client like the communication file descriptor, the network informations and the buffers.
Why can't I just use read on my socket into a fixed size buffer and then pass this buffer to the parsing function ?
Same goes for writing : why can't I just dprintf my response into the socket ?
From my point of view using a circular buffer adds a useless layer of complexity just to be translated back into a string to parse the command or to send back the response.
Did I misunderstood the subject ? Instead of storing individual characters should I store commands and responses as circular buffers of strings ?
Why should I use circular buffers when reading and writing to sockets in C?
The socket interface does not itself provide a reason for using circular buffers (a.k.a. ring buffers). You should be looking instead at the protocol requirements of the application using the socket -- the FTP protocol in this case. This will be colored by the characteristics of the underlying network protocol (TCP for FTP) and their effect on the behavior of the socket layer.
Why can't I just use read on my socket into a fixed size buffer and then pass this buffer to the parsing function ?
You surely could do without circular buffers, but that wouldn't be as simple as you seem to suppose. And that's not the question you should be asking anyway: it's not whether circular buffers are required, but what benefit they can provide that you might not otherwise get. More on that later.
Also, you surely can have fixed size circular buffers -- "circular" and "fixed size" are orthogonal characteristics. However, it is usually among the objectives of using a circular buffer to minimize or eliminate any need for dynamically adjusting the buffer size.
Same goes for writing : why can't I just dprintf my response into the socket ?
Again, you probably could do as you describe. The question is what do you stand to gain from interposing a circular buffer? Again, more later.
From my point of view using a circular buffer adds a useless layer of
complexity just to be translated back into a string to parse the
command or to send back the response.
Did I misunderstood the subject ?
That you are talking about translating to and from strings makes me think that you did indeed misunderstand the subject.
Instead of storing individual
characters should I store commands and responses as circular buffers
of strings ?
Again, where do you think "of strings" comes into it? Why are you supposing that the elements of the buffer(s) would represent (whole) messages?
A circular buffer is more a manner of use of an ordinary, flat, usually fixed-size buffer than it is a separate data structure of its own. There is a little bit of extra bookkeeping data involved, however, so I won't quibble with anyone who wants to call it a data structure in its own right.
Circular buffers for input
Among the main contexts for circular buffers' usefulness is data arriving with stream semantics (such as TCP provides) rather than with message semantics (such as UDP provides). With respect to your assignment, consider this: when the server reads command input, how does it know where the command ends? I suspect you're supposing that you will get one complete command per read(), but that is in no way a safe assumption, regardless of the implementation of the client. You may get partial commands, multiple commands, or both on each read(), and you need to be prepared to deal with that.
So suppose, for example, that you receive one and a half control messages in one read(). You can parse and respond to the first, but you need to read more data before you can act on the second. Where do you put that data? Ok, you read it into the end of the buffer. And what if on the next read() you get not only the rest of a message, but also part of another message?
You cannot keep on indefinitely adding data at the end of the buffer, not even if you dynamically allocate more space as needed. You could at some point move the unprocessed data from the tail of the buffer to the beginning, thus opening up space at the end, but that is costly, and at this point we are well past the simplicity you had in mind. (That simplicity was always imaginary.) Alternatively, you can perform your reads into a circular buffer, so that consuming data from the (logical) beginning of the buffer automatically makes space available at the (logical) end.
Circular buffers for output
Similar applies on the writing side with a stream-oriented network protocol. Consider that you cannot write() an arbitrary amount of data at a time, and it is very hard to know in advance exactly how much you can write. That's more likely to bite you on the data connection than on the control connection, but in principle, it applies to both. If you have only one client to feed at a time then you can keep write()ing in a loop until you've successfully transferred all the data, and this is what dprintf() would do. But that's potentially a blocking operation, so it undercuts your responsiveness when you are serving multiple clients at the same time, and maybe even with just one if (as with FTP) there are multiple connections per client.
You need to buffer data on the server, especially for the data connection, and now you have pretty much the same problem that you did on the reading side: when you've written only part of the data you want to send, and the socket is not ready for you to send more, what do you do? You could just track where you are in the buffer, and send more pieces as you can until the buffer is empty. But then you are wasting opportunities to read more data from the source file, or to buffer more control responses, until you work through the buffer. Once again, a circular buffer can mitigate that, by giving you a place to buffer more data without requiring it to start at the beginning of the buffer or being limited by the available space before the physical end of the buffer.
I need to write a couple of C++ applications on Linux, one to receive data via UDP and the second TCP.
The only thing I'm unsure about is regarding the buffer.
How do I choose what size buffer?
If I make the buffer large enough, am I guaranteed to avoid scenarios where half of a packet is at the end of my buffer and I need to copy the bytes to the beginning and then receive the remaining half of the packet?
I am going to use the Linux socket API functions if it matters.
If I make the buffer large enough, am I guaranteed to avoid scenarios
where half of a packet is at the end of my buffer and I need to copy
the bytes to the beginning and then receive the remaining half of the
packet?
Based on the above paragraph, I'm going to surmise that the buffer you are referring to is the application-space buffer that you pass into your recv() calls, and not the in-kernel buffer that the networking stack maintains on your application's behalf.
For UDP, the answer is simple: Your buffer needs to be large enough to hold the largest possible datagram you expect to receive. Since UDP datagrams are typically less than 1500 bytes (to avoid fragmentation) and in all cases are <= 65507 bytes (since that is the maximum datagram size the UDP protocol supports), you can always make your receive buffer 65507 bytes long, or smaller if you want to save a bit on RAM usage.
For TCP, the protocol is stream-based, so the amount of data written in to your recv-buffer by a given recv() call is unrelated to packet sizes. Another consequence of TCP being stream-based is that it doesn't do any message-framing -- that means you will have to handle partial messages regardless of how big or small you make your buffer. The only advantage of a larger TCP buffer is that it's a bit more efficient to handle more bytes at a time instead of fewer, again at the cost of using a little more RAM.
If I make the buffer large enough, am I guaranteed to avoid scenarios where half of a packet is at the end of my buffer and I need to copy the bytes to the beginning and then receive the remaining half of the packet?
For TCP: It doesn't matter. Packets are an implementation detail. The application doesn't even have to think about them. TCP is a byte-stream protocol and all you ever get from the API is a stream of bytes. Message boundaries are never preserved.
For UDP: Packets are still an implementation detail. You send and receive datagrams. Your read function always gets an entire datagram so long as your buffer is as large as the largest datagram your application protocol supports.
I read How large should my recv buffer be when calling recv in the socket library in order to understand buffer in read. There are yet some points that i wish to know about read buffer in tcp socket connection.
My application is sending video packets. when i set buff to 80000 sender could send the packets but when i set it less for example 8000 after sending few packets it stops with RST.
a)Is this buffer, TCP receive window?
b)Is there any relation between this buffer and .net.ipv4.tcp_rmem , .net.ipv4.tcp_wmem ?if yes, Should i set read buffer based on rmem or wmem?
I would greatly appreciate any responses
a)Is this buffer, TCP receive window?
No, it is just a buffer that you provide for the TCP stack to place bytes into when you call recv().
b)Is there any relation between this buffer and .net.ipv4.tcp_rmem ,
.net.ipv4.tcp_wmem?
No.
if yes, Should i set read buffer based on rmem or women?
You can pass any size buffer you want to recv(); it is unrelated to any of the above, except that there isn't any benefit to making the buffer you pass to recv() larger than the socket's current SO_RCVBUF size, since it's unlikely that recv() would ever return more bytes at once than can be present in the socket's internal buffer.
As for how to decide what size buffer to use -- consider that a larger buffer will (of course) take up more memory, and if you are allocating that buffer on the stack, a very large buffer might cause a stack overflow. On the other hand, a smaller buffer means that you can read fewer bytes with any given call to recv(), so you may have to call recv() more times to read in the same total number of bytes.
Note that number of bytes of data returned by recv() may be any number from 1 byte up to the total size of the buffer that you passed in to recv()'s third argument, and there is no way to predict how many bytes you'll get. In particular, with TCP the number of bytes you receive from any particular call to recv() will not have any correlation to the number of bytes previously passed to any particular call to send() on the sending side. So you just need to use a "reasonably sized" array (for whatever definition of "reasonably sized" you prefer) and recv() as many bytes into it as possible, and then handle that many bytes (based on recv()'s return value).
I want some feedback or suggestion on how to design a server that handles variable size messages.
to simplify the answer lets assume:
single thread epoll() based
the protocol is: data-size + data
data is stored on a ringbuffer
the read code, with some simplification for clarity, looks like this:
if (client->readable) {
if (client->remaining > 0) {
/* SIMPLIFIED FOR CLARITY - assume we are always able to read 1+ bytes */
rd = read(client->sock, client->buffer, client->remaining);
client->buffer += rd;
client->remaining -= rd;
} else {
/* SIMPLIFIED FOR CLARITY - assume we are always able to read 4 bytes */
read(client->sock, &(client->remaining), 4);
client->buffer = acquire_ringbuf_slot(client->remaining);
}
}
please, do not focus on the 4 byte. just assume we have the data size in the beginning compressed or not does not make difference for this discussion.
now, the question is: what is the best way to do the above?
assume both small "data", few bytes and large data MBs
how can we reduce the number of read() calls? e.g. in case we have 4 message of 16 bytes on the stream, it seems a waste doing 8 calls to read().
are there better alternatives to this design?
PART of the solution depends on the transport layer protocol you use.
I assume you are using TCP which provides connection oriented and reliable communication.
From your code I assume you understand TCP is a stream-oriented protocol
(So when a client sends a piece of data, that data is stored in the socket send buffer and TCP may use one or more TCP segments to convey it to the other end (server)).
So the code, looks very good so far (considering you have error checks and other things in the real code).
Now for your questions, I give you my responses, what I think is best based on my experience (but there could be better solutions):
1-This is a solution with challenges similar to how an OS manages memory, dealing with fragmentation.
For handling different message sizes, you have to understand there are always trade-offs depending on your performance goals.
One solution to improve memory utilization and parallelization is to have a list of free buffer chunks of certain size, say 4KB.
You will retrieve as many as you need for storing your received message. In the last one you will have unused data. You play with internal fragmentation.
The drawback could be when you need to apply certain type of processing (maybe a visitor pattern) on the message, like parsing/routing/transformation/etc. It will be more complex and less efficient than a case of a huge buffer of contiguous memory. On the other side, the drawback of a huge buffer is much less efficient memory utilization, memory bottlenecks, and less parallelization.
You can implement something smarter in the middle (think about chunks that could also be contiguous whenever available). Always depending on your goals. Something useful is to implement an abstraction over the fragmented memory so that every function (or visitor) that is applied works as it were dealing with contiguous memory.
If you use these chunks, when the message was processed and dropped/forwarded/eaten/whatever, you return the unused chunks to the list of free chunks.
2-The number of read calls will depend on how fast TCP conveys the data from client to server. Remember this is stream oriented and you don't have much control over it. Of course, I'm assuming you try to read the max possible (remaining) data in each read.
If you use the chunks I mentioned above the max data to read will also depend on the chunk size.
Something you can do at TCP layer is to increase the server receive buffer. Thus, it can receive more data even when server cannot read it fast enough.
3-The ring buffer is OK, if you used chunked, the ring buffer should provide the abstraction. But I don't know why you need a ring buffer.
I like ring buffers because there is a way of implementing producer-consumer synchronization without locking (Linux Kernel uses this for moving packets from L2 to IP layer) but I don't know if that's your goal.
To pass messages to other components and/or upper-layers you could also use ring buffers of pointers to messages.
A better design may be as follows:
Set up your user-space socket read buffer to be the same size as the kernel socket buffer. If your user-space socket read buffer is smaller, then you would need more than one read syscall to read the kernel buffer. If your user-space buffer is bigger, then the extra space is wasted.
Your read function should only read as much data as possible in one read syscall. This function must not know anything about the protocol. This way you do not need to re-implement this function for different wire formats.
When your read function has read into the user-space buffer it should call a callback passing the iterators to the data available in the buffer. That callback is a parser function that should extract all available complete messages and pass these messages to another higher-level callback. Upon return the parser function should return the number of bytes consumed, so that these bytes can be discarded from the user-space socket buffer.
I'm sending a C struct over UDP
struct packet{
int numInt;
int* intList; //malloc'ed as (sizeof(int)*numInt)
}
It will be serialized as [numInt][intList[0]]...[intList[numInt-1]].
My understanding is that calling recvfrom on UDP will read the entire packet, even if the buffer doesn't hold that many bytes. Is using a really large buffer the only option I have?
You could pass MSG_PEEK to recvfrom to find out exactly how big the buffer needs to be. So just recvfrom a few bytes with MSG_PEEK to find numInt and then recvfrom the real thing (this time without MSG_PEEK).
The standard says something about MSG_PEEK, but kernel.org spells it better:
MSG_PEEK
This flag causes the receive operation to return data from the
beginning of the receive queue without removing that data from the
queue. Thus, a subsequent receive call will return the same data.
Obviously at some point you will start wondering if doubling the number of system calls to save memory is worth it. I think it isn't.
UDP packets are sent and received as a whole. if you receive it, the size is right. The only thing you have to do is to supply a big enough buffer on read() or recv() or recfrom(). The length field inside the payload is redundant, since the read() will tell you the correct size. It is also dangerous, since it relies on the sender and reciever having the same byte order.
You could try using a small buffer, just large enough to get numInt, with the MSG_PEEK flag set. Then you can find out the size you actually need, and receive again without MSG_PEEK to get the whole thing.
I'm pretty sure recvfrom will read up to as many bytes as is told to it by its 3rd argument, len. If there are fewer bytes available, it will return what is there. If there are more, it will return up to len bytes. You may have to make additional calls to obtain all the data your are expecting.