I read How large should my recv buffer be when calling recv in the socket library in order to understand buffer in read. There are yet some points that i wish to know about read buffer in tcp socket connection.
My application is sending video packets. when i set buff to 80000 sender could send the packets but when i set it less for example 8000 after sending few packets it stops with RST.
a)Is this buffer, TCP receive window?
b)Is there any relation between this buffer and .net.ipv4.tcp_rmem , .net.ipv4.tcp_wmem ?if yes, Should i set read buffer based on rmem or wmem?
I would greatly appreciate any responses
a)Is this buffer, TCP receive window?
No, it is just a buffer that you provide for the TCP stack to place bytes into when you call recv().
b)Is there any relation between this buffer and .net.ipv4.tcp_rmem ,
.net.ipv4.tcp_wmem?
No.
if yes, Should i set read buffer based on rmem or women?
You can pass any size buffer you want to recv(); it is unrelated to any of the above, except that there isn't any benefit to making the buffer you pass to recv() larger than the socket's current SO_RCVBUF size, since it's unlikely that recv() would ever return more bytes at once than can be present in the socket's internal buffer.
As for how to decide what size buffer to use -- consider that a larger buffer will (of course) take up more memory, and if you are allocating that buffer on the stack, a very large buffer might cause a stack overflow. On the other hand, a smaller buffer means that you can read fewer bytes with any given call to recv(), so you may have to call recv() more times to read in the same total number of bytes.
Note that number of bytes of data returned by recv() may be any number from 1 byte up to the total size of the buffer that you passed in to recv()'s third argument, and there is no way to predict how many bytes you'll get. In particular, with TCP the number of bytes you receive from any particular call to recv() will not have any correlation to the number of bytes previously passed to any particular call to send() on the sending side. So you just need to use a "reasonably sized" array (for whatever definition of "reasonably sized" you prefer) and recv() as many bytes into it as possible, and then handle that many bytes (based on recv()'s return value).
Related
What happens when you call read() (or recv()) on an open socket, and you specify a length that is more the number of bytes ready to be read in the buffer (TCP) or the length of the next datagram (UDP)?
In both cases, if the size of the buffer is larger than the amount of available data, what data is available is read and the number of bytes actually read are returned from the function. That return value is what you should use when operating on the data.
I need to write a couple of C++ applications on Linux, one to receive data via UDP and the second TCP.
The only thing I'm unsure about is regarding the buffer.
How do I choose what size buffer?
If I make the buffer large enough, am I guaranteed to avoid scenarios where half of a packet is at the end of my buffer and I need to copy the bytes to the beginning and then receive the remaining half of the packet?
I am going to use the Linux socket API functions if it matters.
If I make the buffer large enough, am I guaranteed to avoid scenarios
where half of a packet is at the end of my buffer and I need to copy
the bytes to the beginning and then receive the remaining half of the
packet?
Based on the above paragraph, I'm going to surmise that the buffer you are referring to is the application-space buffer that you pass into your recv() calls, and not the in-kernel buffer that the networking stack maintains on your application's behalf.
For UDP, the answer is simple: Your buffer needs to be large enough to hold the largest possible datagram you expect to receive. Since UDP datagrams are typically less than 1500 bytes (to avoid fragmentation) and in all cases are <= 65507 bytes (since that is the maximum datagram size the UDP protocol supports), you can always make your receive buffer 65507 bytes long, or smaller if you want to save a bit on RAM usage.
For TCP, the protocol is stream-based, so the amount of data written in to your recv-buffer by a given recv() call is unrelated to packet sizes. Another consequence of TCP being stream-based is that it doesn't do any message-framing -- that means you will have to handle partial messages regardless of how big or small you make your buffer. The only advantage of a larger TCP buffer is that it's a bit more efficient to handle more bytes at a time instead of fewer, again at the cost of using a little more RAM.
If I make the buffer large enough, am I guaranteed to avoid scenarios where half of a packet is at the end of my buffer and I need to copy the bytes to the beginning and then receive the remaining half of the packet?
For TCP: It doesn't matter. Packets are an implementation detail. The application doesn't even have to think about them. TCP is a byte-stream protocol and all you ever get from the API is a stream of bytes. Message boundaries are never preserved.
For UDP: Packets are still an implementation detail. You send and receive datagrams. Your read function always gets an entire datagram so long as your buffer is as large as the largest datagram your application protocol supports.
the docs say for send:
When the message does not fit into the send buffer of the socket,
send() normally blocks, unless the socket has been placed in non-block-
ing I/O mode. In non-blocking mode it would return EAGAIN in this
case. The select(2) call may be used to determine when it is possible
to send more data.
I am in blocking mode, doing something along the lines of:
buf = malloc(size);
send (socket, buf, size);
free(buf)
Assume but is very large, larger than the buffer can hold at a time (so it would need to go into the buffer as two chunks lets say). Anyways, in blocking mode, which I'm in, after send, can I feel safe that the data is fully copied or dealt with and thus deletable?
In blocking mode, send blocks until I/O is complete, or an error is triggered. You should check the returned value, because a send operation does not guarantee that the number of bytes sent is the same number of bytes passed as third argument.
Only when send returns a value equal to the size of the buffer sent you can be sure that the whole block has been copied into kernel memory, or passed through device memory, or sent to the destination.
The short answer is: Yes, you can free the buffer after the send() call successfully returns (without errors) when the file descriptor is in blocking mode.
The reason for this is based on the blocking concept itself: The send() call (targeting a blocking file descriptor) will only return when an error occur or the requested size bytes of the data in the buf is buffered or transmitted by the underlying layer of the operating system (typically the kernel).
Also note that a successful return of send() doesn't mean that the data was transmitted. It means that it was, at least, buffered by the underlying layer.
I'm working on a client-server app. My app is working with variable size packets, each packet has a header and a payload of variable length.
My dilemma is what is the best approach for handing the packets when doing recv.
Most of the tutorials I've came across suggest using a ring buffer but as far as I can tell it's more efficient to use a buffer whose size is twice the size of the biggest packet you can handle.
If I use a ring buffer I need an additional buffer for recv and then I need to copy the buffer in the ring buffer which means I need to do one or two memcpys to insert the buffer in the ring buffer
If I use the single buffer approach I only need one buffer which I can pass to recv call and a memmove call to move data to the start of the buffer when I got a full packet and there still is data belonging to another packet in the buffer.
Am I getting something wrong ?
PS. If you can point me to any source code/example where variable length packets are handled that would be helpful.
If I use a ring buffer I need an additional buffer for recv and then I need to copy the buffer in the ring buffer which means I need to do one or two memcpys to insert the buffer in the ring buffer
Yeah, two reads and writes, no big deal (*). But you don't need an additional buffer. For read just maximize read to the space left till the end of the ring buffer.
(*): If you are worried about the cost of additional syscall, for scatter/gather read/write there is recvmsg and sendmsg.
I'm sending a C struct over UDP
struct packet{
int numInt;
int* intList; //malloc'ed as (sizeof(int)*numInt)
}
It will be serialized as [numInt][intList[0]]...[intList[numInt-1]].
My understanding is that calling recvfrom on UDP will read the entire packet, even if the buffer doesn't hold that many bytes. Is using a really large buffer the only option I have?
You could pass MSG_PEEK to recvfrom to find out exactly how big the buffer needs to be. So just recvfrom a few bytes with MSG_PEEK to find numInt and then recvfrom the real thing (this time without MSG_PEEK).
The standard says something about MSG_PEEK, but kernel.org spells it better:
MSG_PEEK
This flag causes the receive operation to return data from the
beginning of the receive queue without removing that data from the
queue. Thus, a subsequent receive call will return the same data.
Obviously at some point you will start wondering if doubling the number of system calls to save memory is worth it. I think it isn't.
UDP packets are sent and received as a whole. if you receive it, the size is right. The only thing you have to do is to supply a big enough buffer on read() or recv() or recfrom(). The length field inside the payload is redundant, since the read() will tell you the correct size. It is also dangerous, since it relies on the sender and reciever having the same byte order.
You could try using a small buffer, just large enough to get numInt, with the MSG_PEEK flag set. Then you can find out the size you actually need, and receive again without MSG_PEEK to get the whole thing.
I'm pretty sure recvfrom will read up to as many bytes as is told to it by its 3rd argument, len. If there are fewer bytes available, it will return what is there. If there are more, it will return up to len bytes. You may have to make additional calls to obtain all the data your are expecting.