I am working on a server client application on windows.
My program works perfectly fine when messages sent are really small (around 20-30KB).
But the moment the size of buffer sent is greater than 50 KB, the program doesnt work.
I noticed that send() function sends exact number of bytes I wish to. But recv function doesnt receive that much. I am assuming that the network might have fragmented the buffer internally. But shouldn't I be able to receive the whole buffer after multiple recv() calls?
I am sending the buffer through TCP, so it should ideally guarantee that I receive the whole message.
When I send a buffer of 50KB or greater, I get around 16KB in the first recv(), but the next time recv() is called I get an empty buffer. Why does the message get lost? And is there any way to get it? It happens only for buffers greater than 50KB.
I code in C using windows socket library.
Related
I wrote a simple C socket program that sends an INIT package to the server to indicate to prepare a text transfer. The server does not sends any data back at that time.
After sending the INIT package the client sends a GET package and waits for chunks of data from the server.
So every time the server receives a GET package it will send a chunk of data to the client.
So far so good. The buffer has a size of 512 bytes, a chunk is 100 Bytes plus a little overhead big.
But my problem is that the client does not receive the second message.
So my guess is that read() will blpck until the buffer is full. Is that right or what might be the reason for that?
It depends. For TCP sockets read may return before the buffer is full, and you may need to receive in a loop to get a whole message. For UDP sockets the size you read is typically the size of a single packet (datagram) and then read may block until it has read all the requested data.
The answer is no: read() on a tcp/ip socket will not block until the buffer has the amount of data you requested. read() will return immediately in all cases if any data is available, even if your socket is blocking and you've requested more data than is available.
Keep in mind that TCP/IP is a byte stream protocol and you must treat it as such. The interface is under no obligation to transmit your data together in a single packet, as long as it is presented to you in the order you placed it in the socket.
The answer is no , read is not blocking call , You can refer below points to guess the error
Several Checkpoints you can find :
Find out what read is returning at the second time .
memset the buffer every time in while before recv
use fflush(stdout) if not able to output.
Make sure all three are present . if problem not solved yet .please post source code here
I'm attempting to write a simple server using C system calls that takes unknown byte streams from unknown clients and executes specific actions depending on client input. For example, the client will send a command "multiply 2 2" and the server will multiply the numbers and return the result.
In order to avoid errors where the server reads before the client has written, I have a blocking recv() call to wait for any data using MSG_PEEK. When recv detects data to be read, I move onto non-blocking recv()'s that read the stream byte by byte.
Everything works except in the corner case where the client sends no data (i.e. write(socket, "", 0); ). I was wondering how exactly I would detect that a message with no data is sent. In this case, recv() blocks forever.
Also, this post pretty much sums up my problem, but it doesn't suggest a way to detect a size 0 packet.
What value will recv() return if it receives a valid TCP packet with payload sized 0
When using TCP at the send/recv level you are not privy to the packet traffic that goes into making the stream. When you send a nonzero number of bytes over a TCP stream the sequence number increases by the number of bytes. That's how both sides know where the other is in terms of successful exchange of data. Sending multiple packets with the same sequence number doesn't mean that the client did anything (such as your write(s, "", 0) example), it just means that the client wants to communicate some other piece of information (for example, an ACK of data flowing the other way). You can't directly see things like retransmits, duplicate ACKs, or other anomalies like that when operating at the stream level.
The answer you linked says much the same thing.
Everything works except in the corner case where the client sends no data (i.e. write(socket, "", 0); ).
write(socket, "", 0) isn't even a send in the first place. It's just a local API call that does nothing on the network.
I was wondering how exactly I would detect that a message with no data is sent.
No message is sent, so there is nothing to detect.
In this case, recv() blocks forever.
I agree.
I have a blocking recv() call to wait for any data using MSG_PEEK. When recv detects data to be read, I move onto non-blocking recv()'s that read the stream byte by byte.
Instead of using recv(MSG_PEEK), you should be using select(), poll(), or epoll() to detect when data arrives, then call recv() to read it.
I'm implementing a small application using UDP (in C). A server sends to a client the data from a given file in chunks of given amount (ex. 100 bytes / call). The client downloads the file and saves it somewhere. The catch is that the client can receive a parameter saying how many bytes to read / call.
My problem is when the server sends 100 bytes / call, and the client is set to read only 15 bytes / call. The other 85 bytes are lost, because the message is removed from the UDP queue.
Is there a way to read these messages in chunks without removing them from the queue until they're completely read?
UDP does not allow for chunked reading like TCP does. Reading a UDP message is an all-or-nothing operation, you either read the whole message in full or none of it at all. There is no in-between. Because of that, UDP-based protocols either use fixed-sized messages, or require both parties to dynamically negotiate the message sizes (like TrivialFTP does, for example).
There is no reason for a UDP protocol to require sending a byte size for each message. The message size itself implicitly dictates the size of the data inside of the message.
If you absolutely must determine the message size before actually reading the message, you could try calling recvfrom() with the MSG_PEEK flag, and give it a large buffer to copy data into (at least 64K, which a UDP message will never exceed, unless you are using IPv6 Jumbograms, but that is a separate issue). The output will tell you the actual size of the message that is still in the queue. However, if you go this route, then you may as well just drop the MSG_PEEK flag and always read using 64K buffers so there is no possibility of dropping data due to insufficient buffer sizes.
You can create a Thread to read the data from UDP Buffer infinitely And save the data to a circle-buffer. Than the client consume the data with your speed. If the Buffer is overflow,You can do nothing.Because the server's sending speed is quicker than the client's.
If data is sent to the client but the client is busy executing something else, how long will the data be available to read using recvfrom()?
Also, what happens if a second packet is sent before the first one is read, is the first one lost and the next one sitting there wating to be read?
(windows - udp)
If data is sent to the client but the client is busy executing something else, how long will the data be available to read using recvfrom()?
Forever, or not at all, or until you close the socket or read as much as a single byte.
The reason for that is:
UDP delivers datagrams, or it doesn't. This sounds like nonsense, but it is exactly what it is.
A single UDP datagram relates to either exactly one or several "fragments", which are IP packets (further encapsulated in some "on the wire" protocol, but that doesn't matter). The network stack collects all fragments for a datagram. If the checksum on any of the fragments is not good, or any other thing that makes the network stack unhappy, the complete datagram is discarded, and you get nothing, not even an error. You simply don't know anything happened.
If all goes well, a complete datagram is placed into the receive buffer. Never anything less, and never anything more. If you try to recvfrom later, that is what you'll get.
The receive buffer is obviously necessarily large enough to hold at least one max-size datagram (65535 bytes), but since usually datagrams will not be maximum size, but rather something below 1280 bytes (or 1500 if you will), it can usually hold quite a few of them (on most platforms, the buffer defaults to something around 128-256k, and is configurable).
If there is not enough room left in the buffer, the datagram is discarded, and you get nothing (well, you do still get the ones that are already in the buffer). Again, you don't even know something happened.
Each time you call recvfrom, a complete datagram is removed from the buffer (important detail!), and you get up to the number of bytes that you requested. Which means if you naively try read a few bytes and then a few bytes again, it just won't work. The first read will discard the rest of the datagram, and the subsequent ones read the first bytes of some future datagrams (and possibly block)!
This is very different from how TCP works. Here you can actually read a few bytes and a few bytes again, and it will just work, because the network layer simulates a data stream. You give a crap how it works, because the network stack makes sure it works.
Also, what happens if a second packet is sent before the first one is read, is the first one lost and the next one sitting there waiting to be read?
You probably meant to say "received" rather than "sent". Send and receive have different buffers, so that would not matter at all. About receiving another packet while one is still in the buffer, see the above explanation. If the buffer can hold the second datagram, it will store it, otherwise it silently goes * poof *.
This does not affect any datagrams already in the buffer.
Normally, the data will be buffered until it's read. I suppose if you wait long enough that the driver completely runs out of space, it'll have to do something, but assuming your code works halfway reasonably, that shouldn't be a problem.
A typical network driver will be able to buffer a number of packets without losing any.
If data is sent to the client but the client is busy executing something else, how long will the data be available to read using recvfrom()?
This depends on the OS, in windows, I believe the default for each UDP socket is 8012, this can be raised with setsockopt() Winsock Documentation So, as long as the buffer isn't full, the data will stay there until the socket is closed or it is read.
Also, what happens if a second packet is sent before the first one is read, is the first one lost and the next one sitting there wating to be read?
If the buffer has room, they are both stored, if not, one of them gets discarded. I believe its the newest one but I'm not 100% Sure.
I'm using Linux and trying to send a long message through send(). The message is 1270 bytes, but my client application is only receiving 1024 bytes.
Since 1024 bytes is such a convenient number, I'm guessing that send() can only send 1024 bytes at a time.
I looked at the man page for send, but all it says about long messages is:
When the message does not fit into the send buffer of the socket, send()
normally blocks, unless the socket has been placed in nonblocking I/O
mode. In nonblocking mode it would fail with the error EAGAIN or EWOULD-
BLOCK in this case. The select(2) call may be used to determine when it
is possible to send more data.
I'm using blocking mode, and the man page doesn't say what to do.
My exact call to send looks like this:
send(socket, message, strlen(message), 0);
Would I need to split up the string into 1024 byte chunks and send them separately? And how would my client handle this? If my client needs to do anything, I'll just mention that it's in Java and it uses InputStreamReader to receive data.
There is a misunderstanding on your part as to how send actually works. It does not send whatever you throw at it, it sends at most the number of bytes you pass in as a parameter. That means you have to check the return value of send, remove the number of bytes that actually got send from your send queue and try to send the remaining stuff with another call to send. Same holds true for recv, by the way.
In addition to what Jim Brissom said, it is worth pointing out that SOCK_STREAM sockets (ie. TCP) do not have a concept of message boundaries - the connection is just one long unstructured stream of bytes.
Just because you sent 100 bytes (say) with one send() call does not mean it will arrive as 100 bytes one recv() call. In the extreme, it may arrive in 100 separate recv()s of 1 byte each - or at the other, it may arrive as part of a larger recv(), along with prior or following data.
Your receiver must be able to handle this - if you need to define individual messages within the stream, you must impose some sort of structure upon the stream yourself, at the application layer.