I'm attempting to write a simple server using C system calls that takes unknown byte streams from unknown clients and executes specific actions depending on client input. For example, the client will send a command "multiply 2 2" and the server will multiply the numbers and return the result.
In order to avoid errors where the server reads before the client has written, I have a blocking recv() call to wait for any data using MSG_PEEK. When recv detects data to be read, I move onto non-blocking recv()'s that read the stream byte by byte.
Everything works except in the corner case where the client sends no data (i.e. write(socket, "", 0); ). I was wondering how exactly I would detect that a message with no data is sent. In this case, recv() blocks forever.
Also, this post pretty much sums up my problem, but it doesn't suggest a way to detect a size 0 packet.
What value will recv() return if it receives a valid TCP packet with payload sized 0
When using TCP at the send/recv level you are not privy to the packet traffic that goes into making the stream. When you send a nonzero number of bytes over a TCP stream the sequence number increases by the number of bytes. That's how both sides know where the other is in terms of successful exchange of data. Sending multiple packets with the same sequence number doesn't mean that the client did anything (such as your write(s, "", 0) example), it just means that the client wants to communicate some other piece of information (for example, an ACK of data flowing the other way). You can't directly see things like retransmits, duplicate ACKs, or other anomalies like that when operating at the stream level.
The answer you linked says much the same thing.
Everything works except in the corner case where the client sends no data (i.e. write(socket, "", 0); ).
write(socket, "", 0) isn't even a send in the first place. It's just a local API call that does nothing on the network.
I was wondering how exactly I would detect that a message with no data is sent.
No message is sent, so there is nothing to detect.
In this case, recv() blocks forever.
I agree.
I have a blocking recv() call to wait for any data using MSG_PEEK. When recv detects data to be read, I move onto non-blocking recv()'s that read the stream byte by byte.
Instead of using recv(MSG_PEEK), you should be using select(), poll(), or epoll() to detect when data arrives, then call recv() to read it.
Related
I'm learning about C socket programming and I came across this piece of code in a online tutorial
Server.c:
//some server code up here
recv(sock_fd, buf, 2048, 0);
//some server code below
Client.c:
//some client code up here
send(cl_sock_fd, buf, 2048, 0);
//some client code below
Will the server receive all 2048 bytes in a single recv call or can the send be be broken up into multiple receive calls?
TCP is a streaming protocol, with no message boundaries of packets. A single send might need multiple recv calls, or multiple send calls could be combined into a single recv call.
You need to call recv in a loop until all data have been received.
Technically, the data is ultimately typically handled by the operating system which programs the physical network interface to send it across a wire or over the air or however else applicable. And since TCP/IP doesn't define particulars like how many packets and of which size should compose your data, the operating system is free to decide as much, which results in your 2048 bytes of data possibly being sent in fragments, over a period of time.
Practically, this means that by calling send you may merely be causing your 2048 bytes of data be buffered for sending, much like an e-mail in a queue, except that your 2048 bytes aren't even a single piece of anything to the system that sends it -- it's just 2048 more bytes to chop into packets the network will accept, marked with a destination address and port, among other things. The job of TCP is to only make sure they're the same bytes when they arrive, in same order with relation to each other and other data sent through the connection.
The important thing at the receiving end is that, again, the arriving data is merely queued and there is no information retained as to how it was partitioned when requested sent. Everything that was ever sent through the connection is now either part of a consumable stream or has already been consumed and removed from the stream.
For a TCP connection a fitting analogy would be the connection holding an open water keg, which also has a spout (tap) at the bottom. The sender can pour water into the keg (as much as it can contain, anyway) and the receiver can open the spout to drain the water from the keg into say, a cup (which is an analogy to a buffer in an application that reads from a TCP socket). Both sender and receiver can be doing their thing at the same time, or either may be doing so alone. The sender will have to wait (send call will block) if the keg is full, and the receiver will have to wait (recv call will block) if the keg is empty.
Another, shorter analogy is that sender and receiver sit each at their own end of a opaque pipe, with the former pushing stuff in one end and the latter removing pushed stuff out of the other end.
I wrote a simple C socket program that sends an INIT package to the server to indicate to prepare a text transfer. The server does not sends any data back at that time.
After sending the INIT package the client sends a GET package and waits for chunks of data from the server.
So every time the server receives a GET package it will send a chunk of data to the client.
So far so good. The buffer has a size of 512 bytes, a chunk is 100 Bytes plus a little overhead big.
But my problem is that the client does not receive the second message.
So my guess is that read() will blpck until the buffer is full. Is that right or what might be the reason for that?
It depends. For TCP sockets read may return before the buffer is full, and you may need to receive in a loop to get a whole message. For UDP sockets the size you read is typically the size of a single packet (datagram) and then read may block until it has read all the requested data.
The answer is no: read() on a tcp/ip socket will not block until the buffer has the amount of data you requested. read() will return immediately in all cases if any data is available, even if your socket is blocking and you've requested more data than is available.
Keep in mind that TCP/IP is a byte stream protocol and you must treat it as such. The interface is under no obligation to transmit your data together in a single packet, as long as it is presented to you in the order you placed it in the socket.
The answer is no , read is not blocking call , You can refer below points to guess the error
Several Checkpoints you can find :
Find out what read is returning at the second time .
memset the buffer every time in while before recv
use fflush(stdout) if not able to output.
Make sure all three are present . if problem not solved yet .please post source code here
In a server/client in C, TCP, what happens if I do consecutive read/write in my program? Is this possibile, or do I've always to follow the structure "client write->server read->server write->client read"?
Is it possible to do something like this? Or is it possible that data from the 3rd write in the server are received from the 2nd read in the client, and other bad things like this?
client.c
write (1)
read (2)
read (3)
read (4)
server.c
read (1)
write (2)
write (3)
write (4)
TCP transfers a data stream.
So N calls to write could result in M calls to read (with N>0 and M>0 and N<=number of bytes transferred and M<=number of bytes transferred).
The order in which the bytes were send will however be preserved between the various writes and reads involved.
As a direct consequence to this you cannot just use the plain number of calls to write and read to synchronise processes. You need to add for example the number of bytes to be transferred by each step.
To cut it short: You need an application level protocol on top of the TCP stream.
Yes, this is possible. reads will block (by default) until data are available to read on the socket. There's no need to try to arrange for reads and writes in a particular order, your program will just wait until there is something to read before the read() call returns.
Your code will result in this:
Server blocks in read()
Client write()s
Client blocks in read()
Server receives data, read() returns
Server write()s three times
Client receives data, read(1) returns
Client's read(2) is called and returns as soon as data arrive
Client's read(3) is called and returns as soon as data arrive
On a really fast link it's possible that the server write()s and the client read()s happen at roughly the same time, maybe even interleaved, but the server write()s will always be in order and the client's read()s will always be in order.
Data ordering is preserved if the sockets are SOCK_STREAM, eg. TCP or UNIX sockets like you're asking about.
So read(2) will always return the data written in write(2) for a TCP socket.
If you used a UDP socket (SOCK_DGRAM's default) you might find that the messages are received out-of-order or not at all.
When reading from a socket using read(2) and blocking I/O, when do I know that the other side (the client) has no more data to send? (by "no more data to send" I mean that, as an example, the client is waiting for a response). At first, I thought that this point is reached when less than count bytes are returned by read (as in read(fd, *buf, count)).
But what if the client sends the data fragmented? Reading until read returns 0 would be a solution, but as far as I know 0 is only returned when the client closes the connection - otherwise, read would just block until the connection is closed. I thought of using non-blocking I/O and a timeout for select(2), but this does not seem to be a tidy solution to me.
Are there any known best practices?
The concept of "the other side has no more data to send", without either a timeout or some semantics in the transmitted data, is quite pointless. Normally, code on the client/server will be able to process data faster than the network can transmit it. So if there's no data in the receive buffer when you're trying to read() it, this just means the network has not yet transmitted everything, but you have no way to tell if the next packet will arrive within a millisecond, a second, or a day. You'd probably consider the first case as "there is more data to send", the third as "no more data to send", and the second depends on your application.
If the other side doesn't close the connection, you probably don't know when it's ready to send the next data packet either.
So unless you have specific semantics and knowledge about what the client sends, using select() and non-blocking I/O is the best you can do.
In specific cases, there might be other ways - for example, if you know the client will send and XML tag, some data, and a closing tag, every n seconds. In that case you could start reading n seconds after the last packet you received, then just read on until you receive the closing tag. But as i said, this isn't a general approach since it requires semantics on the channel.
TCP is a byte-stream protocol, not a message protocol. If you want messages you really have to implement them yourself, e.g. with a length-word prefix, lines, XML, etc. You can guess with the FIONREAD option of ioctl(), but guessing is all it is, as you can't know whether the client has paused in the middle of transmission of the message, or whether the network has done so for some reason.
The protocol needs to give you a way to know when the client is finishes sending a message.
Common approaches are to send the length of each message before it, or to send a special terminator after each message (similar to the NUL character at the end of strings in C).
I'm using Linux and trying to send a long message through send(). The message is 1270 bytes, but my client application is only receiving 1024 bytes.
Since 1024 bytes is such a convenient number, I'm guessing that send() can only send 1024 bytes at a time.
I looked at the man page for send, but all it says about long messages is:
When the message does not fit into the send buffer of the socket, send()
normally blocks, unless the socket has been placed in nonblocking I/O
mode. In nonblocking mode it would fail with the error EAGAIN or EWOULD-
BLOCK in this case. The select(2) call may be used to determine when it
is possible to send more data.
I'm using blocking mode, and the man page doesn't say what to do.
My exact call to send looks like this:
send(socket, message, strlen(message), 0);
Would I need to split up the string into 1024 byte chunks and send them separately? And how would my client handle this? If my client needs to do anything, I'll just mention that it's in Java and it uses InputStreamReader to receive data.
There is a misunderstanding on your part as to how send actually works. It does not send whatever you throw at it, it sends at most the number of bytes you pass in as a parameter. That means you have to check the return value of send, remove the number of bytes that actually got send from your send queue and try to send the remaining stuff with another call to send. Same holds true for recv, by the way.
In addition to what Jim Brissom said, it is worth pointing out that SOCK_STREAM sockets (ie. TCP) do not have a concept of message boundaries - the connection is just one long unstructured stream of bytes.
Just because you sent 100 bytes (say) with one send() call does not mean it will arrive as 100 bytes one recv() call. In the extreme, it may arrive in 100 separate recv()s of 1 byte each - or at the other, it may arrive as part of a larger recv(), along with prior or following data.
Your receiver must be able to handle this - if you need to define individual messages within the stream, you must impose some sort of structure upon the stream yourself, at the application layer.