I have a socket which waits for recv and then after receiving data, sends data forward for processing. However, then it again goes for recv, and this time it receives nothing returns -1 and when printed the errno it prints 35 (which is EAGAIN).
This happens only on MAC OS Lion operating system, for other OS this runs perfectly fine
do{
rc = recv(i, buffer, sizeof(buffer), 0);
if (rc < 0){
printf("err code %d", errno);
}
if(rc == 0){
//Code for processing the data in buffer
break;
}
....
}while(1);
EDIT: Corrected indentation and errno
You either set the socket to non-blocking mode or enabled the receive timeout. Here's from recv(2) on a mac:
The calls fail if:
[EAGAIN] The socket is marked non-blocking, and the receive operation would block, or a receive timeout had been set, and the timeout expired before data were received.
Edit 0:
Hmm, apologies for quoting again. This time from intro(2):
11 EDEADLK Resource deadlock avoided. An attempt was made to
lock a system resource that would have resulted in a deadlock
situation.
...
35 EAGAIN Resource temporarily
unavailable. This is a temporary condition and later calls to the
same routine may complete normally.
Just use strerror(3) to figure out the actual issue.
Your socket is in non-blocking mode. EAGAIN is the normal return from recv() (and other system calls) when there is no data available to read. In that sense it's not really an error.
If you meant for your socket to be nonblocking then you need to monitor it to find out when it has data available and only call recv() when there is data available. Use poll() (or kqueue, which is specific to FreeBSD and MacOS) to monitor is. Usually this is done in your application's main event loop.
If you did not mean for your socket to be nonblocking, then you should set it to blocking more with fcntl():
flags = fcntl(i, F_GETFL, 0); /* add error checking here, please */
flags &= ~O_NONBLOCK;
fcntl(i, F_SETFL, flags); /* add more error checking here! */
But you should be aware that the default blocking state of sockets (and all file descriptors) is blocking, so if your socket is in nonblocking mode then that means someone or something has manually made it nonblocking.
In blocking mode, the recv call will block and wait for more data instead of returning EAGAIN (or EWOULDBLOCK which is the same thing as EAGAIN).
Related
Here is the succinct code:
int main(){
/* here ellipsis socket connect malloc etc. */
send(socket,buffer, 1024*1024*1024,0);
return 1;
}
Question: If send() is blocked, no matter how large the buffer is, I think it will send all data before the main function returned. But actually, if the data large enough, the other socket end only receive a part of data which the send() function send. However, if I add this code before 'return 1;'
while(1){}
the other socket end receive intact data.
Does the send() is non_blocked?
or something wrong with send() function?
thanks advance.
When blocking socket is used, send() function blocks until last data is delivered to queue of local TCP-stack.
So send() may return when part of the data is still queued in local TCP-stack.
Because your process exits right after send() call, there can be undelivered data in local TCP stack during exit.
TCP stack may continue the data transfer after exit, if linger is enabled. Or TCP stack may reset the connection without any attempt to transfer undelivered data to the peer, if linger is disabled.
If you close the TCP connection gracefully when linger is enabled, then TCP-stack should (try to) deliver queued data to the peer.
Close the connection gracefully by adding close() call.
And make sure that SO_LINGER is enabled with reasonable timeout:
send(socket,buffer, 1024*1024*1024,0);
const struct linger linger_val = { 1, 600 };
setsockopt(socket, SOL_SOCKET, SO_LINGER, &linger_val, sizeof(linger_val));
close(socket);
return 1;
Usually there is no need to change SO_LINGER. More information about SO_LINGER in man page of socket(7):
SO_LINGER
When enabled, a close(2) or shutdown(2) will not return until
all queued messages for the socket have been successfully sent
or the linger timeout has been reached. Otherwise, the call
returns immediately and the closing is done in the background.
When the socket is closed as part of exit(2), it always
lingers in the background.
Whit 0 as flag, send() is like write() function:
https://linux.die.net/man/2/send
https://linux.die.net/man/2/write
and for write():
The number of bytes written may be less than count if, for example, there is insufficient space on the underlying physical medium, or the RLIMIT_FSIZE resource limit is encountered (see setrlimit(2)), or the call was interrupted by a signal handler after having written less than count bytes. (See also pipe(7).).
Check also this answer:
Blocking sockets: when, exactly, does "send()" return?
Hope this help
send() is blocking call, but it is blocked till all the data is pushed to sendbuffer. You can modify the program to exit when all the data is send from the socket. This is possible by reducing the sendbuffer size. you can use setsockopt(s, SOL_SOCKET, SO_SNDBUF, (char*)send_buffer, send_buffer_sizeof);
I have a multithreaded server written in C, with each client thread looking something like this:
ssize_t n;
struct request request;
// Main loop: receive requests from the client and send responses.
while(running && (n = recv(sockfd, &request, sizeof(request), 0)) == sizeof(request)) {
// Process request and send response.
}
if(n == -1)
perror("Error receiving request from client");
else if(n != sizeof(act))
fprintf(stderr, "Error receiving request from client: Incomplete data\n");
// Clean-up code.
At some point, a client meets a certain criteria where it must be disconnected. If the client is regularly sending requests, this is fine because it can be informed of the disconnection in the responses; However sometimes the clients take a long time to send a request, so the client threads end up blocking in the recv call, and the client does not get disconnected until the next request/response.
Is there a clean way to disconnect the client from another thread while the client thread is blocking in the recv call? I tried close(sockfd) but that causes the error Error receiving request from client: Bad file descriptor to occur, which really isn't accurate.
Alternatively, is there a better way for me to be handling errors here?
So you have at least these possibilities:
(1) pthread_kill will blow the thread out of recv with errno == EINTR and you can clean up and exit the thread on your own. Some people think this is nasty. Depends, really.
(2) Make your client socket(s) non-blocking and use select to wait on input for a specific period of time before checking if a switch used between the threads has been set to indicated they should shut down.
(3) In combo with (2) have each thread share a pipe with the master thread. Add it to the select. If it becomes readable and contains a shutdonw request, the thread shuts itself down.
(4) Look into the pthread_cancel mechanism if none of the above (or variations thereof) do not meet your needs.
Shutdown the socket for input from another thread. That will cause the reading thread to receive an EOS, which should cause it to close the socket and terminate if it is correctly written.
To interrupt the thread, make the socket non-blocking (set O_NONBLOCK using fcntl) and then signal the thread with pthread_kill. This way, recv will fail with either EINTR if it was sleeping, or EAGAIN or EWOULDBLOCK if it wasn’t (also maybe if SA_RESTART is in effect, didn’t check). Note that the socket doesn’t need to, and actually should not, be non-blocking before that. (And of course the signal needs to be handled; empty handler is sufficient).
To be sure to catch the stop-signal but not anything else, use a flag; there are things that may go wrong. For example, recv may fail with EINTR on some spurious signal. Or it may succeed if there was some data available, effectively ignoring the stop request.
And what not to do:
Don’t use pthread_kill alone or with any plain check. It may arrive right before issuing the recv syscall, too early to interrupt it but after all the checks.
Don’t close the socket. That may not even work, and as #R.. pointer out, is dangerous as the socket file descriptor may be reused between close and recv (unless you’re sure nothing opens file descriptors).
I am a newbie dabbling in C and my little project is to write a simple SOCKS4 proxy. Thanks to the help here i got so far to use non-blocking sockets and poll() for my routine. However at this point i seem to have 2 problems:
The outgoing Socket dstSocket doesn't get closed if the incoming Socket rcvSocket gets closed and vice versa. I don't check for this in the loop, but i don't know how. I tried POLLHUP as revents, but that doesn't seem to do the trick. A normal check seems to be whether recv() returns 0, but is that also valid for non-blocking sockets? And if so, how does that work with revents, i can't seem to figure out where i would put this, since if POLLIN | POLLPRI are set it seems to me recv() never should return 0? Also i don't understand what the exact difference is between POLLIN and POLLPRI, seems to me just a check "data is available for reading" in either case?
The proxy seems to work for connections i tested with netcat. However if i use a browser, it says (when i target a website) whether i want to save "binary data". I checked the data in wireshark and what is received from the server is correctly forwarded to the client byte by byte it seems. If anyone maybe has an idea why that could happen with this program it would be nice :)
Attached the relevant code (beware programming newbie):
fds[1].fd = dstSocket;
fds[0].fd = rcvSocket;
fds[1].events = POLLIN | POLLPRI | POLLHUP;
fds[0].events = POLLIN | POLLPRI | POLLHUP;
timer = poll(fds, 2, timeout_msecs); /* i dont use this yet */
fcntl(rcvSocket, F_SETFL, O_NONBLOCK);
fcntl(dstSocket, F_SETFL, O_NONBLOCK);
while (1 == 1)
{
if (fds[0].revents & POLLIN | POLLPRI)
{
recvMsgSize = recv(rcvSocket, rcvBuffer, RCVBUFSIZE, 0);
if (recvMsgSize > 0) {send(dstSocket, rcvBuffer, recvMsgSize, 0);}
}
if (fds[1].revents & POLLIN | POLLPRI)
{
sndMsgSize = recv(dstSocket, sndBuffer, RCVBUFSIZE, 0);
if (sndMsgSize > 0) { send(rcvSocket, sndBuffer, sndMsgSize, 0);}
}
if ((fds[0].revents & POLLHUP) || (fds[1].revents & POLLHUP))
{
close(rcvSocket);
close(dstSocket);
}
}
recv() returns 0 on a clean remote shutdown - this is true even for nonblocking sockets. In this case, POLLIN will be returned - the notification that the remote side has closed the socket is considered a "readable" event.
You shouldn't need to use POLLPRI for SOCKS / HTTP connections - it indicates TCP "urgent data", which isn't used by those protocols (or indeed, much used at all).
Apart from your direct questions, you need to do more to implement a reliable proxy:
You need to be calling poll() on every loop, not just once. The way you have it written it is busy-looping, which is generally not considered acceptable practise.
You should be setting the disposition of SIGPIPE to ignored with signal(SIGPIPE, SIG_IGN);. This allows you to gracefully handle write failures.
You should be checking the result of send(). Note that it can write less than the amount you requested - in this case, you will have to keep the unsent data buffered, return to the poll() and try sending the remaining data again if POLLOUT is raised on the socket. You only want to request POLLOUT if there is unsent data remaining, so you need to make sure .events is set correctly before every poll() call.
You should be checking errno if recv() or send() returns a value less than 0. EINTR and EWOULDBLOCK should be ignored; any other error should be treated as a connection failure.
You should not be closing both directions immediately when one socket closes - you must support asymmetric shutdowns. This means that when you see that fds[0] has been closed by the remote end, you should call shutdown(fds[1], SHUT_WR);, and vice-versa; only when both have been shutdown (or a connection failure has occured) should you call close() on both file descriptors and finish up.
I have an application that is going to work like a p2p-software where all peer are going to talk to each other. Since the communication will be TCP i thought that I could use epool(4) so that multiple connections can be handled. Since each peer will send data very often, I thought that I will establish a persistent connection to each peer that will be used under the applications lifetime.
Now, one thing that I don't know how to handle is that since the connection is never closed how do I know when i should stop receiving data with read() and call epool_wait() again to listen after more packages? Or is there a better way of dealing with persistent TCP connections?
You should set the socket to non-blocking, and when epoll indicates there is data to read
you should call read() in a loop until read() returns -1 and errno is EWOULDBLOCK
That is, your read loop could look sometihng like:
for(;;)
ssize_t ret;
ret = read(...);
if(ret == 0) {
//client disconnected, handle it, remove the fd from the epoll set
break;
} else if(ret == -1) {
if(errno == EWOULDBLOCK) {
// no more data, return to epoll loop
} else {
//error occured, handle it remove the fd from the epoll set
}
break;
}
// handle the read data
}
If you're not using edge triggered mode with epoll, you don't really need the loop - you could get away with doing just 1 read and return to the epoll loop. But handle the return values just like the above code.
That should have been 'epoll', not 'epool'...not familiar with epoll, but have a look here at the Beej's guide to see an example of the sockets using 'poll'...look at section 7.2 in there to see how it is done, also look at the section 9.17 for the usage of 'poll'...
Hope this helps,
Best regards,
Tom.
read() reads as much data as is immediately available (but no more that you request). Just run read() on the active socket, with a big-enough buffer (you probably don't need it bigger than your MTU… 2048 bytes will do) and call epoll_wait() when it completes.
If I got a file descriptor (socket fd), how to check this fd is avaiable for read/write?
In my situation, the client has connected to server and we know the fd.
However, the server will disconnect the socket, are there any clues to check it ?
You want fcntl() to check for read/write settings on the fd:
#include <unistd.h>
#include <fcntl.h>
int r;
r = fcntl(fd, F_GETFL);
if (r == -1)
/* Error */
if (r & O_RDONLY)
/* Read Only */
else if (r & O_WRONLY)
/* Write Only */
else if (r & O_RDWR)
/* Read/Write */
But this is a separate issue from when the socket is no longer connected. If you are already using select() or poll() then you're almost there. poll() will return status nicely if you specify POLLERR in events and check for it in revents.
If you're doing normal blocking I/O then just handle the read/write errors as they come in and recover gracefully.
You can use select() or poll() for this.
In C#, this question is answered here
In general, socket disconnect is asynchronous and needs to be polled for in some manner. An async read on the socket will typically return if it's closed as well, giving you a chance to pick up on the status change quicker. Winsock (Windows) has the ability to register to receive notification of a disconnect (but again, this may not happen for a long time after the other side "goes away", unless you use some type of 'keepalive' (SO_KEEPALIVE, which by default may not notice for hours, or an application-level heartbeat).
I found the recv can check. when socket fd is bad, some errno is set.
ret = recv(socket_fd, buffer, bufferSize, MSG_PEEK);
if(EPIPE == errno){
// something wrong
}
Well, you could call select(). If the server has disconnected, I believe you'll eventually get an error code returned... If not, you can use select() to tell whether you're network stack is ready to send more data (or receive it).