Related
I wonder about the behaviour of socket on blocking and nonblocking actions. What happens to threads blocking on socket when the socket blocking mode changes ? Here is the scenario;
thread1(T1) creates a UDP socket and
fd = socket(AF_INET , SOCK_DGRAM, 0);
T1 waiting(sleeping) for receive
recv(fd, buf , sizeof(buf) , 0);
and thread2(T2) changes socket mode to non-blocking before socket receive any data
fcntl(fd, F_SETFL, O_NONBLOCK);
what happens to T1 ? Is it signalled/waked because the socket is no more blocking ?
The behavior is literally unspecified: fcntl is not required to unblock any threads.
Linux just sets the flag in the file description struct file and returns without unblocking any blocked threads.
A thread already blocked in recv can be scheduled to run only if:
data to read becomes available;
or an error condition on the file descriptor is detected (FIN/RST, socket read timeout, TCP keep-alive failure, the file descriptor is closed by another thread);
or a signal is received and the signal disposition does not include SA_RESTART;
or it is pthread_cancelled.
The fact that you are trying to change flags of a file descriptor of another thread suggests that your design requires a review. Ideally, threads must not share any data and not poke at each other's state, rather they should use message passing to communicate with each other.
You made me curious. First, it's quite obvious that since there is no standard that specifies that the socket should wake up, it will not be awoken because it would be quite a pain to implement this (since the non-blocking flag is in a different layer than where the socket is blocking). So we can say quite confidently that the socket will not wake up until we receive a packet. Or will it?
#include <pthread.h>
#include <signal.h>
#include <stdio.h>
#include <err.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <string.h>
#include <time.h>
#include <fcntl.h>
int sock;
static void
sighand(int s)
{
write(1, "sig\n", 4);
}
static void *
rcv(void *v)
{
struct sigaction sa;
memset(&sa, 0, sizeof(sa));
sa.sa_handler = sighand;
sa.sa_flags = SA_RESTART;
if (sigaction(SIGUSR1, &sa, NULL) == -1)
err(1, "sigaction");
char buf[64];
ssize_t sz = recv(sock, buf, sizeof(buf), 0);
printf("recv %d\n", (int)sz);
return NULL;
}
pthread_t t1, t2;
static void *
beeper(void *v)
{
for (;;) {
nanosleep(&((struct timespec){.tv_sec = 1}), NULL);
if (pthread_kill(t1, SIGUSR1))
errx(1, "pthread_kill");
printf("beep\n");
}
}
int
main(int argc, char **argv)
{
if ((sock = socket(PF_INET, SOCK_DGRAM, 0)) == -1)
err(1, "socket");
struct sockaddr_in sin;
memset(&sin, 0, sizeof(sin));
sin.sin_family = AF_INET;
sin.sin_port = htons(4711);
sin.sin_addr.s_addr = htonl(INADDR_ANY);
if (bind(sock, (struct sockaddr *)&sin, sizeof(sin)) == -1)
err(1, "bind");
if (pthread_create(&t1, NULL, rcv, NULL))
errx(1, "pthread_create");
if (pthread_create(&t2, NULL, beeper, NULL))
errx(1, "pthread_create");
/* pretend that this is correct synchronization. */
nanosleep(&((struct timespec){.tv_sec = 3}), NULL);
printf("setting non-block\n");
if (fcntl(sock, F_SETFL, O_NONBLOCK) == -1)
err(1, "fcntl");
printf("set\n");
nanosleep(&((struct timespec){.tv_sec = 3}), NULL);
return 0;
}
The code above (sorry, couldn't make it shorter). Blocks the thread in recv, waits a bit and then sets non-blocking on the file descriptor. As expected nothing happens. But then I added a twist. The receiving thread has a signal handler that wakes up once a second with SA_RESTART. Of course, since we have SA_RESTART recv should not wake up.
On OSX this will "misbehave" if we consider any behavior correct. I'm pretty sure this will behave the same way on all BSDs. Same on Linux. In fact, the way SA_RESTART is usually implemented I'm pretty sure this will "misbehave" pretty much everywhere.
Funny. This of course doesn't mean that the code above is useful for anything, but it's an interesting curiosity. To answer your question, this is unspecified, but in most cases it won't wake up, unless it will. Please don't do weird things like this.
I think POSIX specification is pretty clear about this:
If no messages are available at the socket and O_NONBLOCK is not set on the socket's file descriptor, recv() shall block until a message arrives. If no messages are available at the socket and O_NONBLOCK is set on the socket's file descriptor, recv() shall fail and set errno to [EAGAIN] or [EWOULDBLOCK].
You call recv when O_NONBLOCK is not yet set, to it should block until message arrives (not until mode changes).
Unblocked thread (didn't block yet due to reading when it was in blocking mode) will behave as if it was always a non-blocking mode.
Reading recv(2) manual page
If no messages are available at the socket, and if the socket is nonblocking, the value -1 is returned and the external
variable errno is set to EAGAIN or EWOULDBLOCK.
Blocked threads (blocked while reading when in blocking mode) before changing to non-blocking.
As pointed by #Maxim sharing the code of the function which doesn't awaken threads, The blocked threads will only be awaken after a write is done (data is available).
I'm building an application that sends requests to a server and I need to implement a way to control how long I try to connect() to the server.
I'd like to end the connect() function after ten seconds. I've read about select() and non-blocking sockets, but I don't completely understand how it works.
Could someone give me a a very simple example of how this could be done? Using C. Thanks.
Also, I'd like to add the same timeouts to the send and recv functions as well.
#include <stdio.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <stdlib.h>
#include <arpa/inet.h>
#include <fcntl.h>
#include <errno.h>
int main(void) {
struct sockaddr_in server;
int sockfd;
sockfd = socket(AF_INET, SOCK_STREAM, 0);
server.sin_addr.s_addr = inet_addr("10.0.0.1");
server.sin_family = AF_INET;
server.sin_port = htons(atoi("80"));
fcntl(sockfd, F_SETFL, O_NONBLOCK);
struct timeval tv;
fd_set writefds;
tv.tv_sec = 5;
tv.tv_usec = 500000;
FD_ZERO(&writefds);
FD_SET(sockfd, &writefds);
connect(sockfd, (struct sockaddr *)&server, sizeof(server));
if (errno == EINPROGRESS)
{
printf("In progress");
}
select(sockfd+1, NULL, &writefds, NULL, &tv);
Can you explain why the if condition for errno does not show until after the timeout? That doesn't make sense to me. And is there any additional code I should look into here to handle this correctly. Thanks.
Rather than write the actual code, I'll give you an outline. (I'm assuming Linux or something similar.)
Create the socket.
Call fcntl on the socket fd and set the
O_NONBLOCK option to make the socket nonblocking. (You may also be
able to do this by giving the SOCK_NONBLOCK option on the socket
call.)
Call connect. It will return immediately. It will most
likely "fail" with the error EINPROGRESS, which means that the
connect operation is in progress.
Call select or poll to wait
for the socket fd to become writable. Both of those calls have a
timeout option.
If the socket becomes writable (i.e., your select or poll
doesn't time out), check the connection status by calling
getsockopt to get the value of SO_ERROR, which will indicate
whether the connect succeeded or failed by returning an error code
or 0 on success.
You can use the same technique to time out sends and recvs as well (although for receive you need to wait for the socket to become readable).
See the following man pages for more information: socket(2), connect(2), select(2), poll(2), socket(7), getsockopt(2).
I have no problems with running the TCP Server and I like the fact that it's in blocking to avoid useless loops and sleeping code and useless cpu cycles.
The problem happens when shutting it down in Linux environment, it stays on, until the connected user sends something, then it turns off.
I figured it's because it's blocking even though the endless while loop is set to exit. But when it's blocking changing the socket id's to NON_BLOCKING doesn't help at all, most likely has to be set to NON_BLOCKING before the block occurs.
#include <pthread.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <stdio.h>
#include <errno.h>
#include <fcntl.h> /* Added for the nonblocking socket */
#define LISTEN_MAX 10 /* Maximum clients that can queue up */
#define LISTEN_PORT 32000
#define MAX_COMMANDS_AT_ONCE 4000
#define NANO_SECOND_MULTIPLIER 1000000 // 1 millisecond = 1,000,000 Nanoseconds
//Global so I can access these where I shut the threads off.
int listenfd, connfd; //sockets that must be set to non-blocking before exiting
int needQuit(pthread_mutex_t *mtx)
{
switch(pthread_mutex_trylock(mtx)) {
case 0: /* if we got the lock, unlock and return 1 (true) */
pthread_mutex_unlock(mtx);
return 1;
case EBUSY: /* return 0 (false) if the mutex was locked */
return 0;
}
return 1;
}
/* this is run on it's own thread */
void *tcplistener(void *arg)
{
pthread_mutex_t *mx = arg;
//keyboard event.
SDLKey key_used;
struct timespec ts;
//int listenfd,connfd,
int n,i, ans;
struct sockaddr_in servaddr,cliaddr;
socklen_t clilen;
pid_t childpid;
char mesg[MAX_COMMANDS_AT_ONCE];
listenfd=socket(AF_INET,SOCK_STREAM,0);
bzero(&servaddr,sizeof(servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_addr.s_addr=htonl(INADDR_ANY);
servaddr.sin_port=htons(LISTEN_PORT);
int option = 1;
if(setsockopt(listenfd, SOL_SOCKET,SO_REUSEADDR,(char*)&option,sizeof(option)) < 0)
{
printf("setsockopt failed\n");
close(listenfd);
}
bind(listenfd,(struct sockaddr *)&servaddr,sizeof(servaddr));
listen(listenfd,LISTEN_MAX);
while( !needQuit(mx) )
{
clilen=sizeof(cliaddr);
connfd = accept(listenfd,(struct sockaddr *)&cliaddr,&clilen);
while( !needQuit(mx) )
{
n = recv(connfd,mesg,MAX_COMMANDS_AT_ONCE,0);
if(n == 0 || n == -1) break;
//...Do Stuff here with mesg...
}
}
close(connfd);
}
close(connfd);
close(listenfd);
return NULL;
}
int main(int argc, char *argv[])
{
/* this variable is our reference to the thread */
pthread_t tcp_listener_thread;
pthread_mutex_t mxq; /* mutex used as quit flag */
/* init and lock the mutex before creating the thread. As long as the
mutex stays locked, the thread should keep running. A pointer to the
mutex is passed as the argument to the thread function. */
pthread_mutex_init(&mxq,NULL);
pthread_mutex_lock(&mxq);
/* create a hread which executes tcplistener(&x) */
if(pthread_create(&tcp_listener_thread, NULL, tcplistener, &mxq)) {
fprintf(stderr, "Error creating TCP Listener thread\n");
//clear thread for tcp listener on exit.
/* unlock mxq to tell the thread to terminate, then join the thread */
fcntl(listenfd, F_SETFL, O_NONBLOCK); /* Change the socket into non-blocking state */
fcntl(connfd, F_SETFL, O_NONBLOCK); /* Change the socket into non-blocking state */
pthread_mutex_unlock(&mxq);
pthread_join(tcp_listener_thread,NULL);
pthread_cancel(tcp_listener_thread);
pthread_exit(NULL);
return 0;
}
//End of the TCP Listener thread.
// Waits 500 milliseconds before shutting down
struct timespec ts;
ts.tv_sec = 0;
ts.tv_nsec = 500 * NANO_SECOND_MULTIPLIER;
nanosleep((&ts, NULL);
//Forces a shutdown of the program and thread.
//clear thread for tcp listener on exit.
/* unlock mxq to tell the thread to terminate, then join the thread */
fcntl(listenfd, F_SETFL, O_NONBLOCK); /* Change the socket into non-blocking state */
fcntl(connfd, F_SETFL, O_NONBLOCK); /* Change the socket into non-blocking state */
pthread_mutex_unlock(&mxq);
pthread_join(tcp_listener_thread,NULL);
pthread_cancel(tcp_listener_thread);
pthread_exit(NULL);
return 0;
}
I tried the fix EJP suggested like so, still hanging..
I've made connfd and listernfd both global scope
pthread_mutex_unlock(&mxq);
close(connfd); //<- this
close(listenfd); //<-- this
pthread_join(tcp_listener_thread,NULL);
pthread_cancel(tcp_listener_thread);
pthread_exit(NULL);
To unblock accept(), just close the listening socket. Make sure the code around accept() handles is correctly.
To unblock recv(), shutdown the receiving socket for input. That will cause recv() to return zero, which again must be handled correctly. Or else just close the socket as above, which might be better if you want the receive code to know that you're closing the application.
Your listener thread will indeed block in the accept().
The nasty way to fix this (almost) is to send a signal to the listener thread with pthread_kill(). This causes accept() to return with errno == EINTR, which you test for and then return.
However, that has a race condition: if the signal is received between testing the while (!needQuit(mx)) condition and entering the accept() then it'll be lost and the accept() will block again.
One correct way to solve this is to use something like select() and a pipe. You select for read over the pipe and the socket. When the main thread wants the listener thread to exit it writes a byte to the pipe. The listener thread's select() call returns either when a byte is readable from the pipe (in which case it exits) and/or when a client can be accepted.
Non-blocking sockets are primarily used to multiplex lots of sockets into one event loop (i.e. thread). That's a good idea for server scalability, but not necessary here.
Upon closely scouring through resources, I'm still not entirely sure how to write a proper and usable timer function in C. I am not working with threads (or parallelizable code). I simply want to write a stopwatch function that I can use to trigger a bit of code after a small amount of time has passed.
This is a very common use of a timer, in the situation of "time-out", where I have a client-server set up where the client is sending some data (UDP style with sendto(...) and recvfrom(...)). I have written my system so that the client sends a chunk of data in a packet struct I have defined, and the server processes it via CRC then sends back an acknowledgement packet (ACK) that the msg was received uncorrupted. However, I want to implement a time-out, where if the client does not receive an ACK in a certain period of time, the client resends the data chunk (of course the server is rigged to check for duplicates). I want to nest this bit of timer code in the client, and for some reason do not think this should be so difficult.
I have dug up old signal handling code from work I had done long ago, as this seems to be the only way I commonly see mentioned as a solution, can someone please guide me as to how I can use the following signal handling code to not just receive a timed signal but trigger an action of some sort. Conceptually, I feel it would be: "send data, start timer, after timer expires execute a resend, reset timer...repeat until that ACK received". Better yet, would be an easier way of writing a timer function, but it doesn't look like there's much hope for that given C is a low-level language....
#include <sys/time.h>
#include <errno.h>
#include <stdio.h>
#include <signal.h>
extern char *strsignal(int sig);
void timer_handler(int a)
{
// handle signal
printf(">>>> signal caught\n");
printf(">>>> int parameter = %s\n", (char*) strsignal(a));
}
int main(int argc, char* argv[])
{
int retval;
struct itimerval timerValue;
struct itimerval oldTimerValue;
struct sigaction action;
action.sa_handler = &timer_handler;
action.sa_flags = SA_NODEFER;
// initialize timer parameters: expires in 5 seconds
timerValue.it_interval.tv_sec = 5;
timerValue.it_interval.tv_usec = 0;
timerValue.it_value.tv_sec = 5;
timerValue.it_value.tv_usec = 0;
// install signal handler to catch SIGALRM
//signal(SIGALRM, timer_handler);
sigaction(SIGALRM, &action, NULL);
retval = setitimer(ITIMER_REAL, &timerValue, &oldTimerValue);
if (-1 == retval)
perror("Could not set timer");
while(1);
return 0;
}
Xymostech provided the exact function I needed and after consulting the API for "select", which includes a small usage example, I modified the code there to fit what I needed and wrote a socket timer (for reads, those it's pretty simple to extend towards writes and such, as "select" has parameters for enabling this kind of check). Make sure you've included the following libraries as specified by the "select" API:
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/time.h>
#include <stdio.h>
The following is the waittoread(...) function I created from the API example and works pretty well. This works well in the domain of my specific problem, however, if one is looking for a more generalized timer (i.e. not just for timing socket read and writes, or file descriptors) please consult signal handling (somewhat in the spirit of the code I posted in my initial question).
#define S1READY 0x01 // necessary for the function's bitwise OR operation
int waittoread(int s1, int timeout_value){
fd_set fds; // create set of sockets to be waited on
struct timeval timeout; // the time-out value
int rc; // # of sockets that are ready before timer expires
int result;
/* Set time limit. */
timeout.tv_sec = timeout_value;
timeout.tv_usec = 0;
/* Create a descriptor set containing the socket. */
FD_ZERO(&fds); // MACRO to reset the socket storage set so new ones can be added
FD_SET(s1, &fds); // add the socket descriptor into the socket set to wait on
rc = select(sizeof(fds)*4, &fds, NULL, NULL, &timeout); // build the socket-wait system
// another way of calling select that would be a better approach:
// rc = select(s1 + 1), &fds, NULL, NULL, &timeout);
if (rc==-1) {
perror("Error: Call to select failed.");
return -1;
}
result = 0;
if (rc > 0){
if (FD_ISSET(s1, &fds)) result |= S1READY; // if the result is non-zero, perform a BIT-wise OR to extract the true socket count #
}
return result;
}
What is the difference between read() and recv(), and between send() and write() in socket programming in terms of performances, speed and other behaviors?
The difference is that recv()/send() work only on socket descriptors and let you specify certain options for the actual operation. Those functions are slightly more specialized (for instance, you can set a flag to ignore SIGPIPE, or to send out-of-band messages...).
Functions read()/write() are the universal file descriptor functions working on all descriptors.
Per the first hit on Google
read() is equivalent to recv() with a flags parameter of 0. Other values for the flags parameter change the behaviour of recv(). Similarly, write() is equivalent to send() with flags == 0.
read() and write() are more generic, they work with any file descriptor.
However, they won't work on Windows.
You can pass additional options to send() and recv(), so you may have to used them in some cases.
I just noticed recently that when I used write() on a socket in Windows, it almost works (the FD passed to write() isn't the same as the one passed to send(); I used _open_osfhandle() to get the FD to pass to write()). However, it didn't work when I tried to send binary data that included character 10. write() somewhere inserted character 13 before this. Changing it to send() with a flags parameter of 0 fixed that problem. read() could have the reverse problem if 13-10 are consecutive in the binary data, but I haven't tested it. But that appears to be another possible difference between send() and write().
Another thing on linux is:
send does not allow to operate on non-socket fd. Thus, for example to write on usb port, write is necessary.
On Linux I also notice that :
Interruption of system calls and library functions by signal handlers
If a signal handler is invoked while a system call or library function call is blocked, then either:
the call is automatically restarted after the signal handler returns; or
the call fails with the error EINTR.
... The details vary across UNIX systems; below, the details for Linux.
If a blocked call to one of the following interfaces is interrupted
by a signal handler, then the call is automatically restarted after
the signal handler returns if the SA_RESTART flag was used; otherwise the call fails with the error EINTR:
read(2), readv(2), write(2), writev(2), and ioctl(2) calls on "slow" devices.
.....
The following interfaces are never restarted after being interrupted by a signal handler, regardless of the use of SA_RESTART; they
always fail with the error EINTR when interrupted by a signal handler:
"Input" socket interfaces, when a timeout (SO_RCVTIMEO) has been set on the socket using setsockopt(2): accept(2), recv(2),
recvfrom(2), recvmmsg(2) (also with a non-NULL timeout argument), and recvmsg(2).
"Output" socket interfaces, when a timeout (SO_RCVTIMEO) has been set on the socket using setsockopt(2): connect(2), send(2),
sendto(2), and sendmsg(2).
Check man 7 signal for more details.
A simple usage would be use signal to avoid recvfrom blocking indefinitely.
An example from APUE:
#include "apue.h"
#include <netdb.h>
#include <errno.h>
#include <sys/socket.h>
#define BUFLEN 128
#define TIMEOUT 20
void
sigalrm(int signo)
{
}
void
print_uptime(int sockfd, struct addrinfo *aip)
{
int n;
char buf[BUFLEN];
buf[0] = 0;
if (sendto(sockfd, buf, 1, 0, aip->ai_addr, aip->ai_addrlen) < 0)
err_sys("sendto error");
alarm(TIMEOUT);
//here
if ((n = recvfrom(sockfd, buf, BUFLEN, 0, NULL, NULL)) < 0) {
if (errno != EINTR)
alarm(0);
err_sys("recv error");
}
alarm(0);
write(STDOUT_FILENO, buf, n);
}
int
main(int argc, char *argv[])
{
struct addrinfo *ailist, *aip;
struct addrinfo hint;
int sockfd, err;
struct sigaction sa;
if (argc != 2)
err_quit("usage: ruptime hostname");
sa.sa_handler = sigalrm;
sa.sa_flags = 0;
sigemptyset(&sa.sa_mask);
if (sigaction(SIGALRM, &sa, NULL) < 0)
err_sys("sigaction error");
memset(&hint, 0, sizeof(hint));
hint.ai_socktype = SOCK_DGRAM;
hint.ai_canonname = NULL;
hint.ai_addr = NULL;
hint.ai_next = NULL;
if ((err = getaddrinfo(argv[1], "ruptime", &hint, &ailist)) != 0)
err_quit("getaddrinfo error: %s", gai_strerror(err));
for (aip = ailist; aip != NULL; aip = aip->ai_next) {
if ((sockfd = socket(aip->ai_family, SOCK_DGRAM, 0)) < 0) {
err = errno;
} else {
print_uptime(sockfd, aip);
exit(0);
}
}
fprintf(stderr, "can't contact %s: %s\n", argv[1], strerror(err));
exit(1);
}
"Performance and speed"? Aren't those kind of ... synonyms, here?
Anyway, the recv() call takes flags that read() doesn't, which makes it more powerful, or at least more convenient. That is one difference. I don't think there is a significant performance difference, but haven't tested for it.
The only difference between recv() and read() is the presence of flags. With a zero flags argument, recv() is generally equivalent to read()
you can use write() and read() instead send() and recv() but send() and recv() offer much greater control over your data transmission