I have a single threaded program. It sends message to four destinations every five seconds. I don't want connect() to be blocked. So I am writing my program like this:
int j, rc, non_blocking=1, sockets[4], max_fd=0;
struct sockaddr server=get_server_addr();
fd_set fdset;
const struct timeval conn_timeout = { 2, 0 }; /* 2 seconds */
for (j=0; j<4; ++j)
{
sockets[j]=socket( AF_INET, SOCK_STREAM, 0 );
ioctl(sockets[j], FIONBIO, (char *)&non_blocking);
connect(sockets[j], &server, sizeof (server));
}
/* prepare fd_set */
FD_ZERO ( &fdset );
for (j=0;j<4;++j)
{
if (sockets[j] != -1 )
{
FD_SET ( sockets[j], &fdset );
if ( sockets[j] > max_fd )
{
max_fd = sockets[j];
}
}
}
rc=select(max_fd + 1, NULL, &fdset, NULL, &conn_timeout );
if(rc > 0)
{
for (j=0;j<4;++j)
{
if(sockets[j]!=-1 && FD_ISSET(sockets[j],&fdset))
{
/* send() */
}
}
}
/* close all valid sockets */
However, it seems select() returns immediately after ONE file descriptor is ready instead of blocking for conn_timeout (2 seconds). So in this case how can I achieve my targets?
The program continues if all sockets are ready.
The program can block there for 2 seconds if any one of sockets are not ready.
Yeah, select was designed on the assumption that you would want to service each socket as soon as it became ready.
If I understand what you're trying to do, then the simplest way to accomplish it will be to remove each socket from the fdset as it becomes ready. If there are any sockets left in the set, use gettimeofday to adjust the timeout downward, and call select again. When the set is empty, all four sockets are usable and you can proceed.
There are three basic approaches:
If you want to stay strictly portable you need to iterate:
calculate end time from current time and timeout of your choice
Cycle:
-- Create fdset with those fds not yet ready
-- calculate max time to wait
-- select()
-- remeber those fds that are now ready
-- break if end time reached or all fds ready
End cycle
Now you have knowledge of the ready fds and the elapsed time
If you want to stay portable, but can use threads:
start n threads
select on one fd per thread
join all threads
If you do not need to be portable: Most OSes have a facility for such a situation, e.g. Windows/.NET has WaitAll (together with async send and an event)
I don't see the connection between your stated targets and your stated problem. You are correct in saying that select() blocks until at least one socket is ready, but according to target #2 above that is exactly what you want. There's nothing in your stated targets about blocking until all four sockets are ready at the same time.
You should also note that sockets are almost always ready for writing, unless the send buffer is full, which means the receiver's receive buffer is full, which means the receiver is slower than the sender. So using select() alone as the underlying write timer isn't a good idea.
Related
I have a question regarding adding new socket file descriptors to an FDSET. Lets say we've already connected to a socket s1:
fd_set readfds;
//s1 = socket(...);
//connect(s1, ...)...
FD_ZERO(&readfds);
FD_SET(s1, &readfds);
and we are waiting for data to come down the socket, by calling select in a thread:
socket_reader_thread() {
for (;;)
{
int rv = select(n, &readfds, NULL, NULL, &tv);
if (rv == -1) {
perror("select"); // error occurred in select()
}
else if (rv == 0) {
printf("Timeout occurred! No data after 10.5 seconds.\n");
}
else {
// one the descriptors have data
.....
}
}
}
If I now wanted to add another socket (or may be two more socket etc) to the readfds set, given that select is blocking, how should I proceed? how can I interrupt select
Is the trick to add a zero timeout and use select like poll?
You need to use the "pipe-trick".
This is where an additional socket or pipe is created add it to the fd_set.
Then to interrupt a running or pending select, send a 1 byte message to it via another thread.
The select will then return and if the special pipe FD is one of the ones that are ready in the set, that means you need to say look at a list or something "do work" - like add any new FDs to the fd_set before returning to the select call.
You can interrupt select by sending (and catching) a signal to your process, for example using raise. select will return in this case with -1 and errno set to EINTR. You can then change the events you want to wait for and call select again.
Is the trick to add a zero timeout and use select like poll?
One can simply use a timeout of 0 in which case it will just do a non-blocking check if any of the events got triggered, i.e. polling. But this should only be done in a few cases since busy polling instead of a blocking wait uses lots of resources of machine. And I would even consider the interrupting of a blocking select a questionable design, although probably not as bad as busy polling.
I am trying to understand how epoll() is different from select() and poll(). select() and poll() are pretty similar. select() allows you to monitor multiple file descriptors and it checks if any of those file descriptors are available for an operation (e.g. read, write) without blocking. When the timeout expires, select() returns the file descriptors that are ready and the program can perform the operations on those file descriptors without blocking.
...
FD_ZERO(&rfds);
FD_SET(0, &rfds);
/* Wait up to five seconds. */
tv.tv_sec = 5;
tv.tv_usec = 0;
retval = select(1, &rfds, NULL, NULL, &tv);
/* Don’t rely on the value of tv now! */
if (retval == -1)
perror("select()");
else if (retval)
printf("Data is available now.\n");
/* FD_ISSET(0, &rfds) will be true. */
else
printf("No data within five seconds.\n");
...
poll() is a little more flexible in that it does not rely on bitmap, but array of file descriptors. Also, since poll() uses separate fields for requested (events) and result (revents), you don't have to worry to refill the sets that were overwritten by kernel.
...
struct pollfd fds[2];
fds[0].fd = open("/dev/dev0", ...);
fds[1].fd = open("/dev/dev1", ...);
fds[0].events = POLLOUT | POLLWRBAND;
fds[1].events = POLLOUT | POLLWRBAND;
ret = poll(fds, 2, timeout_msecs);
if (ret > 0) {
for (i=0; i<2; i++) {
if (fds[i].revents & POLLWRBAND) {
...
However, I read that there is an issue with poll() too since both select() and poll() are stateless; the kernel does not internally maintain the requested sets. I read this:
Suppose that there are 10,000 concurrent connections. Typically, only
a small number of file descriptors among them, say 10, are ready to
read. The rest 9,990 file descriptors are copied and scanned for no
reason, for every select()/poll() call. As mentioned earlier, this
problem comes from the fact that those select()/poll() interfaces are
stateless.
I don't understand what is meant by the file descripters are "copied" and "scanned". Copied where? And I don't know what is meant by "stateless". Thanks for clarification.
"Stateless" means "Does not retain anything between two calls". So kernel need to rebuild many things for mainly nothing in the mentioned example.
I'm working on an embedded processor running Yocto. I have a modified uio_pdrv_genirq.c UIO driver.
I am writing a library to control the DMA. There is one function which writes to the device file and initiates the DMA. A second function is intended to wait for the DMA to complete by calling select(). Whilst DMA is in progress the device file blocks. On completion the DMA controller issues an interrupt which releases the block on the device file.
I have the system working as expected using read() but I want to switch to select() so that I can include a time out. However, when I use select(), it doesn't seem to be recognising the block and always returns immediately (before the DMA has completed). I have included a simple version of the code:
int gannet_dma_interrupt_wait(dma_device_t *dma_device,
dma_direction dma_transfer_direction) {
fd_set rfds;
struct timeval timeout;
int select_res;
/* Initialize the file descriptor set and add the device file */
FD_ZERO(&rfds);
FD_SET(dma_device->fd, &rfds);
/* Set the timeout period. */
timeout.tv_sec = 5;
timeout.tv_usec = 0;
/* The device file will block until the DMA transfer has completed. */
select_res = select(FD_SETSIZE, &rfds, NULL, NULL, &timeout);
/* Reset the channel */
gannet_dma_reset(dma_device, dma_transfer_direction);
if (select_res == -1) {
/* Select has encountered an error */
perror("ERROR <Interrupt Select Failed>\n");
exit(0);
}
else if (select_res == 1) {
/* The device file descriptor block released */
return 0;
}
else {
/* The device file descriptor block exceeded timeout */
return EINTR;
}
}
Is there anything obviously wrong with my code? Or can anyone suggest an alternative to select?
It turns out that the UIO driver contains two counters. One records the
number of events (event_count), the other records how many events the
calling function is aware of (listener->event_count).
When you do a read() on a UIO driver it returns the number of events and
makes listener->event_count equal to event_count. ie. the listener is
now up to date with all the events that have occurred.
When you use poll() or select() on a UIO driver, it checks if these two
numbers are different and returns if they are (if they are the same it
waits until they differ and then returns). It does NOT update the
listener->event_count.
Clearly if you do not do a read() between calls to select() then
the listener->event_count will not match the event_count and the second
select() will return immediately. Therefore it is necessary to call
read() in between calls to select().
With hindsight it seems clear that select() should work in this way but it wasn't obvious to me at the time.
This answer assumes that it is possible to use select() as intented for the specified device file (I use select() for socket descriptors only). As an alternative function to select(), you may want to check out the poll() family of functions. What follows will hopefully at least offer hints as to what can be done to resolve your problem with calling select().
The first parameter to the select() function has to be the maximum despriptor number plus 1. Since you have only one descriptor, you can pass it directly to select() as its first parameter and add 1. Also consider that the file descriptor in dma_device could be invalid. Returning EINTR on a timeout may actually be what you intend to do but should that not be the case and to test for an invalid descriptor, here is a different version for you to consider. The select() call could be interrupted by a signal, in which case, the return value is -1 and errno will be set to EINTR. This could be handled internally by your function as in:
FD_ZERO(&rfds);
FD_SET(dma_device->fd, &rfds);
timeout.tv_sec = 5;
timeout.tv_usec = 0;
// restart select() if it's interrupted by a signal;
do {
select_res = select(dma_device->fd + 1, &rfds, NULL, NULL, &timeout);
}
while( select_res < 0 && errno == EINTR);
if (select_res > 0) {
// a file descriptor is legible
}
else {
if (select_res == 0) {
// select() timed-out
}
else {
// an error other than a signal occurred
if (errno == EBADF) {
// your file descriptor is invalid
}
}
}
I use libpcap to capture packets, and I need to put the packets into a FIFO queue as soon as a packet is available. But the FIFO queue is shared by 2 threads, one thread call pcap_next() and put packet into the FIFO queue. Another thread fetch packet from the fifo queue. so I have to relate it to a mutex. Like below:
u_char* pkt;
for(;;){
pkt = pcap_next();
lock(&mutex);
some_process(pkt);
insert(pkt, list);
unlock(&mutext);
}
The pcap_next() is related to a packet buffer, if there is no packet in the buffer, pcap_next() is blocked. If there is/are packet(s), each call of pcap_next() returns 1 packet.
it can only fetch oen packet for each lock-unlock operation pair, If packet arrival is not frequent, then it is fine. But if packet arrival is frequent, like in the buffer there are many pending packets, it is a bit resource-consuming to don a lock-unlock operation pair for one packet.
what I hope is: after processing and inserting a packet, I immediately can check whether there are packets available in the packet buffer. If there are, continue processing and insertion. Otherwise, unlock mutex and go back to loop.
Is there a workaround for this?
Try something such as
/*
* XXX - this can fail on some platforms and with some devices,
* and there may be issues with select() on this. See the
* pcap_get_selectable_fd() man page for details.
*/
pcap_fd = pcap_get_selectable_fd(p);
pcap_setnonblock(p); /* XXX - check for failure */
for (;;) {
fd_set fdset;
struct timeval timeout;
/*
* Wait for a batch of packets to be available.
*/
FD_ZERO(&fdset);
FD_SET(pcap_fd, &fdset);
timeout.tv_sec = 1;
timeout.tv_usec = 0;
if (select(1, &fdset, NULL, NULL, &timeout) == -1) {
report an error;
} else {
lock(&mutex);
pcap_dispatch(p, -1, callback, pointer-to-stuff);
unlock(&mutex);
}
}
That way, you lock the mutex, process an entire batch of packets, and then unlock the mutex. Many OS capture mechanisms deliver multiple packets in a batch, so there will be one lock/unlock pair per batch in this case.
callback would do the some_process(pkt); and insert(pkt, list); stuff.
It is possible that, once you're done with a batch, the next batch will be immediately available, so this doesn't achieve an absolute minimum of lock/unlock pairs; however, the absolute minimum might lock out the other thread for a significant period of time, so that it can never make any progress, so lock and unlock around each batch might be best.
just use pcap_dispatch(), or select() with non-blocking style pcap_next()
I am trying to make a simple client-server chat program. On the client side I spin off another thread to read any incomming data from the server. The problem is, I want to gracefully terminate that second thread when a person logs out from the main thread. I was trying to use a shared variable 'running' to terminate, problem is, the socket read() command is a blocking command, so if I do while(running == 1), the server has to send something before the read returns and the while condition can be checked again. I am looking for a method (with common unix sockets only) to do a non-blocking read, basically some form of peek() would work, for I can continually check the loop to see if I'm done.
The reading thread loop is below, right now it does not have any mutex's for the shared variables, but I plan to add that later don't worry! ;)
void *serverlisten(void *vargp)
{
while(running == 1)
{
read(socket, readbuffer, sizeof(readbuffer));
printf("CLIENT RECIEVED: %s\n", readbuffer);
}
pthread_exit(NULL);
}
You can make socket not blockable, as suggested in another post plus use select to wait input with timeout, like this:
fd_set input;
FD_ZERO(&input);
FD_SET(sd, &input);
struct timeval timeout;
timeout.tv_sec = sec;
timeout.tv_usec = msec * 1000;
int n = select(sd + 1, &input, NULL, NULL, &timeout);
if (n == -1) {
//something wrong
} else if (n == 0)
continue;//timeout
if (!FD_ISSET(sd, &input))
;//again something wrong
//here we can call not blockable read
fcntl(socket, F_SETFL, O_NONBLOCK);
or, if you have other flags:
int x;
x=fcntl(socket ,F_GETFL, 0);
fcntl(socket, F_SETFL, x | O_NONBLOCK);
then check the return value of read to see whether there was data available.
note: a bit of googling will yield you lots of full examples.
You can also use blocking sockets, and "peek" with select with a timeout. It seems more appropriate here so you don't do busy wait.
The best thing is likely to get rid of the extra thread and use select() or poll() to handle everything in one thread.
If you want to keep the thread, one thing you can do is call shutdown() on the socket with SHUT_RDWR, which will shut down the connection, wake up all threads blocked on it but keep the file descriptor valid. After you have joined the reader thread, you can then close the socket. Note that this only works on sockets, not on other types of file descriptor.
Look for function setsockopt with option SO_RCVTIMEO.