I would like to know if that code:
select(fd,..., NULL);
is less CPU consuming than that one:
struct timeval tv;
tv.tv_sec = X;
tv_tv_usec = Y;
select(fd,..., &tv);
and why. Thank you.
EDIT: I'm asking about one single call. It's a sys call, so it's system dependent and it's up to the system to unblock the select()ing program. So, for the system, is more CPU consuming to accomplish to a select with or without timeout?
Neither is "lighter". select is a system call and will instruct the OS to wake up your task when either an event occurs on one of the watched file descriptors or (if supplied) a timeout occurs. selecting with a NULL timeout will select indefinitely until a watched file descriptor event occurs of the process is interrupted in another way.
Clearly:
while (select(..., NULL) == 0) { /* ... */ }
is lighter than:
while (select(..., tv) == 0) { /* ... */ }
where the time in tv is small, otherwise the difference is likely too small to be noticed.
Related
My question is about exactly when the I/O thread in an asynchronous I/O call returns when a call back function is involved. Specifically, given this very general code for reading a file ...
#include<stdio.h>
#include<aio.h>
...
// callback function:
void finish_aio(sigval_t sigval) {
/* do stuff ... maybe close the file */
}
int main() {
struct aiocb my_aiocb;
int aio_return;
...
//Open file, take care of any other prelims, then
//Fill in file-specific info for my_aiocb, then
//Fill in callback information for my_aiocb:
my_aiocb.aio_sigevent.sigev_notify = SIGEV_THREAD;
my_aiocb.aio_sigevent.sigev_notify_function = finish_aio;
my_aiocb.aio_sigevent.sigev_notify_attributes = NULL;
my_aiocb.aio_sigevent.sigev_value.sival_ptr = &info_on_file;
// then read the file:
aio_return = aio_read(&my_aiocb);
// do stuff that doesn't need data that is being read ...
// then block execution until read is complete:
while(aio_error(&my_aiocb) == EINPROGRESS) {}
// etc.
}
I understand that the callback function is called as soon as the read of the file is completed. But what exactly happens then? Does the I/O thread start running the callback finish_aio()? Or does it spawn a new thread to handle that callback, while it returns to the main thread? Another way to put this would be: When does aio_error(&my_aiocb) stop returning EINPROGRESS? Is it just before the call to the callback, or when the callback is completed?
I understand that the callback function is called as soon as the read of the file is completed. But what exactly happens then?
What happens is that when the IO finishes it "behaves as if" it started a new thread (similar to calling pthread_create(&ignored, NULL, finish_aio, &info_on_file)).
When does aio_error(&my_aiocb) stop returning EINPROGRESS?
I'd expect that aio_error(&my_aiocb) stops returning EINPROGRESS as soon as the IO finishes, then the system (probably the standard library) either begins creating a new thread to call finish_aio() or "unblocks" a "previously created without you knowing" thread. However, I don't think the exact order is documented anywhere ("implementation defined") because it doesn't make much sense to call aio_error(&my_aiocb) from anywhere other than the finish_aio() anyway.
More specifically; if you're using polling (my_aiocb.aio_sigevent.sigev_notify = SIGEV_NONE) then you'd repeatedly check aio_error(&my_aiocb) yourself and you can't care if you're notified before or after this happens because you're not notified at all; and if you aren't using polling you'd wait until you are notified (via. a new thread or a signal) that there's a reason to check aio_error(&my_aiocb).
In other words, your finish_aio() would look more like this:
void finish_aio(sigval_t sigval) {
struct aiocb * my_aiocb = (struct aiocb *)sigval;
int status;
status = aio_error(&my_aiocb);
/* Figure out what to do (handle the error or handle the file's data) */
.. and for main() the while(aio_error(&my_aiocb) == EINPROGRESS) (which may waste a huge amount of CPU time for nothing) would be deleted and/or possibly replaced with something else (e.g. a pthread_cond_wait() to wait until the code in finish_aio() does a pthread_cond_signal() to tell the main thread it can continue).
To understand this, let's take a look at what pure polling would look like:
int main() {
struct aiocb my_aiocb;
int aio_return;
...
//Open file, take care of any other prelims, then
//Fill in file-specific info for my_aiocb, then
my_aiocb.aio_sigevent.sigev_notify = SIGEV_NONE; /* CHANGED! */
// my_aiocb.aio_sigevent.sigev_notify_function = finish_aio;
// my_aiocb.aio_sigevent.sigev_notify_attributes = NULL;
// my_aiocb.aio_sigevent.sigev_value.sival_ptr = &info_on_file;
// then read the file:
aio_return = aio_read(&my_aiocb);
// do stuff that doesn't need data that is being read ...
// then block execution until read is complete:
while(aio_error(&my_aiocb) == EINPROGRESS) {}
finish_aio(sigval_t sigval); /* ADDED! */
}
In this case it behaves almost the same as your original code, except that there's no extra thread (and you can't care if the "thread that doesn't exist" is started before or after aio_error(&my_aiocb) returns a value other than EINPROGRESS).
The problem with pure polling is that the while(aio_error(&my_aiocb) == EINPROGRESS) could waste a huge amount of CPU time constantly checking when nothing has happened yet.
The main purpose of using my_aiocb.aio_sigevent.sigev_notify = SIGEV_THREAD is to avoid wasting a possibly huge amount of CPU time polling when nothing changed (not forgetting that in some cases wasting CPU time polling like this can prevent other threads, including the finish_aio() thread, from getting CPU time). In other words, you want to delete the while(aio_error(&my_aiocb) == EINPROGRESS) loop, so you used SIGEV_THREAD so that you can delete that polling loop.
The new problem is that (if the main thread has to wait until the data is ready) you need some other way for the main thread to wait until the data is ready. However, typically it's not "the aio_read() completed" that you actually care about, it's something else. For example, maybe the raw file data is a bunch of values in a text file (like "12, 34, 56, 78") and you want to parse that data and create an array of integers, and want to notify the main thread that the array of integers is ready (and don't want to notify the main thread if you're starting to parse the file's data). It might be like:
int parsed_file_result = 0;
void finish_aio(sigval_t sigval) {
struct aiocb * my_aiocb = (struct aiocb *)sigval;
int status;
status = aio_error(&my_aiocb);
close(my_aiocb->aio_fildes);
if(status == 0) {
/* Read was successful */
parsed_file_result = parse_file_data(); /* Create the array of integers */
} else {
/* Read failed, handle the error somehow */
parsed_file_result = -1; /* Tell main thread we failed to create the array of integers */
}
/* Tell the main thread it can continue somehow */
}
One of the best ways to tell the main thread it can continue (at the end of finish_aio()) is to use pthread conditional variables (e.g. pthread_cond_signal() called at the end of finish_aio(); with pthread_cond_wait() in the main thread). In this case the main thread will simply block (the kernel/scheduler will not give it any CPU time until pthread_cond_signal() is called) so it wastes no CPU time polling.
Sadly, pthread conditional variables aren't trivial (they require a mutex, initialization, etc), and teaching/showing their use here is a little too far from the original topic. Fortunately; you shouldn't have much trouble finding a good tutorial elsewhere.
The important part is that if you used SIGEV_THREAD (so that you can delete that awful while(aio_error(&my_aiocb) == EINPROGRESS) polling loop) you're left with no reason to call aio_error(&my_aiocb) until after the finish_aio() has already been started; and no reason to care if aio_error(&my_aiocb) would've been changed (or not) before finish_aio() is started.
I'm working on an embedded processor running Yocto. I have a modified uio_pdrv_genirq.c UIO driver.
I am writing a library to control the DMA. There is one function which writes to the device file and initiates the DMA. A second function is intended to wait for the DMA to complete by calling select(). Whilst DMA is in progress the device file blocks. On completion the DMA controller issues an interrupt which releases the block on the device file.
I have the system working as expected using read() but I want to switch to select() so that I can include a time out. However, when I use select(), it doesn't seem to be recognising the block and always returns immediately (before the DMA has completed). I have included a simple version of the code:
int gannet_dma_interrupt_wait(dma_device_t *dma_device,
dma_direction dma_transfer_direction) {
fd_set rfds;
struct timeval timeout;
int select_res;
/* Initialize the file descriptor set and add the device file */
FD_ZERO(&rfds);
FD_SET(dma_device->fd, &rfds);
/* Set the timeout period. */
timeout.tv_sec = 5;
timeout.tv_usec = 0;
/* The device file will block until the DMA transfer has completed. */
select_res = select(FD_SETSIZE, &rfds, NULL, NULL, &timeout);
/* Reset the channel */
gannet_dma_reset(dma_device, dma_transfer_direction);
if (select_res == -1) {
/* Select has encountered an error */
perror("ERROR <Interrupt Select Failed>\n");
exit(0);
}
else if (select_res == 1) {
/* The device file descriptor block released */
return 0;
}
else {
/* The device file descriptor block exceeded timeout */
return EINTR;
}
}
Is there anything obviously wrong with my code? Or can anyone suggest an alternative to select?
It turns out that the UIO driver contains two counters. One records the
number of events (event_count), the other records how many events the
calling function is aware of (listener->event_count).
When you do a read() on a UIO driver it returns the number of events and
makes listener->event_count equal to event_count. ie. the listener is
now up to date with all the events that have occurred.
When you use poll() or select() on a UIO driver, it checks if these two
numbers are different and returns if they are (if they are the same it
waits until they differ and then returns). It does NOT update the
listener->event_count.
Clearly if you do not do a read() between calls to select() then
the listener->event_count will not match the event_count and the second
select() will return immediately. Therefore it is necessary to call
read() in between calls to select().
With hindsight it seems clear that select() should work in this way but it wasn't obvious to me at the time.
This answer assumes that it is possible to use select() as intented for the specified device file (I use select() for socket descriptors only). As an alternative function to select(), you may want to check out the poll() family of functions. What follows will hopefully at least offer hints as to what can be done to resolve your problem with calling select().
The first parameter to the select() function has to be the maximum despriptor number plus 1. Since you have only one descriptor, you can pass it directly to select() as its first parameter and add 1. Also consider that the file descriptor in dma_device could be invalid. Returning EINTR on a timeout may actually be what you intend to do but should that not be the case and to test for an invalid descriptor, here is a different version for you to consider. The select() call could be interrupted by a signal, in which case, the return value is -1 and errno will be set to EINTR. This could be handled internally by your function as in:
FD_ZERO(&rfds);
FD_SET(dma_device->fd, &rfds);
timeout.tv_sec = 5;
timeout.tv_usec = 0;
// restart select() if it's interrupted by a signal;
do {
select_res = select(dma_device->fd + 1, &rfds, NULL, NULL, &timeout);
}
while( select_res < 0 && errno == EINTR);
if (select_res > 0) {
// a file descriptor is legible
}
else {
if (select_res == 0) {
// select() timed-out
}
else {
// an error other than a signal occurred
if (errno == EBADF) {
// your file descriptor is invalid
}
}
}
my question is that i am using rawsocket passing high rate ( larger than 50kpps) traffic, two threads, one is to send ( read from buffer), another one is to receive ( write to buffer).
i have to use while(1) loop to make sure an infinite loop, and i cannot use usleep since then i will loose packet ( i have tried that)... now the cpu usage is 100% and i think i am buring my cpu...
here is the code:
while (1)
{
if (sendIndex == PACKET_COUNT_MAX)
{
sendIndex = 0;
}
else if (ringBuffer[sendIndex].drop == 0)
{if(sendtosocket (ringBuffer, sendIndex, rawout) < 0)
a++;
else
sendIndex++;}
else if (ringBuffer[sendIndex].drop == 1) {
ringBuffer[sendIndex].header.free = 1;
memset (ringBuffer[sendIndex].data, 0, sizeof (ringBuffer[sendIndex].data));
sendIndex++;
}
else
{
a++;
}
//nanosleep((struct timespec[]){{0, 5}}, NULL);
}
Thanks in advance!!!!!!!
Lisa
You need to pass the control over to the kernel. The command you may find useful is select. Check out the whole story on http://manpages.courier-mta.org/htmlman2/select.2.html. For more info, http://www.gnu.org/software/libc/manual/html_node/Waiting-for-I_002fO.html.
It's all about knowing you have nothing else to do except wait for input from the network. Or the file system. Or anything else that is a file descriptor (U*ix lingo). So, you let the kernel awake you once you've got something to process.
You can try
Increasing the receive buffer
Slowing down the sendto with a sleep, that should not lose packets
Use the MSG_WAITALL flag with recvfrom to make it a blocking read, and make sure the socket was not opened with SOCK_NONBLOCK or O_NONBLOCK
You need sane synchronization between your threads. This includes:
Using locks of some kind to ensure that a variable isn't read by one thread while it is, or might be, modified in another.
Using some kind of waiting scheme so that the sending thread can wait for there to be work for it do without spinning.
Check out pthread_mutex_lock and pthread_cond_wait (assuming you're using POSIX threads).
I have a worker thread that gets work from pipe. Something like this
void *worker(void *param) {
while (!work_done) {
read(g_workfds[0], work, sizeof(work));
do_work(work);
}
}
I need to implement a 1 second timer in the same thread do to some book-keeping about the work. Following is what I've in mind:
void *worker(void *param) {
prev_uptime = get_uptime();
while (!work_done) {
// set g_workfds[0] as non-block
now_uptime = get_uptime();
if (now_uptime - prev_uptime > 1) {
do_book_keeping();
prev_uptime = now_uptime;
}
n = poll(g_workfds[0], 1000); // Wait for 1 second else timeout
if (n == 0) // timed out
continue;
read(g_workfds[0], work, sizeof(work));
do_work(work); // This can take more than 1 second also
}
}
I am using system uptime instead of system time because system time can get changed while this thread is running. I was wondering if there is any other better way to do this. I don't want to consider using another thread. Using alarm() is not an option as it already used by another thread in same process. This is getting implemented in Linux environment.
I agree with most of what webbi wrote in his answer. But there is one issue with his suggestion of using time instead of uptime. If the system time is updated "forward" it will work as intended. But if the system time is set back by say 30 seconds, then there will be no book keeping done for 30 seconds as (now_time - prev_time) will be negative (unless an unsigned type is used, in which case it will work anyway).
An alternative would be to use clock_gettime() with CLOCK_MONOTONIC as clockid ( http://linux.die.net/man/2/clock_gettime ). A bit messy if you don't need smaller time units than seconds.
Also, adding code to detect a backwards clock jump isn't hard either.
I have found a better way but it is Linux specific using timerfd_create() system call. It takes care of system time change. Following is possible psuedo code:
void *worker(void *param) {
int timerfd = timerfd_create(CLOCK_MONOTONIC, 0); // Monotonic doesn't get affected by system time change
// set timerfd to non-block
timerfd_settime(timerfd, 1 second timer); // timer starts
while (!work_done) {
// set g_workfds[0] as non-block
n = poll(g_workfds[0] and timerfd, 0); // poll on both pipe and timerfd and Wait indefinetly
if (timerfd is readable)
do_book_keeping();
if (g_workfds[0] is readable) {
read(g_workfds[0], work, sizeof(work));
do_work(work); // This can take more than 1 second also
}
}
}
It seems cleaner and read() on timerfd returns extra time elapsed in case do_work() takes long time which is quite useful as do_book_keeping() expects to get called every second.
I found some things weird in your code...
poll() has 3 args, you are passing 2, the second arg is the number of structs that you are passing in the struct array of first param, the third param is the timeout.
Reference: http://linux.die.net/man/2/poll
Besides that, it's fine for me that workaround, it's not the best of course, but it's fine without involving another thread or alarm(), etc.
You use time and not uptime, it could cause you one error if the system date gets changed, but then it will continue working as it will be updated and continuing waiting for 1 sec, no matter what time is.
I'm using select() on a Linux/ARM platform to see if a udp socket has received a packet. I'd like to know how much time was remaining in the select call if it returns before the timeout (having detected a packet).
Something along the lines of:
int wait_fd(int fd, int msec)
{
struct timeval tv;
fd_set rws;
tv.tv_sec = msec / 1000ul;
tv.tv_usec = (msec % 1000ul) * 1000ul;
FD_ZERO( & rws);
FD_SET(fd, & rws);
(void)select(fd + 1, & rws, NULL, NULL, & tv);
if (FD_ISSET(fd, &rws)) { /* There is data */
msec = (tv.tv_sec * 1000) + (tv.tv_usec / 1000);
return(msec?msec:1);
} else { /* There is no data */
return(0);
}
}
The safest thing is to ignore the ambiguous definition of select() and time it yourself.
Just get the time before and after the select and subtract that from the interval you wanted.
If I recall correctly, the select() function treats the timeout and an I/O parameter and when select returns the time remaining is returned in the timeout variable.
Otherwise, you will have to record the current time before calling, and again after and obtain the difference between the two.
From "man select" on OSX:
Timeout is not changed by select(), and may be reused on subsequent calls, however it
is good style to re-ini-tialize it before each invocation of select().
You'll need to call gettimeofday before calling select, and then gettimeofday on exit.
[Edit] It seems that linux is slightly different:
(ii) The select function may update the timeout parameter to indicate
how much time was left. The pselect function does not change
this parameter.
On Linux, the function select modifies timeout to reflect the amount of
time not slept; most other implementations do not do this. This causes
problems both when Linux code which reads timeout is ported to other
operating systems, and when code is ported to Linux that reuses a
struct timeval for multiple selects in a loop without reinitializing
it. Consider timeout to be undefined after select returns.
Linux select() updates the timeout argument to reflect the time that has past.
Note that this is not portable across other systems (hence the warning in the OS X manual quoted above) but does work with Linux.
Gilad
Do not use select, try with fd larger than 1024 with your code and see what you will get.