So I have just discovered that libuv is a fairly small library as far as C libraries go (compare to FFmpeg). I have spent the past 6 hours reading through the source code to get a feel for the event loop at a deeper level. But still not seeing where the "nonblockingness" is implemented. Where some event interrupt signal or whatnot is being invoked in the codebase.
I have been using Node.js for over 8 years so I am familar with how to use an async non-blocking event loop, but I never actually looked into the implementation.
My question is twofold:
Where exactly is the "looping" occuring within libuv?
What are the key steps in each iteration of the loop that make it non-blocking and async.
So we start with a hello world example. All that is required is this:
#include <stdio.h>
#include <stdlib.h>
#include <uv.h>
int main() {
uv_loop_t *loop = malloc(sizeof(uv_loop_t));
uv_loop_init(loop); // initialize datastructures.
uv_run(loop, UV_RUN_DEFAULT); // infinite loop as long as queue is full?
uv_loop_close(loop);
free(loop);
return 0;
}
The key function which I have been exploring is uv_run. The uv_loop_init function essentially initializes data structures, so not too much fancness there I don't think. But the real magic seems to happen with uv_run, somewhere. A high level set of code snippets from the libuv repo is in this gist, showing what the uv_run function calls.
Essentially it seems to boil down to this:
while (NOT_STOPPED) {
uv__update_time(loop)
uv__run_timers(loop)
uv__run_pending(loop)
uv__run_idle(loop)
uv__run_prepare(loop)
uv__io_poll(loop, timeout)
uv__run_check(loop)
uv__run_closing_handles(loop)
// ... cleanup
}
Those functions are in the gist.
uv__run_timers: runs timer callbacks? loops with for (;;) {.
uv__run_pending: runs regular callbacks? loops through queue with while (!QUEUE_EMPTY(&pq)) {.
uv__run_idle: no source code
uv__run_prepare: no source code
uv__io_poll: does io polling? (can't quite tell what this means tho). Has 2 loops: while (!QUEUE_EMPTY(&loop->watcher_queue)) {, and for (;;) {,
And then we're done. And the program exists, because there is no "work" to be done.
So I think I have answered the first part of my question after all this digging, and the looping is specifically in these 3 functions:
uv__run_timers
uv__run_pending
uv__io_poll
But not having implemented anything with kqueue or multithreading and having dealt relatively little with file descriptors, I am not quite following the code. This will probably help out others along the path to learning this too.
So the second part of the question is what are the key steps in these 3 functions that implement the nonblockingness? Assuming this is where all the looping exists.
Not being a C expert, does for (;;) { "block" the event loop? Or can that run indefinitely and somehow other parts of the code are jumped to from OS system events or something like that?
So uv__io_poll calls poll(...) in that endless loop. I don't think is non-blocking, is that correct? That seems to be all it mainly does.
Looking into kqueue.c there is also a uv__io_poll, so I assume the poll implementation is a fallback and kqueue on Mac is used, which is non-blocking?
So is that it? Is it just looping in uv__io_poll and each iteration you can add to the queue, and as long as there's stuff in the queue it will run? I still don't see how it's non-blocking and async.
Can one outline similar to this how it is async and non-blocking, and which parts of the code to take a look at? Basically, I would like to see where the "free processor idleness" exists in libuv. Where is the processor ever free in the call to our initial uv_run? If it is free, how does it get reinvoked, like an event handler? (Like a browser event handler from the mouse, an interrupt). I feel like I'm looking for an interrupt but not seeing one.
I ask this because I want to implement an MVP event loop in C, but just don't understand how nonblockingness actually is implemented. Where the rubber meets the road.
I think that trying to understand libuv is getting in your way of understanding how reactors (event loops) are implemented in C, and it is this that you need to understand, as opposed to the exact implementation details behind libuv.
(Note that when I say "in C", what I really means is "at or near the system call interface, where userland meets the kernel".)
All of the different backends (select, poll, epoll, etc) are, more-or-less, variations on the same theme. They block the current process or thread until there is work to be done, like servicing a timer, reading from a socket, writing to a socket, or handling a socket error.
When the current process is blocked, it literally is not getting any CPU cycles assigned to it by the OS scheduler.
Part of the issue behind understanding this stuff IMO is the poor terminology: async, sync in JS-land, which don't really describe what these things are. Really, in C, we're talking about non-blocking vs blocking I/O.
When we read from a blocking file descriptor, the process (or thread) is blocked -- prevented from running -- until the kernel has something for it to read; when we write to a blocking file descriptor, the process is blocked until the kernel accepts the entire buffer.
In non-blocking I/O, it's exactly the same, except the kernel won't stop the process from running when there is nothing to do: instead, when you read or write, it tells you how much you read or wrote (or if there was an error).
The select system call (and friends) prevent the C developer from having to try and read from a non-blocking file descriptor over and over again -- select() is, in effect, a blocking system call that unblocks when any of the descriptors or timers you are watching are ready. This lets the developer build a loop around select, servicing any events it reports, like an expired timeout or a file descriptor that can be read. This is the event loop.
So, at its very core, what happens at the C-end of a JS event loop is roughly this algorithm:
while(true) {
select(open fds, timeout);
did_the_timeout_expire(run_js_timers());
for (each error fd)
run_js_error_handler(fdJSObjects[fd]);
for (each read-ready fd)
emit_data_events(fdJSObjects[fd], read_as_much_as_I_can(fd));
for (each write-ready fd) {
if (!pendingData(fd))
break;
write_as_much_as_I_can(fd);
pendingData = whatever_was_leftover_that_couldnt_write;
}
}
FWIW - I have actually written an event loop for v8 based around select(): it really is this simple.
It's important also to remember that JS always runs to completion. So, when you call a JS function (via the v8 api) from C, your C program doesn't do anything until the JS code returns.
NodeJS uses some optimizations like handling pending writes in a separate pthreads, but these all happen in "C space" and you shouldn't think/worry about them when trying to understand this pattern, because they're not relevant.
You might also be fooled into the thinking that JS isn't run to completion when dealing with things like async functions -- but it absolutely is, 100% of the time -- if you're not up to speed on this, do some reading with respect to the event loop and the micro task queue. Async functions are basically a syntax trick, and their "completion" involves returning a Promise.
I just took a dive into libuv's source code, and found at first that it seems like it does a lot of setup, and not much actual event handling.
Nonetheless, a look into src/unix/kqueue.c reveals some of the inner mechanics of event handling:
int uv__io_check_fd(uv_loop_t* loop, int fd) {
struct kevent ev;
int rc;
rc = 0;
EV_SET(&ev, fd, EVFILT_READ, EV_ADD, 0, 0, 0);
if (kevent(loop->backend_fd, &ev, 1, NULL, 0, NULL))
rc = UV__ERR(errno);
EV_SET(&ev, fd, EVFILT_READ, EV_DELETE, 0, 0, 0);
if (rc == 0)
if (kevent(loop->backend_fd, &ev, 1, NULL, 0, NULL))
abort();
return rc;
}
The file descriptor polling is done here, "setting" the event with EV_SET (similar to how you use FD_SET before checking with select()), and the handling is done via the kevent handler.
This is specific to the kqueue style events (mainly used on BSD-likes a la MacOS), and there are many other implementations for different Unices, but they all use the same function name to do nonblocking IO checks. See here for another implementation using epoll.
To answer your questions:
1) Where exactly is the "looping" occuring within libuv?
The QUEUE data structure is used for storing and processing events. This queue is filled by the platform- and IO- specific event types you register to listen for. Internally, it uses a clever linked-list using only an array of two void * pointers (see here):
typedef void *QUEUE[2];
I'm not going to get into the details of this list, all you need to know is it implements a queue-like structure for adding and popping elements.
Once you have file descriptors in the queue that are generating data, the asynchronous I/O code mentioned earlier will pick it up. The backend_fd within the uv_loop_t structure is the generator of data for each type of I/O.
2) What are the key steps in each iteration of the loop that make it non-blocking and async?
libuv is essentially a wrapper (with a nice API) around the real workhorses here, namely kqueue, epoll, select, etc. To answer this question completely, you'd need a fair bit of background in kernel-level file descriptor implementation, and I'm not sure if that's what you want based on the question.
The short answer is that the underlying operating systems all have built-in facilities for non-blocking (and therefore async) I/O. How each system works is a little outside the scope of this answer, I think, but I'll leave some reading for the curious:
https://www.quora.com/Network-Programming-How-is-select-implemented?share=1
The first thing to keep in mind is that work must be added to libuv's queues using its API; one cannot just load up libuv, start its main loop, and then code up some I/O and get async I/O.
The queues maintained by libuv are managed by looping. The infinite loop in uv__run_timers isn't actually infinite; notice that the first check verifies that a soonest-expiring timer exists (presumably, if the list is empty, this is NULL), and if not, breaks the loop and the function returns. The next check breaks the loop if the current (soonest-expiring) timer hasn't expired. If neither of those conditions breaks the loop, the code continues: it restarts the timer, calls its timeout handler, and then loops again to check more timers. Most times when this code runs, it's going to break the loop and exit, allowing the other loops to run.
What makes all this non-blocking is the caller/user following the guidelines and API of libuv: adding your work to queues, and allowing libuv to perform its work on those queues. Processing-intensive work may block these loops and other work from running, so it's important to break your work into chunks.
btw, uv__run_idle, uv__run_check, uv__run_prepare 's source code is defined on src/unix/loop-watcher.c
I find myself in need for a repeating block of code I can execute. If I were in an object I could simply pass self to the NSTimer scheduling. I am in a pure C project at the moment and I don't see any obvious NSTimer analogs. Is there a correct way to approach this?
If you really want the CF way of doing this:
CFRunLoopTimer is the CoreFoundation counter-part you're probably looking for.
You'll need to construct a separate CFRunLoop on a different thread if you want the timer to fire an asynchronous job. Otherwise, you can use use the main application thread by calling CFRunLoopGetCurrent() and set that to be the "parent loop" (called a "mode") that responds to the timer's events.
Here's what you do:
Initialize a CFRunLoopTimerContext struct (on the stack is fine)
Calculate the next time at which the run loop fires using type CFAbsoluteTime and CFAbsoluteTimeGetCurrent() plus some CFTimeInterval (can be a double aka CGFloat)
Create the run loop timer with CFRunLoopTimerCreate, passing in the next fire time and the callback function pointer
Add the run loop timer to either a runloop on a separate thread or the main thread of the executable (see above)
I don't know much of anything about the foundation framework but you can hack together a timer using the mach/POSIX APIs which allow you to tap into kernel services and processes for your C application.
Using a combination of mach_absolute_time() and nanosleep() system calls may allow you to emulate a timer. It's a bit nasty if you've never done any systems programming on a *NIX like platform.
On Linux, it's much easier to implement a POSIX timer or make use of clock_gettime() and clock_nanosleep (POSIX functions) which are not available on the mac.
There's lots of info on Linux/Unix systems programming in C but not so much on a mac, so I'd recommend having a look at this question.
Also, I made my own little emulation of clock_gettime() and clock_nanosleep for the mac (shameless plug) Find it here
I would like to be able to 'capture' an hrtimer interrupt with a linux kernel module and replay the interrupt at a later period in time. Any thoughts on how to go about doing this?
Use case: A program that calls sleep(1). My module will grab the hrtimer interrupt when it fires after 1 second, wait for 'x' amount of time, then re-fire the interrupt, waking the process.
Note: I do not want to hook the sleep system call.
Thanks!
Quite honestly, writing a Linux kernel module just to modify the behavior of sleep() for a single application sounds like overkill.
For most cases you should be able to use a preloadable shared object to intercept/override the sleep() function family with your own implementations. The application will call your implementation and your code may then call the real function with a modified parameter list.
This method is much simpler and less intrusive than anything involving kernel programming, although it will not work if your application is statically linked or if it uses direct system calls instead of library functions.
In an embedded project, we're supposed to implement a task scheduler with different priorities, the project implementation is in C and is run on an Arduino device.
Now that we're in the researching phase, one question popped but nobody had experience enough to have a certain answer:
How is it possible to control the execution time of a function? How do we keep track of time before the function returns so we can interrupt it for example when a time-out occurs?
One suggestion was to use fork(), but since Arduino does not include an operation system, there's no kernel to handle a thread. Or am I wrong?
Any input will be helpful, thanks a bunch,
You need a timer. All non-cooperative multi tasking systems (i.e. those which don't depend on the function to say "you can interrupt me now" all the time) use a timer to stop the execution after some time (say 100ms).
In the interrupt handler, check if there is another "thread" which can run and switch context.
A pretty simple implementation is a "ready list": Whenever a task or thread could do some work, add it to the ready list.
When the timer fires, add the current task at the end of the list and make the head of the list the current task.
In an embedded system a task scheduler is the core of an operating system (usually an RTOS), so you are being asked to implement one not to use one.
A simple example of how such a scheduler works is described in Jean Labrosse's boot Micro C/OS-II. It describes a complete RTOS kernel with scheduling and IPC. For your project you can take the description of this core and implement your own (or you could use the included source code).
Such a kernel works by scheduling at certain OS calls and on a timer interrupt. A context switch involves storing the processor registers for one task and replacing then with teh registers for another. Because this register save/restore includes the stack-pointer and program counter, control is switched between threads.
It may be that simpler forms of scheduling (rather than preemptive) scheduling are called for. One method is to implement task functions that run to completion and where necessary store their own state and are implemented as state-machines, and then have a simple loop that polls a timer and call's each 'task' function according to a schedule table (that includes the periodicity of the task and a pointer to its function, so that say one function will be called every second, while another will be called every millisecond.
select() is a great system call. You can pack any number of file descriptors, socket descriptors, pipes, etc. and get notified in a synchronous fashion when input becomes available.
Is there a way to create an interval/oneshot timer and use it with select()? That would save me from having multiple threads for IO and timing.
timerfd_create does exactly this. It's a fairly recent addition to the linux kernel and might not be available on all distros yet though.
Use the timeout parameter - keep your timer events in a priority queue, check the top item and set the timeout accordingly - if the timeout is reached, then you can check that the event is ready to run, run the event and continue.
At least that's what I do.
Note that poll has a nicer interface (in some ways) and may be more efficient with lots of file descriptors.
MarkR has a nice portable solution, but here's another:
Use a POSIX timer (timer_create) and you can transform the problem into "select-able signals". This problem has a classic solution: writing to a pipe from the signal handler and selecting on the read end of the pipe.
Building on #MarkR, using a sorted structure to store callback+closure with an int and a pointer to an int. If the two ints have the same value then the event is active otherwise it was discarded.
This way events can be discarded simply by increasing an int. Perhaps not the most straightforward solution but it was all I could think of.
https://github.com/cheako/tor2web/tree/6ac67f80daaea01d14a5d07e6026e1af4258dc96/src
hextree.c contains the code for the data structure used.
schedule.c:156 is where the int is changed.
gnutls.c:197 is where the timers are created.