I'm trying to understand how curl_multi_perform works.
The documentation says that:
This function performs transfers on all the added handles that need
attention in an non-blocking fashion. The easy handles have previously
been added to the multi handle with curl_multi_add_handle.
When an application has found out there's data available for the
multi_handle or a timeout has elapsed, the application should call
this function to read/write whatever there is to read or write right
now etc.
Question 1: What does the "application should call" mean? How can an application cause something? Did you mean the programmer should call ?
OK, I found two simple usage examples - "curl_multi_perform":
1 - https://everything.curl.dev/libcurl/drive/multi
int transfers_running;
do {
curl_multi_wait ( multi_handle, NULL, 0, 1000, NULL);
curl_multi_perform ( multi_handle, &transfers_running );
} while (transfers_running);
2 - enter link description here
int still_running;
do {
CURLMcode mc = curl_multi_perform(multi_handle, &still_running);
if(!mc && still_running)
/* wait for activity, timeout or "nothing" */
mc = curl_multi_poll(multi_handle, NULL, 0, 1000, NULL);
if(mc) {
fprintf(stderr, "curl_multi_poll() failed, code %d.\n", (int)mc);
break;
}
/* if there are still transfers, loop! */
} while(still_running);
-Firstly:
in the first example curl_multi_perform is called after curl_multi_wait.
in the second example curl_multi_perform is called before curl_multi_wait.
Nothing is clear.
Secondly:
Why do I need to call curl_multi_perform in a loop ?? I do not understand.
Why is one call not enough ?
Question 1: What does the "application should call" mean? How can an application cause something? Did you mean the programmer should call ?
Programmers don't call functions. Programmers write programs that tell the computer what to do. So this means that the programmer should write code that tells the application to call the function.
in the first example curl_multi_perform is called after curl_multi_wait.
in the second example curl_multi_perform is called before curl_multi_wait.
Either order works. As the documentation says:
This function does not require that there actually is any data available for reading or that data can be written, it can be called just in case.
If there's nothing available, it will simply return immediately, updating transfers_running.
Why do I need to call curl_multi_perform in a loop ?? I do not understand.
Because multiple transfers are in progress. curl_multi_wait() returns as soon as data is available on any of them. After you process that data, you need to continue waiting for other transfers.
Also, this doesn't wait for transfers to be complete, it processes partial data as it arrives. So you have to keep calling it until you've sent or received everything.
Related
So I have just discovered that libuv is a fairly small library as far as C libraries go (compare to FFmpeg). I have spent the past 6 hours reading through the source code to get a feel for the event loop at a deeper level. But still not seeing where the "nonblockingness" is implemented. Where some event interrupt signal or whatnot is being invoked in the codebase.
I have been using Node.js for over 8 years so I am familar with how to use an async non-blocking event loop, but I never actually looked into the implementation.
My question is twofold:
Where exactly is the "looping" occuring within libuv?
What are the key steps in each iteration of the loop that make it non-blocking and async.
So we start with a hello world example. All that is required is this:
#include <stdio.h>
#include <stdlib.h>
#include <uv.h>
int main() {
uv_loop_t *loop = malloc(sizeof(uv_loop_t));
uv_loop_init(loop); // initialize datastructures.
uv_run(loop, UV_RUN_DEFAULT); // infinite loop as long as queue is full?
uv_loop_close(loop);
free(loop);
return 0;
}
The key function which I have been exploring is uv_run. The uv_loop_init function essentially initializes data structures, so not too much fancness there I don't think. But the real magic seems to happen with uv_run, somewhere. A high level set of code snippets from the libuv repo is in this gist, showing what the uv_run function calls.
Essentially it seems to boil down to this:
while (NOT_STOPPED) {
uv__update_time(loop)
uv__run_timers(loop)
uv__run_pending(loop)
uv__run_idle(loop)
uv__run_prepare(loop)
uv__io_poll(loop, timeout)
uv__run_check(loop)
uv__run_closing_handles(loop)
// ... cleanup
}
Those functions are in the gist.
uv__run_timers: runs timer callbacks? loops with for (;;) {.
uv__run_pending: runs regular callbacks? loops through queue with while (!QUEUE_EMPTY(&pq)) {.
uv__run_idle: no source code
uv__run_prepare: no source code
uv__io_poll: does io polling? (can't quite tell what this means tho). Has 2 loops: while (!QUEUE_EMPTY(&loop->watcher_queue)) {, and for (;;) {,
And then we're done. And the program exists, because there is no "work" to be done.
So I think I have answered the first part of my question after all this digging, and the looping is specifically in these 3 functions:
uv__run_timers
uv__run_pending
uv__io_poll
But not having implemented anything with kqueue or multithreading and having dealt relatively little with file descriptors, I am not quite following the code. This will probably help out others along the path to learning this too.
So the second part of the question is what are the key steps in these 3 functions that implement the nonblockingness? Assuming this is where all the looping exists.
Not being a C expert, does for (;;) { "block" the event loop? Or can that run indefinitely and somehow other parts of the code are jumped to from OS system events or something like that?
So uv__io_poll calls poll(...) in that endless loop. I don't think is non-blocking, is that correct? That seems to be all it mainly does.
Looking into kqueue.c there is also a uv__io_poll, so I assume the poll implementation is a fallback and kqueue on Mac is used, which is non-blocking?
So is that it? Is it just looping in uv__io_poll and each iteration you can add to the queue, and as long as there's stuff in the queue it will run? I still don't see how it's non-blocking and async.
Can one outline similar to this how it is async and non-blocking, and which parts of the code to take a look at? Basically, I would like to see where the "free processor idleness" exists in libuv. Where is the processor ever free in the call to our initial uv_run? If it is free, how does it get reinvoked, like an event handler? (Like a browser event handler from the mouse, an interrupt). I feel like I'm looking for an interrupt but not seeing one.
I ask this because I want to implement an MVP event loop in C, but just don't understand how nonblockingness actually is implemented. Where the rubber meets the road.
I think that trying to understand libuv is getting in your way of understanding how reactors (event loops) are implemented in C, and it is this that you need to understand, as opposed to the exact implementation details behind libuv.
(Note that when I say "in C", what I really means is "at or near the system call interface, where userland meets the kernel".)
All of the different backends (select, poll, epoll, etc) are, more-or-less, variations on the same theme. They block the current process or thread until there is work to be done, like servicing a timer, reading from a socket, writing to a socket, or handling a socket error.
When the current process is blocked, it literally is not getting any CPU cycles assigned to it by the OS scheduler.
Part of the issue behind understanding this stuff IMO is the poor terminology: async, sync in JS-land, which don't really describe what these things are. Really, in C, we're talking about non-blocking vs blocking I/O.
When we read from a blocking file descriptor, the process (or thread) is blocked -- prevented from running -- until the kernel has something for it to read; when we write to a blocking file descriptor, the process is blocked until the kernel accepts the entire buffer.
In non-blocking I/O, it's exactly the same, except the kernel won't stop the process from running when there is nothing to do: instead, when you read or write, it tells you how much you read or wrote (or if there was an error).
The select system call (and friends) prevent the C developer from having to try and read from a non-blocking file descriptor over and over again -- select() is, in effect, a blocking system call that unblocks when any of the descriptors or timers you are watching are ready. This lets the developer build a loop around select, servicing any events it reports, like an expired timeout or a file descriptor that can be read. This is the event loop.
So, at its very core, what happens at the C-end of a JS event loop is roughly this algorithm:
while(true) {
select(open fds, timeout);
did_the_timeout_expire(run_js_timers());
for (each error fd)
run_js_error_handler(fdJSObjects[fd]);
for (each read-ready fd)
emit_data_events(fdJSObjects[fd], read_as_much_as_I_can(fd));
for (each write-ready fd) {
if (!pendingData(fd))
break;
write_as_much_as_I_can(fd);
pendingData = whatever_was_leftover_that_couldnt_write;
}
}
FWIW - I have actually written an event loop for v8 based around select(): it really is this simple.
It's important also to remember that JS always runs to completion. So, when you call a JS function (via the v8 api) from C, your C program doesn't do anything until the JS code returns.
NodeJS uses some optimizations like handling pending writes in a separate pthreads, but these all happen in "C space" and you shouldn't think/worry about them when trying to understand this pattern, because they're not relevant.
You might also be fooled into the thinking that JS isn't run to completion when dealing with things like async functions -- but it absolutely is, 100% of the time -- if you're not up to speed on this, do some reading with respect to the event loop and the micro task queue. Async functions are basically a syntax trick, and their "completion" involves returning a Promise.
I just took a dive into libuv's source code, and found at first that it seems like it does a lot of setup, and not much actual event handling.
Nonetheless, a look into src/unix/kqueue.c reveals some of the inner mechanics of event handling:
int uv__io_check_fd(uv_loop_t* loop, int fd) {
struct kevent ev;
int rc;
rc = 0;
EV_SET(&ev, fd, EVFILT_READ, EV_ADD, 0, 0, 0);
if (kevent(loop->backend_fd, &ev, 1, NULL, 0, NULL))
rc = UV__ERR(errno);
EV_SET(&ev, fd, EVFILT_READ, EV_DELETE, 0, 0, 0);
if (rc == 0)
if (kevent(loop->backend_fd, &ev, 1, NULL, 0, NULL))
abort();
return rc;
}
The file descriptor polling is done here, "setting" the event with EV_SET (similar to how you use FD_SET before checking with select()), and the handling is done via the kevent handler.
This is specific to the kqueue style events (mainly used on BSD-likes a la MacOS), and there are many other implementations for different Unices, but they all use the same function name to do nonblocking IO checks. See here for another implementation using epoll.
To answer your questions:
1) Where exactly is the "looping" occuring within libuv?
The QUEUE data structure is used for storing and processing events. This queue is filled by the platform- and IO- specific event types you register to listen for. Internally, it uses a clever linked-list using only an array of two void * pointers (see here):
typedef void *QUEUE[2];
I'm not going to get into the details of this list, all you need to know is it implements a queue-like structure for adding and popping elements.
Once you have file descriptors in the queue that are generating data, the asynchronous I/O code mentioned earlier will pick it up. The backend_fd within the uv_loop_t structure is the generator of data for each type of I/O.
2) What are the key steps in each iteration of the loop that make it non-blocking and async?
libuv is essentially a wrapper (with a nice API) around the real workhorses here, namely kqueue, epoll, select, etc. To answer this question completely, you'd need a fair bit of background in kernel-level file descriptor implementation, and I'm not sure if that's what you want based on the question.
The short answer is that the underlying operating systems all have built-in facilities for non-blocking (and therefore async) I/O. How each system works is a little outside the scope of this answer, I think, but I'll leave some reading for the curious:
https://www.quora.com/Network-Programming-How-is-select-implemented?share=1
The first thing to keep in mind is that work must be added to libuv's queues using its API; one cannot just load up libuv, start its main loop, and then code up some I/O and get async I/O.
The queues maintained by libuv are managed by looping. The infinite loop in uv__run_timers isn't actually infinite; notice that the first check verifies that a soonest-expiring timer exists (presumably, if the list is empty, this is NULL), and if not, breaks the loop and the function returns. The next check breaks the loop if the current (soonest-expiring) timer hasn't expired. If neither of those conditions breaks the loop, the code continues: it restarts the timer, calls its timeout handler, and then loops again to check more timers. Most times when this code runs, it's going to break the loop and exit, allowing the other loops to run.
What makes all this non-blocking is the caller/user following the guidelines and API of libuv: adding your work to queues, and allowing libuv to perform its work on those queues. Processing-intensive work may block these loops and other work from running, so it's important to break your work into chunks.
btw, uv__run_idle, uv__run_check, uv__run_prepare 's source code is defined on src/unix/loop-watcher.c
Is there a non-blocking function which returns the current rx queue length of a serial port in Windows, using C?
All examples I've seen simply call ReadFile which blocks until the specified timeout, so I was wondering if it's possible to check if there is anything in the buffer before reading?
E.g. I can simply do this for each character:
void ReadCharacter(char *theCharacter)
{
DWORD numBytesRead = 0;
while (numBytesRead == 0)
{
ReadFile(comPorthandle,
theCharacter,
sizeof(char),
&numBytesRead,
NULL);
}
}
But is it possible to have something like
int numBytesRx = GetNumBytesRx(&portHandle);
if (numBytesRx > 0)
Read(&portHandle, targetBuffer, numBytesRead);
To perform async IO with COM port via ReadFile consider using last parameter of function, LPOVERLAPPED and OVERLAPPED structure. OVERLAPPED is common practice for async IO in Windows.
Here you can find examples
ReadFile will always return whatever is already in the rx buffer. So if you set no timeouts, you'll get the contents instantly.
Though please note that no professional application puts ReadFile in a busy-wait loop. Not only will this needlessly use up CPU, it will also block the thread where the loop exists.
So you should put ReadFile inside a thread of its own. This is common practice with all I/O functions. This will solve the blocking problem, but you'll still have the problem with high CPU use.
As an alternative you can use what Windows calls "asynchronous I/O" (*), by using the ReadFileEx function. It lets you specify a callback function, which will get triggered whenever you actually receive some data.
Now if you combine "asynchronous I/O" with threading, you get a non-blocking communication which consumes no CPU when there is no data to process. Your I/O thread could either wait for I/O with SleepEx, or it could WaitFor an event that you set manually from inside the callback.
(*) "Asynchronous I/O" is a nonsense Windows term, since technically, all serial port communication is always asynchronous. Would you send data synchronously with no pause in between, there would just be no way for a slow desktop PC to keep up.
When writing code I often have checks to see if errors occurred. An example would be:
char *x = malloc( some_bytes );
if( x == NULL ){
fprintf( stderr, "Malloc failed.\n" );
exit(EXIT_FAILURE);
}
I've also used strerror( errno ) in the past.
I've only ever written small desktop appications where it doesn't matter if the program exit()ed in case of an error.
Now, however, I'm writing C code for an embedded system (Arduino) and I don't want the system to just exit in case of an error. I want it to go to a particular state/function where it can power down systems, send error reports and idle safely.
I could simply call an error_handler() function, but I could be deep in the stack and very low on memory, leaving error_handler() inoperable.
Instead, I'd like execution to effectively collapse the stack, free up a bunch of memory and start sorting out powering down and error reporting. There is a serious fire risk if the system doesn't power down safely.
Is there a standard way that safe error handling is implemented in low memory embedded systems?
EDIT 1:
I'll limit my use of malloc() in embedded systems. In this particular case, the errors would occur when reading a file, if the file was not of the correct format.
Maybe you're waiting for the Holy and Sacred setjmp/longjmp, the one who came to save all the memory-hungry stacks of their sins?
#include <setjmp.h>
jmp_buf jumpToMeOnAnError;
void someUpperFunctionOnTheStack() {
if(setjmp(jumpToMeOnAnError) != 0) {
// Error handling code goes here
// Return, abort(), while(1) {}, or whatever here...
}
// Do routinary stuff
}
void someLowerFunctionOnTheStack() {
if(theWorldIsOver)
longjmp(jumpToMeOnAnError, -1);
}
Edit: Prefer not to do malloc()/free()s on embedded systems, for the same reasons you said. It's simply unhandable. Unless you use a lot of return codes/setjmp()s to free the memory all the way up the stack...
If your system has a watchdog, you could use:
char *x = malloc( some_bytes );
assert(x != NULL);
The implementation of assert() could be something like:
#define assert (condition) \
if (!(condition)) while(true)
In case of a failure the watchdog would trigger, the system would make a reset. At restart the system would check the reset reason, if the reset reason was "watchdog reset", the system would goto a safe state.
update
Before entering the while loop, assert cold also output a error message, print the stack trace or save some data in non volatile memory.
Is there a standard way that safe error handling is implemented in low memory embedded systems?
Yes, there is an industry de facto way of handling it. It is all rather simple:
For every module in your program you need to have a result type, such as a custom enum, which describes every possible thing that could go wrong with the functions inside that module.
You document every function properly, stating what codes it will return upon error and what code it will return upon success.
You leave all error handling to the caller.
If the caller is another module, it too passes on the error to its own caller. Possibly renames the error into something more suitable, where applicable.
The error handling mechanism is located in main(), at the bottom of the call stack.
This works well together with classic state machines. A typical main would be:
void main (void)
{
for(;;)
{
serve_watchdog();
result = state_machine();
if(result != good)
{
error_handler(result);
}
}
}
You should not use malloc in bare bone or RTOS microcontroller applications, not so much because of safety reasons, but simple because it doesn't make any sense whatsoever to use it. Apply common sense when programming.
Use setjmp(3) to set a recovery point, and longjmp(3) to jump to it, restoring the stack to what it was at the setjmp point. It wont free malloced memory.
Generally, it is not a good idea to use malloc/free in an embedded program if it can be avoided. For example, a static array may be adequate, or even using alloca() is marginally better.
to minimize stack usage:
write the program so the calls are in parallel rather than function calls sub function that calls sub function that calls sub function.... I.E. top level function calls sub function where sub function promptly returns, with status info. top level function then calls next sub function... etc
The (bad for stack limited) nested method of program architecture:
top level function
second level function
third level function
forth level function
should be avoided in embedded systems
the preferred method of program architecture for embedded systems is:
top level function (the reset event handler)
(variations in the following depending on if 'warm' or 'cold' start)
initialize hardware
initialize peripherals
initialize communication I/O
initialize interrupts
initialize status info
enable interrupts
enter background processing
interrupt handler
re-enable the interrupt
using 'scheduler'
select a foreground function
trigger dispatch for selected foreground function
return from interrupt
background processing
(this can be, and often is implemented as a 'state' machine rather than a loop)
loop:
if status info indicates need to call second level function 1
second level function 1, which updates status info
if status info indicates need to call second level function 2
second level function 2, which updates status info
etc
end loop:
Note that, as much as possible, there is no 'third level function x'
Note that, the foreground functions must complete before they are again scheduled.
Note: there are lots of other details that I have omitted in the above, like
kicking the watchdog,
the other interrupt events,
'critical' code sections and use of mutex(),
considerations between 'soft real-time' and 'hard real-time',
context switching
continuous BIT, commanded BIT, and error handling
etc
I am a bit unsure about this question, but I constantly running into troubles with my current design, and would be really greatful, if someone could show be a different approach to this.
My program writes async commands to a device via rs232, while constantly reading and reacting to received data.
This works all nice and neat, but during my init-phase, I have to send a bunch of commands, which would have to wait for a response, before letting the program countinue.
Now, this part I have to do in plain C, and all I could come up with was to use global vars and while loops. But I find this really not pretty.
Take this as an pseudo-code example:
OnReceive(data){
switch determineCommand(data)
case CMD1:
config.value1 = data.value1
case CMD2:
config.value2 = data.value2
default:
print data
}
DoCommandChain(){
Send(CMD1)
If("Send(CMD1)" got its response){
Send(CMD2)
}
}
Now, the problem is "If XY got its response" - because I don't want to use some vars to detect this and I can't rely on a return value for Send(CMD1), because this is just an indicator, that CMD1 was send - not there was something corresponding received.
I am asking here, because I would like to know what I could search for / read about, to solve this mess in a nice way.
Right now, my best idea for this would be to somehow setup a timer-guarded-function to monitor, if a specific response is received. Then, depending on the status of this response, go to the next Send or retry the last.
Like this:
Instead of `Send(CMD1)` -> `DoUntilResponse(Send(CMD1),Timeout,NumberOfRetries)`
DoUntilResponse(function,Timeout,NumberOfRetries){
registerExpectedResponse(CMD1, gotResponse) //awaiting some response for CMD1
for(i=0 ; i!=Numberofretries; i++){
if (Send(CMD1) == successfulSend ){
while(not timeout){
if gotResponse then break;
}
if gotResponse then break;
}
}
}
Edit:
Just to clarify: I am not worried about the serial-connection, or about how to fire the OnReceive function - this is all working already. What I can't get a clear idea of, is how to solve the above pseudo-code without the use of polling and preferably in pure C.
Assuming that write and read operations are executed in different threads, use the following algorithm:
Write thread.
Register expected response.
Send packet to the device
Wait for Received event with timeout. If event is set, continue. Timeout -
report error and exit (or make additional trial).
Read thread.
For every received packet:
Add it to the input buffer.
Analyze buffer. If it contains valid response, set "Received" event,
releasing the Write thread, clear input buffer, and continue reading.
If input buffer contains unrecognized data, report error, clear buffer
and continue reading.
If input buffer contains the beginning of expected response,
continue reading.
Threading and event notification is OS-specific. If your C++ compiler doesn't support multithreading, use portable library like Boost, or use OS-specific API. Notice that Read thread implements stream parsing logic, since serial communication is stream-oriented.
Edit.
"Register expected response" means: set some program variable(s) than mean "Command of type X is sent to device". According to this variable, Read thread expects to receive the packet, which should be sent by device according to communication protocol (application-specific). Another received packet, which may be generally valid, should be treated as error, because this is not response to the command just sent to device.
Another way: Set expected response size. In this case, once Read thread received expected amount of data, it sets Received event, leaving packet recognition task to the sender.
It is operating system specific. On Linux or Unix you should consider using a multiplexing syscall like poll(2) (or perhaps select(2) which I feel is obsolete, because it limits the maximal file descriptor number, see the C10K problem).
The C standard don't know about serial ports.
I have to send a command over serial and receive back an answer based on the command and do something based on the message received. I was told that I have to use callbacks as this is an asynchronous operation.
I have a 2 threads, one that can send messages and one that receives the messages.
Example:
//Thread 1
sendMessage("Initialize");
//Thread 2
while(1)
{
checkForMessages();
}
How can I write a function that is initialized for a specific message and handles the message recieved.
Example:
CommHandle(Command,MsgReceived)
{
if(command)
{
if(MsgReceived == ok)
...
if(MsgReceived == error)
...
}
}
I was told that I have to use callbacks as this is an asynchronous operation.
Not necessarily. There is something in Windows called "asynchronous I/O", this is to be regarded as an internal Windows term, which is synonymous with "overlapped I/O" (explanation here). When you are using overlapped I/O, you will get a callback when the transmission is finished. This is nice, since it reduces CPU load, but it is not really necessary if your program has nothing better to do while waiting. So it depends on the nature of your application.
But no matter the nature of your application, you should indeed handle all serial communication through threads, so that you won't cause the main GUI thread to freeze up in embarrassing ways.
Having one rx and one tx thread gives you a dilemma though: they are using the same port handle and they cannot freely access it, because that wouldn't be thread-safe. The solution is to either make one single super-thread handling all transmissions, or to protect the port handle through a mutex.
I'm not sure which method that is best, I have no recommendation. I have only used the "super-thread" one myself: one obvious advantage was that I could centralize WaitFor instructions like "kill thread", "port is open", "port is closed" at one place. But at the same time the code turned out rather complex.
How can I write a function that is initialized for a specific message and handles the message recieved.
Let your thread(s) shovel their received data into some buffers. A tx buffer and a rx buffer. Depending on your serial protocol and performance, you might have to use double buffers: one that is used for the current transmission and one that contains the most recently received data.
Then from main, pick up the data from the buffers. They need to be thread-safe. Once you have gotten that far, simply write a parser like you would with any form of data and take actions from there