Let's say I have 2 sockets I am watching for events and I really want socket 1 to be preferred (event at the cost of starving socket 2). How would I do that with libuv ?
As an example when working with libevent one can use :
int event_priority_set(struct event *event, int priority);
When multiple events of multiple priorities become active, the
low-priority events are not run. Instead, Libevent runs the high
priority events, then checks for events again. Only when no
high-priority events are active are the low-priority events run.
There is no way to do that in libuv. In libuv there is no abstraction for an event (so no struct event or some equivalent stuff), all i/o callbacks are dispatched as they happen.
Related
Tell me please, here is a quote from reference section 3 ("Running an event loop") of Nick Mathewson's book on Libevent:
By default, the event_base_loop() function runs an event_base until
there are no more events registered in it.
Honestly, I just can't understand this phrase.
What means - "until there are no more events registered".
How does a loop know that there are no more new events ?? There can be a time interval between events - 1 second, 1 hour ...
How does Libevent know that events are over?
PS: I'm trying to compare the work of the event_base_loop () function with the GetQueuedCompletionStatus () function on windows and I don't get it. Because GetQueuedCompletionStatus works in an infinite loop - whether there are events or not - GetQueuedCompletionStatus waits for an infinitely new event and does not complete if there are none.
PS1:I was in a hurry and did not finish the chapter on "Working with an event loop".
It says that you can redefine the behavior of the event_base_loop () function by passing it a flag in the second parameter - EVLOOP_NO_EXIT_ON_EMPTY.
This will mean that the function will wait for new events indefinitely or until I want to exit the event_base_loop () loop manually via event_base_loopbreak / event_base_loopexit.
Note it says "no more events registered", rather than "no more events". By default, the event loop will exit when the event base runs empty, even though more events could happen later. This is restated in a different way both later in the chapter:
Ordinarily, the loop will exit as soon as it has no pending or active events.
and in the API documentation for event_base_loop():
By default, this loop will run the event base until either there are no more pending or active events, or until something calls event_base_loopbreak() or event_base_loopexit().
As noted in the question, EVLOOP_NO_EXIT_ON_EMPTY can change this behavior to work more like a typical event queue, waiting for events (unless EVLOOP_NONBLOCK was set) and dispatching them when they arrive.
"Running an event loop" contains pseudocode for event_base_loop() that should also help clarify the behavior.
In comparison, GetQueuedCompletionStatus will wait for and operate on a single packet (note: the wait is not likely a busyloop but based on some other multitasking technologies, such as interrupts and thread synchronization primitives; the exact method is an implementation detail, and shouldn't affect use). It's most similar to event_base_loop() with the flags EVLOOP_ONCE | EVLOOP_NO_EXIT_ON_EMPTY, though event_base_loop may process multiple events of the same priority if they happen to come in before the first event is fully handled. Coming from the other direction, if 0 is passed for the dwMilliseconds argument to GetQueuedCompletionStatus, it behaves like event_base_loop()'s default behavior, exiting immediately if there are no packets in the I/O completion queue.
Description:
Currently I am working on using Flink with an IOT setup. Essentially, devices are sending data such as (device_id, device_type, event_timestamp, etc) and I don't have any control over when the messages get sent. I then key the steam by device_id and device_type to preform aggregations. I would like to use event-time given that is ensures the timers which are set trigger in a deterministic nature given a failure. However, given that this isn't always a high throughput stream a window could be opened for a 10 minute aggregation period, but not have its next point come until approximately 40 minutes later. Although the calculation would aggregation would eventually be completed it would output my desired result extremely late.
So my work around for this is to create an additional external source that does nothing other than pump fake messages. By having these fake messages being pumped out in alignment with my 10 minute aggregation period, even if a device hadn't sent any data, the event time windows would have something to force the windows closed. The critical part here is to make it possible that all parallel instances / operators have access to this fake message because I need to close all the windows with this single fake message. I was thinking that Broadcast state might be the most appropriate way to accomplish this goal given: "Broadcast state is replicated across all parallel instances of a function, and might typically be used where you have two streams, a regular data stream alongside a control stream that serves rules, patterns, or other configuration messages." Quote Source
Questions:
Is broadcast state the best method for ensuring all parallel instances (e.g. windows) receive my fake messages?
Once the operators have access to this fake message via the broadcast state can this fake message then be used to advance the event time watermark?
You can make this work with broadcast state, along the lines you propose, but I'm not convinced it's the best solution.
In an ideal world I'd suggest you arrange for the devices to send occasional keepalive messages, but assuming that's not possible, I think a custom Trigger would work well here. You can extend the EventTimeTrigger so that in addition to the event time timer it creates via
ctx.registerEventTimeTimer(window.maxTimestamp());
you also create a processing time timer, as a fallback, and you FIRE the window if the window still exists when that processing time timer fires.
I'm recommending this approach because it's simpler and more directly addresses the specific need. With the broadcast state approach you'll have to introduce a source for these messages, add a broadcast state descriptor and stream, add special fake watermarks for the non-broadcast stream (set to Watermark.MAX_WATERMARK), connect the broadcast and non-broadcast streams and implement a BroadcastProcessFunction (that probably doesn't really do anything), etc. It's a lot of moving parts spread across several different operators.
When running an event loop in libuv using the uv_run function, there's a "mode" parameter that is used with the following values:
UV_RUN_DEFAULT
UV_RUN_ONCE
UV_RUN_NOWAIT
The first two are obvious. UV_RUN_DEFAULT runs the event loop until there are no more events, and UV_RUN_ONCE processing a single event from the loop. However, UV_RUN_NOWAIT doesn't seem to be a separate mode, but rather a flag that can be ORed with one of the other two values.
By default, this function blocks until events are done processing, and UV_RUN_NOWAIT makes it nonblocking, but any documentation I can find on it ends there. My question is, if you run the event loop nonblocking, how are callbacks handled?
The libuv event model is single-threaded (reactor pattern), so I'd assume it needs to block to be able to call the callbacks, but if the main thread is occupied, what happens to an event after it's processed? Will the callback be "queued" until libuv gets control of the main thread again? Or will the callbacks be dispatched on another thread?
Callbacks are handled in the same manner. They will run within the thread that is in uv_run().
Per the documentation:
UV_RUN_DEFAULT: Runs the event loop until the reference count drops to zero. Always returns zero.
UV_RUN_ONCE: Poll for new events once. Note that this function blocks if there are no pending events. Returns zero when done (no active handles or requests left), or non-zero if more events are expected (meaning you should run the event loop again sometime in the future).
UV_RUN_NOWAIT: Poll for new events once but don't block if there are no pending events.
Consider the case where a program has a single watcher listening to a socket. In this scenario, an event would be created when the socket has received data.
UV_RUN_DEFAULT will block the caller even if the socket does not have data. The caller will return from uv_run(), when either:
The loop has been explicitly stopped, via uv_stop()
No more watchers are running in the loop. For example, the only watcher has been stopped.
UV_RUN_ONCE will block the caller even if the socket does not have data. The caller will return from uv_run(), when any of the following occur:
The loop has been explicitly stopped, via uv_stop()
No more watchers are running in the loop. For example, the only watcher has been stopped.
It has handled a max of one event. For example, the socket received data, and the user callback has been invoked. Additional events may be ready to be handled, but will not be handled in the current uv_run() call.
UV_RUN_NOWAIT will return if the socket does not have data.
Often times, running an event-loop in a non-blocking manner is done to integrate with other event-loops. Consider an application that has two event loops: libuv for backend work and Qt UI (which is driven by its own event loop). Being able to run the event loop in a non-blocking manner allows for a single thread to dispatch events on both event-loops. Here is a simplistic overview showing two libuv loops being handled by a single thread:
uv_loop_t *loop1 = uv_loop_new();
uv_loop_t *loop2 = uv_loop_new();
// create, initialize, and start a watcher for each loop.
...
// Handle two event loops with a single thread.
while (uv_run(loop1, UV_RUN_NOWAIT) || uv_run(loop2, UV_RUN_NOWAIT));
Without using UV_RUN_NOWAIT, loop2 would only run once loop1 or loop1's watchers have been stopped.
For more information, consider reading the Advanced Event Loops and Processes sections of An Introduction to libuv.
I want to simulate a FAM event. The API doesn't seem to support this.
Why? Because I have a thread waiting on a FAMNextEvent() and I want to wake it up from another thread.
I guess I could touch one of the files being monitored but that seems very crude.
Instead of using FAMNextEvent directly, how about using select or poll on
FAMCONNECTION_GETFD(FAMConnection)
When it triggers, check FAMPending before calling FAMNextEvent.
a self-pipe
The thread will wait for either a FAM event or a byte written to the self-pipe.
I would like to model an event-driven finite state machine in C as proposed here :
http://en.wikipedia.org/wiki/Event-driven_finite_state_machine
But I would also like the 'external' events to be handled in various threads.
Can I find such a code somewhere ? Or advices ?
Message queues are a way to solve your problem.
If you want to feed your state machine with external events from other threads, they can write these events in a message queue that will be read by your state machine.
If you want that other threads trigger on actions from your state machine, it can write to various message queues each associated to a thread that will read from its MQ.
One drawback is that events get sorted in chronological order. If your state machine is not in the mood of handling the event it just read from the queue, you have to decide what to do with this event: destroy it, put it back to the queue, remember it for future use...
Maybe the Quantum Framework is what you are looking for? See http://state-machine.com/ for further information. There are ports for many microcontrollers as well as for linux and windows.