I am migrating some code of CYBOI from Xlib to XCB.
CYBOI uses a couple of threads for different communication channels like:
serial_port, terminal, socket, x_window_system.
However, it uses these threads only for signal/event/data detection;
the actual receiving and sending is done in the main thread,
in order to avoid any multi-threading conflicts of address space.
For the x_window_system channel, I previously detected events in a thread:
int n = XEventsQueued(display, QueuedAfterReading);
Upon detection of an event, an "interrupt flag" was set.
Afterwards, the main thread was reading the actual event using:
XNextEvent(display, &event);
When no more events were available, the main thread stopped receiving events
and the x_window_system channel thread started listening with XEventsQueued again.
Now, I am migrating the code to X C Binding (XCB).
There is a blocking function "xcb_wait_for_event" which is fine for reading an event.
What I miss is some function "peeking ahead" if there are events pending,
WITHOUT actually returning/removing the event from the queue.
I was reading the web for a couple of hours now, but am not able to find such a function.
The "xcb_poll_for_event" does not help. Blocking is fine for me,
since my event detection runs in its own thread.
The "xcb_request_check" as third input function does not seem to be what I want.
Could somebody help me out?
Thanks,
Christian
Are you looking for xcb_poll_for_queued_event(xcb_connection_t *c) which returns the next event without reading from the connection?
First, thanks to Julien for his reply.
I have studied the XCB 1.9 sources and found out that the
"xcb_poll_for_queued_event" function is not what I need.
The functions "xcb_poll_for_event" and "xcb_poll_for_queued_event"
both call "poll_for_next_event".
The functions "poll_for_next_event" and "xcb_wait_for_event"
both call "get_event".
If "get_event" finds an event, it changes the internal
linked list to point to the next event. However, I would
prefer NOT to change the event queue AT ALL, independent
from whether or not events are available.
I therefore propose to add a function like the following to XCB:
void* NULL_POINTER = (void*) 0;
int xcb_test_for_event(xcb_connection_t* c) {
int r = 0;
if (c != NULL_POINTER) {
struct _xcb_in in = c->in;
struct event_list* l = in.events;
if (l != NULL_POINTER) {
xcb_generic_event_t* e = l->event;
if (e != NULL_POINTER) {
r = 1;
}
}
}
return r;
}
This would allow me to write an endless loop like:
while (!xcb_test_for_event(connection)) {
sleep(t);
}
This is comparable to the old Xlib function:
int n = XEventsQueued(d, QueuedAfterReading);
which just checked the number of events in the event queue.
The "XEventsQueued" function always returns immediately WITHOUT
input/output, if there are events already in the queue.
Thanks
Christian
Related
I've been looking through the NAPI documentation to try and understand how it deals with multithreading. According to the documentation napi_create_threadsafe_function() and napi_call_threadsafe_function() are used to create and call js functions from multiple threads. The issue is that the documentation is not that straight forward, and there are no examples and I can't find any anywhere else.
If anyone has any experience using napi_create_threadsafe_function() and napi_call_threadsafe_function() or know where to find examples of them being used. Please if you could help out with a a basic example so I can just understand how to use them correctly.
I'm writting a C addon not C++ and need to use these functions. I am not using the wrapper node-addon-api, but napi directly
As a summery tag we may say, the N-API ThreadSafeFunctions acts as a safe tunnel between the asynchronous C/C++ code executing on a worker thread and the JavaScript layer for information exchange.
Before going technical let us consider a scenario that we have a very long running process heavy task to be completed. We all know putting this task on node.js main thread is not a good choice, it will chock the event loop and block all other task in the queue. So a good choice could be to consider this task in a separate thread (let us call this thread as a worker thread). JavaScript asynchronous callback and Promise are doing exactly this approach.
Let us say we have deployed the task on a worker thread and we are ready with a portion of result and we would like it to be send to JavaScript layer. Then the process involve are, converting the result into napi_value and then call the Callback JavaScript function from C/C++. Unfortunately neither of the operation can be performed from a worker thread; these operations should be exclusively done from the main thread. The JavaScript Promise and Callback, wait till the task completion and then switch over to main thread along with the task result in a normal C/C++ storage facility such as structure etc. Then do the napi_value conversion and call the JavaScript callback function from the main thread.
Since our task is extremely long running probably we don't want to wait till the end of the task before exchanging the result with JavaScript layer.
Let us consider a scenario where we are searching objects in a very large video where we prefer to get the detected objects send to JavaScript layer as on when it is found.
In such a scenario we will have to start sending task result while the task is still in progress. This is the scenario where Asynchronous Thread-safe Function Calls come for our help. It acts as a safe tunnel between the worker thread and the JavaScript layer for information exchange. Let us consider the following function snippet
napi_value CAsyncStreamSearch(napi_env env, napi_callback_info info)
{
// The native addon function exposed to JavaScript
// This will be the funciton a node.js application calling.
}
void ExecuteWork(napi_env env, void* data)
{
// We will use this function to get the task done.
// This code will be executed on a worker thread.
}
void OnWorkComplete(napi_env env, napi_status status, void* data)
{
// after the `ExecuteWork` function exits, this
// callback function will be called on the main thread
}
void ThreadSafeCFunction4CallingJS(napi_env env, napi_value js_cb,
void* context, void* data)
{
// This funcion acts as a safe tunnel between the asynchronous C/C++ code
// executing the worker thread and the JavaScript layer for information exchange.
}
In this first three functions are nearly same as JavaScript Promise and Callback that we are familiar with. The fourth one is specifically for the Asynchronous Thread-safe Function Calls. In this, our long running task is being executed by ExecuteWork() function on a worker thread. Let us say it has instructed us not to call JavaScript (and also any napi_value conversion of result) from ExecuteWork() but permitted to do so from ThreadSafeCFunction4CallingJS as long as we are calling ThreadSafeCFunction4CallingJS with an napi equivalent of C/C++ function pointer. Then we could pack the JavaScript calls inside this ThreadSafeCFunction4CallingJS() function. Then when ExecuteWork() function could pass the result to ThreadSafeCFunction4CallingJS() while it is being invoked in a plain C/C++ storage units such as structure etc. The ThreadSafeCFunction4CallingJS() convert this result to napi_value and call JavaScript function.
Under the cover the ThreadSafeCFunction4CallingJS() function is being queue to the event loop, and eventually it get executed by main thread.
The following code snippet packed inside CAsyncStreamSearch() is responsible for creating a C/C++ function pointer equivalent of N-API by usng napi_create_threadsafe_function() and it is being done from the native addon's main thread itself. Similarly the request for creation of worker thread by using napi_create_async_work() function then placing the work int the event queue by using napi_queue_async_work() so that a worker thread will pickup this item in the future.
napi_value CAsyncStreamSearch(napi_env env, napi_callback_info info)
{
-- -- -- --
-- -- -- --
// Create a thread-safe N-API callback function correspond to the C/C++ callback function
napi_create_threadsafe_function(env,
js_cb, NULL, work_name, 0, 1, NULL, NULL, NULL,
ThreadSafeCFunction4CallingJS, // the C/C++ callback function
// out: the asynchronous thread-safe JavaScript function
&(async_stream_data_ex->tsfn_StreamSearch));
// Create an async work item, that can be deployed in the node.js event queue
napi_create_async_work( env, NULL,
work_name,
ExecuteWork,
OnWorkComplete,
async_stream_data_ex,
// OUT: THE handle to the async work item
&(async_stream_data_ex->work_StreamSearch);)
// Queue the work item for execution.
napi_queue_async_work(env, async_stream_data_ex->work_StreamSearch);
return NULL;
}
Then during the asynchronous execution of task (ExecuteWork() function) invokes ThreadSafeCFunction4CallingJS() by calling napi_call_threadsafe_function() function as shown bellow.
static void ExecuteWork(napi_env env, void *data)
{
// tsfn is napi equivalent of point to ThreadSafeCFunction4CallingJS
// function that we created at CAsyncStreamSearch function
napi_acquire_threadsafe_function( tsfn )
Loop
{
// this will eventually invoke ThreadSafeCFunction4CallingJS()
// we may call any number of time (in fact it can be called from any thread)
napi_call_threadsafe_function( tsfn, WorkResult, );
}
napi_release_threadsafe_function( tsfn,);
}
The example you pointed out is one of the best source of information and it is directly form node.js team itself. When I was learning this concept I too was referring the same example, during my study the example has been recreated by extracting original idea from it, hope you may find this much simplified. and it is available at
https://github.com/msatyan/MyNodeC/blob/master/src/mync1/ThreadSafeAsyncStream.cpp
https://github.com/msatyan/MyNodeC/blob/master/test/ThreadSafeAsyncStream.js
If anyone else gets stuck with this issue. I finally managed to hunt down an example here.
Once I understand it better and have gotten a working sample, I will update here. Hopefully someone needing this in the future will have an easier time than me.
See Satyan's answer
The solution from this site worked for me here
struct ThreadCtx {
ThreadCtx(Napi::Env env) {};
std::thread nativeThread;
Napi::ThreadSafeFunction tsfn;
};
void Target::Connect(const Napi::CallbackInfo& info) {
Napi::Env env = info.Env();
threadCtx = new ThreadCtx(env);
// Create a ThreadSafeFunction
threadCtx->tsfn = Napi::ThreadSafeFunction::New(env, info[0].As<Napi::Function>(), "Resource Name", 0 /* Unlimited queue */, 1 /* Only 1 thread */, threadCtx,
[&]( Napi::Env, void *finalizeData, ThreadCtx *context ) {
printf("Thread cleanup\n");
threadCtx->nativeThread.join();
},
(void*)nullptr
);
// Create a native thread
threadCtx->nativeThread = std::thread([&] {
auto callback = [](Napi::Env env, Napi::Function cb, char* buffer) {
cb.Call({Napi::String::New(env, buffer)});
};
char reply[1024];
memset(reply, 0, sizeof(reply));
while(true)
{
size_t reply_length = boost::asio::read(s, boost::asio::buffer(reply, sizeof(reply)));
if(reply_length <= 0) {
printf("Bad read from boost asio\n");
break;
}
// Callback (blocking) to JS
napi_status status = threadCtx->tsfn.BlockingCall(reply, callback);
if (status != napi_ok)
{
// Handle error
break;
}
}
// Release the thread-safe function
threadCtx->tsfn.Release();
});
}
addon.cc - (tested and 100% working)
#include <napi.h>
Napi::Value SAFE_THREAD(const Napi::CallbackInfo& info) {
std::thread([](Napi::ThreadSafeFunction tsfn){
struct output_data{
int arg1;
std::string arg2;
};
auto data = new output_data();
///---------------
///fill output data
data->arg1=1;
data->arg2="string data";
std::this_thread::sleep_for(std::chrono::milliseconds(2000));
///---------------
///output thread result to nodejs
napi_status status = tsfn.BlockingCall(data,[](Napi::Env env, Napi::Function jsCallback,output_data* data){
jsCallback.Call({Napi::Number::New(env, data->arg1), String::New(env, data->arg2)});
delete data;
});
if(status != napi_ok) { std::cout << "error!" << "\n"; }
tsfn.Release();
},Napi::ThreadSafeFunction::New(info.Env(), info[0].As<Function>(), "TSFN", 0, 1,[](Napi::Env env, void *finalizeData){},(void *)nullptr)).detach();
return info.Env().Null();
}
index.js
const ADDON = require('./THREAD/build/Release/addon');
function time_sec(){return (new Date()).getTime()/1000;}
var t = time_sec();
ADDON.SAFE_THREAD((arg1,arg2)=>{
console.log(time_sec()-t, 'arg1 = '+arg1)
console.log(time_sec()-t, 'arg2 = '+arg2)
});
console.log(time_sec()-t, 'fin')
output:
0.00099992752075 fin
2.00499987602233 arg1 = 1
2.00600004196167 arg2 = string data
see also how to emit data from thread
I'm making a GTK+3 application in C and I want a spinner to show when the program is processing the data. Here's what I generally have:
main()
{
//Some statements
g_signal_connect(G_OBJECT(btnGenerate), "clicked", G_CALLBACK(Generate), &mainform);
}
void Generate(GtkWidget *btnGenerate, form_widgets *p_main_form)
{
gtk_spinner_start(GTK_SPINNER(p_main_form->spnProcessing));
Begin_Lengthy_Processing(Parameters, Galore, ...);
//gtk_spinner_stop(GTK_SPINNER(p_main_form->spnProcessing));
}
I have the stop function commented out so I can see the spinner spin even after the function has finished, but the spinner starts after the function is finished, and I suspect it turns on in the main loop.
I also found out that the entire interface freezes during the execution of the long going function.
Is there a way to get it to start and display inside the callback function? I found the same question, but it uses Python and threads. This is C, not Python, so I would assume things are different.
You need to run your lengthy computation in a separate thread, or break it up into chunks and run each of them separately as idle callbacks in the main thread.
If your lengthy computation takes a single set of inputs and doesn’t need any more inputs until it’s finished, then you should construct it as a GTask and use g_task_run_in_thread() to start the task. Its result will be delivered back to the main thread via the GTask’s GAsyncReadyCallback. There’s an example here.
If it takes more input as it progresses, you probably want to use a GAsyncQueue to feed it more inputs, and a GThreadPool to provide the threads (amortising the cost of creating threads over multiple calls to the lengthy function, and protecting against denial of service).
The GNOME developer docs give an overview of how to do threading.
This is what I got:
int main()
{
// Statements...
g_signal_connect(G_OBJECT(btnGenerate), "clicked", G_CALLBACK(Process), &mainform);
// More statements...
}
void Process(GtkWidget *btnGenerate, form_widgets *p_main_form)
{
GError *processing_error;
GThread *start_processing;
gtk_spinner_start(GTK_SPINNER(p_main_form->spnProcessing));
active = true;
if((start_processing = g_thread_try_new(NULL, (GThreadFunc)Generate, p_main_form, &processing_error)) == NULL)
{
printf("%s\n", processing_error->message);
printf("Error, cannot create thread!?!?\n\n");
exit(processing_error->code);
}
}
void Generate(form_widgets *p_main_form)
{
// Long process
active = false;
}
My program, once cleaned up and finished, as there are many other bugs in the program, will be put on GitHub.
Thank you all for your help. This answer comes from looking at all of your answers and comments as well as reading some more documentation, but mostly your comments and answers.
I did something similar in my gtk3 program. It's not that difficult. Here's how I would go about it.
/**
g_idle_add_full() expects a pointer to a function with the signature below:
(*GSourceFunc) (gpointer user_data).
So your function signature must adhere to that in order to be called.
But you might want to pass variables to the function.
If you don't want to have the variables in the global scope
then you can do this:
typedef struct myDataType {
char* name;
int age;
} myDataType;
myDataType person = {"Max", 25};
then when calling g_idle_add_full() you do it this way:
g_idle_add_full(G_PRIORITY_HIGH_IDLE, myFunction, person, NULL);
*/
int main()
{
// Assumming there exist a pointer called data
g_idle_add_full(G_PRIORITY_HIGH_IDLE, lengthyProcessCallBack, data, NULL);
// GTK & GDK event loop continues and window should be responsive while function runs in background
}
gboolean lengthyProcessCallBack(gpointer data)
{
myDataType person = (myDataType) *data;
// Doing lenghthy stuff
while(;;) {
sleep(3600); // hypothetical long process :D
}
return FALSE; // removed from event sources and won't be called again.
}
Attempting to use mbed OS scheduler for a small project.
As mbed os is Asynchronous I need to avoid blocking code.
However the library for my wireless receiver uses a blocking line of:
while (!(wireless.isRxData()));
Is there an alternative way to do this that won't block all the code until a message is received?
static void listen(void) {
wireless.quickRxSetup(channel, addr1);
sprintf(ackData,"Ack data \r\n");
wireless.acknowledgeData(ackData, strlen(ackData), 1);
while (!(wireless.isRxData()));
len = wireless.getRxData(msg);
}
static void motor(void) {
pc.printf("Motor\n");
m.speed(1);
n.speed(1);
led1 = 1;
wait(0.5);
m.speed(0);
n.speed(0);
}
static void sendData() {
wireless.quickTxSetup(channel, addr1);
strcpy(accelData, "Robot");
wireless.transmitData(accelData ,strlen(accelData));
}
void app_start(int, char**) {
minar::Scheduler::postCallback(listen).period(minar::milliseconds(500)).tolerance(minar::milliseconds(1000));
minar::Scheduler::postCallback(motor).period(minar::milliseconds(500));
minar::Scheduler::postCallback(sendData).period(minar::milliseconds(500)).delay(minar::milliseconds(3000));
}
You should remove the while (!(wireless.isRxData())); loop in your listen function. Replace it with:
if (wireless.isRxData()) {
len = wireless.getRxData(msg);
// Process data
}
Then, you can process your data in that if statement, or you can call postCallback on another function that will do your processing.
Instead of looping until data is available, you'll want to poll for data. If RX data is not available, exit the function and set a timer to go off after a short interval. When the timer goes off, check for data again. Repeat until data is available. I'm not familiar with your OS so I can't offer any specific code. This may be as simple as adding a short "sleep" call inside the while loop, or may involve creating another callback from the scheduler.
I am relatively new to working with threads in Win32 api and have reached a problem that i am unable to work out.
Heres my problem, i have 4 threads (they work as intended) that allow the operator to test 4 terminals. In each thread i am trying to send a message to the main windows form with either Pass or Fail, this is placed within a listbox. Below is one of the threads, the remaining are exactly the same.
void Thread1(PVOID pvoid)
{
for(int i=0;i<numberOfTests1;i++) {
int ret;
double TimeOut = 60.0;
int Lng = 1;
test1[i].testNumber = getTestNumber(test1[i].testName);
unsigned char Param[255] = {0};
unsigned char Port1 = port1;
ret = PSB30_Open(Port1, 16);
ret = PSB30_SendOrder (Port1, test1[i].testNumber, &Param[0], &Lng, &TimeOut);
ret = PSB30_Close (Port1);
if(*Param == 1) SendDlgItemMessage(hWnd,IDT_RESULTLIST1,LB_ADDSTRING,i,(LPARAM)"PASS");
else SendDlgItemMessage(hWnd,IDT_RESULTLIST1,LB_ADDSTRING,i,(LPARAM)"FAIL");
}
_endthread();
}
I have debugged the code and it does everything except populate the listbox, i assume because its a thread i am missing something as the same code works outwith the thread. Do i need to put the thread to sleep while it sends the message to the main window?
Any help is appreciated.
Cheers
You don't want your secondary threads trying to manipulate your UI elements directly (such as the SendDlgItemMessage). Instead, you normally want to post something like a WM_COMMAND or WM_USER+N to the main window, and let that manipulate the UI elements accordingly.
I am building a FreeRTOS application. I created a module which registers a freeRTOS queue handle from another module and when an interrupt in this module module occurs, it sends a message to all the registered queues. But it seems I am able to send the message from the queue but not able to receive it at the other module.
Here is my code.
remote module:-
CanRxMsg RxMessage;
can_rx0_queue = xQueueCreate( 10, sizeof(CanRxMsg) ); // can_rx0_queue is globally defined
// Register my queue with can module
if (registerRxQueueWithCAN(can_rx0_queue) == -1)
{
TurnLedRed();
}
while(1)
{
if(can_rx0_queue){
while( xQueueReceive( can_rx0_queue, ( void * ) &RxMessage, portMAX_DELAY))
{
}
.....
Here is the registration module
#define MAX_NUMBER_OF_RX_QUEUES 2
//xQueueHandle rxQueueStore[MAX_NUMBER_OF_RX_QUEUES];
typedef struct QUEUE_REGISTRY_ITEM
{
// signed char *pcQueueName;
xQueueHandle xHandle;
} xQueueRegistryItem;
xQueueRegistryItem rxQueueStore[MAX_NUMBER_OF_RX_QUEUES];
int numberOfQueuesRegistered;
#define cError -1
#define cSuccess 0
void processInterrupt()
{
for(int i=0; i < numberOfQueuesRegistered; i++)
{
if(xQueueSendFromISR(rxQueueStore[i].xHandle,(void *) &RxMessage,&tmp) != pdTRUE)
TurnLedRed();
if(tmp)resched_needed = pdTRUE;
}
portEND_SWITCHING_ISR(resched_needed);
}
int registerRxQueueWithCAN(xQueueHandle myQueue)
{
if(numberOfQueuesRegistered == MAX_NUMBER_OF_RX_QUEUES)
{
// Over Flow of registerations
TurnLedRed();
return cError;
}else
{
rxQueueStore[numberOfQueuesRegistered].xHandle = myQueue;
numberOfQueuesRegistered++;
}
return cSuccess;
}
Few points:-
xQuehandle is typdefed to "void *"
The code works if remove the registration thing and just do with directly pointer of queue in xQueueSendFromISR if I take the pointer by extern.
Any advice or information required?
At first glance I cannot see anything obviously wrong. The problem might be outside of the code you have shown, like how is can_rx0_queue declared, how is the interrupt entered, which port are you using, etc.
There is a FreeRTOS support forum, linked to from the FreeRTOS home page http://www.FreeRTOS.org
Regards.
I think Richard is right. The problem could be issues that are not within your code that you have posted here.
Are you calling any form of suspension on the receiving Task that is waiting on the Queue? When you invoke a vTaskSuspend() on a Task that is blocked waiting on a Queue, the Task that is suspended will be moved to the pxSuspendedTaskList and it will "forget" that it is waiting on an Event Queue because the pvContainer of xEventListItem in that Task will be set to NULL.
You might want to check if your receiving Task is ever suspended while waiting on a Queue. Hope that helped. Cheers!
Your shared memory should at least be declared volatile:
volatile xQueueRegistryItem rxQueueStore[MAX_NUMBER_OF_RX_QUEUES] ;
volatile int numberOfQueuesRegistered ;
otherwise the compiler may optimise out read or writes to these because it has no concept of different threads of execution (between the ISR and the main thread).
Also I recall that some PIC C runtime start-up options do not apply zero-initialisation of static data in order to minimise start-up time, if you are using such a start-up, you should explicitly initialise numberOfQueuesRegistered. I would suggest that to do so would be a good idea in any case.
It is not clear from your code that RxMessage in the ISR is not the same as RxMessage in the 'remote module'; they should not be shared, since that would allow the ISR to potentially modify the data while the receiving thread was processing it. If they could be shared, there would ne no reason to have a queue in the first place, since shared memory and a semaphore would suffice.
As a side-note, there is never any need to cast a pointer to void*, and you should generally avoid doing so, since it will prevent the compiler from issuing an error if you were to pass something other than a pointer. The whole point of a void* is rather that it can accept any pointer type.