Problem with Array of Queues in FreeRTOS - c

I am building a FreeRTOS application. I created a module which registers a freeRTOS queue handle from another module and when an interrupt in this module module occurs, it sends a message to all the registered queues. But it seems I am able to send the message from the queue but not able to receive it at the other module.
Here is my code.
remote module:-
CanRxMsg RxMessage;
can_rx0_queue = xQueueCreate( 10, sizeof(CanRxMsg) ); // can_rx0_queue is globally defined
// Register my queue with can module
if (registerRxQueueWithCAN(can_rx0_queue) == -1)
{
TurnLedRed();
}
while(1)
{
if(can_rx0_queue){
while( xQueueReceive( can_rx0_queue, ( void * ) &RxMessage, portMAX_DELAY))
{
}
.....
Here is the registration module
#define MAX_NUMBER_OF_RX_QUEUES 2
//xQueueHandle rxQueueStore[MAX_NUMBER_OF_RX_QUEUES];
typedef struct QUEUE_REGISTRY_ITEM
{
// signed char *pcQueueName;
xQueueHandle xHandle;
} xQueueRegistryItem;
xQueueRegistryItem rxQueueStore[MAX_NUMBER_OF_RX_QUEUES];
int numberOfQueuesRegistered;
#define cError -1
#define cSuccess 0
void processInterrupt()
{
for(int i=0; i < numberOfQueuesRegistered; i++)
{
if(xQueueSendFromISR(rxQueueStore[i].xHandle,(void *) &RxMessage,&tmp) != pdTRUE)
TurnLedRed();
if(tmp)resched_needed = pdTRUE;
}
portEND_SWITCHING_ISR(resched_needed);
}
int registerRxQueueWithCAN(xQueueHandle myQueue)
{
if(numberOfQueuesRegistered == MAX_NUMBER_OF_RX_QUEUES)
{
// Over Flow of registerations
TurnLedRed();
return cError;
}else
{
rxQueueStore[numberOfQueuesRegistered].xHandle = myQueue;
numberOfQueuesRegistered++;
}
return cSuccess;
}
Few points:-
xQuehandle is typdefed to "void *"
The code works if remove the registration thing and just do with directly pointer of queue in xQueueSendFromISR if I take the pointer by extern.
Any advice or information required?

At first glance I cannot see anything obviously wrong. The problem might be outside of the code you have shown, like how is can_rx0_queue declared, how is the interrupt entered, which port are you using, etc.
There is a FreeRTOS support forum, linked to from the FreeRTOS home page http://www.FreeRTOS.org
Regards.

I think Richard is right. The problem could be issues that are not within your code that you have posted here.
Are you calling any form of suspension on the receiving Task that is waiting on the Queue? When you invoke a vTaskSuspend() on a Task that is blocked waiting on a Queue, the Task that is suspended will be moved to the pxSuspendedTaskList and it will "forget" that it is waiting on an Event Queue because the pvContainer of xEventListItem in that Task will be set to NULL.
You might want to check if your receiving Task is ever suspended while waiting on a Queue. Hope that helped. Cheers!

Your shared memory should at least be declared volatile:
volatile xQueueRegistryItem rxQueueStore[MAX_NUMBER_OF_RX_QUEUES] ;
volatile int numberOfQueuesRegistered ;
otherwise the compiler may optimise out read or writes to these because it has no concept of different threads of execution (between the ISR and the main thread).
Also I recall that some PIC C runtime start-up options do not apply zero-initialisation of static data in order to minimise start-up time, if you are using such a start-up, you should explicitly initialise numberOfQueuesRegistered. I would suggest that to do so would be a good idea in any case.
It is not clear from your code that RxMessage in the ISR is not the same as RxMessage in the 'remote module'; they should not be shared, since that would allow the ISR to potentially modify the data while the receiving thread was processing it. If they could be shared, there would ne no reason to have a queue in the first place, since shared memory and a semaphore would suffice.
As a side-note, there is never any need to cast a pointer to void*, and you should generally avoid doing so, since it will prevent the compiler from issuing an error if you were to pass something other than a pointer. The whole point of a void* is rather that it can accept any pointer type.

Related

How to use napi_threadsafe_function for NodeJS Native Addon

I've been looking through the NAPI documentation to try and understand how it deals with multithreading. According to the documentation napi_create_threadsafe_function() and napi_call_threadsafe_function() are used to create and call js functions from multiple threads. The issue is that the documentation is not that straight forward, and there are no examples and I can't find any anywhere else.
If anyone has any experience using napi_create_threadsafe_function() and napi_call_threadsafe_function() or know where to find examples of them being used. Please if you could help out with a a basic example so I can just understand how to use them correctly.
I'm writting a C addon not C++ and need to use these functions. I am not using the wrapper node-addon-api, but napi directly
As a summery tag we may say, the N-API ThreadSafeFunctions acts as a safe tunnel between the asynchronous C/C++ code executing on a worker thread and the JavaScript layer for information exchange.
Before going technical let us consider a scenario that we have a very long running process heavy task to be completed. We all know putting this task on node.js main thread is not a good choice, it will chock the event loop and block all other task in the queue. So a good choice could be to consider this task in a separate thread (let us call this thread as a worker thread). JavaScript asynchronous callback and Promise are doing exactly this approach.
Let us say we have deployed the task on a worker thread and we are ready with a portion of result and we would like it to be send to JavaScript layer. Then the process involve are, converting the result into napi_value and then call the Callback JavaScript function from C/C++. Unfortunately neither of the operation can be performed from a worker thread; these operations should be exclusively done from the main thread. The JavaScript Promise and Callback, wait till the task completion and then switch over to main thread along with the task result in a normal C/C++ storage facility such as structure etc. Then do the napi_value conversion and call the JavaScript callback function from the main thread.
Since our task is extremely long running probably we don't want to wait till the end of the task before exchanging the result with JavaScript layer.
Let us consider a scenario where we are searching objects in a very large video where we prefer to get the detected objects send to JavaScript layer as on when it is found.
In such a scenario we will have to start sending task result while the task is still in progress. This is the scenario where Asynchronous Thread-safe Function Calls come for our help. It acts as a safe tunnel between the worker thread and the JavaScript layer for information exchange. Let us consider the following function snippet
napi_value CAsyncStreamSearch(napi_env env, napi_callback_info info)
{
// The native addon function exposed to JavaScript
// This will be the funciton a node.js application calling.
}
void ExecuteWork(napi_env env, void* data)
{
// We will use this function to get the task done.
// This code will be executed on a worker thread.
}
void OnWorkComplete(napi_env env, napi_status status, void* data)
{
// after the `ExecuteWork` function exits, this
// callback function will be called on the main thread
}
void ThreadSafeCFunction4CallingJS(napi_env env, napi_value js_cb,
void* context, void* data)
{
// This funcion acts as a safe tunnel between the asynchronous C/C++ code
// executing the worker thread and the JavaScript layer for information exchange.
}
In this first three functions are nearly same as JavaScript Promise and Callback that we are familiar with. The fourth one is specifically for the Asynchronous Thread-safe Function Calls. In this, our long running task is being executed by ExecuteWork() function on a worker thread. Let us say it has instructed us not to call JavaScript (and also any napi_value conversion of result) from ExecuteWork() but permitted to do so from ThreadSafeCFunction4CallingJS as long as we are calling ThreadSafeCFunction4CallingJS with an napi equivalent of C/C++ function pointer. Then we could pack the JavaScript calls inside this ThreadSafeCFunction4CallingJS() function. Then when ExecuteWork() function could pass the result to ThreadSafeCFunction4CallingJS() while it is being invoked in a plain C/C++ storage units such as structure etc. The ThreadSafeCFunction4CallingJS() convert this result to napi_value and call JavaScript function.
Under the cover the ThreadSafeCFunction4CallingJS() function is being queue to the event loop, and eventually it get executed by main thread.
The following code snippet packed inside CAsyncStreamSearch() is responsible for creating a C/C++ function pointer equivalent of N-API by usng napi_create_threadsafe_function() and it is being done from the native addon's main thread itself. Similarly the request for creation of worker thread by using napi_create_async_work() function then placing the work int the event queue by using napi_queue_async_work() so that a worker thread will pickup this item in the future.
napi_value CAsyncStreamSearch(napi_env env, napi_callback_info info)
{
-- -- -- --
-- -- -- --
// Create a thread-safe N-API callback function correspond to the C/C++ callback function
napi_create_threadsafe_function(env,
js_cb, NULL, work_name, 0, 1, NULL, NULL, NULL,
ThreadSafeCFunction4CallingJS, // the C/C++ callback function
// out: the asynchronous thread-safe JavaScript function
&(async_stream_data_ex->tsfn_StreamSearch));
// Create an async work item, that can be deployed in the node.js event queue
napi_create_async_work( env, NULL,
work_name,
ExecuteWork,
OnWorkComplete,
async_stream_data_ex,
// OUT: THE handle to the async work item
&(async_stream_data_ex->work_StreamSearch);)
// Queue the work item for execution.
napi_queue_async_work(env, async_stream_data_ex->work_StreamSearch);
return NULL;
}
Then during the asynchronous execution of task (ExecuteWork() function) invokes ThreadSafeCFunction4CallingJS() by calling napi_call_threadsafe_function() function as shown bellow.
static void ExecuteWork(napi_env env, void *data)
{
// tsfn is napi equivalent of point to ThreadSafeCFunction4CallingJS
// function that we created at CAsyncStreamSearch function
napi_acquire_threadsafe_function( tsfn )
Loop
{
// this will eventually invoke ThreadSafeCFunction4CallingJS()
// we may call any number of time (in fact it can be called from any thread)
napi_call_threadsafe_function( tsfn, WorkResult, );
}
napi_release_threadsafe_function( tsfn,);
}
The example you pointed out is one of the best source of information and it is directly form node.js team itself. When I was learning this concept I too was referring the same example, during my study the example has been recreated by extracting original idea from it, hope you may find this much simplified. and it is available at
https://github.com/msatyan/MyNodeC/blob/master/src/mync1/ThreadSafeAsyncStream.cpp
https://github.com/msatyan/MyNodeC/blob/master/test/ThreadSafeAsyncStream.js
If anyone else gets stuck with this issue. I finally managed to hunt down an example here.
Once I understand it better and have gotten a working sample, I will update here. Hopefully someone needing this in the future will have an easier time than me.
See Satyan's answer
The solution from this site worked for me here
struct ThreadCtx {
ThreadCtx(Napi::Env env) {};
std::thread nativeThread;
Napi::ThreadSafeFunction tsfn;
};
void Target::Connect(const Napi::CallbackInfo& info) {
Napi::Env env = info.Env();
threadCtx = new ThreadCtx(env);
// Create a ThreadSafeFunction
threadCtx->tsfn = Napi::ThreadSafeFunction::New(env, info[0].As<Napi::Function>(), "Resource Name", 0 /* Unlimited queue */, 1 /* Only 1 thread */, threadCtx,
[&]( Napi::Env, void *finalizeData, ThreadCtx *context ) {
printf("Thread cleanup\n");
threadCtx->nativeThread.join();
},
(void*)nullptr
);
// Create a native thread
threadCtx->nativeThread = std::thread([&] {
auto callback = [](Napi::Env env, Napi::Function cb, char* buffer) {
cb.Call({Napi::String::New(env, buffer)});
};
char reply[1024];
memset(reply, 0, sizeof(reply));
while(true)
{
size_t reply_length = boost::asio::read(s, boost::asio::buffer(reply, sizeof(reply)));
if(reply_length <= 0) {
printf("Bad read from boost asio\n");
break;
}
// Callback (blocking) to JS
napi_status status = threadCtx->tsfn.BlockingCall(reply, callback);
if (status != napi_ok)
{
// Handle error
break;
}
}
// Release the thread-safe function
threadCtx->tsfn.Release();
});
}
addon.cc - (tested and 100% working)
#include <napi.h>
Napi::Value SAFE_THREAD(const Napi::CallbackInfo& info) {
std::thread([](Napi::ThreadSafeFunction tsfn){
struct output_data{
int arg1;
std::string arg2;
};
auto data = new output_data();
///---------------
///fill output data
data->arg1=1;
data->arg2="string data";
std::this_thread::sleep_for(std::chrono::milliseconds(2000));
///---------------
///output thread result to nodejs
napi_status status = tsfn.BlockingCall(data,[](Napi::Env env, Napi::Function jsCallback,output_data* data){
jsCallback.Call({Napi::Number::New(env, data->arg1), String::New(env, data->arg2)});
delete data;
});
if(status != napi_ok) { std::cout << "error!" << "\n"; }
tsfn.Release();
},Napi::ThreadSafeFunction::New(info.Env(), info[0].As<Function>(), "TSFN", 0, 1,[](Napi::Env env, void *finalizeData){},(void *)nullptr)).detach();
return info.Env().Null();
}
index.js
const ADDON = require('./THREAD/build/Release/addon');
function time_sec(){return (new Date()).getTime()/1000;}
var t = time_sec();
ADDON.SAFE_THREAD((arg1,arg2)=>{
console.log(time_sec()-t, 'arg1 = '+arg1)
console.log(time_sec()-t, 'arg2 = '+arg2)
});
console.log(time_sec()-t, 'fin')
output:
0.00099992752075 fin
2.00499987602233 arg1 = 1
2.00600004196167 arg2 = string data
see also how to emit data from thread

Linux Kernel - How to match a jprobe to kretprobe?

I am writing a kernel module to monitor a few syscalls wanting to return the function arguments to user-land (via netlink socket) if the call was successful.
jprobe.kp.symbol_name = "rename";
jprobe.entry = rename_handler;
kretprobe.kp.symbol_name = "rename";
kretprobe.handler = rename_ret_handler;
static rename_obj_t _g_cur_rename = NULL;
static void _rename_handler(const char *oldpath, const char *newpath)
{
_g_cur_rename = create_rename(oldpath, newpath);
jprobe_return();
}
static void _rename_ret_handler(struct kretprobe_instance *ri, struct pt_regs *regs)
{
/* Send only if successful */
if (regs_return_value(regs) == 0) {
add_send_queue(_g_cur_rename);
}
return 0;
}
I worry that another rename syscall may preempt[1] the current one after the jprobe and I will send incorrect return codes and arguments.
jprobe: rename(a, b)
jprobe rename(c, d)
kretprobe
kretprobe
Edit: This article[2] states that interrupts are disabled during a kprobe handler. But does that mean that interrupts are disable throughout the whole chain (jprobe -> kprobe -> kretprobe) or just for that single kprobe?
https://unix.stackexchange.com/questions/186355/few-questions-about-system-calls-and-kernel-modules-kernel-services-in-parallel
https://lwn.net/Articles/132196/
Interrupts are disabled for each jprobe call: not for the entire sequence.
How many calls are you expecting in the time it will take the application to process them? There are different approaches depending on how fast you expect the calls to come in. The simplest method, if you are only expecting maybe a few hundred calls before you can process them and you will dedicate the static memory to the purpose, is to implement a static array of rename_obj_t objects in memory and then use atomic_add from the kernel asm includes to point to the next entry (mod the size of your array).
This way you are returning a unique static reference each time, so long as the counter doesn't wrap around before you process the returned values. atomic_add is guaranteed to have the correct memory barriers in place so you don't have to worry about things like cache coherency.

Alternative to blocking code

Attempting to use mbed OS scheduler for a small project.
As mbed os is Asynchronous I need to avoid blocking code.
However the library for my wireless receiver uses a blocking line of:
while (!(wireless.isRxData()));
Is there an alternative way to do this that won't block all the code until a message is received?
static void listen(void) {
wireless.quickRxSetup(channel, addr1);
sprintf(ackData,"Ack data \r\n");
wireless.acknowledgeData(ackData, strlen(ackData), 1);
while (!(wireless.isRxData()));
len = wireless.getRxData(msg);
}
static void motor(void) {
pc.printf("Motor\n");
m.speed(1);
n.speed(1);
led1 = 1;
wait(0.5);
m.speed(0);
n.speed(0);
}
static void sendData() {
wireless.quickTxSetup(channel, addr1);
strcpy(accelData, "Robot");
wireless.transmitData(accelData ,strlen(accelData));
}
void app_start(int, char**) {
minar::Scheduler::postCallback(listen).period(minar::milliseconds(500)).tolerance(minar::milliseconds(1000));
minar::Scheduler::postCallback(motor).period(minar::milliseconds(500));
minar::Scheduler::postCallback(sendData).period(minar::milliseconds(500)).delay(minar::milliseconds(3000));
}
You should remove the while (!(wireless.isRxData())); loop in your listen function. Replace it with:
if (wireless.isRxData()) {
len = wireless.getRxData(msg);
// Process data
}
Then, you can process your data in that if statement, or you can call postCallback on another function that will do your processing.
Instead of looping until data is available, you'll want to poll for data. If RX data is not available, exit the function and set a timer to go off after a short interval. When the timer goes off, check for data again. Repeat until data is available. I'm not familiar with your OS so I can't offer any specific code. This may be as simple as adding a short "sleep" call inside the while loop, or may involve creating another callback from the scheduler.

Need for XEventsQueued(display, QueuedAfterReading) in XCB

I am migrating some code of CYBOI from Xlib to XCB.
CYBOI uses a couple of threads for different communication channels like:
serial_port, terminal, socket, x_window_system.
However, it uses these threads only for signal/event/data detection;
the actual receiving and sending is done in the main thread,
in order to avoid any multi-threading conflicts of address space.
For the x_window_system channel, I previously detected events in a thread:
int n = XEventsQueued(display, QueuedAfterReading);
Upon detection of an event, an "interrupt flag" was set.
Afterwards, the main thread was reading the actual event using:
XNextEvent(display, &event);
When no more events were available, the main thread stopped receiving events
and the x_window_system channel thread started listening with XEventsQueued again.
Now, I am migrating the code to X C Binding (XCB).
There is a blocking function "xcb_wait_for_event" which is fine for reading an event.
What I miss is some function "peeking ahead" if there are events pending,
WITHOUT actually returning/removing the event from the queue.
I was reading the web for a couple of hours now, but am not able to find such a function.
The "xcb_poll_for_event" does not help. Blocking is fine for me,
since my event detection runs in its own thread.
The "xcb_request_check" as third input function does not seem to be what I want.
Could somebody help me out?
Thanks,
Christian
Are you looking for xcb_poll_for_queued_event(xcb_connection_t *c) which returns the next event without reading from the connection?
First, thanks to Julien for his reply.
I have studied the XCB 1.9 sources and found out that the
"xcb_poll_for_queued_event" function is not what I need.
The functions "xcb_poll_for_event" and "xcb_poll_for_queued_event"
both call "poll_for_next_event".
The functions "poll_for_next_event" and "xcb_wait_for_event"
both call "get_event".
If "get_event" finds an event, it changes the internal
linked list to point to the next event. However, I would
prefer NOT to change the event queue AT ALL, independent
from whether or not events are available.
I therefore propose to add a function like the following to XCB:
void* NULL_POINTER = (void*) 0;
int xcb_test_for_event(xcb_connection_t* c) {
int r = 0;
if (c != NULL_POINTER) {
struct _xcb_in in = c->in;
struct event_list* l = in.events;
if (l != NULL_POINTER) {
xcb_generic_event_t* e = l->event;
if (e != NULL_POINTER) {
r = 1;
}
}
}
return r;
}
This would allow me to write an endless loop like:
while (!xcb_test_for_event(connection)) {
sleep(t);
}
This is comparable to the old Xlib function:
int n = XEventsQueued(d, QueuedAfterReading);
which just checked the number of events in the event queue.
The "XEventsQueued" function always returns immediately WITHOUT
input/output, if there are events already in the queue.
Thanks
Christian

Arduino EthernetServer read() only works when Serial is initialized and read characters are printed

I have an Arduino project where I read data from a webserver.
I have an EthernetClient that reads the data character by character in a callback function.
My working code looks like (only the relevant parts):
void setup() {
Serial.begin(9600);
...
}
void loop() {
char* processedData = processData(callback); // this is in a external lib
}
boolean callback(char* buffer, int& i) {
...
if (Client.available()) {
char c = client.read();
buffer[i++] = c;
Serial.print(c);
}
...
}
This works without any problems (reading and processing the data), but when I remove Serial.begin(9600); and Serial.print(c); it stops working and I don't know why? The only thing changed is that the char c is not printed. What could be the problem?
A common reason why callback functions change their behavior when seemingly unrelated code is altered, is optimizer-related bugs.
Many embedded compilers fail to understand that a callback function (or an interrupt service routine) will ever be called in the program. They see no explicit call to that function and then assumes it is never called.
When the compiler has made such an assumption, it will optimize variables that are changed by the callback function, because it fails to see that the variable is changed by the program, between the point of initialization and the point of access.
// Bad practice example:
int x;
void main (void)
{
x=5;
...
if(x == 0) /* this whole if statement will get optimized away,
the compiler assumes that x has never been changed. */
{
do_stuff();
}
}
void callback (void)
{
x = 0;
}
When this bug strikes, it is nearly impossible to find, it can cause any kind of weird symptoms.
The solution is to always declare all file scope ("global") variables shared between main() and an interrupt/callback/thread as volatile. This makes it impossible for the compiler to make incorrect optimizer assumptions.
(Please note that the volatile keyword cannot be used to achieve synchronization nor does it guarantee any memory barriers. This answer is not in the slightest related to such issues!)
A guess: Because without the serial driver started, there is no data to process, and therefore your callback is not hit.
What were you hoping the serial callback to be doing in the absence of data?
Providing more information about Client and processData may help.

Resources