Flutter (Dart) ffi - Aplication freezes during processing external library methohd - c

I am using C library iperf3 to measure network. When I start network testing my aplication freezes and wait for results. I tried async and threads but any progress. Any advise? I'd like to run my test and asynchronously call another methods (at best, call this library again, but other methods). Is it possible?
My network.dart
final DynamicLibrary iperfLib = Platform.isAndroid
? DynamicLibrary.open("libiperf.so")
: DynamicLibrary.process();
typedef RunTestFunc = ffi.Pointer<ffi.Uint8> Function(
ffi.Pointer<ffi.Uint8> context);
typedef RunTest = ffi.Pointer<ffi.Uint8> Function(
ffi.Pointer<ffi.Uint8> context);
RunTest _run_test = iperfLib
.lookup<ffi.NativeFunction<RunTestFunc>>('run_test')
.asFunction<RunTest>();
ffi.Pointer<ffi.Uint8> runTest(ffi.Pointer<ffi.Uint8> context) {
return _run_test(context);
}
and iperf.c
Iperf* run_test(Iperf* test) {
__android_log_print( ANDROID_LOG_INFO, "DONE ", "server_hostname %s", test->server_hostname );
int cc = iperf_run_client( test ) ;
__android_log_print( ANDROID_LOG_INFO, "DONE ", " %d",cc );
iperf_free_test( test );
return test
}

Async Callbacks
The problem is that C routines called from dart are blocking and therefore congest the single existing dart isolate, consequently freezing the UI.
To work around this problem you have to open a port on the dart isolate through which your C routines can asynchronously send messages to the dart isolate. To signal to the dart compiler that this is a non-blocking operation, simply delay the completion of the function until a message on the designated port has been received.
Future<int> asyncData() async {
var receiveData;
bool receivedCallback = false;
var receivePort = ReceivePort()..listen((data) {
print('Received data from c');
receiveData = data;
receivedCallback = true;
});
var nativeSendPort = receivePort.sendPort.nativePort;
nativeTriggerFunction(nativeSendPort);
while(!receivedCallback) {
await Future.delayed(Duration(milliseconds: 100));
}
receivePort.close();
return receiveData;
}
In C, you need to create a trigger function which should ideally be as lightweight as possible, passing the port number to your C code and calling the actual function you want to execute on a different thread.
The trigger function will finish almost instantly, allowing your dart thread to do other work and as soon as the newly created thread is done, it sends its result through the native port back to the dart isolate which can pick up where it left off.
void native_trigger_function(Dart_Port port) {
pthread_t t;
Dart_Port *args = (Dart_Port *) malloc(sizeof(Dart_Port));
*args = port;
pthread_create(&t, NULL, _native_function, args);
}
void *_native_function(void *args) {
Dart_Port port = *(Dart_Port *) args;
int rc = 0;
// do some heavy work
// send return code to dart
Dart_CObject obj;
obj.type = Dart_CObject_kInt32;
obj.value.as_int32 = rc;
Dart_PostCObject_DL(port, &obj);
free(args);
pthread_exit(NULL);
}
Note: This logic relies on the native dart api to work which can be found here. Before use, the interface needs to be attached to the current dart isolate which can be achieved by calling Dart_InitializeApiDL(dart_api_data) from C where dart_api_data is a void pointer which can be obtained from your dart code using the dart:ffi package through NativeApi.initializeApiData.
Update: Thanks #fdollack for fixing the example snippets!

Thank you #Lucas Aschenbach!
This minimum example was so hard to find.
2 small additions.
First, the allocated pointer should be casted to (Dart_Port*),
and the port argument from dart has to be assigned/copied to where the pointer is at!
void native_trigger_function(Dart_Port port) {
pthread_t t;
Dart_Port *args= (Dart_Port*)malloc(sizeof(Dart_Port));
*args = port; // assign port
pthread_create(&t, NULL, _native_function, args);
}
The second thing is inside the _native_function the response to Dart has to be
Dart_PostCObject_DL(port, &obj);
instead of
Dart_PostCObject_DL(args_c.send_port, &obj);

Related

Managing data to send for MQTTAsync_sendMessage

Trying to write simple MQTT application with Paho library. I'm planning to create simple fire and forget not blocking function :
void send(char* data)
{
..
}
For this purpose I'm planning to use MQTTAsync client. But how to pass data to connect event. I suppose it is not good style to define char * dataToSend globally.
void onConnect(void* context, MQTTAsync_successData* response)
{
MQTTAsync client = (MQTTAsync)context;
MQTTAsync_responseOptions opts = MQTTAsync_responseOptions_initializer;
MQTTAsync_message pubmsg = MQTTAsync_message_initializer;
int rc;
printf("Successful connection\n");
opts.onSuccess = onSend;
opts.context = client;
pubmsg.payload = dataToSend;
pubmsg.payloadlen = strlen(dataToSend);
pubmsg.qos = QOS;
pubmsg.retained = 0;
deliveredtoken = 0;
if ((rc = MQTTAsync_sendMessage(client, TOPIC, &pubmsg, &opts)) != MQTTASYNC_SUCCESS)
{
printf("Failed to start sendMessage, return code %d\n", rc);
exit(EXIT_FAILURE);
}
}
I afraid of the fact that when I pass pointer of data and return from send function content of data string will not be available since the fact that it is local variable of function that calls send one.
How to build some mechanism that allocates memory for data to send, copies to it data and deallocates when send complete. User that call send are not planning to handle memory management, they will use somehing like send("Hello wrorld") ; I suppose I need something like list of data_to_send for this purpose.
You probably need to add some more context to the question, but the short answer is you don't.
You connect the client once when you launch the application and reuse the same client object for the life of the application. So when you want to send a message you should be already connected, no need to use the onConnect callback.
What you might do is use the onConnect callback to set a flag so the send function knows that the client has finished connecting if it is going to be called very soon after the application starts, just to make sure the client has finished connecting first.
It is possible to pass it using a structure in the context.
Something like:
typedef MQTTContext {
MQTTAsync client;
char* dataToSend;
};
This will allow to get buffer (and MQTTclient) from this context structure.
void onConnect(void* context, MQTTAsync_successData* response)
{
MQTTContext* ctx = (MQTTContext*)context;
// retreive client
MQTTAsync client = ctx->client;
...
// retreive message to publish
pubmsg.payload = ctx->dataToSend;
...
}
In your send function you could store message in this context structure
void send(char* data)
{
MQTTAsync client;
// store in context structure client and message to publish
MQTTContext* ctx = (MQTTContext*)malloc(sizeof(MQTTContext));
ctx->client = client;
ctx->dataToSend = strdup(data);
MQTTAsync_connectOptions conn_opts = MQTTAsync_connectOptions_initializer;
...
conn_opts.context = ctx;
MQTTAsync_connect(client, &conn_opts);
}

Windows Filtering Platform: ClassifyFn BSOD at DISPATCH_LEVEL

I'm trying to implement a simple firewall which filters network connections made by Windows processes.
The firewall should either allow/block the connection.
In order to intercept connections by any process, I created a kernel driver which makes use of Windows Filtering Platform.
I registered a ClassifyFn (FWPS_CALLOUT_CLASSIFY_FN1) callback at the filtering layer FWPM_LAYER_ALE_AUTH_CONNECT_V4:
FWPM_CALLOUT m_callout = { 0 };
m_callout.applicableLayer = FWPM_LAYER_ALE_AUTH_CONNECT_V4;
...
status = FwpmCalloutAdd(filter_engine_handle, &m_callout, NULL, NULL);
The decision regarding connection allow/block should be taken by userlevel.
I communicate with Userlevel using FltSendMessage,
which cannot be used at IRQL DISPATCH_LEVEL.
Following the instructions of the Microsoft documentation regarding how to process callouts asynchronously,
I do call FwpsPendOperation0 before calling FltSendMessage.
After the call to FltSendMessage, I resume packet processing by calling FwpsCompleteOperation0.
FwpsPendOperation0 documentation states that calling this function should make possible to operate calls at PASSIVE_LEVEL:
A callout can pend the current processing operation on a packet when
the callout must perform processing on one of these layers that may
take a long interval to complete or that should occur at IRQL =
PASSIVE_LEVEL if the current IRQL > PASSIVE_LEVEL.
However, when the ClassifyFn callback is called at DISPATCH_LEVEL, I do sometimes still get a BSOD on FltSendMessage (INVALID_PROCESS_ATTACH_ATTEMPT).
I don't understand what's wrong.
Thank you in advance for any advice which could point me to the right direction.
Here is the relevant code of the ClassifyFn callback:
/*************************
ClassifyFn Function
**************************/
void example_classify(
const FWPS_INCOMING_VALUES * inFixedValues,
const FWPS_INCOMING_METADATA_VALUES * inMetaValues,
void * layerData,
const void * classifyContext,
const FWPS_FILTER * filter,
UINT64 flowContext,
FWPS_CLASSIFY_OUT * classifyOut)
{
NTSTATUS status;
BOOLEAN bIsReauthorize = FALSE;
BOOLEAN SafeToOpen = TRUE; // Value returned by userlevel which signals to allow/deny packet
classifyOut->actionType = FWP_ACTION_PERMIT;
remote_address = inFixedValues->incomingValue[FWPS_FIELD_ALE_AUTH_CONNECT_V4_IP_REMOTE_ADDRESS].value.uint32;
remote_port = inFixedValues->incomingValue[FWPS_FIELD_ALE_AUTH_CONNECT_V4_IP_REMOTE_PORT].value.uint16;
bIsReauthorize = IsAleReauthorize(inFixedValues);
if (!bIsReauthorize)
{
// First time receiving packet (not a reauthorized packet)
// Communicate with userlevel asynchronously
HANDLE hCompletion;
status = FwpsPendOperation0(inMetaValues->completionHandle, &hCompletion);
//
// FltSendMessage call here
// ERROR HERE:
// INVALID_PROCESS_ATTACH_ATTEMP BSOD on FltMessage call when at IRQL DISPATCH_LEVEL
//
FwpsCompleteOperation0(hCompletion, NULL);
}
if (!SafeToOpen) {
// Packet blocked
classifyOut->actionType = FWP_ACTION_BLOCK;
}
else {
// Packet allowed
}
return;
}
You need to invoke FltSendMessage() on another thread running at PASSIVE_LEVEL. You can use IoQueueWorkItem() or implement your own mechanism to process it on a system worker thread created via PsCreateSystemThread().

Windows WFP Driver: Getting BSOD when processing packets in ClassifyFn callback

I am trying to code a simple firewall application which can allow or block network connection attempts made from userlevel processes.
To do so, following the WFPStarterKit tutorial, I created a WFP Driver which is set to intercept data at FWPM_LAYER_OUTBOUND_TRANSPORT_V4 layer.
The ClassifyFn callback function is responsible for intercepting the connection attempt, and either allow or deny it.
Once the ClassifyFn callback gets hit, the ProcessID of the packet is sent, along with a few other info, to a userlevel process through the FltSendMessage function.
The userlevel process receives the message, checks the ProcessID, and replies a boolean allow/deny command to the driver.
While this approach works when blocking a first packet, in some cases (expecially when allowing multiple packets) the code generates a BSOD with the INVALID_PROCESS_ATTACH_ATTEMPT error code.
The error is triggered at the call to FltSendMessage.
While I am still unable to pinpoint the exact problem,
it seems that making the callout thread wait (through FltSendMessage) for a reply from userlevel can generate this BSOD error on some conditions.
I would be very grateful if you can point me to the right direction.
Here is the function where I register the callout:
NTSTATUS register_example_callout(DEVICE_OBJECT * wdm_device)
{
NTSTATUS status = STATUS_SUCCESS;
FWPS_CALLOUT s_callout = { 0 };
FWPM_CALLOUT m_callout = { 0 };
FWPM_DISPLAY_DATA display_data = { 0 };
if (filter_engine_handle == NULL)
return STATUS_INVALID_HANDLE;
display_data.name = EXAMPLE_CALLOUT_NAME;
display_data.description = EXAMPLE_CALLOUT_DESCRIPTION;
// Register a new Callout with the Filter Engine using the provided callout functions
s_callout.calloutKey = EXAMPLE_CALLOUT_GUID;
s_callout.classifyFn = example_classify;
s_callout.notifyFn = example_notify;
s_callout.flowDeleteFn = example_flow_delete;
status = FwpsCalloutRegister((void *)wdm_device, &s_callout, &example_callout_id);
if (!NT_SUCCESS(status)) {
DbgPrint("Failed to register callout functions for example callout, status 0x%08x", status);
goto Exit;
}
// Setup a FWPM_CALLOUT structure to store/track the state associated with the FWPS_CALLOUT
m_callout.calloutKey = EXAMPLE_CALLOUT_GUID;
m_callout.displayData = display_data;
m_callout.applicableLayer = FWPM_LAYER_OUTBOUND_TRANSPORT_V4;
m_callout.flags = 0;
status = FwpmCalloutAdd(filter_engine_handle, &m_callout, NULL, NULL);
if (!NT_SUCCESS(status)) {
DbgPrint("Failed to register example callout, status 0x%08x", status);
}
else {
DbgPrint("Example Callout Registered");
}
Exit:
return status;
}
Here is the callout function:
/*************************
ClassifyFn Function
**************************/
void example_classify(
const FWPS_INCOMING_VALUES * inFixedValues,
const FWPS_INCOMING_METADATA_VALUES * inMetaValues,
void * layerData,
const void * classifyContext,
const FWPS_FILTER * filter,
UINT64 flowContext,
FWPS_CLASSIFY_OUT * classifyOut)
{
UNREFERENCED_PARAMETER(layerData);
UNREFERENCED_PARAMETER(classifyContext);
UNREFERENCED_PARAMETER(flowContext);
UNREFERENCED_PARAMETER(filter);
UNREFERENCED_PARAMETER(inMetaValues);
NETWORK_ACCESS_QUERY AccessQuery;
BOOLEAN SafeToOpen = TRUE;
classifyOut->actionType = FWP_ACTION_PERMIT;
AccessQuery.remote_address = inFixedValues->incomingValue[FWPS_FIELD_OUTBOUND_TRANSPORT_V4_IP_REMOTE_ADDRESS].value.uint32;
AccessQuery.remote_port = inFixedValues->incomingValue[FWPS_FIELD_OUTBOUND_TRANSPORT_V4_IP_REMOTE_PORT].value.uint16;
// Get Process ID
AccessQuery.ProcessId = (UINT64)PsGetCurrentProcessId();
if (!AccessQuery.ProcessId)
{
return;
}
// Here we connect to our userlevel application using FltSendMessage.
// Some checks are done and the SafeToOpen variable is populated with a BOOLEAN which indicates if to allow or block the packet.
// However, sometimes, a BSOD is generated with an INVALID_PROCESS_ATTACH_ATTEMPT error on the FltSendMessage call
QueryUserLevel(QUERY_NETWORK, &AccessQuery, sizeof(NETWORK_ACCESS_QUERY), &SafeToOpen, NULL, 0);
if (!SafeToOpen) {
classifyOut->actionType = FWP_ACTION_BLOCK;
}
return;
}
WFP drivers communicate to user-mode applications using the inverted call model. In this method, you keep an IRP from the user-mode pending at your kernel-mode driver instance and whenever you want to send data back to the user-mode you complete the IRP along with the data you want to send back.
The problem was that sometimes the ClassifyFn callback function can be called at IRQL DISPATCH_LEVEL.
FltSendMessage does not support DISPATCH_LEVEL, as it can only be run at IRQL <= APC_LEVEL.
Running at DISPATCH_LEVEL can cause this function to generate a BSOD.
I solved the problem by invoking FltSendMessage from a worker thread which runs at IRQL PASSIVE_LEVEL.
The worker thread can be created using IoQueueWorkItem.

How to use napi_threadsafe_function for NodeJS Native Addon

I've been looking through the NAPI documentation to try and understand how it deals with multithreading. According to the documentation napi_create_threadsafe_function() and napi_call_threadsafe_function() are used to create and call js functions from multiple threads. The issue is that the documentation is not that straight forward, and there are no examples and I can't find any anywhere else.
If anyone has any experience using napi_create_threadsafe_function() and napi_call_threadsafe_function() or know where to find examples of them being used. Please if you could help out with a a basic example so I can just understand how to use them correctly.
I'm writting a C addon not C++ and need to use these functions. I am not using the wrapper node-addon-api, but napi directly
As a summery tag we may say, the N-API ThreadSafeFunctions acts as a safe tunnel between the asynchronous C/C++ code executing on a worker thread and the JavaScript layer for information exchange.
Before going technical let us consider a scenario that we have a very long running process heavy task to be completed. We all know putting this task on node.js main thread is not a good choice, it will chock the event loop and block all other task in the queue. So a good choice could be to consider this task in a separate thread (let us call this thread as a worker thread). JavaScript asynchronous callback and Promise are doing exactly this approach.
Let us say we have deployed the task on a worker thread and we are ready with a portion of result and we would like it to be send to JavaScript layer. Then the process involve are, converting the result into napi_value and then call the Callback JavaScript function from C/C++. Unfortunately neither of the operation can be performed from a worker thread; these operations should be exclusively done from the main thread. The JavaScript Promise and Callback, wait till the task completion and then switch over to main thread along with the task result in a normal C/C++ storage facility such as structure etc. Then do the napi_value conversion and call the JavaScript callback function from the main thread.
Since our task is extremely long running probably we don't want to wait till the end of the task before exchanging the result with JavaScript layer.
Let us consider a scenario where we are searching objects in a very large video where we prefer to get the detected objects send to JavaScript layer as on when it is found.
In such a scenario we will have to start sending task result while the task is still in progress. This is the scenario where Asynchronous Thread-safe Function Calls come for our help. It acts as a safe tunnel between the worker thread and the JavaScript layer for information exchange. Let us consider the following function snippet
napi_value CAsyncStreamSearch(napi_env env, napi_callback_info info)
{
// The native addon function exposed to JavaScript
// This will be the funciton a node.js application calling.
}
void ExecuteWork(napi_env env, void* data)
{
// We will use this function to get the task done.
// This code will be executed on a worker thread.
}
void OnWorkComplete(napi_env env, napi_status status, void* data)
{
// after the `ExecuteWork` function exits, this
// callback function will be called on the main thread
}
void ThreadSafeCFunction4CallingJS(napi_env env, napi_value js_cb,
void* context, void* data)
{
// This funcion acts as a safe tunnel between the asynchronous C/C++ code
// executing the worker thread and the JavaScript layer for information exchange.
}
In this first three functions are nearly same as JavaScript Promise and Callback that we are familiar with. The fourth one is specifically for the Asynchronous Thread-safe Function Calls. In this, our long running task is being executed by ExecuteWork() function on a worker thread. Let us say it has instructed us not to call JavaScript (and also any napi_value conversion of result) from ExecuteWork() but permitted to do so from ThreadSafeCFunction4CallingJS as long as we are calling ThreadSafeCFunction4CallingJS with an napi equivalent of C/C++ function pointer. Then we could pack the JavaScript calls inside this ThreadSafeCFunction4CallingJS() function. Then when ExecuteWork() function could pass the result to ThreadSafeCFunction4CallingJS() while it is being invoked in a plain C/C++ storage units such as structure etc. The ThreadSafeCFunction4CallingJS() convert this result to napi_value and call JavaScript function.
Under the cover the ThreadSafeCFunction4CallingJS() function is being queue to the event loop, and eventually it get executed by main thread.
The following code snippet packed inside CAsyncStreamSearch() is responsible for creating a C/C++ function pointer equivalent of N-API by usng napi_create_threadsafe_function() and it is being done from the native addon's main thread itself. Similarly the request for creation of worker thread by using napi_create_async_work() function then placing the work int the event queue by using napi_queue_async_work() so that a worker thread will pickup this item in the future.
napi_value CAsyncStreamSearch(napi_env env, napi_callback_info info)
{
-- -- -- --
-- -- -- --
// Create a thread-safe N-API callback function correspond to the C/C++ callback function
napi_create_threadsafe_function(env,
js_cb, NULL, work_name, 0, 1, NULL, NULL, NULL,
ThreadSafeCFunction4CallingJS, // the C/C++ callback function
// out: the asynchronous thread-safe JavaScript function
&(async_stream_data_ex->tsfn_StreamSearch));
// Create an async work item, that can be deployed in the node.js event queue
napi_create_async_work( env, NULL,
work_name,
ExecuteWork,
OnWorkComplete,
async_stream_data_ex,
// OUT: THE handle to the async work item
&(async_stream_data_ex->work_StreamSearch);)
// Queue the work item for execution.
napi_queue_async_work(env, async_stream_data_ex->work_StreamSearch);
return NULL;
}
Then during the asynchronous execution of task (ExecuteWork() function) invokes ThreadSafeCFunction4CallingJS() by calling napi_call_threadsafe_function() function as shown bellow.
static void ExecuteWork(napi_env env, void *data)
{
// tsfn is napi equivalent of point to ThreadSafeCFunction4CallingJS
// function that we created at CAsyncStreamSearch function
napi_acquire_threadsafe_function( tsfn )
Loop
{
// this will eventually invoke ThreadSafeCFunction4CallingJS()
// we may call any number of time (in fact it can be called from any thread)
napi_call_threadsafe_function( tsfn, WorkResult, );
}
napi_release_threadsafe_function( tsfn,);
}
The example you pointed out is one of the best source of information and it is directly form node.js team itself. When I was learning this concept I too was referring the same example, during my study the example has been recreated by extracting original idea from it, hope you may find this much simplified. and it is available at
https://github.com/msatyan/MyNodeC/blob/master/src/mync1/ThreadSafeAsyncStream.cpp
https://github.com/msatyan/MyNodeC/blob/master/test/ThreadSafeAsyncStream.js
If anyone else gets stuck with this issue. I finally managed to hunt down an example here.
Once I understand it better and have gotten a working sample, I will update here. Hopefully someone needing this in the future will have an easier time than me.
See Satyan's answer
The solution from this site worked for me here
struct ThreadCtx {
ThreadCtx(Napi::Env env) {};
std::thread nativeThread;
Napi::ThreadSafeFunction tsfn;
};
void Target::Connect(const Napi::CallbackInfo& info) {
Napi::Env env = info.Env();
threadCtx = new ThreadCtx(env);
// Create a ThreadSafeFunction
threadCtx->tsfn = Napi::ThreadSafeFunction::New(env, info[0].As<Napi::Function>(), "Resource Name", 0 /* Unlimited queue */, 1 /* Only 1 thread */, threadCtx,
[&]( Napi::Env, void *finalizeData, ThreadCtx *context ) {
printf("Thread cleanup\n");
threadCtx->nativeThread.join();
},
(void*)nullptr
);
// Create a native thread
threadCtx->nativeThread = std::thread([&] {
auto callback = [](Napi::Env env, Napi::Function cb, char* buffer) {
cb.Call({Napi::String::New(env, buffer)});
};
char reply[1024];
memset(reply, 0, sizeof(reply));
while(true)
{
size_t reply_length = boost::asio::read(s, boost::asio::buffer(reply, sizeof(reply)));
if(reply_length <= 0) {
printf("Bad read from boost asio\n");
break;
}
// Callback (blocking) to JS
napi_status status = threadCtx->tsfn.BlockingCall(reply, callback);
if (status != napi_ok)
{
// Handle error
break;
}
}
// Release the thread-safe function
threadCtx->tsfn.Release();
});
}
addon.cc - (tested and 100% working)
#include <napi.h>
Napi::Value SAFE_THREAD(const Napi::CallbackInfo& info) {
std::thread([](Napi::ThreadSafeFunction tsfn){
struct output_data{
int arg1;
std::string arg2;
};
auto data = new output_data();
///---------------
///fill output data
data->arg1=1;
data->arg2="string data";
std::this_thread::sleep_for(std::chrono::milliseconds(2000));
///---------------
///output thread result to nodejs
napi_status status = tsfn.BlockingCall(data,[](Napi::Env env, Napi::Function jsCallback,output_data* data){
jsCallback.Call({Napi::Number::New(env, data->arg1), String::New(env, data->arg2)});
delete data;
});
if(status != napi_ok) { std::cout << "error!" << "\n"; }
tsfn.Release();
},Napi::ThreadSafeFunction::New(info.Env(), info[0].As<Function>(), "TSFN", 0, 1,[](Napi::Env env, void *finalizeData){},(void *)nullptr)).detach();
return info.Env().Null();
}
index.js
const ADDON = require('./THREAD/build/Release/addon');
function time_sec(){return (new Date()).getTime()/1000;}
var t = time_sec();
ADDON.SAFE_THREAD((arg1,arg2)=>{
console.log(time_sec()-t, 'arg1 = '+arg1)
console.log(time_sec()-t, 'arg2 = '+arg2)
});
console.log(time_sec()-t, 'fin')
output:
0.00099992752075 fin
2.00499987602233 arg1 = 1
2.00600004196167 arg2 = string data
see also how to emit data from thread

calling IO Operations from thread in ruby c extension will cause ruby to hang

I have a problem with using threads in a C Extension to run ruby code async.
I have the following C code:
struct DATA {
VALUE callback;
pthread_t watchThread;
void *ptr;
};
void *executer(void *ptr) {
struct DATA *data = (struct DATA *) ptr;
char oldVal[20] = "1";
char newVal[20] = "1";
pthread_cleanup_push(&threadGarbageCollector, data);
while(1) {
if(triggerReceived) {
rb_funcall(data->callback, rb_intern("call"), 0);
}
}
pthread_cleanup_pop(1);
return NULL;
}
VALUE spawn_thread(VALUE self) {
VALUE block;
struct DATA *data;
Data_Get_Struct(self, struct DATA, data);
block = rb_block_proc();
data->callback = block;
pthread_create(&data->watchThread, NULL, &executer, data);
return self;
}
I am using this because I want to provide ruby-code as a callback, which will be executed, once the Thread receives a signal.
In general this is working fine, if the callback is something like this ruby-code:
1 + 1
But, if the callbacks ruby-code looks like this:
puts "test"
than the main ruby process will stop responding, once the callback is getting executed.
The thread is still running and able to react to signals and puts the "test" everytime, the thread receives a message.
Can somebody maybe tell me, how to fix this?
Thanks a lot
From the Ruby C API docs:
As of Ruby 1.9, Ruby supports native 1:1 threading with one kernel
thread per Ruby Thread object. Currently, there is a GVL (Global VM
Lock) which prevents simultaneous execution of Ruby code which may be
released by the rb_thread_call_without_gvl and
rb_thread_call_without_gvl2 functions. These functions are
tricky-to-use and documented in thread.c; do not use them before
reading comments in thread.c.
TLDR; the Ruby VM is not currently (at the time of writing) thread safe. Check out this nice write-up on Ruby Threading for a better overall understanding of how to work within these confines.
You can use Ruby's native_thread_create(rb_thread_t *th) which will use pthread_create behind the scenes. There are some drawbacks that you can read about in the documentation above the method definition. You can then run your callback with Ruby's rb_thread_call_with_gvl method. Also, I haven't done it here, but it might be a good idea to create a wrapper method so you can use rb_protect to handle exceptions your callback may raise (otherwise they will be swallowed by the VM).
VALUE execute_callback(VALUE callback)
{
return rb_funcall(callback, rb_intern("call"), 0);
}
// execute your callback when the thread receives signal
rb_thread_call_with_gvl(execute_callback, data->callback);

Resources