In my C application I have a single ZMQ context, which I would like to share with all Lua States. I use Lua version 5.2 and ZMQ version 3.2.
I would like to use an already available binding for Lua, like lzmq.
For example:
// create ZMQ context
void *ctx = zmq_ctx_new();
...
// create Lua State
lua_State *L = luaL_newstate();
...
// push the context or something
lua_setglobal(L, "MY_ZMQ_CONTEXT");
then being able to somehow use that ZMQ context in Lua (example using lzmq):
local zmq = require "lzmq"
require "utils"
print_version(zmq)
local ctx = MY_ZMQ_CONTEXT -- ???
local skt = ctx:socket{zmq.REQ,
linger = 0, rcvtimeo = 1000;
connect = "inproc://hello";
}
skt:send("hello from cli")
print_msg("recv: ",skt:recv())
skt:close()
How would I do something like this? either using lzmq or any other ZMQ Lua bindings?
You could set lightuserda and use init_ctx function.
lua_pushlightuserdata(L, ctx);
lua_setglobal(L, "MY_ZMQ_CONTEXT");
local zmq = require "lzmq"
local ctx = zmq.init_ctx(MY_ZMQ_CONTEXT)
In this case you can not close context from Lua.
I have idea add this functionality in next version.
lzmq also has C functoin LUAZMQ_EXPORT int luazmq_context (lua_State *L, void *ctx, unsigned char own).
You can wrap contex like this.
luazmq_contex(L, ctx, 0);
// or if you want hase ability to destroy contex from Lua
// luazmq_contex(L, ctx, 1);
// MY_ZMQ_CONTEXT is lzmq context
lua_setglobal(L, "MY_ZMQ_CONTEXT");
Related
I need to attach file descriptors to the GLIB mainloop. My issue is that the list of file descriptors is not fixed during execution.
According to GLIB documentation, I can:
create a GIOChannel for each FD using g_io_channel_unix_new and attach it to the context with g_io_add_watch
Use a Gsource created with g_io_create_watch and set a callback g_source_set_callback
My question is : is it possible to modify dynamically a source or a context. And how can I do it ? I find the GSourceFuncs ability, but that doesn't fit my issue.
Thanks for your help.
g_io_add_watch returns an event source ID which you can later use to dynamically remove the watch again, using g_source_remove. Use one event source per FD and instead of modifying existing watches, remove the old ones and create appropriate new ones.
I digged more into GLIB and now:
I create a source with callbacks functions (prepare, check, dispatch, finalize)
In the prepare callback, FD are deleted using g_source_remove_unix_fd() and then added to the current source using g_source_add_unix_fd().
I returned FALSE to set the timeout (1s for my example)
My issue is that without the FD, the prepare callback is called each 1s as expected. When FD is added, the prepare callback is called without timeout. the poll exit directly.
I have a look into GLIB source code, but don't understand the reason why ?
Help please
Regards
amenophiks' answer is the best.
If you want your code to work with an older glib you can use:
g_source_add_poll()
g_source_remove_poll()
Have you read the Main Event Loop documentation? The description section has a pretty good explanation of how things work.
Have you looked at the Custom GSource tutorial? This allows you to extend a GSource object to include your own state. You also can write your own prepare, dispatch, query, and check functions.
Whenever I really want to see how something should be done with GLib, GTK, etc the first place I look is the test code that lives in their git repository. Be sure to checkout the proper tag for the version of that you are targeting.
For example I currently target 2.48.2
Here are two pretty good examples
https://gitlab.gnome.org/GNOME/glib/blob/2.48.2/tests/mainloop-test.c
https://gitlab.gnome.org/GNOME/glib/blob/2.48.2/glib/tests/mainloop.c
The other nice feature is it's a git repository so you can search it very easily.
Seems, I found a diminutive hook. Try this:
struct source {
GSource gsrc;
GPollFD *gpfd;
};
struct data {
/* A something data. */
};
static gboolean gsrc_dispatch(GSource *gsrc, GSourceFunc cb, gpointer data);
static struct data * data_alloc(void);
static GSourceFuncs gsf = {
.prepare = NULL,
.check = NULL,
.dispatch = gsrc_dispatch,
.finalize = NULL
};
int main(void)
{
struct source *src;
int fd;
struct data *data = data_alloc();
/* Something other. */
/* For example, we are want to capture video from a camera. */
fd = open("/dev/video0", O_RDWR);
if (fd < 0) {
perror("open()");
return -1;
}
src = (struct source *) g_source_new(&gsf, sizeof(struct source));
src->gpfd = g_source_add_unix_fd((GSource *) src, fd, G_IO_IN);
g_source_set_callback((GSource *) src, NULL, data, NULL);
g_source_attach((GSource *) src, NULL);
/* Something other and free. */
return 0;
}
static gboolean
gsrc_dispatch(GSource *gsrc, GSourceFunc cb, gpointer data)
{
struct source *src = (struct source *) gsrc;
struct data *d = data;
if (src->gpfd != NULL) {
if (src->gpfd->revents & G_IO_IN) {
/* Capture a frame. */
}
}
g_main_context_iteration(NULL, TRUE);
return G_SOURCE_CONTINUE;
}
static struct data *
data_alloc(void)
{
/* Allocate a data. */
}
Yes, you can use the double gpfd pointer.
I am using C library iperf3 to measure network. When I start network testing my aplication freezes and wait for results. I tried async and threads but any progress. Any advise? I'd like to run my test and asynchronously call another methods (at best, call this library again, but other methods). Is it possible?
My network.dart
final DynamicLibrary iperfLib = Platform.isAndroid
? DynamicLibrary.open("libiperf.so")
: DynamicLibrary.process();
typedef RunTestFunc = ffi.Pointer<ffi.Uint8> Function(
ffi.Pointer<ffi.Uint8> context);
typedef RunTest = ffi.Pointer<ffi.Uint8> Function(
ffi.Pointer<ffi.Uint8> context);
RunTest _run_test = iperfLib
.lookup<ffi.NativeFunction<RunTestFunc>>('run_test')
.asFunction<RunTest>();
ffi.Pointer<ffi.Uint8> runTest(ffi.Pointer<ffi.Uint8> context) {
return _run_test(context);
}
and iperf.c
Iperf* run_test(Iperf* test) {
__android_log_print( ANDROID_LOG_INFO, "DONE ", "server_hostname %s", test->server_hostname );
int cc = iperf_run_client( test ) ;
__android_log_print( ANDROID_LOG_INFO, "DONE ", " %d",cc );
iperf_free_test( test );
return test
}
Async Callbacks
The problem is that C routines called from dart are blocking and therefore congest the single existing dart isolate, consequently freezing the UI.
To work around this problem you have to open a port on the dart isolate through which your C routines can asynchronously send messages to the dart isolate. To signal to the dart compiler that this is a non-blocking operation, simply delay the completion of the function until a message on the designated port has been received.
Future<int> asyncData() async {
var receiveData;
bool receivedCallback = false;
var receivePort = ReceivePort()..listen((data) {
print('Received data from c');
receiveData = data;
receivedCallback = true;
});
var nativeSendPort = receivePort.sendPort.nativePort;
nativeTriggerFunction(nativeSendPort);
while(!receivedCallback) {
await Future.delayed(Duration(milliseconds: 100));
}
receivePort.close();
return receiveData;
}
In C, you need to create a trigger function which should ideally be as lightweight as possible, passing the port number to your C code and calling the actual function you want to execute on a different thread.
The trigger function will finish almost instantly, allowing your dart thread to do other work and as soon as the newly created thread is done, it sends its result through the native port back to the dart isolate which can pick up where it left off.
void native_trigger_function(Dart_Port port) {
pthread_t t;
Dart_Port *args = (Dart_Port *) malloc(sizeof(Dart_Port));
*args = port;
pthread_create(&t, NULL, _native_function, args);
}
void *_native_function(void *args) {
Dart_Port port = *(Dart_Port *) args;
int rc = 0;
// do some heavy work
// send return code to dart
Dart_CObject obj;
obj.type = Dart_CObject_kInt32;
obj.value.as_int32 = rc;
Dart_PostCObject_DL(port, &obj);
free(args);
pthread_exit(NULL);
}
Note: This logic relies on the native dart api to work which can be found here. Before use, the interface needs to be attached to the current dart isolate which can be achieved by calling Dart_InitializeApiDL(dart_api_data) from C where dart_api_data is a void pointer which can be obtained from your dart code using the dart:ffi package through NativeApi.initializeApiData.
Update: Thanks #fdollack for fixing the example snippets!
Thank you #Lucas Aschenbach!
This minimum example was so hard to find.
2 small additions.
First, the allocated pointer should be casted to (Dart_Port*),
and the port argument from dart has to be assigned/copied to where the pointer is at!
void native_trigger_function(Dart_Port port) {
pthread_t t;
Dart_Port *args= (Dart_Port*)malloc(sizeof(Dart_Port));
*args = port; // assign port
pthread_create(&t, NULL, _native_function, args);
}
The second thing is inside the _native_function the response to Dart has to be
Dart_PostCObject_DL(port, &obj);
instead of
Dart_PostCObject_DL(args_c.send_port, &obj);
Is there a way to attach data to a coroutine, or at least to somehow identify different coroutines?
I am trying to implement a timer API where the timers are controlled by the host, that looks something like the following on the Lua side:
function callback()
local timer = ElapsedTimer()
...
end
function main()
local timer = CreateTimer(...)
StartTimer(timer, callback, ...)
end
The StartTimer() call sends the timer and callback to the C side, and the C side will eventually call the callback in a new coroutine.
The call to ElapsedTimer() needs to return data that is specific to this coroutine/thread, i.e. in this case the timer.
In pseudo code:
int StartTimer(lua_State* L) {
auto timer = ...;
auto coroutine = ???
coroutine.userdata = &timer; // But probably store an actual structure with more pointers
return 0;
}
int ElapsedTimer(lua_State* L) {
auto coroutine = ???
auto timer = (Timer*)coroutine.userdata;
lua_pushlightuserdata(L, timer)
return 1;
}
This is what userdata is for:
int StartTimer(lua_State* L) {
TimerStruct timer = (TimerStruct*)lua_newuserdata(L, sizeof(TimerStruct));
//allocates some memory and returns pointer to it
timer->//...
// lua_setuservalue will let you bind the callback to the timer
return 1; //actually return it
}
Also it is going to be easier to pass the timer along
function callback(timer)
...
end
That way the lua code doesn't need to query that value. It already got it.
I just now realized I need to use lua_newthread to create the coroutine, which means I have a separate lua_State object.
This means I can always create a mapping between the coroutine's state and whatever data I want in the host language (e.g. in a map data structure)
I initiate Redis connection pool in redis.lua, by calling from C, I got a redis_lua_state, this Lua state is global initiated once and other thread only get from it.
While there comes a HTTP request(worker thread), I need to fetch a redis connection from redis_lua_state, then new another Lua state to load other Lua script, and these scripts will use this redis connection to communicate with Redis, how to do this? Or how to design my Lua scripts to make it simple?
Code Sample:
/* on main thread, to init redis pool connection */
lua_State *g_ls = NULL;
lua_State *init_redis_pool(void) {
int ret = 0;
g_ls = luaL_newstate();
lua_State *ls = g_ls;
luaL_openlibs(ls);
ret = luaL_loadfile(ls, "redis.lua");
const char *err;
(void)err;
/* preload */
ret = lua_pcall(ls, 0, 0, 0);
lua_getglobal(ls, "init_redis_pool");
ret = lua_pcall(ls, 0, 0, 0);
return ls;
}
/* worker thread */
int worker() {
...
lua_State *ls = luaL_newstate();
ret = luaL_loadfile(ls, "run.lua");
/* How to fetch data from g_ls? */
...
lua_getglobal(ls, "run")
ret = lua_pcall(ls, 0, 0, 0)
lua_close(ls);
...
return 0;
}
If your Lua states are separate, then there's no way to do this. Your worker thread will have to initialize the Redis connection and do processing on it.
One way to do it is to implement copying variables between Lua states on C side. I have done a similar thing in my ERP project, but it requires a bit of a hassle, especially when it comes to copying user data.
What I did was to implement some sort of a super global variable (a C class in my instance) system that's implemented as __index and __newindex of the global table, which is a proxy to default Lua global table. When setting a global variable, __newindex would copy that to that super global. When another Lua state would try to access that global, it would retrieve it from the same structure.
And then the redis connection could be a mutex locked shared, so when one state accesses it, the other cannot, for instance.
Of course there's the issue of accessing Lua default globals that you also have to take care of.
Alternatively, you can check out Lua lanes, if that's an option (I've no Redis experience, so don't know how open the Lua is, but I see that you have full access to C api, so it should work).
I have researched this subject and tried various approaches but I can't implement the behavior I have in mind (I'm not even sure it's possible). Basically, I have several userdata objects created in C that can be accessed by their metatable, like this:
Main.lua
config.display_width = 1280
What I'd like to do is to "force" the config namespace to a specific script. You've guessed it, I need to protect a configuration file so that users are restricted to deal only with the config metatable. Like this:
Config.lua
display_width = 1280
And I know I have to do something like this in C:
// Register the config metatable and its methods
luaL_loadfile(L, "my_config.cfg");
lua_getglobal(L, "config"); // Is this necessary?
lua_setfenv(L, -2); // I know this has to be used, but how?
lua_pcall(L, 0, 0, 0);
Thank you in advance, this one is driving me crazy!
PS: For the record, I really need to keep the config userdata as it is because it's binded to a C structure. In consequence, I'm not concerned about "losing" the Lua state or declared variables between different environments.
Adding the following information. This is how the config userdata is being created:
const struct luaL_Reg metaconfig[] =
{
{"__index", l_get},
{"__newindex", l_set},
{NULL, NULL}
};
lua_newuserdata(L, sizeof(void *));
luaL_newmetatable(L, "metaconfig");
luaL_register(L, NULL, metaconfig);
lua_setmetatable(L, -2);
lua_setglobal(L, "config");
So every time the user sets or gets values from the config userdata I update the C structure via the __index or__newindex methods.
you don't really need a global representing the config table, you can do with a lua_ref too.
Here this works as expected (I guess):
#include <lua.h>
#include <lauxlib.h>
#include <stdlib.h>
int main (void){
int idxConfig, res;
lua_State *L = luaL_newstate();
if ((res = luaL_loadfile(L,"my_config.cfg")) != 0){//Load file
printf("Got error code %d loading file my_config.cfg, exiting",res);
exit(-1);
}
lua_newtable(L); // new config table
lua_pushvalue(L,-1);// duplicate table
idxConfig = lua_ref(L,LUA_REGISTRYINDEX); // take a reference to the table (pops it)
lua_setfenv(L,-2); // pop table, set as environment for loaded chunk
lua_call(L,0,0); // load config -- nothing on stack
lua_rawgeti(L,LUA_REGISTRYINDEX,idxConfig); //push config table
lua_getfield(L,1,"display"); //read out "display"
lua_Integer disp_width = lua_tointeger(L,-1);
printf("Display width = %d",(int) disp_width);
lua_close(L);
exit(0);
}