I initiate Redis connection pool in redis.lua, by calling from C, I got a redis_lua_state, this Lua state is global initiated once and other thread only get from it.
While there comes a HTTP request(worker thread), I need to fetch a redis connection from redis_lua_state, then new another Lua state to load other Lua script, and these scripts will use this redis connection to communicate with Redis, how to do this? Or how to design my Lua scripts to make it simple?
Code Sample:
/* on main thread, to init redis pool connection */
lua_State *g_ls = NULL;
lua_State *init_redis_pool(void) {
int ret = 0;
g_ls = luaL_newstate();
lua_State *ls = g_ls;
luaL_openlibs(ls);
ret = luaL_loadfile(ls, "redis.lua");
const char *err;
(void)err;
/* preload */
ret = lua_pcall(ls, 0, 0, 0);
lua_getglobal(ls, "init_redis_pool");
ret = lua_pcall(ls, 0, 0, 0);
return ls;
}
/* worker thread */
int worker() {
...
lua_State *ls = luaL_newstate();
ret = luaL_loadfile(ls, "run.lua");
/* How to fetch data from g_ls? */
...
lua_getglobal(ls, "run")
ret = lua_pcall(ls, 0, 0, 0)
lua_close(ls);
...
return 0;
}
If your Lua states are separate, then there's no way to do this. Your worker thread will have to initialize the Redis connection and do processing on it.
One way to do it is to implement copying variables between Lua states on C side. I have done a similar thing in my ERP project, but it requires a bit of a hassle, especially when it comes to copying user data.
What I did was to implement some sort of a super global variable (a C class in my instance) system that's implemented as __index and __newindex of the global table, which is a proxy to default Lua global table. When setting a global variable, __newindex would copy that to that super global. When another Lua state would try to access that global, it would retrieve it from the same structure.
And then the redis connection could be a mutex locked shared, so when one state accesses it, the other cannot, for instance.
Of course there's the issue of accessing Lua default globals that you also have to take care of.
Alternatively, you can check out Lua lanes, if that's an option (I've no Redis experience, so don't know how open the Lua is, but I see that you have full access to C api, so it should work).
Related
Is there a way to attach data to a coroutine, or at least to somehow identify different coroutines?
I am trying to implement a timer API where the timers are controlled by the host, that looks something like the following on the Lua side:
function callback()
local timer = ElapsedTimer()
...
end
function main()
local timer = CreateTimer(...)
StartTimer(timer, callback, ...)
end
The StartTimer() call sends the timer and callback to the C side, and the C side will eventually call the callback in a new coroutine.
The call to ElapsedTimer() needs to return data that is specific to this coroutine/thread, i.e. in this case the timer.
In pseudo code:
int StartTimer(lua_State* L) {
auto timer = ...;
auto coroutine = ???
coroutine.userdata = &timer; // But probably store an actual structure with more pointers
return 0;
}
int ElapsedTimer(lua_State* L) {
auto coroutine = ???
auto timer = (Timer*)coroutine.userdata;
lua_pushlightuserdata(L, timer)
return 1;
}
This is what userdata is for:
int StartTimer(lua_State* L) {
TimerStruct timer = (TimerStruct*)lua_newuserdata(L, sizeof(TimerStruct));
//allocates some memory and returns pointer to it
timer->//...
// lua_setuservalue will let you bind the callback to the timer
return 1; //actually return it
}
Also it is going to be easier to pass the timer along
function callback(timer)
...
end
That way the lua code doesn't need to query that value. It already got it.
I just now realized I need to use lua_newthread to create the coroutine, which means I have a separate lua_State object.
This means I can always create a mapping between the coroutine's state and whatever data I want in the host language (e.g. in a map data structure)
I'm declaring array of pthread_rwlock_t static global.
e.g. static pthread_rwlock_t cm[255];
Inside constructor I want to initialize one of the 255 mutex( I keep track with static counter)
Now I'm confused with
1) I don't want to re-initialize lock again, that is bad!
I thought reinitialize should return some error code, but it doesn't:
#include<stdio.h>
#include <pthread.h>
static pthread_rwlock_t cm[2];
int main()
{
int ret;
ret = pthread_rwlock_init(&cm[0], NULL);
ret = pthread_rwlock_wrlock(&cm[0]);
printf("Ret: %d\n", ret);
ret = pthread_rwlock_init(&cm[0], NULL);
printf("Ret: %d\n", ret);
ret = pthread_rwlock_wrlock(&cm[0]);
printf("Ret: %d\n", ret);
}
Result:
Ret: 0
Ret: 0
Ret: 0
Can anyone help, 1) If this is possible, then how? 2) If not what should be alternative approach?
EDIT 1:
I'm updating from comments/answers I got:
Instead, just put the rwlocks inside the objects they protect.
So I have n # of objects getting called, and will be using that many pthread_lock .. so disadvantage is memory. Hence I'm trying to improve on that part with global array of locks. Picking 256 to get good distribution.
It's undefined behavior to call pthread_rwlock_init (or analogously any of the pthread primitive init functions) more than once on the same object, and logically there's no way it would make sense to do so anyway since (as you've demonstrated) the object is already in use. You said in the comments on 2501's answer that you can't use pthread_once, but this makes no sense. If you're able to call pthread_rwlock_init, you can instead just call pthread_once using an init function which performs the call to pthread_rwlock_init.
However I really think you're experiencing an XY problem. There is no sense in maintaining a "global pool" of rwlocks and handing them out dynamically in constructors. Instead, just put the rwlocks inside the objects they protect. If you really want to hand them out from a global pool like you're doing, you need to keep track of which ones have been handed out independently of the job of initializing them, and have the task of initializing them after obtaining one, and destroying one before giving it back to the pool, be handled by the constructor/destructor for the object using them.
If you need static initialization, use PTHREAD_RWLOCK_INITIALIZER on your array.
static pthread_rwlock_t cm[2] = { PTHREAD_RWLOCK_INITIALIZER ,
PTHREAD_RWLOCK_INITIALIZER} ;
This is equivalent as calling pthread_rwlock_init() on every element with attr parameter specified as NULL, except that no error checking is performed.
In my C application I have a single ZMQ context, which I would like to share with all Lua States. I use Lua version 5.2 and ZMQ version 3.2.
I would like to use an already available binding for Lua, like lzmq.
For example:
// create ZMQ context
void *ctx = zmq_ctx_new();
...
// create Lua State
lua_State *L = luaL_newstate();
...
// push the context or something
lua_setglobal(L, "MY_ZMQ_CONTEXT");
then being able to somehow use that ZMQ context in Lua (example using lzmq):
local zmq = require "lzmq"
require "utils"
print_version(zmq)
local ctx = MY_ZMQ_CONTEXT -- ???
local skt = ctx:socket{zmq.REQ,
linger = 0, rcvtimeo = 1000;
connect = "inproc://hello";
}
skt:send("hello from cli")
print_msg("recv: ",skt:recv())
skt:close()
How would I do something like this? either using lzmq or any other ZMQ Lua bindings?
You could set lightuserda and use init_ctx function.
lua_pushlightuserdata(L, ctx);
lua_setglobal(L, "MY_ZMQ_CONTEXT");
local zmq = require "lzmq"
local ctx = zmq.init_ctx(MY_ZMQ_CONTEXT)
In this case you can not close context from Lua.
I have idea add this functionality in next version.
lzmq also has C functoin LUAZMQ_EXPORT int luazmq_context (lua_State *L, void *ctx, unsigned char own).
You can wrap contex like this.
luazmq_contex(L, ctx, 0);
// or if you want hase ability to destroy contex from Lua
// luazmq_contex(L, ctx, 1);
// MY_ZMQ_CONTEXT is lzmq context
lua_setglobal(L, "MY_ZMQ_CONTEXT");
I want to use the GWAN API Key-Value to record and read a number of data (in a multi-threaded way). The problem is that my recordings are only available on the current page and therefore can not be used on my other pages.
Can you show me an example or explain how to create a persistent KV store (which will be accessible on all my subdomains) ?
Here is an example that I currently use:
kv_t store;
kv_init(&store, "users", 10, 0, 0, 0);
kv_item item;
item.key = "pierre";
item.klen = sizeof("pierre") - 1;
item.val = "pierre#example.com";
item.flags = 0;
kv_add(&store, &item);
char *p = kv_get(&store, "pierre", sizeof("pierre") - 1);
xbuf_xcat(get_reply(argv), "<br>pierre's email address: %s<br>", p);
but is not persistent.
As G-WAN scripts are compiled and linked independently, 'global' variables are 'static' (to each script) rather than available for all scripts.
So, you have to attach the KV store to a persistent pointer. G-WAN offers persistent pointers with different scopes:
US_REQUEST_DATA = 200, // Request-wide pointer
US_HANDLER_DATA, // Listener-wide pointer
US_VHOST_DATA, // Virtual-Host-wide pointer
US_SERVER_DATA, // global pointer (for maintenance script)
There are several G-WAN script examples demonstrating how to do that:
http://gwan.ch/source/persistence.c
http://gwan.ch/source/stream1.c
http://gwan.ch/source/forum.c
etc.
I have researched this subject and tried various approaches but I can't implement the behavior I have in mind (I'm not even sure it's possible). Basically, I have several userdata objects created in C that can be accessed by their metatable, like this:
Main.lua
config.display_width = 1280
What I'd like to do is to "force" the config namespace to a specific script. You've guessed it, I need to protect a configuration file so that users are restricted to deal only with the config metatable. Like this:
Config.lua
display_width = 1280
And I know I have to do something like this in C:
// Register the config metatable and its methods
luaL_loadfile(L, "my_config.cfg");
lua_getglobal(L, "config"); // Is this necessary?
lua_setfenv(L, -2); // I know this has to be used, but how?
lua_pcall(L, 0, 0, 0);
Thank you in advance, this one is driving me crazy!
PS: For the record, I really need to keep the config userdata as it is because it's binded to a C structure. In consequence, I'm not concerned about "losing" the Lua state or declared variables between different environments.
Adding the following information. This is how the config userdata is being created:
const struct luaL_Reg metaconfig[] =
{
{"__index", l_get},
{"__newindex", l_set},
{NULL, NULL}
};
lua_newuserdata(L, sizeof(void *));
luaL_newmetatable(L, "metaconfig");
luaL_register(L, NULL, metaconfig);
lua_setmetatable(L, -2);
lua_setglobal(L, "config");
So every time the user sets or gets values from the config userdata I update the C structure via the __index or__newindex methods.
you don't really need a global representing the config table, you can do with a lua_ref too.
Here this works as expected (I guess):
#include <lua.h>
#include <lauxlib.h>
#include <stdlib.h>
int main (void){
int idxConfig, res;
lua_State *L = luaL_newstate();
if ((res = luaL_loadfile(L,"my_config.cfg")) != 0){//Load file
printf("Got error code %d loading file my_config.cfg, exiting",res);
exit(-1);
}
lua_newtable(L); // new config table
lua_pushvalue(L,-1);// duplicate table
idxConfig = lua_ref(L,LUA_REGISTRYINDEX); // take a reference to the table (pops it)
lua_setfenv(L,-2); // pop table, set as environment for loaded chunk
lua_call(L,0,0); // load config -- nothing on stack
lua_rawgeti(L,LUA_REGISTRYINDEX,idxConfig); //push config table
lua_getfield(L,1,"display"); //read out "display"
lua_Integer disp_width = lua_tointeger(L,-1);
printf("Display width = %d",(int) disp_width);
lua_close(L);
exit(0);
}