How to get metatable set by lua from Lua C API - c

Question

Lua:
a = {
b = "c",
d = {
e = "f",
g = "h"
}
}
setmetatable(a.d, {__ismt = true})
cfun(a) --call C function to iterate over table a
C:
int cfun(lua_State *L)
{
lua_pushnil(L);
while (lua_next(L, -2) != 0)
{
// iterate over table
lua_pop(L, 1);
}
}
How do you know if there is a metatable when the host client iterates over the table? And then how do you get the metatable?

Answer 1

The table is in a form of a tree, and you need to traverse the tree in iterative mode. Lua already has a stack implementation, so this makes the job easier.
On entrance, the stack has the table at top, you will push a nil element since lua_next() will consume one element from the stack before checking the table. So the stack will look like table -> nil.
Next, we are calling lua_next() which will consume one element from the stack and will add two new key-value pair from the table. The stack will look like table -> key -> value. If there is no next element, the return value of the call is 0.
If the return value is 1, and the value on stack is a nested table, you will push nil on the stack, so now the stack will look like table -> key -> table -> nil. Now you are almost like on the begining, so with the loop, you will start traversing the nested table.
if the return value is 1, and if the value is not table, to your stuff with the value
if the return value is 0, we can check if this was a meta table. After the check, you will pop the value and check if the stack is table -> key or any -> key. If the second element on the stack is not a table, you have finished the traversing and you will break the loop.
Here is the C code that implements the algorithm. I have left the printf in order to help debugging. The printf() should be removed.
static int cfun(lua_State *L)
{
luaL_checktype(L, 1, LUA_TTABLE);
lua_pushnil(L); // Add extra space for the first lua_next to pop
int loop=1;
do {
if ( lua_next(L,-2) != 0 ) {
if (lua_istable(L,-1)) {
printf("Table [%s] \n", lua_tostring(L, -2));
lua_pushnil(L); // Start iterating this sub-table
} else {
// The Key and Value are on the stack. We can get their type
printf("(%s - %s)\n",
lua_tostring(L, -2),
lua_typename(L, lua_type(L, -1)));
lua_pop(L,1);
}
} else {
printf("table finished, still on stack (%s -> %s -> %s)\n",
lua_typename(L, lua_type(L, -3)),
lua_typename(L, lua_type(L, -2)),
lua_typename(L, lua_type(L, -1)));
if (lua_getmetatable(L,-1)) {
// The table has metatable. Now the metatable is on stack
printf("Metatable detected\n");
lua_pop(L,1); // remove the metatable from stack
}
lua_pop(L,1); // Pop the current table from stack
if (!lua_istable(L, -2)) {
loop = 0; // No more tables on stack, breaking the loop
}
}
} while (loop);
lua_pop(L,1); // Clear the last element
lua_pushnumber(L,0); // Return 0
return 1;
}