I'm evaluating Lua for a bare metal project (most of it is already running) and it looks like what we need.
I need to put it into my code and be able to call into Lua without hanging in there at all. I need to call into Lua repeatedly like it's a state machine and have it return a status every time I call it to say either "Keep calling me, I'm not finished", "Stop, I have an error", or "Stop, no errors, script has completed".
I've seen the hooks that allow a callback from Lua to c for every line or byte of bytecode that's evaluated, but I cannot use a callback.
Does a state machine implementation of Lua exist?
You're looking for the lua_resume C function, which uses coroutines. From its documentation:
lua_resume returns LUA_YIELD if the coroutine yields, 0 if the coroutine finishes its execution without errors, or an error code in case of errors (see lua_pcall).
That sounds to me like exactly what you're describing.
Related
At first glance (see the evidence below), it looks like while a Tcl_CmdProc has control, the interpreter is waiting for it to return and can't accept any other calls in the meantime.
So, how do I make any calls into Tcl before returning like e.g. a user-defined function would do? I guess I may need to set up a new call stack frame in the interpreter or something (and unwind it later). Tcl_CreateCommand man page says nothing on this matter.
The big picture is like this:
I'm fixing https://bugs.python.org/issue33257 . The TkinterHandlers.py example uses Python event handlers that are implemented as custom Tcl commands under the hood. Currently, their implementation releases the "Tcl lock" (a Python-specific lock that it wraps all Tcl calls with) while executing Python code and reacquires it to Tcl_SetObjResult at the end -- thus allowing other calls to the same interpreter in the meantime.
Now, if another call into the interpreter is actually made during this time frame, Tcl aborts shortly with a message on stderr: TclStackFree: incorrect freePtr. Call out of sequence?
And if I make the custom command hold on to the Tcl lock, it later freezes trying to acquire the lock again because it itself also needs to make a Tcl call sometimes. Now, I can make the lock reentrant, but without knowing how to handle the interpreter right, I'll probably break it, too.
To keep this question on topic, I'm specifically asking about how to handle the interpreter, and make Tcl calls in particular, from a Tcl_CmdProc. The specific situation is solely for exposition to illustrate my needs. If this is actually explained in some doc that I couldn't find, linking to it and reciting some key points would be sufficient.
To call a Tcl command from C code, you've got a choice between two API function families. One is Tcl_EvalObjv, and the other is Tcl_Eval. Each has a number of variants, but the only variant I'll mention is Tcl_EvalObjEx.
Tcl_EvalObjv
This function invokes a single Tcl command, with no processing of substitutions in arguments (unless the command itself does them, of course). It has this signature:
int Tcl_EvalObjv(Tcl_Interp *interp,
int objc,
Tcl_Obj *const objv[],
int flags);
It takes the description of what command to call and what arguments to pass to it as a C array of Tcl value references (in argument objv) where the array is of length objc; Tcl guarantees to not modify the array itself, but might transform the values if it does type conversions. The values must all have a non-zero reference count (and all values start with a zero reference count from their birthing Tcl_NewObj call). The interp is the interpreter context, and flags can usually be zero.
The result is a Tcl exception code; if it is TCL_OK, the result of the call can be retrieved from the interpreter using Tcl_GetObjResult, and if the exception code is TCL_ERROR then there was an error and you should usually pass that on out (perhaps adding to the stack trace with Tcl_AddErrorInfo). Other exception codes are possible; it's usually best to just pass those straight on out without doing any further processing (unless you're making something loop-like, when you should pay attention to TCL_BREAK and TCL_CONTINUE).
Tcl_Eval
This function evaluates a Tcl script, not just a single command, and that includes processing substitutions in arguments. It has this signature:
int Tcl_Eval(Tcl_Interp *interp,
const char *script);
The script is any old C string; Tcl won't modify it, but it will parse, bytecode-compile, and execute it. It's up to you to provide the script in a form that will execute a single command without surprises. The interp argument and the result of the function call are the same as for Tcl_EvalObjv.
If you're interested in using this for running a single command, you're actually better off using Tcl_EvalObjv or…
Tcl_EvalObjEx.
This is like Tcl_Eval except it takes the script as a Tcl value reference (and takes flags too).
int Tcl_EvalObjEx(Tcl_Interp *interp,
Tcl_Obj *objPtr,
int flags);
Again, make sure the objPtr has a non-zero reference count before passing it into this function. (It may adjust the reference count during execution.) Again, interp and the result are as documented for Tcl_EvalObjv, and flags is too.
The advantage of this for calling single commands is that you can call Tcl_NewListObj (or any other list-building function) to make the script value; doing so guarantees that there will be no surprise substitutions. But you could also go directly to invoking the command with Tcl_EvalObjv. But if you want to process anything more complex than a single simple call to a command, this is a good place to start as it has a key advantage that plain Tcl_Eval doesn't: it can make the type of the script passed in via objPtr be one that caches the compiled bytecode, allowing quite a reasonable performance gain in some circumstances.
Note that Tcl_EvalObjv is effectively the API that Tcl calls internally to invoke all user code and perform all I/O. (“Effectively” because things get more complex in Tcl 8.6.)
Within a Tcl_CmdProc, all these functions can be called as usual, no special processing or "handling of the interpreter" is needed. If this doesn't work for you, causing crashes or whatever, the interpreter is not at fault, something else must be wrong with your code.
I was looking at the manual for longjmp and in the Errors part it says this:
ERRORS
If the contents of the env are corrupted, or correspond to an environment that has already returned, the longjmp() routine calls the routine longjmperror(3). If longjmperror()
returns, the program is aborted (see abort(3)). The default version of longjmperror() prints the message ``longjmp botch'' to standard error and returns. User programs wishing to exit more gracefully should write their own versions of longjmperror().
How would i write my own version of longjmperror? From what i know in C you can't override functions and i really need the long jump to exit in a specific way when it doesn't find the point to jump to.
On Mac OS X (10.9.2, Mavericks) at any rate, the prototype for longjmperror() is:
void longjmperror(void);
You write a function with that signature. It must not return (or, rather, if it does, the program will be abort()ed). What you do in that function is your business, but bear in mind that things have gone moderately catastrophically wrong for the function to be called at all). It might log an error to your log file, or just write a more meaningful message before exiting (instead of aborting and perhaps core dumping).
You link the object file containing the function ahead of the system library. You are normally not expected to replace system functions, but this is one you are intended to override.
How is WinMain() actually called? I remember a function used by pro-hackers that started with (something) that looked like __startupWinMain().
The problem is, I have a Win32 EXE(compiled with /SUBSYSTEM:WINDOWS) but gets arguments from command-line. If the command line is incorrect, the process should print a help message to the console.
How can I manually deallocate(or FreeConsole()) from an exe with /SUBSYSTEM:WINDOWS linker option?
As the very first act of your program, check the parameters. If they are fine, continue as normal.
Otherwise call AttachConsole passing ATTACH_PARENT_PROCESS. If that succeeds, then you can print your error to stdout and quit. If it doesn't, then you'll have to show the error in a message box.
Perhaps you should consider having the program pop up a message box when the command line is incorrect. Something like this:
MessageBox( NULL, "(description of command line error)",
"MyProg - Command Line Error",
MB_OK|MB_ICONEXCLAMATION );
This will open a message box in the center of the display and wait for the user to acknowledge it before actually terminating your program.
On the other hand, you could build your program as a console app and use printf() to write to the console. A console program may still create windows, but the console itself will hang around unless you figure out how to detach from it (and then, of course, you will no longer be able to use printf().)
How does the compiler know to invoke wWinMain instead of the standard main function? What actually happens is that the Microsoft C runtime library (CRT) provides an implementation of main that calls either WinMain or wWinMain.
Note The CRT does some additional work inside main. For example, any static initializers are called before wWinMain. Although you can tell the linker to use a different entry-point function, use the default if you link to the CRT. Otherwise, the CRT initialization code will be skipped, with unpredictable results. (For example, global objects will not be initialized correctly.)
How is WinMain() actually called?
If you single-step to the first line of your program in a debugger, and then look at the stack, you can see how WinMain gets called. The actual start function for a typical build is a function pulled in from the run-time library. For me, it's _WinMainCRTStartup, but I suppose it might vary depending on the version of the compiler, linker, and library you build with. The startup function from the run-time library does some initialization and then calls WinMain.
Using dumpbin /headers (or another program that can inspect a PE binary), you can confirm which function is the "entry point" to your executable. Unless you've done something to change it, you'll probably see _WinMainCRTStartup, which is consistent with what the stack trace tells us.
That should answer your question, but it doesn't solve your problem. It looks like some others have posted good solutions.
How does one call Go code in C from threads that weren't created by Go?
What do I assign to a C function pointer such that threads not created by Go can call that pointer and enter into Go code?
Update0
I don't want to use SWIG.
The callbacks will be coming from threads Go hasn't seen before. Neither cgo/life nor anything in pkg/runtime demonstrates this behaviour AFAICT.
You can do this, but the solution is relatively slow (about 22µs per call on my machine).
The answer is for the C code to use C thread primitives to communicate with another goroutine that will actually run the callback.
I have created a Go package that provides this functionality: rog-go.googlecode.com/hg/exp/callback.
There is an example package demonstrating its use here. The example demonstrates a call back to an arbitrary Go closure from a thread created outside of the Go runtime. Another example is here. This demonstrates a typical C callback interface and layers a Go callback on top of it.
To try out the first example:
goinstall rog-go.googlecode.com/hg/exp/example/looper
cd $GOROOT/src/pkg/rog-go.googlecode.com/hg/exp/example/looper
gotest
To try out the second example:
goinstall rog-go.googlecode.com/hg/exp/example/event
cd $GOROOT/src/pkg/rog-go.googlecode.com/hg/exp/example/event
gotest
Both examples assume that pthreads are available. Of course, this is just a stop-gap measure until cgo is fixed, but the technique for calling arbitrary Go closures in a C callback will be applicable even then.
Here is the documentation for the callback package:
PACKAGE
package callback
import "rog-go.googlecode.com/hg/exp/callback"
VARIABLES
var Func = callbackFunc
Func holds a pointer to the C callback function.
When called, it calls the provided function f in a
a Go context with the given argument.
It can be used by first converting it to a function pointer
and then calling from C.
Here is an example that sets up the callback function:
//static void (*callback)(void (*f)(void*), void *arg);
//void setCallback(void *c){
// callback = c;
//}
import "C"
import "rog-go.googlecode.com/hg/exp/callback"
func init() {
C.setCallback(callback.Func)
}
I'll assume you mean from C code compiled with gcc?
IIRC, this either can't be done or can't easily be done using 6g+cgo and friends. Go uses a different calling convention (as well as the segmented stacks and such).
However, you can write C code for [685]c (or even [685]a) and call into go easily using package·function() (you can even call methods IIRC). See the Source of the runtime package for examples.
Update:
Coming back to this question after the update, and giving it some more thought. This can't be done in a standard fashion using 6c or cgo. Especially because the threads are not started by the go runtime, the current implementation would fail. The scheduler would suddenly have a thread under its control that it does not know about; additionally, that thread would be missing some thread-local variables the go runtime uses for managing stacks and some other things. Also, if the go function returns a value (or several) the C code can't access it on the currently supported platforms, as go returns values on the stack (you could access them with assembly though). With these things in mind, I do believe you could still do this using channels. It would require your C code to be a little too intimate with the inner workings of the go runtime, but it would work for a given implementation. While using channels may not be the solution you're looking for, it could possibly fit more nicely with the concepts of Go than callbacks. If your C code reimplemented at least the sending methods in The channel implementation (that code is written for 6c, so it would have to be adapted for gcc most likely, and it calls the go runtime, which we've determined can't be done from a non-go thread), you should be able to lock the channel and push a value to it. The go scheduler can continue to manage it's own threads, but now it can receive data from other threads started in C.
Admittedly, it's a hack; I haven't looked close enough, but it would probably take a few other hacks to get it working (I believe the channels themselves maintain a list of the goroutines that are waiting on them [EDIT: confirmed: runtime·ready(gp);], so you'd need something in your go code to wake up the receiving channel or to warranty the go code won't receive on the channel until you've already pushed a value). However, I can't see any reason this can't work, whereas there are definite reasons that running code generated by 6g on a thread created in C can't.
My original answer still holds though: barring an addition to the language or runtime, this can't yet be done the way you'd like (I'd love to be proven wrong here).
You can find a real-world application of rog's callback package in these bindings for the PortAudio audio I/O library: http://code.google.com/p/portaudio-go/. Might make it easier to understand..
(Thanks for implementing that, rog. It's just what I needed!)
I would like to have a detailed explanation.
How would I use the Lua source code to create a Lua interpreter that will execute given blocks of Lua code? The blocks of Lua code would be sent as a char.
you need a call to lua_load to compile the block of code, and then a call to lua_call to run it. For a really good example of how this is done, take a look at the example provided here:.
The first argument to any Lua api function is always an interpreter state, which is the return value of lua_open()
The example actually uses luaL_loadbuffer which wraps the call to lua_load to make compiling a c string a bit easier. you can read how to use it in the chapter of the reference manual that covers the The Auxiliary Library. This leaves a lua chunk at the top of the lua stack, which can then be invoked with lua_call, but the example uses lua_pcall, which provides a bit of error trapping. since the chunk you just compiled doesn't take any arguments (it's a chunk not a function) and doesn't have any return value you'd be interested in, and you want to see the error exactly as it was produced, all of the arguments besides the first (which is always the lua interpreter state) can be zeros.
http://www.lua.org/manual/2.1/subsection3_7_6.html
http://lua-users.org/lists/lua-l/2006-10/msg00405.html an example
http://www.debian-administration.org/articles/264 c++, same type thing
This will tell you how to call Lua from C.