At first glance (see the evidence below), it looks like while a Tcl_CmdProc has control, the interpreter is waiting for it to return and can't accept any other calls in the meantime.
So, how do I make any calls into Tcl before returning like e.g. a user-defined function would do? I guess I may need to set up a new call stack frame in the interpreter or something (and unwind it later). Tcl_CreateCommand man page says nothing on this matter.
The big picture is like this:
I'm fixing https://bugs.python.org/issue33257 . The TkinterHandlers.py example uses Python event handlers that are implemented as custom Tcl commands under the hood. Currently, their implementation releases the "Tcl lock" (a Python-specific lock that it wraps all Tcl calls with) while executing Python code and reacquires it to Tcl_SetObjResult at the end -- thus allowing other calls to the same interpreter in the meantime.
Now, if another call into the interpreter is actually made during this time frame, Tcl aborts shortly with a message on stderr: TclStackFree: incorrect freePtr. Call out of sequence?
And if I make the custom command hold on to the Tcl lock, it later freezes trying to acquire the lock again because it itself also needs to make a Tcl call sometimes. Now, I can make the lock reentrant, but without knowing how to handle the interpreter right, I'll probably break it, too.
To keep this question on topic, I'm specifically asking about how to handle the interpreter, and make Tcl calls in particular, from a Tcl_CmdProc. The specific situation is solely for exposition to illustrate my needs. If this is actually explained in some doc that I couldn't find, linking to it and reciting some key points would be sufficient.
To call a Tcl command from C code, you've got a choice between two API function families. One is Tcl_EvalObjv, and the other is Tcl_Eval. Each has a number of variants, but the only variant I'll mention is Tcl_EvalObjEx.
Tcl_EvalObjv
This function invokes a single Tcl command, with no processing of substitutions in arguments (unless the command itself does them, of course). It has this signature:
int Tcl_EvalObjv(Tcl_Interp *interp,
int objc,
Tcl_Obj *const objv[],
int flags);
It takes the description of what command to call and what arguments to pass to it as a C array of Tcl value references (in argument objv) where the array is of length objc; Tcl guarantees to not modify the array itself, but might transform the values if it does type conversions. The values must all have a non-zero reference count (and all values start with a zero reference count from their birthing Tcl_NewObj call). The interp is the interpreter context, and flags can usually be zero.
The result is a Tcl exception code; if it is TCL_OK, the result of the call can be retrieved from the interpreter using Tcl_GetObjResult, and if the exception code is TCL_ERROR then there was an error and you should usually pass that on out (perhaps adding to the stack trace with Tcl_AddErrorInfo). Other exception codes are possible; it's usually best to just pass those straight on out without doing any further processing (unless you're making something loop-like, when you should pay attention to TCL_BREAK and TCL_CONTINUE).
Tcl_Eval
This function evaluates a Tcl script, not just a single command, and that includes processing substitutions in arguments. It has this signature:
int Tcl_Eval(Tcl_Interp *interp,
const char *script);
The script is any old C string; Tcl won't modify it, but it will parse, bytecode-compile, and execute it. It's up to you to provide the script in a form that will execute a single command without surprises. The interp argument and the result of the function call are the same as for Tcl_EvalObjv.
If you're interested in using this for running a single command, you're actually better off using Tcl_EvalObjv or…
Tcl_EvalObjEx.
This is like Tcl_Eval except it takes the script as a Tcl value reference (and takes flags too).
int Tcl_EvalObjEx(Tcl_Interp *interp,
Tcl_Obj *objPtr,
int flags);
Again, make sure the objPtr has a non-zero reference count before passing it into this function. (It may adjust the reference count during execution.) Again, interp and the result are as documented for Tcl_EvalObjv, and flags is too.
The advantage of this for calling single commands is that you can call Tcl_NewListObj (or any other list-building function) to make the script value; doing so guarantees that there will be no surprise substitutions. But you could also go directly to invoking the command with Tcl_EvalObjv. But if you want to process anything more complex than a single simple call to a command, this is a good place to start as it has a key advantage that plain Tcl_Eval doesn't: it can make the type of the script passed in via objPtr be one that caches the compiled bytecode, allowing quite a reasonable performance gain in some circumstances.
Note that Tcl_EvalObjv is effectively the API that Tcl calls internally to invoke all user code and perform all I/O. (“Effectively” because things get more complex in Tcl 8.6.)
Within a Tcl_CmdProc, all these functions can be called as usual, no special processing or "handling of the interpreter" is needed. If this doesn't work for you, causing crashes or whatever, the interpreter is not at fault, something else must be wrong with your code.
Related
Is it possible call a separate C program (.exe file) within a C program, like if it was a function?
I would like to be able to pass arguments of any kind (like any other function) to this separate program, and get the return value (so it can be used in the host program).
I imagine that the arguments can be passed by using int argc, char *argv[], but I don't know if it's possible to pass integers, arrays, pointers to structures and so on.
On the other hand, I've read that the return value from the main function is system specific. Since I'm using Windows, is there any limitations to this return value (type, size, etc.)? Can it be anything that could be used as a return value in any normal function?
Thanks!
What you describe, is the basic premise of the Unix operating system. Unix was designed to allow accomplishing very complex tasks by chaining several commands, piping the (text) output of a command as the input of the next one (this was pretty revolutionary back then).
As klutt already suggested, you can accomplish the same with a Windows executable. To his list, I would add learning how to redirect the input/output of a program to a file handle.
The Windows PowerShell extended this concept to allow passing different data-types other than text, to some special executables known as cmdlets, however, to write your own, you need support from the .Net Framework or the .Net Core infrastructure, so you must do so from a managed language such as C# or C++/CLI.
Keep in mind that spawning a whole process is an extremely expensive operation (compared to simply calling a linked function), so there is some significant overhead you need to be aware of.
I am trying to call a C program from my Ruby script, parsing it an argument (file object) and then store some variables the C program would return.
The idea is that my Ruby script allows me to easily cycle through the files & folders of a parent folder but it is way too slow to efficiently process all the files in that folder. Hence the use of a C program that I want to call to process each file.
My problem is that I can't find a method to call that C program from Ruby (and how to parse it the file argument, I'm not even sure it is possible as I don't know if Ruby files objects and C streams are "compatible")
Thank you in advance for your help !
You say you are trying to call a program so I assume you are not trying to statically or dynamically load a library and call a function. (If you are trying to load a library to call a function then look to the DL::Importer module.)
As for calling an external program from Ruby and receiving its result (from stdout, in this case), regardless of whether it was written in C or not, an easy way to do it is:
value = `program arg1 arg2 ...`
e.g. if the program you want to call compresses a given file and outputs the compressed size.
size = `mycompressionprogram filename.txt`
puts "compressed result is: #{size}"
Note those are back ticks " ` ".
So this is one easy way to code your computationally heavy stuff in C and wrap it up in a Ruby script.
One simple traditional way for a Ruby process to interact with unrelated C code is popen, which will allow your Ruby process to invoke the (compiled) code as a separate process, passing your choice of arguments into the traditional space the operating system allocates for that (accessible in argv in your process's int main(int argc, char** argv)), and then interacting with its standard input and standard output over a pipe. However, this technique launches another process and requires that you serialize/deserialize any ongoing interprocess communication so that it can run over the pipe, which may be an impediment.
So you can also write the C code as a Ruby extension, which will allow you to return values more readily, and moreover avoids the overhead associated with having a separate process involved. However, note that if you perform extensive work with Ruby objects in your C code you may still incur the performance penalties you'd hoped to avoid. The canonical document on how to write Ruby extensions is README.EXT.
I was looking at the manual for longjmp and in the Errors part it says this:
ERRORS
If the contents of the env are corrupted, or correspond to an environment that has already returned, the longjmp() routine calls the routine longjmperror(3). If longjmperror()
returns, the program is aborted (see abort(3)). The default version of longjmperror() prints the message ``longjmp botch'' to standard error and returns. User programs wishing to exit more gracefully should write their own versions of longjmperror().
How would i write my own version of longjmperror? From what i know in C you can't override functions and i really need the long jump to exit in a specific way when it doesn't find the point to jump to.
On Mac OS X (10.9.2, Mavericks) at any rate, the prototype for longjmperror() is:
void longjmperror(void);
You write a function with that signature. It must not return (or, rather, if it does, the program will be abort()ed). What you do in that function is your business, but bear in mind that things have gone moderately catastrophically wrong for the function to be called at all). It might log an error to your log file, or just write a more meaningful message before exiting (instead of aborting and perhaps core dumping).
You link the object file containing the function ahead of the system library. You are normally not expected to replace system functions, but this is one you are intended to override.
Say I replace a function by creating a shared object and using LD_PRELOAD to load it first. Is it possible to have parameters to that function different from the one in original library?
For example, if I replace pthread_mutex_lock, such that instead of parameter pthread_mutex_t it takes pthread_my_mutex_t. Is it possible?
Secondly, besides function, is it possible to change structure declarations using LD_PRELOAD? For example, one may add one more field to a structure.
Although you can arrange to provide your modified pthread_mutex_lock() function, the code will have been compiled to call the standard function. This will lead to problems when the replacement is called with the parameters passed to the standard function. This is a polite way of saying:
Expect it to crash and burn
Any pre-loaded function must implement the same interface — same name, same arguments in, same values out — as the function it replaces. The internals can be implemented as differently as you need, but the interface must be the same.
Similarly with structures. The existing code was compiled to expect one size for the structure, with one specific layout. You might get away with adding an extra field at the end, but the non-substituted code will probably not work correctly. It will allocate space for the original size of structure, not the enhanced structure, etc. It will never access the extra element itself. It probably isn't quite impossible, but you must have designed the program to handle dynamically changing structure sizes, which places severe enough constraints on when you can do it that the answer "you can't" is probably apposite (and is certainly much simpler).
IMNSHO, the LD_PRELOAD mechanism is for dire emergencies (and is a temporary band-aid for a given problem). It is not a mechanism you should plan to use on anything remotely resembling a regular basis.
LD_PRELOAD does one thing, and one thing only. It arranges for a particular DSO file to be at the front of the list that ld.so uses to look up symbols. It has nothing to do with how the code uses a function or data item once found.
Anything you can do with LD_PRELOAD, you can simulate by just linking the replacement library with -l at the front of the list. If, on the other hand, you can't accomplish a task with that -l, you can't do it with LD_PRELOAD.
The effects of what you're describing are conceptually the same as the effects of providing a mismatching external function at normal link time: undefined behavior.
If you want to do this, rather than playing with fire, why don't you make your replacement function also take pthread_mutex_t * as its argument type, and then just convert the pointer to pthread_my_mutex_t * in the function body? Normally this conversion will take place only at the source level anyway; no code should be generated for it.
I am trying to fully understand the process pro writing code in some language to execution by OS. In my case, the language would be C and the OS would be Windows. So far, I read many different articles, but I am not sure, whether I understand the process right, and I would like to ask you if you know some good articles on some subjects I couldn´t find.
So, what I think I know about C (and basically other languages):
C compiler itself handles only data types, basic math operations, pointers operations, and work with functions. By work with functions I mean how to pass argument to it, and how to get output from function. During compilation, function call is replaced by passing arguments to stack, and than if function is not inline, its call is replaced by some symbol for linker. Linker than find the function definition, and replace the symbol to jump adress to that function (and of course than jump back to program).
If the above is generally true and I get it right, where to final .exe file actually linker saves the functions? After the main() function? And what creates the .exe header? Compiler or Linker?
Now, additional capabilities of C, today known as C standart library is set of functions and the declarations of them, that other programmers wrote to extend and simplify use of C language. But these functions like printf() were (or could be?) written in different language, or assembler. And there comes my next question, can be, for example printf() function be written in pure C without use of assembler?
I know this is quite big question, but I just mostly want to know, wheather I am right or not. And trust me, I read a lots of articles on the web, and I would not ask you, If I could find these infromation together on one place, in one article. Insted I must piece by piece gather informations, so I am not sure if I am right. Thanks.
I think that you're exposed to some information that is less relevant as a beginning C programmer and that might be confusing you - part of the goal of using a higher level language like this is to not have to initially think about how this process works. Over time, however, it is important to understand the process. I think you generally have the right understanding of it.
The C compiler merely takes C code and generates object files that contain machine language. Most of the object file is taken by the content of the functions. A simple function call in C, for example, would be represented in the compiled form as low level operators to push things into the stack, change the instruction pointer, etc.
The C library and any other libraries you would use are already available in this compiled form.
The linker is the thing that combines all the relevant object files, resolves all the dependencies (e.g., one object file calling a function in the standard library), and then creates the executable.
As for the language libraries are written in: Think of every function as a black box. As long as the black box has a standard interface (the C calling convention; that is, it takes arguments in a certain way, returns values in a certain way, etc.), how it is written internally doesn't matter. Most typically, the functions would be written in C or directly in assembly. By the time they make it into an object file (or as a compiled library), it doesn't really matter how they were initially created, what matters is that they are now in the compiled machine form.
The format of an executable depends on the operating system, but much of the body of the executable in windows is very similar to that of the object files. Imagine as if someone merged together all the object files and then added some glue. The glue does loading related stuff and then invokes the main(). When I was a kid, for example, people got a kick out of "changing the glue" to add another function before the main() that would display a splash screen with their name.
One thing to note, though is that regardless of the language you use, eventually you have to make use of operating system services. For example, to display stuff on the screen, to manage processes, etc. Most operating systems have an API that is also callable in a similar way, but its contents are not included in your EXE. For example, when you run your browser, it is an executable, but at some point there is a call to the Windows API to create a window or to load a font. If this was part of your EXE, your EXE would be huge. So even in your executable, there are "missing references". Usually, these are addressed at load time or run time, depending on the operating system.
I am a new user and this system does not allow me to post more than one link. To get around that restriction, I have posted some idea at my blog http://zhinkaas.blogspot.com/2010/04/how-does-c-program-work.html. It took me some time to get all links, but in totality, those should get you started.
The compiler is responsible for translating all your functions written in C into assembly, which it saves in the object file (DLL or EXE, for example). So, if you write a .c file that has a main function and a few other function, the compiler will translate all of those into assembly and save them together in the EXE file. Then, when you run the file, the loader (which is part of the OS) knows to start running the main function first. Otherwise, the main function is just like any other function for the compiler.
The linker is responsible for resolving any references between functions and variables in one object file with the references in other files. For example, if you call printf(), since you do not define the function printf() yourself, the linker is responsible for making sure that the call to printf() goes to the right system library where printf() is defined. This is done at compile-time.
printf() is indeed be written in pure C. What it does is call a system call in the OS which knows how to actually send characters to the standard output (like a window terminal). When you call printf() in your program, at compile time, the linker is responsible for linking your call to the printf() function in the standard C libraries. When the function is passed at run-time, printf() formats the arguments properly and then calls the appropriate OS system call to actually display the characters.