It's often that you need to extern some C APIs like this:
set_module_callback(module, some_func, func_args);
But the module need the some_func to be thread-safe, i.e., there maybe more than 1 threads to call this function, but I don't know whether it was thread-safe, is there some way to test that while calling set_module_callback?
Thread-safety is very complicated domain. You can get unsafety from many aspects, even well protected code can potentially have a holes in communication with another piece of codes. That is why it is better to assume that some_func is unsafe and provide (for example) critical section around shared resources.
So answer is 'no'
I see no way to automatically have some piece of code test another piece of code.
In order for you to really be sure ti is thread-safe, you will have to go in and look at it/analyze what it does and where side-effects could arise/add a mutex where needed.
If your application is a POSIX one, you could give a try to the Helgrind tool which does some concurrent access detection.
But it is based on how your software calls the APIs. So you won't have an absolute general answer if yes or no your APIs are thread safe. You will only have this answer given a specific application.
Related
I am now porting an single-threaded library to support multi-threads, and I need the whole list of functions that use local static or global variables.
Any information is appreciated.
Check the manual page for each function you use ... the non-thread-safe ones will be identified as such, and the manual page will mention a thread safe version when there is one (e.g., readdir_r). You could extract the list by running a script over the man pages.
Edit: Although my answer has been accepted, I fear that it is inaccurate and possibly dangerous. For example, while strerror_r mentions that it is a thread safe version of strerror, strerror itself says nothing about thread safety ... what it says instead is "the string might be overwritten", which merely implies that it isn't thread-safe. So you need to search for at least "might be overwritten" as well as "thread", but there's no guarantee that even that will be complete.
Its always a good idea to know if a particular function is reentrant or not, but you must also consider the situation when you may call several reentrant functions from a shared piece of code from multiple threads, which could also lead to problems when using shared data.
So, if you have any data shared between threads, the data must be "protected" irregardless of the fact that the functions being called are reentrant.
Consider the following function:
void yourFunc(CommonObject *o)
{
/* This function is NOT thread safe */
reentrant_func1(o->propertyA);
reentrant_func2(o->propertyA);
}
If this function is not mutex protected, you will get undesired behavior in a multithreaded application, irregardless of the fact that func1 and func2 are reentrant.
Hi I'm need to jump from a place to another...
But I would like to know which is better to use, setjmp or ucontext, things like:
Are setjmp and ucontext portable?
My code is thread safe using these library?
Why use one instead another?
Which is fast and secure?
...(Someone please, can answer future question that I forgot to put here?)
Please give a little more information that I'm asking for, like examples or some docs...
I had searching on the web, but I only got exception handling in C like example of setjmp, and I got nothing about ucontex.h, I got that it was used for multitask, what's the difference of it and pthread?
Thanks a lot.
setjmp is portable (ISO C89 and C99) and ucontext (obsolescent in SUSv3 and removed from SUSv4/POSIX 2008) is not. However ucontext was considerably more powerful in specification. In practice, if you used nasty hacks with setjmp/longjmp and signal handlers and alternate signal handling stacks, you could make these just about as powerful as ucontext, but they were not "portable".
Neither should be used for multithreading. For that purpose POSIX threads (pthread functions). I have several reasons for saying this:
If you're writing threaded code, you might as well make it actually run concurrently. We're hitting the speed limits of non-parallel computing and future machines will be more and more parallel, so take advantage of that.
ucontext was removed from the standards and might not be supported in future OS's (or even some present ones?)
Rolling your own threads cannot be made transparent to library code you might want to use. It might break library code that makes reasonable assumptions about concurrency, locking, etc. As long as your multithreading is cooperative rather than async-signal-based there are probably not too many issues like this, but once you've gotten this deep into nonportable hacks things can get very fragile.
...and probably some more reasons I can't think of right now. :-)
On the portability matter, setjmp() is portable to all hosted C implementations; the <ucontext.h> functions are part of the XSI extensions to POSIX - this makes setjmp() significantly more portable.
It is possible to use setjmp() in a thread-safe manner. It doesn't make much sense to use the ucontext functions in a threaded program - you would use multiple threads rather than multiple contexts.
Use setjmp() if you want to quickly return from a deeply-nested function call (this is why you find that most examples show its use for exception handling). Use the ucontext functions for implementing user-space threads or coroutines (or don't use them at all).
The "fast and secure" question makes no sense. The implementations are typically as fast as it is practical to make them, but they perform different functions so cannot be directly compared (the ucontext functions do more work, so will typically be slightly slower).
Note that the ucontext functions are listed as obsolescent in the two most recent editions of POSIX. The pthreads threading functions should generally be used instead.
setjmp/longjmp are only intended to restore a "calling" context, so you can use it only to do a "fast exit" from a chain of subroutines. Different uses may or may not work depending on the system, but in general these functions are not intended to do this kind of things. So "ucontext" is better. Also have a look to "fibers" (native on Windows). Here a link to an article that may be helpful:
How to implement a practical fiber scheduler?
Bye!
How does one call Go code in C from threads that weren't created by Go?
What do I assign to a C function pointer such that threads not created by Go can call that pointer and enter into Go code?
Update0
I don't want to use SWIG.
The callbacks will be coming from threads Go hasn't seen before. Neither cgo/life nor anything in pkg/runtime demonstrates this behaviour AFAICT.
You can do this, but the solution is relatively slow (about 22µs per call on my machine).
The answer is for the C code to use C thread primitives to communicate with another goroutine that will actually run the callback.
I have created a Go package that provides this functionality: rog-go.googlecode.com/hg/exp/callback.
There is an example package demonstrating its use here. The example demonstrates a call back to an arbitrary Go closure from a thread created outside of the Go runtime. Another example is here. This demonstrates a typical C callback interface and layers a Go callback on top of it.
To try out the first example:
goinstall rog-go.googlecode.com/hg/exp/example/looper
cd $GOROOT/src/pkg/rog-go.googlecode.com/hg/exp/example/looper
gotest
To try out the second example:
goinstall rog-go.googlecode.com/hg/exp/example/event
cd $GOROOT/src/pkg/rog-go.googlecode.com/hg/exp/example/event
gotest
Both examples assume that pthreads are available. Of course, this is just a stop-gap measure until cgo is fixed, but the technique for calling arbitrary Go closures in a C callback will be applicable even then.
Here is the documentation for the callback package:
PACKAGE
package callback
import "rog-go.googlecode.com/hg/exp/callback"
VARIABLES
var Func = callbackFunc
Func holds a pointer to the C callback function.
When called, it calls the provided function f in a
a Go context with the given argument.
It can be used by first converting it to a function pointer
and then calling from C.
Here is an example that sets up the callback function:
//static void (*callback)(void (*f)(void*), void *arg);
//void setCallback(void *c){
// callback = c;
//}
import "C"
import "rog-go.googlecode.com/hg/exp/callback"
func init() {
C.setCallback(callback.Func)
}
I'll assume you mean from C code compiled with gcc?
IIRC, this either can't be done or can't easily be done using 6g+cgo and friends. Go uses a different calling convention (as well as the segmented stacks and such).
However, you can write C code for [685]c (or even [685]a) and call into go easily using package·function() (you can even call methods IIRC). See the Source of the runtime package for examples.
Update:
Coming back to this question after the update, and giving it some more thought. This can't be done in a standard fashion using 6c or cgo. Especially because the threads are not started by the go runtime, the current implementation would fail. The scheduler would suddenly have a thread under its control that it does not know about; additionally, that thread would be missing some thread-local variables the go runtime uses for managing stacks and some other things. Also, if the go function returns a value (or several) the C code can't access it on the currently supported platforms, as go returns values on the stack (you could access them with assembly though). With these things in mind, I do believe you could still do this using channels. It would require your C code to be a little too intimate with the inner workings of the go runtime, but it would work for a given implementation. While using channels may not be the solution you're looking for, it could possibly fit more nicely with the concepts of Go than callbacks. If your C code reimplemented at least the sending methods in The channel implementation (that code is written for 6c, so it would have to be adapted for gcc most likely, and it calls the go runtime, which we've determined can't be done from a non-go thread), you should be able to lock the channel and push a value to it. The go scheduler can continue to manage it's own threads, but now it can receive data from other threads started in C.
Admittedly, it's a hack; I haven't looked close enough, but it would probably take a few other hacks to get it working (I believe the channels themselves maintain a list of the goroutines that are waiting on them [EDIT: confirmed: runtime·ready(gp);], so you'd need something in your go code to wake up the receiving channel or to warranty the go code won't receive on the channel until you've already pushed a value). However, I can't see any reason this can't work, whereas there are definite reasons that running code generated by 6g on a thread created in C can't.
My original answer still holds though: barring an addition to the language or runtime, this can't yet be done the way you'd like (I'd love to be proven wrong here).
You can find a real-world application of rog's callback package in these bindings for the PortAudio audio I/O library: http://code.google.com/p/portaudio-go/. Might make it easier to understand..
(Thanks for implementing that, rog. It's just what I needed!)
The user thread functions in <ucontext.h> are deprecated because they use a deprecated C feature (they use a function declaration with empty parentheses for an argument).
Is there a standard replacement for them? I don't feel full-fledged threads are good at implementing cooperative threading.
If you really want to do something like what the ucontext.h functions allow, I would keep using them. Anything else will be less portable. Marking them obsolescent in POSIX seems to have been a horrible mistake of pedantry by someone on the committee. POSIX itself requires function pointers and data pointers to be the same size and for function pointers to be representable cast to void *, and C itself requires a cast between function pointer types and back to be round-trip safe, so there are many ways this issue could have been solved.
There is one real problem, that converting the int argc, ... passed into makecontext into a form to pass to the function cannot be done without major assistance from the compiler unless the calling convention for variadic and non-variadic functions happens to be the same (and even then it's rather questionable whether it can be done robustly). This problem however could have been solved simply by deprecating the use of makecontext in any form other than makecontext(ucp, func, 1, (void *)arg);.
Perhaps a better question though is why you think ucontext.h functions are the best way to handle threading. If you do want to go with them, I might suggest writing a wrapper interface that you can implement either with ucontext.h or with pthreads, then comparing the performance and bloat. This will also have the advantage that, should future systems drop support for ucontext.h, you can simply switch to compiling with the pthread-based implementation and everything will simply work. (By then, the bloat might be less important, the benefit of multi-core/SMP will probably be huge, and hopefully pthread implementations will be less bloated.)
Edit (based on OP's request): To implement "cooperative threading" with pthreads, you need condition variables. Here's a decent pthreads tutorial with information on using them:
https://computing.llnl.gov/tutorials/pthreads/#ConditionVariables
Your cooperative multitasking primitive of "hand off execution to thread X" would go something like:
self->flag = 0;
other_thread->flag = 1;
pthread_mutex_lock(other_thread->mutex);
pthread_cond_signal(other_thread->cond);
pthread_mutex_unlock(other_thread->mutex);
pthread_mutex_lock(self->mutex);
while (!self->flag)
pthread_cond_wait(self->cond, self->mutex);
pthread_mutex_unlock(self->mutex);
Hope I got that all right; at least the general idea is correct. If anyone sees mistakes please comment so I can fix it. Half of the locking (other_thread's mutex) is probably entirely unnecessary with this sort of usage, so you could perhaps make the mutex a local variable in the task_switch function. All you'd really be doing is using pthread_cond_wait and pthread_cond_signal as "go to sleep" and "wake up other thread" primitives.
For what it's worth, there's a Boost.Context library that was recently accepted and needs only to be merged into an official Boost release. Boost.Context addresses the same use cases as the POSIX ucontext family: low-overhead cooperative context switching. The author has taken pains with performance issues.
No, there is no standard replacement for them.
You options are
continue to use <ucontext.h>even though they contain obsolete C.
switch to pthreads
write your own co-thread library
use an existing (and possibly not-so-portable) co-thread library such as http://swtch.com/libtask/ , though many of such libraries are implemented on top of ucontext.h
The Open Group Base Specifications Issue 6
IEEE Std 1003.1, 2004 Edition
Still lists makecontext() and swapcontext() with the same deprecated syntax. I have not seen anything more recent.
Just as in title. Is suspect it is, but I couldn't find it anywhere explicitly stated. And for this property I wouldn't like to rely on speculations.
If you use the multithreaded version of the CRT, all functions are thread safe, because any thread-specific information is stored in TLS. rand_s actually doesn't use state information in the first place, since it just calls an OS API, so question of thread-safety doesn't arise for rand_s. rand(), however depends on a seed value to generate a random number.
Chris said: rand() is not thread-safe because its internal state is static, but rand_s() should be thread-safe, however.
Jeff added however that with the multithreaded version of MSVCRT, rand()'s state is held in thread-local storage, so it's okay still.
Visual Studio comes with the source to the runtime library. While some of it can be rather painful to wade through, rand_s() is pretty simple.
All rand_s() does is call SystemFunction036() in ADVAPI32.DLL to get the random value. Anything in ADVAPI32.DLL should be thread-safe.
For its part, rand_s() gets the pointer to that function in a thread-safe manner.
I don't know if rand_s is thread-safe, but it seems like it probably is, since it seems to make a round-trip to the OS for entropy. (as long as you link to the VC++ multi-thread CRT, all bets are off if you link to the single-thread one)
If it's supported by windows CRT, you can try a call to rand_r which is the posix reentrant version of rand. OR even better boost::random, if you're already using boost.
considering how pervasive multi-threading will be soon, no one should be using rand() anymore in new code - always try to use rand_r/rand_s/boost/various platform-dependent secure rands/etc.
I can't think of any reason why rand_s() or even rand() wouldn't be thread safe.