Taken from: https://www.gnu.org/software/libc/manual/html_node/Nonreentrancy.html
For example, suppose that the signal handler uses gethostbyname. This function returns its value in a static object, reusing the same object each time. If the signal happens to arrive during a call to gethostbyname, or even after one (while the program is still using the value), it will clobber the value that the program asked for.
I fail to see how the above scenario is non-reentrant. It seems to me that gethostbyname is a (read-only) getter function that merely reads from memory (as opposed to modifying memory). Why is gethostbyname non-reentrant?
As the word says, reentrancy is the capability of a function to be able to be called again while it is being called in anothe thread. The scenario you propose is the exact place in which reentrancy is exercised. asume the function has some static or global variable (as the gethostbyname(3) function does) As the return buffer for the structure is being written by one, the other call can be overwriting it to completely destroy the first writing. When the in execution instance of the function (the interrupted one, not the interrumpting one) gets control again, all it's data has been overwritten by the interrupting one, and destroyed it.
A common solution to solve this problem with interruptions is to disable interrupts while the function is executing. This way it doesn't get interrupted by a new call to itself.
If two threads call the same piece of code, and all the parameters and local variables are stored in the stack, each thread has a copy of its own data, so there's no problem in calling both at the same time, as the data they touch is in different stacks. This will not happen with static variables, being those local scope, compilation unit scope or global scope (think that the problem comes when calling the same piece of code, so everywhere one call has access to, the other has also)
Static data, like buffers (look at stdio buffered packages) etc. means in general, the routines will not be reentrant.
Related
I want to determine whether certain function in C can be called from multiple threads at the same time, in order to understand if I need to protect it with mutexes. The file where the function is implemented and defined does not have any mutex mechanism, so there is a chance that only one thread ever accesses the function but there is a chance that multiple threads do.
I thought to add a thread local storage variable which I increment upon starting the function and decrement upon exiting the function. If, after decrementing, the value of the variable is greater than 0, then multiple threads access the function.
This is my code:
#include <stdio.h>
static __thread int threadCounter = 0;
void f(void)
{
threadCounter++;
// do something
threadCounter--;
printf("threadCounter: %d\n", threadCounter);
}
a'm wondering if this solution is sufficient to determine whether multiple threads access a function and whether there are better ways to accomplish this.
From the GCC documentation :
"Thread-local storage (TLS) is a mechanism by which variables are allocated such that there is one instance of the variable per extant thread."
Thus, your solution will always indicate that only one thread access your function at the time even if it's not the case. You should use a variable that is shared between thread. But using a volatile one is still not a good solution because if there are multiple thread accessing it at the time, the value might not be the good one.
In conclusion, I think the better way of doing this would be to setup a mutex and using the pthread_mutex_trylock function to detect if there are multiple threads trying to call your function.
A thread local variable is by definition only visible for the current thread, your solution won't work. But your approach is good. Instead of using a thread local variable you should use a variable protected by a mutex.
This test either is using a local variable, or it isn't thread-safe in itself. In either case, it won't be useful as proof. If you want to use a counter, you have to protect it with something like a mutex or critical section. There's pretty much no way around this.
But there's a another way to do this better though, giving you exact information of who called the function, while at the same time not having to modify the actual function. You can create a "mock" function:
#define f() ( print(__func__), f() )
This prints the name of the thread callback function, then calls the actual function f(). This works because the pre-processor token f() is evaluated before any function call.
I wrote the function as a custom one print, since you'll still have the problem with multiple thread trying to access stdout at once if you use printf etc. So the custom print function must contain the means of thread-safety.
I know it's a very specific question and it's not very interesting for a high level programmer, but I would like to know when exactly are allocated the local variables of a thread function, in other words after
pthread_create(&thread, &function, ...)
is executed, can I say that they exists in memory or not (considering that the scheduler could have not executed the thread yet)?
I tried to search in the posix library code but it's not easy to understand, I arrive at the clone function, written in assembly, but than I cannot find che code of the system call service routine sys_clone to understand what exactly it does. I see in the clone code the invocation of the thread function, but I think this should happen only in the created thread (which could have never been executed by the scheduler when pthread_create is terminated) and not in the creator.
in other words after
pthread_create(&thread, &function, ...)
is executed, can I say that they exists in memory or not (considering
that the scheduler could have not executed the thread yet)?
POSIX does not give you any reason for confidence that the local variables of the initial call to function function() in the created thread will have been allocated by the time pthread_create() returns. They might or might not have been, and indeed, the answer might not even be well defined inasmuch as different threads do not necessarily have a consistent view of machine state.
There is no special significance to the local variables of a thread's start function relative to the local variables of any other function called in that thread. Moreover, although pthread_create() will not return successfully until the new thread has been created, that's a separate question from whether the start function has even been entered, much less whether its local variables have been allocated.
I read about this in Advanced Programming in the UNIX Environment (3rd Edition),11.5,Thread Termination:
If we run the same program on FreeBSD or Mac OS X, we see that the
program incurs a segmentation violation and drops core. This happens
because on these systems, pthread_cleanup_push is implemented as a
macro that stores some context on the stack. When thread 1 returns in
between the call to pthread_cleanup_push and the call to
pthread_cleanup_pop, the stack is overwritten and these platforms try
to use this (now corrupted) context when they invoke the cleanup
handlers. In the Single UNIX Specification, returning while in between
a matched pair of calls to pthread_cleanup_push and
pthread_cleanup_pop results in undefined behavior. The only portable
way to return in between these two functions is to call pthread_exit.
So,what I want to know is what is so called context on the stack, and what is overwritten and corrupted context like, and why return can't work as expected?
When a C function is called, it stores some information (like local variables) on the "stack" section of memory. Likewise when this function calls another, this new function stores it's information just after the calling function's.
When the function call is over, it's memory is released/free'd, similar to taking a plate off a stack. This space can now be reused.
memory http://www.firmcodes.com/wp-content/uploads/2014/08/memory.png
If pthread_cleanup_push is implemented as a macro that stores information (context) on the stack, this memory will be free'd, and likely overwritten by the next function call.
If this happens before pthread_cleanup_pop is called, there is a good chance that the "context" will have been overwritten by successive function calls thus corrupting the memory pthread_cleanup_pop was expecting to read from.
Lua docs say:
The Lua library defines no global variables at all. It keeps all its state in the dynamic structure lua_State and a pointer to this
structure is passed as an argument to all functions inside Lua. This
implementation makes Lua reentrant and ready to be used in
multithreaded code.
But is it true? Has anyone tried to verify this? What if a signal is raised and caught by a custom handler during the execution of lua? That is, is lua itself (never mind the system calls it makes) truly reentrant?
EDIT:
A well-known problem in lua is the lack of a timer implementation. These can be implemented using POSIX timers, that raise a signal. But raising such a signal may interrupt the execution of lua itself. The canonical solution to solve this problem is the masking/unmasking of a signal, but if lua were truly re-entrant this would not be needed.
AFAICT, re-entrancy is a single threaded concept, and somewhat independent from multi-threading. Multi-thread safety relates to data coherence when concurrent read/write shared data, whereas re-entrancy relates to state coherence of function pre/post signal, within one thread.
A function is either multi-thread safe, or it is not. There is no in-between. However, it is not so simple with regards to re-entrancy: there are conditions under which a function is re-entrant, and conditions under which it is not; for some functions, there are no conditions under which it is re-entrant. I'm not a computer scientist but my guess is that there are very few functions, if any, that would be re-entrant under all conditions. Like void f() {} would be one, but it's not very useful :)
The following are probably true:
A required condition for a function to be re-entrant is that it must not use any static or global data or data that can be set from outside itself (such as registers or DMA).
Another required condition for re-entrancy is that the function only call re-entrant functions. In this case the function is re-entrant with the sum of all conditions required for the called functions to be considered re-entrant. So if A calls B and C, and B is re-entrant if condition b is true, and C is re-entrant if condition c is true, then a necessary condition for A to be re-entrant is conditions b and c must be true.
A function that accepts at least one argument is only re-entrant if 1 and 2 are true and the signal handler does not call, directly or indirectly, the function with the same argument.
An API is re-entrant in the same manner as the totality of its functions. This means that there may be only a subset of the API that can be said to be re-entrant, under certain specific conditions (1-3), and other functions are not re-entrant. This does not mean the API is not re-entrant; just that a subset of it is re-entrant, under certain conditions.
If the above is correct, then you have to be more specific when asking (or stating) whether Lua is re-entrant, to ask which subset of Lua functions are known to be re-entrant, under what conditions. Apparently all Lua functions satisfy 1, but which ones satisfy 2? Almost all Lua API functions accept at least one argument, so under the condition that your signal handler does not call directly or indirectly the same Lua function with the same Lua state variable, you could say that Lua is re-entrant for those functions that don't call non-reentrant functions.
Update 1: why condition 3:
Consider
void f(const Foo& foo) {
if (foo.bar)
do stuff
signal happens here, calling isr()
modify fo
}
Foo* isrFoo;
void g() {
Foo foo;
isrFoo = & foo;
f(foo)
}
void isr() {
f(*isrFoo)
}
Although f(const Foo&) does not use globals or static (although strictly speaking it doesn't know if a is a ref to such var), the object received can be shared by multiple objects and hence, in isr(), can be modified, such that when f() resumes, foo is no longer same as when interrupted. One could say that f() is re-entrant (in single-thread) but here isr() is interfering, making f() non-re-entrant in that particular case. Assuming that an object copy op could be made atomic, f() could be made re-entrant even for this particular design of isr() if foo was copied into a local variable of f before being used, or if isr() made a local copy, or foo was pass-by-value.
Update 2: russian roulette
Russian roulette is a game of chance. So no, re-entrancy is not game of chance: given the above, the manual says basically that if your signal handler does not call (directly or indirectly) Lua C API functions, then you can consider the Lua C API functions re-entrant because of the way the API was designed and implemented.
For example if you have a timer that ticks (signals) every 100 ms, but the handler just sets a flag to true for "do something ASAP", and your code loops endlessly, calling a Lua function (via lua_pcall) at every iteration, to check the flag, you shouldn't have any problems: if the Lua function is interrupted by the timer before the flag is checked, the flag will get set, then upon return from signal the flag will be seen as true and your function will take action as designed.
However, if you are not careful, your Lua function (not the C API that calls it) may not be re-entrant, thus causing lua_pcall to not be re-entrant when calling your Lua function. For example if your Lua function (called via lua_pcall) checks the flag in two places:
function checkTimerFlagSet()
if flag then ... end
... do stuff ...
if flag then ... end
and the timer signal occurs between the two checks, then the flag could be seen as false before signal and true after, during the same function call, which could lead to inconsistent behavior of your Lua function. But this is merely rule #1 not being followed (no choice since your signal handler can only set global variable) by your function, not by the Lua C API: this "bad" (i.e. non-reentrant) design of your Lua function is what caused one of the Lua C API functions (lua_pcall) to no-longer be re-entrant. It is re-entrant otherwise.
It is true that lua keeps all its variables in lua_State. If a signal occurs, that signal will be handled in C. You cannot call lua safely from your signal handler, just as you can't call even some thread safe functions from a signal handler.
What the documentation is saying is that if you have different threads with different lua_State variables, they can each safely run lua without the need to synchronise between them.
How does one modify a threads data from outside a thread?
If a thread is running a function that loops for the runtime of the application, how can its data be set, changed?
How does one call functions which modify a specific threads functions?
Where do these functions belong?
The advantage and disadvantage of threads is that they share the memory space with every other thread in the process. You can use any form of data transfer you would use in single threaded applications to pass data betweens segments of you application. However, in a multi-threaded application you must use some type of synchronization to assure data integrity and prevent deadlocks.
If the "thread's data" you want to modify from outside is in the form of local variables in a function running in the thread, or thread-specific data created with the __thread extension, then the only way you can modify them from outside (modulo code with UB that's technically just trashing memory) is by having the thread take the addresses of its variables and store that somewhere where other threads can see it (either in a global variable, or at a location passed in via the thread start function's void * argument.
Also note that, as rerun pointed out, you have to use some method of synchronization if multiple threads are accessing the same data. The only standard/portable synchronization methods are the pthread ones: pthread_mutex_lock etc., but you can also use assembly or compiler intrinsics (like __sync_* in gcc).