Is thread local storage used anywhere else other than making global and static variables local to a thread?Is it useful in any new code that we write?
TLS can certainly be useful in new code. If you ever want a global variable which needs to be specific to each thread, (like errno in C/C++), thread-local-storage is the way to go.
Thread specific singleton objects? A multi-threaded web server where each thread is handling one request, there is quite a good amount of possibility of some TLS data (like request URL or some database connections, essentially some resources intended to be used at any point during request handling if required) so that they can be easily accessed anywhere in the code when required.
These days errno is typically put in thread-local storage.
There are some situations (eg: shared libraries like DLLs that require startup code) where using thread-local storage can be a problem.
I've only needed it for thread-specific error handling, and optimization (in C):
__thread int cpfs_errno;
static __thread struct Cpfs *g_cpfs;
In this example, this saves me passing a context pointer of struct Cpfs * through dozens of functions in which it never changes.
Related
If I have a threaded function, and from inside it I call another function (that is located in a separate .c file), does each instance of the threaded function get it's own version of the called function? Example:
file 1
void* threaded_func(void* data) {
function2(...);
}
file 2
int a;
int function2(...) {
int b;
}
In my example, would int a be the same int a for all threads? What about for variables I define inside function2 (i.e int b)?
Some context:
I'm creating a simple HTTP server and my function2 is a serve_client function which should only return when the connection is closed. I'm having a problem that I think is due to some variables in file2 (int a) being overwritten for each new thread. I think my problem is me expecting a C source file to behave like a class would in Java.
does each instance of the threaded function get it's own version of the called function?
Loosely speaking, that's correct, provided the function doesn't use any global variables (e.g., int a; in your example) or any variable with static storage duration. If a is "shared" by multiple threads, that's likely to the source of your problem.
If a needs to be shared among threads, then you need to synchronize its access. If you a needs to be unique to each of threads, you need thread local storage. E.g., using C11's __Thread_local or gcc's __thread.
A C function is just machine code, and could be run by several threads (or processes). What matters is the data used by that function. C don't have closures (you could emulate them with callbacks, that is some function using additional client data). As answered by usr read also about thread local storage duration with thread_local. Each thread has its own call stack.
You should first read some POSIX thread tutorial.
You need to care about synchronization (and semaphores), notably using mutexes (and lock so serialize access to global shared variables). Read also about ACID transactions. Sometimes you can use atomic types and atomic operations.
BTW, you might want to use some HTTP server library like libonion.
You should look into the source code of existing free software projects (e.g. on github).
On Linux, read also about pthreads(7), nptl(7) and futex(7) and sem_overview(7) and clone(2).
Is CryptGenRandom() thread-safe with a single global program-wide HCRYPTPROV instance?
MSDN appears to lack any info on this: https://msdn.microsoft.com/en-us/library/windows/desktop/aa379942(v=vs.85).aspx
Creating a separate HCRYPTPROV per thread and destroying it again would significantly complicate matters (and also risk more security-relevant bugs on my side), so this would be really useful to know. Sharing one global HCRYPTPROV would be a lot easier for sure.
So does anyone here know about the thread-safety of CryptGenRandom(), particularly with a single HCRYPTPROV instance?
Creating a separate HCRYPTPROV per thread doesn't make much sense. This is pointer to memory block from heap in all current implementations, primarily saved pointers to CSP entry points which used to call actual provider implementation (CPGenRandom in our case). The references themselves do not contain state of the CSP, unlike for example HCRYPTKEY which containing actual key state. So even if you create a separate HCRYPTPROV for every thread - this changes nothing.
There may be some global variables / data used by CSP internally during this call; this is however unknown as these would be implementation details. Of course we can serialize calls to CryptGenRandom in the code. However we cannot control that some other dll in our process also call CryptGenRandom concurrently. So serializing all calls to CryptGenRandom also impossible.
As result I think the CPGenRandom must be design to be thread-safe. and it my tests with a well known Microsoft CSP this is true. Internal synchronization is used in function, when need access global data and if multiple threads call CPGenRandom concurrently; every thread receives unique random data.
So my conclusion - CryptGenRandom is thread-safe, at least for all Microsoft CSP
I can think of two ways a thread-safe library can be used:
One is having a global instance of the library protected by a mutex, which is initialised by the main thread and used by worker threads, like so:
mutex g_lib_mutex;
lib_t g_lib;
thread:
lock(&g_lib_mutex);
/* use lib */
unlock(&g_lib_mutex);
main:
lib_init(&g_lib);
start_threads(thread);
lock(&g_lib_mutex);
/* use lib */
unlock(&g_lib_mutex);
join_threads();
lib_close(&g_lib);
The other, is for every thread to have a local instance of the library, something like this:
thread:
lib_t g_lib;
lib_init(&g_lib);
/* use lib */
lib_close(&g_lib);
main:
start_threads(thread);
lib_t g_lib;
lib_init(&g_lib);
/* use lib */
lib_close(&g_lib);
Which of these ways is more correct / preferable?
Do I need to protect library calls with a global mutex in both cases?
I was trying to use libmysql and POSIX message queues in a multi-threaded application when this question crossed my mind.
Generally, only initialize a library once. Remember, all threads happen in the same process' memory space, so whatever you do to any global variables in thread X is true for all threads. Library initialization should happen only once per process.
Now, whether library calls are thread safe or must be protected by mutexes is a question of your library. Modern libraries should have definite documentation on what functions you're allowed to call from multiple threads. If that info is missing you can either
assume the worst and encapsulate everything that changes something that the library deals with, or calls into the library, with a single global mutex, or
read the source code of the library to figure out what might go wrong where, introduce security measures (mutexes/conditions) accordingly, and make sure that no one uses a different version of the library (where things might be different), or
improve the documentation, send that patch to the upstream developers asking them to verify that what you document in thread-(un)safety is intentional and matches reality, (documentation patches are, for any project that I know of, always welcome) or
modify the library itself to be thread safe (making yourself a hero).
We can pass argument to a thread when using pthread_create /(not key_), if it is over using value per thread. If about using static storage, we use mutexes, so, what is it saying we use pthread_key_create? With example & emphasize on why it is born (application & need of it)?
I heard it is useful for migrating single threded applications, but they still use mutexes. So, what is the point?
Thanks!
pthread_key_create is used to create a unique identifier for a piece of thread-local storage.
Each thread can then store a different value for each key. This can be useful when migrating code that initially uses globals, but where you want each thread of execution to have its own independent version of the global value. You can use thread-local storage in place of the global.
You don't need to use mutexes when accessing thread-local data belonging to the thread you're in.
all the threads share memory location. For example a global variable changes in one thread will reflect in another thread. Since each thread has its own stack, the local
variables that are created inside the thread is unique. In this case, why do we need
to go for thread specific data mechanism?. Can't it be achieved by auto storage varibles
inside the thread function ?
Kindly clarify!!!.
BR
Rj
Normal globals are shared between threads. Local variables are specific to a particular invocation of a function. If you want something that (for example) is visible to a number of functions running in the same thread, but unique to that thread, then thread specific data is what you're looking for.
It's not required but it's rather handy. Some functions like rand and strtok use static storage duration information which is likely to be problematic when shared among threads.
Say you have a random number function where you want to maintain a different sequence (hence seed) for each thread. You have two approaches.
You can use something like the kludgy:
int seed;
srand (&seed, time (NULL));
int r = rand_r (void *seed);
where the seed has to be created by the caller and passed in each time.
Or you can use the rather nicer, ISO-compliant:
srand (time (NULL));
int r = rand();
that uses thread-local storage to maintain a thread-specific seed. Similarly with the information used by strtok regarding the locations within the string it's processing.
That way, you don't have to muck about with changing your code between threaded and non-threaded versions.
Now you could create that information in the thread function but how is the rand function going to know about it's address without it being passed down. And what if rand is called 87 stack levels down? That's an awful lot of levels to be transferring a pointer through.
And, even if you do something like:
void pthread_fn (void *unused) {
int seed;
rand_set_seed_location (&seed);
:
}
and rand subsequently uses that value regardless of how deep it is in the stack, that's still a code change from the standard. It may work but so may writing an operating system in COBOL. That doesn't make it a good idea :-)
Yes, the stack is one way of allocating thread-local storage (including handles to heap allocations local to the particular thread).
The best example for thread specific data is the "errno". When a call to some function in c library failed, the errno is set, and you can check it out to find the reason of the failure. If there's no thread specific data, it's impossible to port these functions to multi-thread environment because the errno could be set by other threads before you check it.
As a general rule, most uses of TSD should be avoided in new APIs. If a function needs some information, it should be passed to it.
However, sometimes you need TSD to 'paper over' an API defect. A good example is 'gmtime'. The 'gmtime' function returns a pointer to a structure that is valid until the next call to 'gmtime'. But that would make 'gmtime' awfully hard to use in a multi-threaded program. What if some library called 'gmtime' when you didn't expect it, trashing your structure? One simple workaround is make the structure returned thread-specific. (The long-term solution, of course, is to create a more suitable API such as 'gmtime_r'.)
One case where it's perfectly reasonable to use TSD in new designs is for information that won't be accessed frequently that would clutter the API. For example, if a critical error is discovered, it might be nice to log certain context information from higher-level code (Which client were you serving? What command did they send?). Your choices are basically to pass this context information from function to function to function (which isn't even always possible if some of the functions are outside your control) or to store it in TSD.