I got one implementation of read/write locks which is below. Notice that in the beginning of the functions, there is a pthread_mutex_lock call. If it uses pthread_mutex_lock once anyway, then what is the benefit of using read/write locks. How is it better than simply using pthread_mutex_lock?
int pthread_rwlock_rlock_np(pthread_rwlock_t *rwlock)
{
pthread_mutex_lock(&(rwlock->mutex));
rwlock->r_waiting++;
while (rwlock->r_wait > 0)
{
pthread_cond_wait(&(rwlock->r_ok), &(rwlock->mutex));
}
rwlock->reading++;
rwlock->r_waiting--;
pthread_mutex_unlock(&(rwlock->mutex));
return 0;
}
int pthread_rwlock_wlock_np(pthread_rwlock_t *rwlock)
{
pthread_mutex_lock(&(rwlock->mutex));
if(pthread_mutex_trylock(&(rwlock->w_lock)) == 0)
{
rwlock->r_wait = 1;
rwlock->w_waiting++;
while (rwlock->reading > 0)
{
pthread_cond_wait(&(rwlock->w_ok), &(rwlock->mutex));
}
rwlock->w_waiting--;
pthread_mutex_unlock(&(rwlock->mutex));
return 0;
}
else
{
rwlock->wu_waiting++;
while (pthread_mutex_trylock(&(rwlock->w_lock)) != 0)
{
pthread_cond_wait(&(rwlock->w_unlock), &(rwlock->mutex));
}
rwlock->wu_waiting--;
rwlock->r_wait = 1;
rwlock->w_waiting++;
while (rwlock->reading > 0)
{
pthread_cond_wait(&(rwlock->w_ok), &(rwlock->mutex));
}
rwlock->w_waiting--;
pthread_mutex_unlock(&(rwlock->mutex));
return 0;
}
}
The rwlock->mutex mutex is used to protect the state of the rwlock structure itself, not the state of whatever the reader/writer lock may be protecting in your target program. This mutex is held only during the time the lock is acquired or released. It is entered into just briefly, to avoid corrupting the state that is needed for "bookkeeping" of the reader/writer lock itself. In contrast, the reader/writer lock may be held for an extended period of time by the callers performing the actual reads and writes on the structure the lock protects.
In both functions, rwlock->mutex is released before returning. That means that just because you hold an rwlock as reader or writer, doesn't mean you hold the mutex.
Half the point of a rwlock is that multiple readers can operate simultaneously, so that's the immediate advantage over just using a mutex. Those readers only hold the mutex briefly, in order to acquire the reader lock. They don't hold the mutex while they do their actual work.
It will allows multiple read OR a single write at a time, which is better than one read OR one write operation at a time.
How is it better than simply using pthread_mutex_lock?
Even with this implementation, the readers will run simultaneously in the critical section.
There could be a better (less portable) implementation that would use atomics in the "fast path" (locking would still be needed when waiting).
Related
#include <stdio.h>
#include <pthread.h>
long mails = 0;
int lock = 0;
void *routine()
{
printf("Thread Start\n");
for (long i = 0; i < 100000; i++)
{
while (lock)
{
}
lock = 1;
mails++;
lock = 0;
}
printf("Thread End\n");
}
int main(int argc, int *argv[])
{
pthread_t p1, p2;
if (pthread_create(&p1, NULL, &routine, NULL) != 0)
{
return 1;
}
if (pthread_create(&p2, NULL, &routine, NULL) != 0)
{
return 2;
}
if (pthread_join(p1, NULL) != 0)
{
return 3;
}
if (pthread_join(p2, NULL) != 0)
{
return 4;
}
printf("Number of mails: %ld \n", mails);
return 0;
}
In the above code each thread runs a for loop to increase the value
of mails by 100000.
To avoid race condition is used lock variable
along with while loop.
Using while loop in routine function does not
help to avoid race condition and give correct output for mails
variable.
In C, the compiler can safely assume a (global) variable is not modified by other threads unless in few cases (eg. volatile variable, atomic accesses). This means the compiler can assume lock is not modified and while (lock) {} can be replaced with an infinite loop. In fact, this kind of loop cause an undefined behaviour since it does not have any visible effect. This means the compiler can remove it (or generate a wrong code). The compiler can also remove the lock = 1 statement since it is followed by lock = 0. The resulting code is bogus. Note that even if the compiler would generate a correct code, some processor (eg. AFAIK ARM and PowerPC) can reorder instructions resulting in a bogus behaviour.
To make sure accesses between multiple threads are correct, you need at least atomic accesses on lock. The atomic access should be combined with proper memory barriers for relaxed atomic accesses. The thing is while (lock) {} will result in a spin lock. Spin locks are known to be a pretty bad solution in many cases unless you really know what you are doing and all the consequence (in doubt, don't use them).
Generally, it is better to uses mutexes, semaphores and wait conditions in this case. Mutexes are generally implemented using an atomic boolean flag internally (with right memory barriers so you do not need to care about that). When the flag is mark as locked, an OS sleeping function is called. The sleeping function wake up when the lock has been released by another thread. This is possible since the thread releasing a lock can send a wake up signal. For more information about this, please read this. In old C, you can use pthread for that. Since C11, you can do that directly using this standard API. For pthread, it is here (do not forget the initialization).
If you really want a spinlock, you need something like:
#include <stdatomic.h>
atomic_flag lock = ATOMIC_FLAG_INIT;
void *routine()
{
printf("Thread Start\n");
for (long i = 0; i < 100000; i++)
{
while (atomic_flag_test_and_set(&lock)) {}
mails++;
atomic_flag_clear(&lock);
}
printf("Thread End\n");
}
However, since you are already using pthreads, you're better off using a pthread_mutex
Jérôme Richard told you about ways in which the compiler could optimize the sense out of your code, but even if you turned all the optimizations off, you still would be left with a race condition. You wrote
while (lock) { }
lock=1;
...critical section...
lock=0;
The problem with that is, suppose lock==0. Two threads racing toward that critical section at the same time could both test lock, and they could both find that lock==0. Then they both would set lock=1, and they both would enter the critical section...
...at the same time.
In order to implement a spin lock,* you need some way for one thread to prevent other threads from accessing the lock variable in between when the first thread tests it, and when the first thread sets it. You need an atomic (i.e., indivisible) "test and set" operation.
Most computer architectures have some kind of specialized op-code that does what you want. It has names like "test and set," "compare and exchange," "load-linked and store-conditional," etc. Chris Dodd's answer shows you how to use a standard C library function that does the right thing on whatever CPU you happen to be using...
...But don't forget what Jérôme said.*
* Jérôme told you that spin locks are a bad idea.
I have written this code in C language and there are two pthreads that are using this code and trying to access the mutex "firstSection" (in both of them we are sure that the mutex passed to function is the same). The code suppose to check two mutexes, and if both of them were available, performs some actions which take place in function safeUnlockTwoMutexes(), and if failed to acquire at least one of them, it has to wait for two seconds and tries again. ("intersection" mutex is the main-lock to safe check the situation of the other mutexes)
void twoSectionRoute(pthread_mutex_t firstSection, pthread_mutex_t secondSection){
bool pathClear = false;
while (!pathClear){
pthread_mutex_lock(&intersection);
if (pthread_mutex_trylock(&firstSection) == 0){
if (pthread_mutex_trylock(&secondSection) == 0){
pathClear = true;
pthread_mutex_unlock(&intersection);
} else {
pthread_mutex_unlock(&firstSection);
pthread_mutex_unlock(&intersection);
sleep(2);
}
} else {
pthread_mutex_unlock(&intersection);
sleep(2);
}
}
safeUnlockTwoMutexes(firstSection, secondSection, 1);
}
Now the problem with this code is both threads are able to lock the mutex "firstSectio" at almost same time and I don't know why. (maybe because its type is recursive mutex?! I've used "PTHREAD_MUTEX_INITIALIZER" in the beginning of the file as global variables)
I'm wondering how can I fix this issue, and the threads access this sections one after another?
Your function signature passes pthread_mutex_t values firstSection and secondSection by value. You need to pass mutexes by pointer.
void twoSectionRoute(pthread_mutex_t* firstSection, pthread_mutex_t* secondSection){
Then, within the function use just firstSection and secondSection rather than &firstSection and &secondSection.
If you pass the mutex by value (as here), and it compiles, then the mutex itself is copied, so you end up with undefined behaviour and the mutex locks do not operate on the same state.
I have a hash table implementation in C where each location in the table is a linked list (to handle collisions). These linked lists are inherently thread safe and so no additional thread-safe code needs to be written at the hash table level if the table is a constant size - the hash table is thread-safe.
However, I would like the hash table to dynamically expand as values were added so as to maintain a reasonable access time. For the table to expand though, it needs additional thread-safety.
For the purposes of this question, procedures which can safely occur concurrently are 'benign' and the table resizing procedure (which cannot occur concurrently) is 'critical'. Threads currently using the list are known as 'users'.
My first solution to this was to put 'preamble' and 'postamble' code for all the critical function which locks a mutex and then waits until there are no current users proceeding. Then I added preamble and postamble code to the benign functions to check if a critical function was waiting, and if so to wait at the same mutex until the critical section is done.
In pseudocode the pre/post-amble functions SHOULD look like:
benignPreamble(table) {
if (table->criticalIsRunning) {
waitUntilSignal;
}
incrementUserCount(table);
}
benignPostamble(table) {
decrementUserCount(table);
}
criticalPreamble(table) {
table->criticalIsRunning = YES;
waitUntilZero(table->users);
}
criticalPostamble(table) {
table->criticalIsRunning = NO;
signalCriticalDone();
}
My actual code is shown at the bottom of this question and uses (perhaps unnecessarily) caf's PriorityLock from this SO question. My implementation, quite frankly, smells awful. What is a better way to handle this situation? At the moment I'm looking for a way to signal to a mutex that it is irrelevant and 'unlock all waiting threads' simultaneously, but I keep thinking there must be a simpler way. I am trying to code it in such a way that any thread-safety mechanisms are 'ignored' if the critical process is not running.
Current Code
void startBenign(HashTable *table) {
// Ignores if critical process can't be running (users >= 1)
if (table->users == 0) {
// Blocks if critical process is running
PriorityLockLockLow(&(table->lock));
PriorityLockUnlockLow(&(table->lock));
}
__sync_add_and_fetch(&(table->users), 1);
}
void endBenign(HashTable *table) {
// Decrement user count (baseline is 1)
__sync_sub_and_fetch(&(table->users), 1);
}
int startCritical(HashTable *table) {
// Get the lock
PriorityLockLockHigh(&(table->lock));
// Decrement user count BELOW baseline (1) to hit zero eventually
__sync_sub_and_fetch(&(table->users), 1);
// Wait for all concurrent threads to finish
while (table->users != 0) {
usleep(1);
}
// Once we have zero users (any new ones will be
// held at the lock) we can proceed.
return 0;
}
void endCritical(HashTable *table) {
// Increment back to baseline of 1
__sync_add_and_fetch(&(table->users), 1);
// Unlock
PriorityLockUnlockHigh(&(table->lock));
}
It looks like you're trying to reinvent the reader-writer lock, which I believe pthreads provides as a primitive. Have you tried using that?
More specifically, your benign functions should be taking a "reader" lock, while your critical functions need a "writer" lock. The end result will be that as many benign functions can execute as desired, but when a critical function starts executing it will wait until no benign functions are in process, and will block additional benign functions until it has finished. I think this is what you want.
I have some thread to write resource and some to read it.But pthread_rwlock cause a lot of context switch. So I imagine a way to avoid it. But I'm not sure it is safe or not.
This is the code:
sig_atomic_t slot = 0;
struct resource {
sig_atomic_t in_use; /*Counter,if in_use, not zero*/
.....
} xxx[2];
int read_thread()
{
i = slot; /*avoid slot changes in process */
xxx[i].in_use++;
read(xxx[i]);
xxx[i].in_use--;
}
int write_thread()
{
mutex_lock; /*mutex between write threads */
if (slot == 0) {
while(xxx[1].in_use != 0); /*wait last read thread in slot 1*/
clear(xxx[1]);
write(xxx[1]);
slot = 1;
} else if (slot == 1) {
while(xxx[0].in_use != 0);
clear(xxx[0]);
write(xxx[0]);
slot = 0;
}
mutex_unlock;
}
Will that works? The cost is 2 times storage and 3 atomic variable.
Thanks a lot!
Your algorithm is not lock-free; the writers use a spin lock.
Is it really necessary to do double-buffering and spin locks? Could you instead use (slot ^ 1) as the writing slot and slot as the reading slot? After writing, the writer would atomically change the value of slot, thus "publishing" its write. You may read the same slot many times consecutively this way, but if that's not the semantics you want then you should be using a queue.
By the way, a sig_atomic_t does not provide the type of atomicity you need for multiple threads. At a minimum, you should declare slot as volatile sig_atomic_t, and use memory barriers when reading and writing.
Your strategy is to have writers write to a different slot than what the readers are reading from. And you are switching the reading slot number after a write is completed. However, you will have a race.
slot reader writer1 writer2
---- ------ ------- -------
0 mutex_lock
i = 0
... slot=1
1 mutex_unlock mutex_lock
... clear(xxx[0])
xxx[0].in_use++
read(xxx[0]) write(xxx[0])
In general, though, this strategy could lead to starvation of writers (that is a writer may spin forever).
However, if you are willing to tolerate that, it would be safer to let xxx[] be an array of 2 pointers to resource. Let the reader always read from xxx[0], and let the writers contend for updates on xxx[1]. When a writer is finished updating xxx[1], it uses CAS on xxx[0] and xxx[1].
struct resource {
sig_atomic_t in_use; /*Counter,if in_use, not zero*/
sig_atomic_t writer;
.....
} *xxx[2];
void read_thread()
{
resource *p = xxx[0];
p->in_use++;
while (p->writer) {
p->in_use--;
p = xxx[0];
p->in_use++;
}
read(*p);
p->in_use--;
}
void write_thread()
{
resource *p;
mutex_lock; /*mutex between write threads */
xxx[1]->writer = 1;
while(xxx[1]->in_use != 0); /*wait last read thread in slot 1*/
clear(xxx[1]);
write(xxx[1]);
xxx[1] = CAS(&xxx[0], p = xxx[0], xxx[1]);
assert(p == xxx[1]);
xxx[0]->writer = 0;
mutex_unlock;
}
If you want to avoid writer starvation, but you want the performance of spinlocks, you are looking at implementing your own reader/writer locks using spinlocks instead of mutex locks. A google search for "read write spinlock implementation" pointed to this page which I found to be an interesting read.
So, I am trying to implement a concurrent queue in C. I have split the methods into "read methods" and "write methods". So, when accessing the write methods, like push() and pop(), I acquire a writer lock. And the same for the read methods. Also, we can have several readers but only one writer.
In order to get this to work in code, I have a mutex lock for the entire queue. And two condition locks - one for the writer and the other for the reader. I also have two integers keeping track of the number of readers and writers currently using the queue.
So my main question is - how to implement several readers accessing the read methods at the same time?
At the moment this is my general read method code: (In psuedo code - not C. I am actually using pthreads).
mutex.lock();
while (nwriter > 0) {
wait(&reader);
mutex.unlock();
}
nreader++;
//Critical code
nreader--;
if (nreader == 0) {
signal(&writer)
}
mutex.unlock
So, imagine we have a reader which holds the mutex. Now any other reader which comes along, and tries to get the mutex, would not be able to. Wouldn't it block? Then how are many readers accessing the read methods at the same time?
Is my reasoning correct? If yes, how to solve the problem?
If this is not for an exercise, use read-write lock from pthreads (pthread_rwlock_* functions).
Also note that protecting individual calls with a lock stil might not provide necessary correctness guarantees. For example, a typical code for popping an element from STL queue is
if( !queue.empty() ) {
data = queue.top();
queue.pop();
}
And this will fail in concurrent code even if locks are used inside the queue methods, because conceptually this code must be an atomic transaction, but the implementation does not provide such guarantees. A thread may pop a different element than it read by top(), or attempt to pop from empty queue, etc.
Please find the following read\write functions.
In my functions, I used canRead and canWrite mutexes and nReads for number of readers:
Write function:
lock(canWrite) // Wait if mutex if not free
// Write
unlock(canWrite)
Read function:
lock(canRead) // This mutex protect the nReaders
nReaders++ // Init value should be 0 (no readers)
if (nReaders == 1) // No other readers
{
lock(canWrite) // No writers can enter critical section
}
unlock(canRead)
// Read
lock(canRead)
nReaders--;
if (nReaders == 0) // No more readers
{
unlock(canWrite) // Writer can enter critical secion
}
unlock(canRead)
A classic solution is multiple-readers, single-writer.
A data structure begins with no readers and no writers.
You permit any number of concurrent readers.
When a writer comes along, you block him till all current readers complete; then you let him go (any new readers and writers which come along which the writer is blocked queue up behind him, in order).
You may try this library it is built in c native, lock free, suitable for cross-platform lfqueue,
For Example:-
int* int_data;
lfqueue_t my_queue;
if (lfqueue_init(&my_queue) == -1)
return -1;
/** Wrap This scope in other threads **/
int_data = (int*) malloc(sizeof(int));
assert(int_data != NULL);
*int_data = i++;
/*Enqueue*/
while (lfqueue_enq(&my_queue, int_data) == -1) {
printf("ENQ Full ?\n");
}
/** Wrap This scope in other threads **/
/*Dequeue*/
while ( (int_data = lfqueue_deq(&my_queue)) == NULL) {
printf("DEQ EMPTY ..\n");
}
// printf("%d\n", *(int*) int_data );
free(int_data);
/** End **/
lfqueue_destroy(&my_queue);