I know that this isn't a "homework helper website", but I got insane in the last days because i have to implement the access to resource avoiding starvation and i can't figure out how to do that. Can anyone help me with some application examples or documentation? The assignment is: a resource may be used by 2 types of processes: black and white. When the resource is used by the white processes, it can not be used by the black processes and vice-versa. Implement the access to the resource avoiding starvation. Is this a producer-consumer case?
Let's make a few assumptions (for the sake of discussion):
Our processes will be threads -- not actual software processes, there's a difference which may be important in your assignment.
White processes are Readers.
Black processes are Writers.
Our common resource is particular Variable.
Mutual exclusion locks (mutex):
A mutex is a type of exclusive lock, it has a binary state, it's either locked or unlocked. You can lock it, unlock it or check to see if it's locked or not.
Threads can lock each other out using mutex (mutual exclusion locks) just as processes can lock each other out using semaphores.
When you want to protect a variable from being used by two threads at once you create a mutex for that variable and write every thread so that it attempts to lock the mutex before attempting to use the variable and unlock it after they're done.
This makes any first thread lock the mutex and any subsequent thread block until the first thread unlocks the mutex basically forcing all of these threads to line up and operate on that particular variable sequentially.
This is a bit ineffective when you just want to read the variable, not change its value, because two threads reading the same content doesn't create any conflict or invalid data. Two threads writing at the same time might however corrupt the data.
Readers/Writers locks (RWL):
Most implementations of Readers/Write locks will use a shared lock and an exclusive lock, but they expose a simple usage approach: if you want to read grab a "read lock", if you want to write grab a "write lock".
"Read locks" are not exclusive and they allow multiple readers to be reading at one particular time (without blocking).
"Write locks" are exclusive and only one writer can be writing at one particular time (without blocking).
Starvation:
First step: Readers/Writers Locks is the event when a first (read) thread grabs a "read lock" on the variable, a second (write) tries to grab a "write lock" but is blocked until all readers finish reading.
Second step: before the first thread finishes reading, a third (read) thread grabs a "read lock" on the variable; this means the second (write) thread has to wait for this third thread to finish.
Repeat the second step until starvation is achieved.
Avoiding starvation with Seqlock:
A seqlock is implemented with one mutex and some counters. It always allows reading, even while the writers are writing to the variable but it gives the readers a means of checking if the data has been written to during the time it was being read, if so it may be corrupt so the readers will have to reread the data and check for consistency again.
The "read & consistency check" phase runs in a loop until the check confirms consistency of the data, at which point the reader can continue with its usual task.
The writers use the mutex to grab exclusive access so they never overlap their operations.
This is good for high read low write situations. If there would be too many writers the readers would continuously loop rereading the data.
Your particular situation:
If black processes need to be able to share the resource among themselves and white processes need to be able to share the resource among themselves but white processes can't share the resource with black processes then the solution will not be either RWL or Seqlock.
A variation on the Seqlock algorithm might be your solution.
Generally, is a problem in which it comes to access a shared resource (or mutex).
If you have two object of the same class, both threads:
pseudo-code:
loop
if shared_resource is free
lock shared_resource
do something
free shared_resource
This in VERY broad terms!
Related
What is the difference between semaphores and mutex provided by pthread library ?
semaphores have a synchronized counter and mutex's are just binary (true / false).
A semaphore is often used as a definitive mechanism for answering how many elements of a resource are in use -- e.g., an object that represents n worker threads might use a semaphore to count how many worker threads are available.
Truth is you can represent a semaphore by an INT that is synchronized by a mutex.
I am going to talk about Mutex vs Binary-Semaphore. You obviously use mutex to prevent data in one thread from being accessed by another thread at the same time.
(Assume that you have just called lock() and in the process of accessing a data. This means that, you don’t expect any other thread (or another instance of the same thread-code) to access the same data locked by the same mutex. That is, if it is the same thread-code getting executed on a different thread instance, hits the lock, then the lock() should block the control flow.)
This applies to a thread that uses a different thread-code, which is also accessing the same data and which is also locked by the same mutex.
In this case, you are still in the process of accessing the data and you may take, say, another 15 secs to reach the mutex unlock (so that the other thread that is getting blocked in mutex lock would unblock and would allow the control to access the data).
Do you ever allow another thread to just unlock the same mutex, and in turn, allow the thread that is already waiting (blocking) in the mutex lock to unblock and access the data? (Hope you got what I am saying here.)
As per agreed-upon universal definition,
with “mutex” this can’t happen. No other thread can unlock the lock
in your thread
with “binary-semaphore” this can happen. Any other thread can unlock
the lock in your thread
So, if you are very particular about using binary-semaphore instead of mutex, then you should be very careful in “scoping” the locks and unlocks, I mean, that every control-flow that hits every lock should hit an unlock call and also there shouldn’t be any “first unlock”, rather it should be always “first lock”.
The Toilet Example
Mutex:
Is a key to a toilet. One person can have the key - occupy the toilet - at the time. When finished, the person gives (frees) the key to the next person in the queue.
"Mutexes are typically used to serialise access to a section of re-entrant code that cannot be executed concurrently by more than one thread. A mutex object only allows one thread into a controlled section, forcing other threads which attempt to gain access to that section to wait until the first thread has exited from that section."
(A mutex is really a semaphore with value 1.)
Semaphore:
Is the number of free identical toilet keys.
For Example, say we have four toilets with identical locks and keys. The semaphore count - the count of keys - is set to 4 at beginning (all four toilets are free), then the count value is decremented as people are coming in. If all toilets are full, ie. there are no free keys left, the semaphore count is 0. Now, when eq. one person leaves the toilet, semaphore is increased to 1 (one free key), and given to the next person in the queue.
"A semaphore restricts the number of simultaneous users of a shared resource up to a maximum number. Threads can request access to the resource (decrementing the semaphore), and can signal that they have finished using the resource (incrementing the semaphore)."
Source
mutex is used to avoid race condition between multiple threads.
whereas semaphore is used as synchronizing element used across multiple process.
mutex can't be replaced with binary semaphore since, one process waits for semaphore while other process releases semaphore. In case mutex both acquisition and release is handled by same.
The difference between the semaphore and mutex is the difference between mechanism and pattern. The difference is in their purpose (intent)and how they work(behavioral).
The mutex, barrier, pipeline are parallel programming patterns. Mutex is used(intended) to protect a critical section and ensure mutual exclusion. Barrier makes the agents(thread/process) keep waiting for each other.
One of the feature(behavior) of mutex pattern is that only allowed agent(s)(process or thread) can enter a critical section and only that agent(s) can voluntarily get out of that.
There are cases when mutex allows single agent at a time. There are cases where it allows multiple agents(multiple readers) and disallow some other agents(writers).
The semaphore is a mechanism that can be used(intended) to implement different patterns. It is(behavior) generally a flag(possibly protected by mutual exclusion). (One interesting fact is even mutex pattern can be used to implement semaphore).
In popular culture, semaphores are mechanisms provided by kernels, and mutexes are provided by user-space library.
Note, there are misconceptions about semaphores and mutexes. It says that semaphores are used for synchronization. And mutexes has ownership. This is due to popular OS books. But the truth is all the mutexes, semaphores and barriers are used for synchronization. The intent of mutex is not ownership but mutual exclusion. This misconception gave the rise of popular interview question asking the difference of the mutexes and binary-semaphores.
Summary,
intent
mutex, mutual exclusion
semaphore, implement parallel design patterns
behavior
mutex, only the allowed agent(s) enters critical section and only it(they) can exit
semaphore, enter if the flag says go, otherwise wait until someone changes the flag
In design perspective, mutex is more like state-pattern where the algorithm that is selected by the state can change the state. The binary-semaphore is more like strategy-pattern where the external algorithm can change the state and eventually the algorithm/strategy selected to run.
This two articles explain great details about mutex vs semaphores
Also this stack overflow answer tells the similar answer.
Semaphore is more used as flag, for which your really don't need to bring RTOS / OS. Semaphore can be accidentally or deliberately changed by other threads (say due to bad coding).
When you thread use mutex, it owns the resources. No other thread can ever access it, before resource get free.
Mutexes can be applied only to threads in a single process and do not work between processes as do semaphores.
Mutex is like sempaphore with with S=1.
You can control number of concurrent accesses with semaphore but with mutex only one process at a time can access it.
See the implemenation of these two below: (all functions are atomic)
Semaphore:
wait(S) {
while (S <= 0 )
; // busy wait
S--;
}
signal(S) {
S++;
}
Mutex:
acquire() {
while (!available)
; // busy wait
available = false;
}
release() {
available = true;
}
I am writing a program in which a memory array is modified by one thread under 2 possible operations (modify the array content, or dealloc the array and replace it by allocating a new array). The memory array can be read by many threads except when the array is modified or deallocated and replaced.
I know how to use mutex lock to allow the memory to be modified by only one thread at all time. How can I use it (or other multithreading tools in c) to allow arbitrary number of read threads to access the memory, except when the write thread modifies the memory?
The best solution to achieve this is using read-write locks i.e pthread_rwlock_* as already answered in above comments. Providing a little more detail about it.
Read-write locks are used for shared read access or exclusive write access. A thread that needs read access can't continue while any thread currently has write access. A thread that needs write access can't continue when any other thread has either write access or read access. When both readers and writers are waiting for the access at the same time, there is default action to give precedence to any of them, this rule can be changed.
Read-write lock functions with arguments are much clearly explained here:
https://docs.oracle.com/cd/E19455-01/806-5257/6je9h032u/index.html
There is a post in stackoverflow itself about the same:
concurrent readers and mutually excluding writers in C using pthreads
This (read-write locks) may cause the writer thread to starve if precedence are not properly defined and implementation has not taken care of case when too many readers are waiting for writer to finish. So read about it too:
How to prevent writer starvation in a read write lock in pthreads
I have a threaded server that can add/append/read files and relay data to the client.
If a file is being added, no other thread can append/read it. If a file is being appended, no threads can append/read it. If a file is being read, no other thread can append to it. However, if a file is being read, other files can read it.
Currently I have a mutex system that will do this, except it won't allow multiple reads.
To fix this, in the read method, I will change:
pthread_mutex_lock(&(fm->mutex));//LOCK
//do some things`
...
pthread_mutex_unlock(&(fm->mutex));
to
pthread_mutex_trylock(&(fm->mutex));//TRYLOCK [NonBlocking, so the thread can continue the read]
//do some things`
...
pthread_mutex_unlock(&(fm->mutex));
Question
How can I unlock the file without allowing the other methods (just append really) to begin writing to the file before all the other read()'s have finished?
Example
For example, if the reading thread that originally locked the file completes and unlocks the file and there are still other threads trying to read the file, then an appending thread gets the chance to lock the file and begin appending while the others are still reading, which is a no-no.
Idea
I want to keep a count of the number of threads currently reading a file. When a thread finishes, reduce the count. If the count is 0, meaning no threads are still reading, unlock the file. But, I'm worried that this would not be thread safe. If this is a viable solution, how could I make it thread safe? Another but, I believe only the original thread can successfully unlock the mutex.
It sounds like you may be looking for a read-write lock, which is provided by pthreads. It allows two modes of locking: a shared/read-lock mode, which can be locked by multiple threads at once, and an exclusive/write-lock mode, where the lock call won't return until all other threads (readers and writers) have given up their hold on the lock.
You could use a semaphore instead of the mutex (see this link about the differences). The semaphore does thread-safe synchronized counting for you.
You can live without an additional mutex to lock the file for writing if you limit the number of simultaneous read accesses to a (sufficient large) number N and require the semaphore to be increased by that number for write access. This way you can only gain write access if the number of readers is zero and all other readers will be locked out until your writer has finished.
Note that the POSIX documentation for pthread_mutex_lock() says:
If successful, the pthread_mutex_lock(), pthread_mutex_trylock(), and pthread_mutex_unlock() functions shall return zero; otherwise, an error number shall be returned to indicate the error.
Since you don't show your code testing the return values, you don't know whether your lock operations (in particular) succeeded or not.
Separately, since you want a read/write lock, why not use one:
pthread_rwlock_rdlock()
pthread_rwlock_wrlock()
pthread_rwlock_unlock()
pthread_rwlock_init()
pthread_rwlock_destroy()
There are four pthread_rwlockattr_*() functions and a total of 9 pthread_rwlock_*() functions; I only listed the most important functions in the family.
Let's say two semaphores are protecting a critical piece of code, and you only want a critical piece of code to execute if both of them are available. Is there a pattern for writing this?
In other words, is there a statement that reads, "If semaphore a and b are available, then run... otherwise sleep"?
The simplest way to implement this is to use a single pthread_mutex_t to protect some state, and a single pthread_cond_t to notify all threads when the state has changed. If you always broadcast on the condvar, then you will always wake all waiting threads. The threads can then perform arbitrarily complex tests and updates to the shared state.
Of course, this is not the most efficient solution since it potentially wakes threads when the state does not satisfy the condition they are waiting for (and they have to go back to sleep). It could also lead to starvation since a thread may always find itself at the back of the queue whenever it waits on the condvar, and never find an acceptable state when it awakens.
Without knowing more details of the problem you are trying to solve, it is hard to give an air tight answer.
pthreads does not allow you to acquire multiple locks/semaphores atomically; however, as pointed out by #Greg, you can avoid deadlock by assigning an order to the locks/semaphores, and having the threads always acquire them in that order. Of course, you have to know which locks you intend to acquire before you start to acquire any of them. It will not work if you cannot determine the next lock to acquire until you have acquired the current one, since you may be required to take a lock out of order. If you release all of the locks and start over, you may find the state has changed, requiring you to acquire a different set of locks, which could lead to livelock.
What is the difference between semaphores and mutex provided by pthread library ?
semaphores have a synchronized counter and mutex's are just binary (true / false).
A semaphore is often used as a definitive mechanism for answering how many elements of a resource are in use -- e.g., an object that represents n worker threads might use a semaphore to count how many worker threads are available.
Truth is you can represent a semaphore by an INT that is synchronized by a mutex.
I am going to talk about Mutex vs Binary-Semaphore. You obviously use mutex to prevent data in one thread from being accessed by another thread at the same time.
(Assume that you have just called lock() and in the process of accessing a data. This means that, you don’t expect any other thread (or another instance of the same thread-code) to access the same data locked by the same mutex. That is, if it is the same thread-code getting executed on a different thread instance, hits the lock, then the lock() should block the control flow.)
This applies to a thread that uses a different thread-code, which is also accessing the same data and which is also locked by the same mutex.
In this case, you are still in the process of accessing the data and you may take, say, another 15 secs to reach the mutex unlock (so that the other thread that is getting blocked in mutex lock would unblock and would allow the control to access the data).
Do you ever allow another thread to just unlock the same mutex, and in turn, allow the thread that is already waiting (blocking) in the mutex lock to unblock and access the data? (Hope you got what I am saying here.)
As per agreed-upon universal definition,
with “mutex” this can’t happen. No other thread can unlock the lock
in your thread
with “binary-semaphore” this can happen. Any other thread can unlock
the lock in your thread
So, if you are very particular about using binary-semaphore instead of mutex, then you should be very careful in “scoping” the locks and unlocks, I mean, that every control-flow that hits every lock should hit an unlock call and also there shouldn’t be any “first unlock”, rather it should be always “first lock”.
The Toilet Example
Mutex:
Is a key to a toilet. One person can have the key - occupy the toilet - at the time. When finished, the person gives (frees) the key to the next person in the queue.
"Mutexes are typically used to serialise access to a section of re-entrant code that cannot be executed concurrently by more than one thread. A mutex object only allows one thread into a controlled section, forcing other threads which attempt to gain access to that section to wait until the first thread has exited from that section."
(A mutex is really a semaphore with value 1.)
Semaphore:
Is the number of free identical toilet keys.
For Example, say we have four toilets with identical locks and keys. The semaphore count - the count of keys - is set to 4 at beginning (all four toilets are free), then the count value is decremented as people are coming in. If all toilets are full, ie. there are no free keys left, the semaphore count is 0. Now, when eq. one person leaves the toilet, semaphore is increased to 1 (one free key), and given to the next person in the queue.
"A semaphore restricts the number of simultaneous users of a shared resource up to a maximum number. Threads can request access to the resource (decrementing the semaphore), and can signal that they have finished using the resource (incrementing the semaphore)."
Source
mutex is used to avoid race condition between multiple threads.
whereas semaphore is used as synchronizing element used across multiple process.
mutex can't be replaced with binary semaphore since, one process waits for semaphore while other process releases semaphore. In case mutex both acquisition and release is handled by same.
The difference between the semaphore and mutex is the difference between mechanism and pattern. The difference is in their purpose (intent)and how they work(behavioral).
The mutex, barrier, pipeline are parallel programming patterns. Mutex is used(intended) to protect a critical section and ensure mutual exclusion. Barrier makes the agents(thread/process) keep waiting for each other.
One of the feature(behavior) of mutex pattern is that only allowed agent(s)(process or thread) can enter a critical section and only that agent(s) can voluntarily get out of that.
There are cases when mutex allows single agent at a time. There are cases where it allows multiple agents(multiple readers) and disallow some other agents(writers).
The semaphore is a mechanism that can be used(intended) to implement different patterns. It is(behavior) generally a flag(possibly protected by mutual exclusion). (One interesting fact is even mutex pattern can be used to implement semaphore).
In popular culture, semaphores are mechanisms provided by kernels, and mutexes are provided by user-space library.
Note, there are misconceptions about semaphores and mutexes. It says that semaphores are used for synchronization. And mutexes has ownership. This is due to popular OS books. But the truth is all the mutexes, semaphores and barriers are used for synchronization. The intent of mutex is not ownership but mutual exclusion. This misconception gave the rise of popular interview question asking the difference of the mutexes and binary-semaphores.
Summary,
intent
mutex, mutual exclusion
semaphore, implement parallel design patterns
behavior
mutex, only the allowed agent(s) enters critical section and only it(they) can exit
semaphore, enter if the flag says go, otherwise wait until someone changes the flag
In design perspective, mutex is more like state-pattern where the algorithm that is selected by the state can change the state. The binary-semaphore is more like strategy-pattern where the external algorithm can change the state and eventually the algorithm/strategy selected to run.
This two articles explain great details about mutex vs semaphores
Also this stack overflow answer tells the similar answer.
Semaphore is more used as flag, for which your really don't need to bring RTOS / OS. Semaphore can be accidentally or deliberately changed by other threads (say due to bad coding).
When you thread use mutex, it owns the resources. No other thread can ever access it, before resource get free.
Mutexes can be applied only to threads in a single process and do not work between processes as do semaphores.
Mutex is like sempaphore with with S=1.
You can control number of concurrent accesses with semaphore but with mutex only one process at a time can access it.
See the implemenation of these two below: (all functions are atomic)
Semaphore:
wait(S) {
while (S <= 0 )
; // busy wait
S--;
}
signal(S) {
S++;
}
Mutex:
acquire() {
while (!available)
; // busy wait
available = false;
}
release() {
available = true;
}