The Producer-Consumer problem is solved using this Semaphore algorithm where the Semaphore is decremented by the buffer size then the Semaphore is decremented by 1 signifying the critical section.
If these operations are happening consecutively back-to-back why is it not correct to decrement by 1 (the mutex) and then decrement the buffer size?
I know both the producer and consumer will go to sleep at the same time creating a deadlock but why does this minor switch cause the entire algorithm to fail?
BufferSize = 3;
semaphore mutex = 1; // Controls access to critical section
semaphore empty = BufferSize; // counts number of empty buffer slots
semaphore full = 0; // counts number of full buffer slots
Producer()
{
int widget;
while (TRUE) { // loop forever
make_new(widget); // create a new widget to put in the buffer
down(&empty); // decrement the empty semaphore
down(&mutex); // enter critical section
put_item(widget); // put widget in buffer
up(&mutex); // leave critical section
up(&full); // increment the full semaphore
}
}
Consumer()
{
int widget;
while (TRUE) { // loop forever
down(&full); // decrement the full semaphore
down(&mutex); // enter critical section
remove_item(widget); // take a widget from the buffer
up(&mutex); // leave critical section
up(&empty); // increment the empty semaphore
consume_item(widget); // consume the item
}
}
Code Source: Resource
Its not correct because what if the producer enters the code section by wait(mutex) but then it finds out there are no empty blocks left so it has to wait on the empty semaphore. At the same time when a consumer tries to enter it has to wait on mutex as producer has locked mutex. Thus there is a deadlock as producer is waiting for consumer to consume and consumer is waiting for producer to signal(mutex)
This is not the case with original arrangement of mutex and semaphores where a producer or consumer is only allowed to wait on mutex when it has decremented the semaphore value. Hence no deadlock.
Related
I'm currently learning concurrent programming. I have two threads and I want them to act like this:
the first thread runs N times while the second one waits
when the first one is done, the second does its job and the first waits
when the second thread is done, repeat.
I'm trying to do it in C using pthread library. Here's some pseudocode (hopefully understandable)
int cnt = 0;
void* thread1(){
while(1){
// thread 1 code
cnt ++;
if(cnt == N){
let_thread2_work();
wait_thread2();
}
}
}
void* thread2(){
while(1){
wait_thread1();
// thread2 code
cnt = 0;
let_thread1_work();
}
}
Can anyone please help me ?
Like David Schwartz commented, this doesn't make a ton of sense with just thread1 and thread2 waiting on each other, not actually doing any work in parallel. But maybe eventually you want multiple "thread1"s, all processing jobs at the same time until they finish a batch of N jobs, then they all stop and wait for "thread2" to do some kind of post-processing before the pool of worker threads start back up.
In this situation I would consider using a couple condition variables, one for your worker threads to communicate to your post-processing thread that they're waiting, and one for your post-processing thread to tell the workers to start working again. You can declare a condition variable and a helper mutex in the global scope right next to your "cnt" variable, which I'm calling "jobs_done" for clarity:
#include<pthread.h>
#DEFINE NUM_WORKERS 1 // although it doesn't make sense to just have 1
#DEFINE BATCH_SIZE 50 // this is "N"
// we're going to keep track of how many jobs we have done in
// this variable
int jobs_done = 0;
// when a worker thread checks jobs_done and it's N or greater,
// that means we have to wait for the post-processing thread to set
// jobs_done back to 0. so the worker threads "wait" on the
// condition variable, and the post-processing thread "broadcasts"
// to the condition variable to wake them all up again once it's
// done its work
pthread_cond_t jobs_ready_cv;
// we're going to use this helper mutex. whenever any thread
// reads or writes to the jobs_done variable, we have to lock this
// mutex. that includes the worker threads when they check to see if
// they're ready to wake up again.
pthread_mutex_t jobs_mx;
// here's how the worker threads will communicate to the post-process
// thread that a batch is done. to make sure that all N jobs are fully
// complete before postprocessing happens, we'll use this variable to
// keep track of how many threads are waiting for postprocessing to finish.
int workers_done = 0;
// we'll also use a separate condition variable and separate mutex to
// communicate to the postprocess thread.
pthread_cond_t workers_done_cv;
pthread_mutex_t workers_done_mx;
Then in your setup code, initialize the condition variables and helper mutexes:
int main() { // or something
pthread_cond_init(&jobs_ready_cv, NULL);
pthread_mutex_init(&jobs_mx, NULL);
pthread_cond_init(&workers_done_cv, NULL);
pthread_mutex_init(&workers_done_mx, NULL);
...
}
So, your worker threads (or "thread1"), before taking a job, will check to see how many jobs have been taken. If N (here, BATCH_SIZE) have been taken, then it updates a variable to indicate that it has no work left to do. If it finds that all of the worker threads are done, then it signals the postprocess thread ("thread2") through workers_done_cv. Then, the thread waits for a signal from the postprocess thread through `
void* worker_thread(){
while(1){
/* first, we check if the batch is complete. we do this first
* so we don't accidentally take an extra job.
*/
pthread_mutex_lock(&jobs_mx);
if (jobs_done == BATCH_SIZE) {
/* if BATCH_SIZE jobs have been done, first let's increment workers_done,
* and if all workers are done, let's notify the postprocess thread.
* after that, we release the workers_done mutex so the postprocess
* thread can wake up from the workers_done condition variable.
*/
pthread_mutex_lock(&workers_done_mx);
++workers_done;
if (workers_done == NUM_WORKERS) {
pthread_cond_broadcast(&workers_done_cv);
}
pthread_mutex_unlock(&workers_done_mx);
/* now we wait for the postprocess thread to do its work. this
* unlocks the mutex. when we get the signal to start doing jobs
* again, the mutex will relock automatically when we wake up.
*
* note that we use a while loop here to check the jobs_done
* variable after we wake up. That's because sometimes threads
* can wake up on accident even if no signal or broadcast happened,
* so we need to make sure that the postprocess thread actually
* reset the variable. google "spurious wakeups"
*/
while (jobs_done == BATCH_SIZE) {
pthread_cond_wait(&jobs_ready_cv, &jobs_mx);
}
}
/* okay, now we're ready to take a job.
*/
++jobs_done;
pthread_mutex_unlock(&jobs_mx);
// thread 1 code
}
}
Meanwhile, your postprocess thread waits on the workers_done_cv immediately, and doesn't wake up until the last worker thread is done and calls pthread_cond_broadcast(&workers_done_cv). Then, it does whatever it needs to, resets the counts, and broadcasts to the worker threads to wake them back up.
void* postprocess_thread(){
while(1){
/* first, we wait for our worker threads to be done
*/
pthread_mutex_lock(&workers_done_mx);
while (workers_done != NUM_WORKERS) {
pthread_cond_wait(&workers_done_cv, &workers_done_mx);
}
// thread2 code
/* reset count of stalled worker threads, release mutex */
workers_done = 0;
pthread_mutex_unlock(&workers_done_mx);
/* reset number of jobs done and wake up worker threads */
pthread_mutex_lock(&jobs_mx);
jobs_done = 0;
pthread_mutex_unlock(&jobs_mx);
pthread_cond_broadcast(&jobs_ready_cv);
}
}
Also take heed of David Schwartz's advice that you probably don't actually need the postprocessing thread to wait on the worker threads. If you don't need this, then you can get rid of the condition variable that makes the worker threads wait for the postprocess thread, and this implementation becomes a lot simpler.
edit: mutex protected the assignment to jobs_done in postprocess_thread(), added a forgotten ampersand
One solution is to use mutex:
https://www.geeksforgeeks.org/mutex-lock-for-linux-thread-synchronization/
Another solution, but using semaphore:
https://www.geeksforgeeks.org/difference-between-binary-semaphore-and-mutex/?ref=rp
Or creating own lightthread schedule.
for exemple:
http://www.dunkels.com/adam/pt/ [edited]
Multiple threads will be “busy waiting” for the next_action variable to be set. Ideally one thread would call perform_action whenever the main sets it to a nonzero.
// choose a time-consuming activity based on action ...
void perform_action(int action);
int next_action = 0;
void* threadfunc(void*)
{
while (1)
{
while (next_action == 0);
int my_action = next_action;
next_action = 0;
perform_action(my_action);
}
}
int main()
{
// assume we've spawned threads executing threadfunc ...
while (1)
{
// determine what action should be dispatched to a thread next
next_action = read_action_from_user();
// exit the program
if (next_action == -1) { break; }
}
}
For this issue I would use a semaphore.
Using a while loop will waste processor time by constantly checking for changes in variable.
Here we can use synchronization methods that can alert when is available.
A semaphore has two options: acquire and release.
Main thread initially acquires the semaphore, and the thread is enqueued to acquire it. Thread will wait until semaphore becomes available.
When main thread sets it releases the semaphore to signal that a nonzero value has been set to it.
The thread will now wake up, acquire the semaphore, perform requested operation and release the semaphore.
When main thread needs to change the semaphore, it must acquire the semaphore again, set the and release the semaphore.
To acquire the semaphore, main thread must necessarily wait until the thread has finished.
I would not use a mutex because you also need a signaling mechanism to wake up your thread, and not only a protection for a shared variable.
See also Conditional Variable vs Semaphore
I'm currently learning about concurrency at my University. In this context I have to implement the reader/writer problem in C, and I think I'm on the right track.
My thought on the problem is, that we need two locks rd_lock and wr_lock. When a writer thread wants to change our global variable, it tries to grab both locks, writes to the global and unlocks. When a reader wants to read the global, it checks if wr_lock is currently locked, and then reads the value, however one of the reader threads should grab the rd_lock, but the other readers should not care if rd_lock is locked.
I am not allowed to use the implementation already in the pthread library.
typedef struct counter_st {
int value;
} counter_t;
counter_t * counter;
pthread_t * threads;
int readers_tnum;
int writers_tnum;
pthread_mutex_t rd_lock;
pthread_mutex_t wr_lock;
void * reader_thread() {
while(true) {
pthread_mutex_lock(&rd_lock);
pthread_mutex_trylock(&wr_lock);
int value = counter->value;
printf("%d\n", value);
pthread_mutex_unlock(&rd_lock);
}
}
void * writer_thread() {
while(true) {
pthread_mutex_lock(&wr_lock);
pthread_mutex_lock(&rd_lock);
// TODO: increment value of counter->value here.
counter->value += 1;
pthread_mutex_unlock(&rd_lock);
pthread_mutex_unlock(&wr_lock);
}
}
int main(int argc, char **args) {
readers_tnum = atoi(args[1]);
writers_tnum = atoi(args[2]);
pthread_mutex_init(&rd_lock, 0);
pthread_mutex_init(&wr_lock, 0);
// Initialize our global variable
counter = malloc(sizeof(counter_t));
counter->value = 0;
pthread_t * threads = malloc((readers_tnum + writers_tnum) * sizeof(pthread_t));
int started_threads = 0;
// Spawn reader threads
for(int i = 0; i < readers_tnum; i++) {
int code = pthread_create(&threads[started_threads], NULL, reader_thread, NULL);
if (code != 0) {
printf("Could not spawn a thread.");
exit(-1);
} else {
started_threads++;
}
}
// Spawn writer threads
for(int i = 0; i < writers_tnum; i++) {
int code = pthread_create(&threads[started_threads], NULL, writer_thread, NULL);
if (code != 0) {
printf("Could not spawn a thread.");
exit(-1);
} else {
started_threads++;
}
}
}
Currently it just prints a lot of zeroes, when run with 1 reader and 1 writer, which means, that it never actually executes the code in the writer thread. I know that this is not going to work as intended with multiple readers, however I don't understand what is wrong, when running it with one of each.
Don't think of the locks as "reader lock" and "writer lock".
Because you need to allow multiple concurrent readers, readers cannot hold a mutex. (If they do, they are serialized; only one can hold a mutex at the same time.) They can take one for a short duration (before they begin the access, and after they end the access), to update state, but that's it.
Split the timeline for having a rwlock into three parts: "grab rwlock", "do work", "release rwlock".
For example, you could use one mutex, one condition variable, and a counter. The counter holds the number of active readers. The condition variable is signaled on by the last reader, and by writers just before they release the mutex, to wake up a waiting writer. The mutex protects both, and is held by writers for the whole duration of their write operation.
So, in pseudocode, you might have
Function rwlock_rdlock:
Take mutex
Increment counter
Release mutex
End Function
Function rwlock_rdunlock:
Take mutex
Decrement counter
If counter == 0, Then:
Signal_on cond
End If
Release mutex
End Function
Function rwlock_wrlock:
Take mutex
While counter > 0:
Wait_on cond
End Function
Function rwlock_unlock:
Signal_on cond
Release mutex
End Function
Remember that whenever you wait on a condition variable, the mutex is atomically released for the duration of the wait, and automatically grabbed when the thread wakes up. So, for waiting on a condition variable, a thread will have the mutex both before and after the wait, but not during the wait itself.
Now, the above approach is not the only one you might implement.
In particular, you might note that in the above scheme, there is a different "unlock" operation you must use, depending on whether you took a read or a write lock on the rwlock. In POSIX pthread_rwlock_t implementation, there is just one pthread_rwlock_unlock().
Whatever scheme you design, it is important to examine it whether it works right in all situations: a lone read-locker, a lone-write-locker, several read-lockers, several-write-lockers, a lone write-locker and one read-locker, a lone write-locker and several read-lockers, several write-lockers and a lone read-locker, and several read- and write-lockers.
For example, let's consider the case when there are several active readers, and a writer wants to write-lock the rwlock.
The writer grabs the mutex. It then notices that the counter is nonzero, so it starts waiting on the condition variable. When the last reader -- note how the order of the readers exiting does not matter, since a simple counter is used! -- unlocks its readlock on the rwlock, it signals on the condition variable, which wakes up the writer. The writer then grabs the mutex, sees the counter is zero, and proceeds to do its work. During that time, the mutex is held by the writer, so all new readers will block, until the writer releases the mutex. Because the writer will also signal on the condition variable when it releases the mutex, it is a race between other waiting writers and waiting readers, who gets to go next.
We have studied that if we dealing with multi-threaded problems then we use one of thread synchronizing method called mutex which allows to lock critical section so that other threads do not interfere and goes into block state until mutex unlocks the critical section.
But I am doing this thing in my program but the output of this program is not matches with the concept of mutex. If I am wrong so correct me.
Code
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <signal.h>
#include <pthread.h>
#define MAX 10
pthread_mutex_t the_mutex;
pthread_cond_t condc, condp;
int toConsume=0;
int i;
void* consumer(void *ptr) {
pthread_mutex_lock(&the_mutex);
while(i<MAX)
{
/* protect buffer */
while (toConsume <= 0) /* If there is nothing in the buffer then wait */
{
printf("Waiting Thread id:%lu \n",pthread_self());
pthread_cond_wait(&condc, &the_mutex);
}
pthread_mutex_unlock(&the_mutex); /* release the buffer */
sleep(2);
pthread_mutex_lock(&the_mutex); /* protect buffer */
toConsume--;
i++;
}
pthread_mutex_unlock(&the_mutex); /* release the buffer */
pthread_exit(0);
}
int main(int argc, char **argv) {
pthread_t pro, con[3];
pthread_mutex_init(&the_mutex, NULL);
pthread_cond_init(&condc, NULL); /* Initialize consumer condition variable */
pthread_cond_init(&condp, NULL); /* Initialize producer condition variable */
// Create the threads
for(int i=0 ;i<3;i++)
pthread_create(&con[i], NULL, consumer, NULL);
for(int i=0 ;i<3;i++)
pthread_join(con[i], NULL);
return 0;
}
OUTPUT
$ ./ex
Waiting Thread id:140580582618880
Waiting Thread id:140580574226176
Waiting Thread id:140580565833472
All thread enters into critical section even the mutex remain its lock.
The function pthread_cond_wait will unlock the held mutex while the thread is waiting. This allows another thread to enter the critical section.
The purpose of using pthread_cond_wait is that the thread needs to wait for some condition to become true before really performing the work inside the critical section. To test the condition in the first place requires locking the mutex. But, if the condition is false, it must wait for some event to make the condition true. If it waits with the lock held, no other thread will be able to update the state for the condition to become true, since updating the condition will also require locking the same mutex.
Thus, when waiting on a condition variable, the mutex is unlocked to allow another thread to grab the lock to perform something that would make the condition true.
As an example, consider a job queue. A thread will lock a mutex to get a job from the queue. However, if the queue is empty, it must wait for a job to appear in the queue. This is the condition that is must wait on, and a condition variable can be used for that purpose. When it calls pthread_cond_wait on the condition variable, the associated mutex is unlocked.
Another thread wishes to place a job into the queue. That thread can lock the mutex, add the job to the queue, signal the condition variable, then unlock the mutex.
When the condition variable is signaled, the waiting thread is woken up and pthread_cond_wait returns with the lock on the mutex held again. It detects that the queue is non-empty, and can enter the critical section of dequeueing a job from the queue.
I'm trying to make a producer/ consumer application. The problem is, my producer is fillin the whole buffer THEN the consumer is removing the whole buffer... wasn't the consumer supposed to remove an item from the buffer as soon as the producer make a post to the semaphore ?
my code:
void* producerFunc(void* arg)
{
while(n_insertions < N_PRODUCTS)
{
sem_wait(&sem_postAvaliable);
sem_wait(&mutex);
//Insert Item
sem_post(&mutex);
sem_post(&sem_posTaken);
}
pthread_exit(NULL);
}
void* consumerFunc(void* arg)
{
while(n_consumed < N_PRODUCTS)
{
sem_wait(&sem_posTaken);
sem_wait(&mutex);
//Remove from bufer
sem_post(&mutex);
sem_post(&sem_postAvaliable);
}
pthread_exit(NULL);
}
n_insertionsis a global integer that counts the number of items inserted into the buffer;
n_consumed is a global integer that counts the number of times the consumer consumed an item from the buffer.
At a very high level, when you do a sem_post(&sem_posTaken) you are telling the consumers "Hey!, you can consume something", but after that depending on the processor scheduler the producers may continue producing until the counter of the semaphore doesn't allow them, so maybe a consumer consume right after the producer insert something or maybe after 3 inserts, who knows? The thread is blocked when you try to make a sem_wait of a semaphore whose counter is 0.
If you want to alternate between an insert and a remove, try initializing sem_postAvaliable to 1, that way you would allowing only one insertion at first, because the other threads would be blocked waiting a sem_post(&sem_postAvaliable), then only one consumer could consume whatever you inserted in the buffer and so on, but note that in this case a buffer with size larger than 1 would be meaningless.