I'm tring to solve producer consumer problem by using monitoring thread.
Basically, when we try to solve this problem, we can easly solve it by using semaphore and mutex(semaphore detect empty or full state keep stable critical section, and mutex keep shared part where producer producing, consumer consuming)
In this problem I want to using monitoring thread to check how many item is in shared memory and this thread must enter critical section after producer or consumer doing their work. It kind of acknowledgement thing
For example
Produce
Ack, buffer have 1 item
Consume
Ack, buffer have 0 item
Produce
Ack, buffer have 1 item
Produce
Ack, buffer have 2 item
I was thinking use two more sem_t,
sem_t m1;
sem_t m2;
Void *produce()
{
sem_wait(&m1);
sem_wait(&m2);
....
sem_signal(&m1);
}
Void *consumer()
{
sem_wait(&m1);
sem_wait(&m2);
....
sem_signal(&m2);
}
Void *monitor()
{
if(m1==o && m2==1)
sem_wait();
else if(m1==1 && m2==0)
sem_wait(m1);
....
sem_signal(m1);
sem_signal(m2);
}
producer and consumer only can enter critical section when both m1 and m2 is bigger then 0, and when they done their tasks they call sem_signal just one time for m1 or m2. this block producer and consumer, only monitoring thread can enter critical section. after monitoring thread finishing own task, it calls sem_signal for m1, m2, then producer or consumer can enter the critical section (each of sem object is kind of mutex, binary semaphore)
but it doesn't work..
I can't find what is my problem.
Is Sem object can't use if statement?
or, my idea is have something problem?
thanks for reading my long text
Related
I'm currently learning concurrent programming. I have two threads and I want them to act like this:
the first thread runs N times while the second one waits
when the first one is done, the second does its job and the first waits
when the second thread is done, repeat.
I'm trying to do it in C using pthread library. Here's some pseudocode (hopefully understandable)
int cnt = 0;
void* thread1(){
while(1){
// thread 1 code
cnt ++;
if(cnt == N){
let_thread2_work();
wait_thread2();
}
}
}
void* thread2(){
while(1){
wait_thread1();
// thread2 code
cnt = 0;
let_thread1_work();
}
}
Can anyone please help me ?
Like David Schwartz commented, this doesn't make a ton of sense with just thread1 and thread2 waiting on each other, not actually doing any work in parallel. But maybe eventually you want multiple "thread1"s, all processing jobs at the same time until they finish a batch of N jobs, then they all stop and wait for "thread2" to do some kind of post-processing before the pool of worker threads start back up.
In this situation I would consider using a couple condition variables, one for your worker threads to communicate to your post-processing thread that they're waiting, and one for your post-processing thread to tell the workers to start working again. You can declare a condition variable and a helper mutex in the global scope right next to your "cnt" variable, which I'm calling "jobs_done" for clarity:
#include<pthread.h>
#DEFINE NUM_WORKERS 1 // although it doesn't make sense to just have 1
#DEFINE BATCH_SIZE 50 // this is "N"
// we're going to keep track of how many jobs we have done in
// this variable
int jobs_done = 0;
// when a worker thread checks jobs_done and it's N or greater,
// that means we have to wait for the post-processing thread to set
// jobs_done back to 0. so the worker threads "wait" on the
// condition variable, and the post-processing thread "broadcasts"
// to the condition variable to wake them all up again once it's
// done its work
pthread_cond_t jobs_ready_cv;
// we're going to use this helper mutex. whenever any thread
// reads or writes to the jobs_done variable, we have to lock this
// mutex. that includes the worker threads when they check to see if
// they're ready to wake up again.
pthread_mutex_t jobs_mx;
// here's how the worker threads will communicate to the post-process
// thread that a batch is done. to make sure that all N jobs are fully
// complete before postprocessing happens, we'll use this variable to
// keep track of how many threads are waiting for postprocessing to finish.
int workers_done = 0;
// we'll also use a separate condition variable and separate mutex to
// communicate to the postprocess thread.
pthread_cond_t workers_done_cv;
pthread_mutex_t workers_done_mx;
Then in your setup code, initialize the condition variables and helper mutexes:
int main() { // or something
pthread_cond_init(&jobs_ready_cv, NULL);
pthread_mutex_init(&jobs_mx, NULL);
pthread_cond_init(&workers_done_cv, NULL);
pthread_mutex_init(&workers_done_mx, NULL);
...
}
So, your worker threads (or "thread1"), before taking a job, will check to see how many jobs have been taken. If N (here, BATCH_SIZE) have been taken, then it updates a variable to indicate that it has no work left to do. If it finds that all of the worker threads are done, then it signals the postprocess thread ("thread2") through workers_done_cv. Then, the thread waits for a signal from the postprocess thread through `
void* worker_thread(){
while(1){
/* first, we check if the batch is complete. we do this first
* so we don't accidentally take an extra job.
*/
pthread_mutex_lock(&jobs_mx);
if (jobs_done == BATCH_SIZE) {
/* if BATCH_SIZE jobs have been done, first let's increment workers_done,
* and if all workers are done, let's notify the postprocess thread.
* after that, we release the workers_done mutex so the postprocess
* thread can wake up from the workers_done condition variable.
*/
pthread_mutex_lock(&workers_done_mx);
++workers_done;
if (workers_done == NUM_WORKERS) {
pthread_cond_broadcast(&workers_done_cv);
}
pthread_mutex_unlock(&workers_done_mx);
/* now we wait for the postprocess thread to do its work. this
* unlocks the mutex. when we get the signal to start doing jobs
* again, the mutex will relock automatically when we wake up.
*
* note that we use a while loop here to check the jobs_done
* variable after we wake up. That's because sometimes threads
* can wake up on accident even if no signal or broadcast happened,
* so we need to make sure that the postprocess thread actually
* reset the variable. google "spurious wakeups"
*/
while (jobs_done == BATCH_SIZE) {
pthread_cond_wait(&jobs_ready_cv, &jobs_mx);
}
}
/* okay, now we're ready to take a job.
*/
++jobs_done;
pthread_mutex_unlock(&jobs_mx);
// thread 1 code
}
}
Meanwhile, your postprocess thread waits on the workers_done_cv immediately, and doesn't wake up until the last worker thread is done and calls pthread_cond_broadcast(&workers_done_cv). Then, it does whatever it needs to, resets the counts, and broadcasts to the worker threads to wake them back up.
void* postprocess_thread(){
while(1){
/* first, we wait for our worker threads to be done
*/
pthread_mutex_lock(&workers_done_mx);
while (workers_done != NUM_WORKERS) {
pthread_cond_wait(&workers_done_cv, &workers_done_mx);
}
// thread2 code
/* reset count of stalled worker threads, release mutex */
workers_done = 0;
pthread_mutex_unlock(&workers_done_mx);
/* reset number of jobs done and wake up worker threads */
pthread_mutex_lock(&jobs_mx);
jobs_done = 0;
pthread_mutex_unlock(&jobs_mx);
pthread_cond_broadcast(&jobs_ready_cv);
}
}
Also take heed of David Schwartz's advice that you probably don't actually need the postprocessing thread to wait on the worker threads. If you don't need this, then you can get rid of the condition variable that makes the worker threads wait for the postprocess thread, and this implementation becomes a lot simpler.
edit: mutex protected the assignment to jobs_done in postprocess_thread(), added a forgotten ampersand
One solution is to use mutex:
https://www.geeksforgeeks.org/mutex-lock-for-linux-thread-synchronization/
Another solution, but using semaphore:
https://www.geeksforgeeks.org/difference-between-binary-semaphore-and-mutex/?ref=rp
Or creating own lightthread schedule.
for exemple:
http://www.dunkels.com/adam/pt/ [edited]
I'm trying to make a producer/ consumer application. The problem is, my producer is fillin the whole buffer THEN the consumer is removing the whole buffer... wasn't the consumer supposed to remove an item from the buffer as soon as the producer make a post to the semaphore ?
my code:
void* producerFunc(void* arg)
{
while(n_insertions < N_PRODUCTS)
{
sem_wait(&sem_postAvaliable);
sem_wait(&mutex);
//Insert Item
sem_post(&mutex);
sem_post(&sem_posTaken);
}
pthread_exit(NULL);
}
void* consumerFunc(void* arg)
{
while(n_consumed < N_PRODUCTS)
{
sem_wait(&sem_posTaken);
sem_wait(&mutex);
//Remove from bufer
sem_post(&mutex);
sem_post(&sem_postAvaliable);
}
pthread_exit(NULL);
}
n_insertionsis a global integer that counts the number of items inserted into the buffer;
n_consumed is a global integer that counts the number of times the consumer consumed an item from the buffer.
At a very high level, when you do a sem_post(&sem_posTaken) you are telling the consumers "Hey!, you can consume something", but after that depending on the processor scheduler the producers may continue producing until the counter of the semaphore doesn't allow them, so maybe a consumer consume right after the producer insert something or maybe after 3 inserts, who knows? The thread is blocked when you try to make a sem_wait of a semaphore whose counter is 0.
If you want to alternate between an insert and a remove, try initializing sem_postAvaliable to 1, that way you would allowing only one insertion at first, because the other threads would be blocked waiting a sem_post(&sem_postAvaliable), then only one consumer could consume whatever you inserted in the buffer and so on, but note that in this case a buffer with size larger than 1 would be meaningless.
I'm trying to do my homework but I'm stuck with these threads.. This function is called when a thread is created:
size_t* mines, gold = 0, gold_collected = 0;
pthread_mutex_t mine_mutex;
int last_mine = 0;
void* dig(void *mine_start) {
int current_worker = (int)mine_start;
int mine = (int)mine_start;
// printf("Hello, it's me, thread %d\n", current_worker);
while(gold != 0) {
if(mine > last_mine - 1) {
mine = 0;
}
pthread_mutex_lock(&mine_mutex);
if(mines[mine] != 0) {
//printf("All gold %zd\n", gold);
//printf("Gold in mine %zd with number %d\n", mines[mine], mine);
printf("Worker %d entered mine %d\n", current_worker, mine);
gold -= 10;
mines[mine] -= 10;
gold_collected += 10;
//sleep(1);
}
pthread_mutex_unlock(&mine_mutex);
++mine;
}
pthread_exit(NULL);
}
My problem is that when I have 5 mines and 2 workers, only one worker gets in the mine and digs the gold. How can I rotate my threads so all of them can dig from the mine?
If you want 1 miner per mine, but you have more miners than mines, then you have to decide what the idle miners will do when all the mines are in use. Furthermore, if you have a mutex per mine, and everyone tries to take the first mutex, only one miner will win, and the others will still block. You can use a try lock, but then the miners will busy wait when all the mines are full.
You could use a semaphore that is initialized with the number of mines. Each miner, upon successfully acquiring the semaphore would know that there is a mine available for them, but they wouldn't know which one. You could use a single mutex to protect all of the mines' in-use state. After acquiring the semaphore, you then acquire the mutex, hunt for an available mine, mark it as in-use, release the mutex and start mining. Then, when you are done, re-acquire the mutex, mark the mine as available, release the mutex, and then release the semaphore.
Finally, instead of the semaphore, you could use a condition variable and a mutex. Acquire the mutex, and hunt for an available mine. If you cannot find one, block on the condvar. If you do find one, mark it as in use, release the mutex, and start mining. When you are done, re-acquire the mutex, mark the mine as available, signal the condvar, and release the mutex. A thread that awakes on the condvar will have automatically re-acquired the mutex and should loop around and re-hunt for an available mine. In this case, it should be sufficient to signal the condvar instead of broadcasting; although broadcasting can be safer.
Also, once you have parallel miners, you're going to have to rethink the global gold and gold_collected. Since your miners will be doing the actual mining without holding a mutex, they cannot update these globals while mining. They should keep a local tally of the amount of gold they mined, and update the global once the mutex has been re-acquired. Maybe gold can be deducted before the miner enters the mine, and gold_collected updated after leaving the mine (both while holding the mutex). It's also a little iffy reading gold when not holding the mutex, since it could change underneath you...
There is only one mutex for both mines, so only one worker can be working in the mines. If you have one mutex per mine, then two workers can enter in mines at same time (one worker in each mine).
Lets say I have a buffer that has 3 producer threads and 5 consumer threads inserting and consuming to/from the buffer.
I only want to allow 1 producer or up to 3 consumer threads access the buffer at any given time.
Up to 3 consumers can peek at the top element in the buffer, only, if no producer is accessing it. If more than 1 consumer thread does access the buffer, the last thread to leave must delete the top element.
Now this is part of a class assignment, and the assignment explicitly states to use semaphores. However, I can't think of a way to really implement this wording exactly using only semaphores.
The pseudo code -I think- should look like this: (I'm not worrying about an empty or full buffer, just this sub-part of the problem)
sem_init(&binary, 0, 1); //Init binary semaphore to 1
sem_init(&consumerCount, 0 , 3); //Allows 3 consumers to access
producer()
{
createItem()
sem_wait(&binary)
appendItem()
sem_post(&binary)
}
//The above assures nothing else can access buffer while appending an item
consumer()
{
while( binary unlocked)
{
sem_wait(&binary) and sem_wait(&consumerCount) //Locks the producers out
//Allows 3 consumers in
peek() //Gets copy of top item
if( last consumer out )
{
delete() //Deletes top item
sem_post(&binary) //Allow producer access back since last one out
}
sem_post(&consumerCount)
}
}
I think that's the gist of the logic, problem is how to implement this with just semaphores. How do I allow only 1 producer in with a semaphore but allow 3 consumers in on the other side? It seems like I would need to use something besides a semaphore.
Also, please correct any of the logic if needed, this is meant to just be a general idea.
You can solve the problem with two semaphores. The first semaphore is used for exclusive access by producers. The second semaphore is used for the shared access. The producer tries to acquire all three permits in order to lock out the consumers.
sem_init(&exclusive, 0, 1);
sem_init(&shared, 0, 3);
void producer()
{
sem_wait(&exclusive);
sem_wait(&shared);
sem_wait(&shared);
sem_wait(&shared);
// critical section
sem_post(&shared);
sem_post(&shared);
sem_post(&shared);
sem_post(&exclusive);
}
void consumer()
{
sem_wait(&shared);
// critical section
sem_post(&shared);
}
I'm reading stevens's book: apue. 2e. I have encountered a problem, about Mutex.
Let's see a peice of code first: (the following code comes from figure 11.10 of apue 2e)
#include <stdlib.h>
#include <pthread.h>
struct foo {
int f_count;
pthread_mutex_t f_lock;
/* ... more stuff here ... */
};
struct foo *
foo_alloc(void) /* allocate the object */
{
struct foo *fp;
if((fp = malloc(sizeof(struct foo))) != NULL){
fp->f_count = 1;
if(pthread_mutex_init(&fp->f_lock, NULL) != 0){
free(fp);
return NULL;
}
/* ... continue initialization ... */
}
return (fp);
}
void
foo_hold(struct foo *fp) /* add a reference to the object */
{
pthread_mutex_lock(&fp->f_lock);
fp->f_count++;
pthread_mutex_unlock(&fp->f_lock);
}
void
foo_rele(struct foo *fp) /* release a reference to the object */
{
pthread_mutex_lock(&fp->f_lock);
if(--fp->f_count == 0){ /* last reference */
pthread_mutex_unlock(&fp->f_lock); /* step 1 */
pthread_mutex_destroy(&fp->f_lock); /* step 2 */
free(fp);
}else{
pthread_mutex_unlock(&fp->f_lock);
}
}
Assuming we have two threads: thread1 and thread2. thread1 is running right now.
it calls function foo_rele and has finished the execution of step 1(see above),
and prepare to execute step2.
however, context switch occurs at this moment. and thread2 starts to execute.
thread2 locks the lock fp->f_lock then does something. but before thread2 unlocks the lock, context switch occurs again.
thread2 stops executing and thread1 starts to execute. thread1 destroy the lock and it's known to us all that error generates.
so, my problem is: Is the situation mentioned above possible to happen?
how can we avoid it and is there any interface(API) that can unlock and destroy a mutex atomically?
I agree with usr (+1), the design is wrong.
Generally speaking, before one thread can destroy anything it must establish that all other threads have finished using it. That requires some other form of inter-thread synchronisation. What you need is a way in which thread 1 can tell thread 2 that the resource needs to be freed, and for thread 2 to acknowledge that so that thread 1 can call foo_rele() knowing for sure that thread 2 will not try to use fp ever again. Or, for thread 2 telling thread 1 that it no longer needs fp and thread 1 calling foo_rele() as a consequence.
This can be accomplished several ways. One is for thread 1 to set a flag under the protection of the lock, and then wait for thread 2 to quit. Meanwhile, thread 2 eventually gets round to seeing the flag, and then quits. This releases thread 1 which then calls foo_rele(). Another way is message passing over pipes (my preferred means of inter-thread synchronisation); something along the lines of thread 1 -> thread 2, "Please stop using fp": thread 2 -> thread 1, "Ok" (though of course a better defined message set based on enums is advisable, but you get my meaning).
What you can't ever have is thread 1 calling foo_rele() without thread 2 either knowing that it must never touch fp ever again or thread 2 having already quit.
This is not well-designed because thread2 accesses an object which might have been destroyed already. Add a reference (increment f_count) before starting thread2. That way thread2 already starts out with the guarantee that the object is stable while it is running.