So i'm working on this code that is a producer-consumer code. It runs completely through the program, it just never actually executes anything in the critical section because it never really wakes up from sleep! I've added print statements everywhere to try to figure out where the code is executing, and it enters both producer and consumer functions, but never enters any portion of it after the sleep() function.
Here is my main:
/* 1. Get command line arguments argv[1], argv[2], argv[3] */
/* n1=arg[2], n2=arg[3]
/* 2. Initialize buffer, mutex, semaphores, and other global vars */
/*create the mutex lock */
/* create the semaphore and initialize it to 3 */
/*creating full semaphore and initializing it to 0 */
/*** critical section ***/
/* get the default attribute */
pthread_attr_init(&attr);
/* 3. Create producer thread(s) */
while (count < n1)
{
pthread_t tid;
/* create a new thread */
pthread_create(&tid, &attr, producer, NULL);
count++;
}
printf("OUTSIDE OF PRODUCER CREATION\n");
/* 4. Create consumer thread(s) */
count = 0;
while(count < n2)
{
pthread_t tid2;
/* create a new thread */
pthread_create(&tid2, &attr, consumer, NULL);
count++;
}
printf("after the crit section \n");
/* 5. Sleep */
/* 6. Realease resources, e.g destroy mutex and semaphores */
i've included for the most part just the code that I know i'm having a problem with and the rest are comments. and here is the code for my producer. consumer is basically the same:
void *producer(void *param) {
buffer_item rand;
unsigned int *seed;
seed = (unsigned int *)malloc(sizeof(unsigned int));
*seed = 10;
while (1) {
printf("Inside producer function\n");
/* Sleep for a random period of time */
r = (rand_r(seed))/divide;
sleep(r);
printf("producer slept\n");
//wait(empty)
//wait(mutex)
printf("producer locked mutex\n");
//crit section - add item to buffer
/*Generate a random number */
/*insert random number*/
printf("producer inserted item \n");
if (resp < 0)
printf("Producer error condition\n"); //Report error condition
//signal mutex
//signal empty
}
}
so when i run a.out 4 4 4, i get this as my output:
Inside producer function
Inside producer function
Inside producer function
OUTSIDE OF PRODUCER CREATION
Inside producer function
inside consumer function
inside consumer function
inside consumer function
after the crit section
inside consumer function
I'm not sure if its normal that things look like they are running out of order... it is executing 4 times but as you can see if never hits that print statement that happens after my sleep function (for both producer and consumer)
This is homework so i'm just looking for a bit more direction on this...
From the discussion, your error was that your sleep() values were too large, one you quoted coming out to 22 minutes. Rather than division(/) to limit the values, modulo(%) will put your values within a range. I recommend %10, or some other value which will limit the sleeps to a reasonable range.
Related
So, I know you guys love to go into details and I promise to provide more as much as they seem to be really relevant, but for my case what you should bear in mind is the following:
I have P threads that are producers
I have C threads that are consumers
I have a buffer of N elements
I want to finish the program after it consumes M elements.
How can I do that?
Below you have my main function
int main(int argc, char *argv[]) {
int i;
/* Verify the correct number of arguments were passed in */
if(argc != 4) {
fprintf(stderr, "USAGE:./main.out <INT> <INT> <INT>\n");
}
int mainSleepTime = atoi(argv[1]); /* Time in seconds for main to sleep */
int numProd = atoi(argv[2]); /* Number of producer threads */int numCons = atoi(argv[3]); /* Number of consumer threads */
/* Initialize the app */
initializeData();
/* Create the producer threads */
for(i = 0; i < numProd; i++) {
/* Create the thread */
pthread_create(&tid,&attr,producer,NULL);
}
/* Create the consumer threads */
for(i = 0; i < numCons; i++) {
/* Create the thread */
pthread_create(&tid,&attr,consumer,NULL);
}
/* Sleep for the specified amount of time in milliseconds */
/* Exit the program */
sleep(mainSleepTime);
printf("Exit the program\n");
exit(0);
}
Right now the program computes during the sleeping time. But I want to measure the different times it will take to do M executions if I vary N. InitializeData is just a function to work on the buffer. I believe this trick should be solved in the main. Can anybody help me?
Please see the following as an example:
struct
{
int nProducednProduced;
pthread_mutex_t producerMutex;
int nConsumed;
pthread_mutex_t consuerMutex;
} monitor;
The monitor is a global object visible from all the threads and initalized by main() before thread creation.
The producers should increment monitor.nProduced and stop/exit when it reaches M.
The consumers increment monitor.nConsumed and stop/exit when it reaches M.
main() either monitors monitor.nConsumed and exit when it reaches M or simply waits until all the producers and consumers finished.
Just write a global function accessible by both producer and consumer and guard this function's entry and exit points with mutex in order to safe guard this critical section from multiple simultaneous access. Now inside this function you can use a counters to check how many items are produced by producer and how many items are consumed by consumer. This can happen if you call this function from producer code at a point before producer produces item. Similarly you can call this function after consumer consumes a item.
I have a problem with multithreading, since I'm new to this topic. Code below is code I've been given from my University. It was in few versions, and I understood most of them. But I don't really understand the nready.nready variable and all this thread condition. Can anyone describe how those two work here? And why can't I just synchronise work of threads via mutex?
#include "unpipc.h"
#define MAXNITEMS 1000000
#define MAXNTHREADS 100
/* globals shared by threads */
int nitems; /* read-only by producer and consumer */
int buff[MAXNITEMS];
struct {
pthread_mutex_t mutex;
pthread_cond_t cond;
int nput;
int nval;
int nready;
} nready = { PTHREAD_MUTEX_INITIALIZER, PTHREAD_COND_INITIALIZER };
void *produce(void *), *consume(void *);
/* include main */
int
main(int argc, char **argv)
{
int i, nthreads, count[MAXNTHREADS];
pthread_t tid_produce[MAXNTHREADS], tid_consume;
if (argc != 3)
err_quit("usage: prodcons5 <#items> <#threads>");
nitems = min(atoi(argv[1]), MAXNITEMS);
nthreads = min(atoi(argv[2]), MAXNTHREADS);
Set_concurrency(nthreads + 1);
/* 4create all producers and one consumer */
for (i = 0; i < nthreads; i++) {
count[i] = 0;
Pthread_create(&tid_produce[i], NULL, produce, &count[i]);
}
Pthread_create(&tid_consume, NULL, consume, NULL);
/* wait for all producers and the consumer */
for (i = 0; i < nthreads; i++) {
Pthread_join(tid_produce[i], NULL);
printf("count[%d] = %d\n", i, count[i]);
}
Pthread_join(tid_consume, NULL);
exit(0);
}
/* end main */
void *
produce(void *arg)
{
for ( ; ; ) {
Pthread_mutex_lock(&nready.mutex);
if (nready.nput >= nitems) {
Pthread_mutex_unlock(&nready.mutex);
return(NULL); /* array is full, we're done */
}
buff[nready.nput] = nready.nval;
nready.nput++;
nready.nval++;
nready.nready++;
Pthread_cond_signal(&nready.cond);
Pthread_mutex_unlock(&nready.mutex);
*((int *) arg) += 1;
}
}
/* include consume */
void *
consume(void *arg)
{
int i;
for (i = 0; i < nitems; i++) {
Pthread_mutex_lock(&nready.mutex);
while (nready.nready == 0)
Pthread_cond_wait(&nready.cond, &nready.mutex);
nready.nready--;
Pthread_mutex_unlock(&nready.mutex);
if (buff[i] != i)
printf("buff[%d] = %d\n", i, buff[i]);
}
return(NULL);
}
/* end consume */
pthread_mutex_lock(&nready.mutex);
while (nready.nready == 0)
pthread_cond_wait(&nready.cond, &nready.mutex);
nready.nready--;
pthread_mutex_unlock(&nready.mutex);
The whole point of this structure is to guarantee that the condition (nready.nready == 0) is still true when you execute the corresponding action (nready.nready--) or - if the condition is not satisfied - to wait until it is without using CPU time.
You could use a mutex only, to check that the condition is correct and to perform the corresponding action atomically. But if the condition is not satisfied, you wouldn't know what to do. Wait? Until when? Check it again? Release the mutex and re-check immediately after? That would be wasting CPU time...
pthread_cond_signal() and pthread_cond_wait() are here to solve this problem. You should check their man pages.
Briefly, what pthread_cond_wait does, is it puts the calling thread to sleep and release the mutex in an atomic way until it's signaled. So this is a blocking function. The thread can then be re-scheduled by calling signal or broadcast from a different thread. When the thread is signaled, it grabs the mutex again and exit the wait() function.
Ath this point you know that
your condition is true and
you hold the mutex.
So you can do whatever you need to do with your data.
Be careful though, you shouldn't call wait, if you're not sure that another thread will signal. This is a very common source of deadlocks.
When a thread received a signal, it's put on the list of threads that are ready to be scheduled. By the time the thread is actually executed, your condition (i.e. nread.nready == 0) may be false again. Hence the while (to recheck if the thread is waked).
"But I don't really understand the nready.nready variable"
this results from the struct instance being named 'nready' and there
is a field within the struct named 'nready'
IMO: a very poor design to have two different objects being given the same name
the nready field of the nready struct seems to be keeping track of the number of
items that have been 'produced'
1) The nready filed of struct nready is used to tack how many tasks are ready to consume, i.e., the remaining tasks in array buff. The nready.nready++; statement is only executed when producers put one new item in array buff, and the nready.nready--; is only executed when consume gets item out of buff. With is variable, programmer can always track how many tasks are there left to process.
2)
pthread_mutex_lock(&nready.mutex);
while (nready.nready == 0)
pthread_cond_wait(&nready.cond, &nready.mutex);
nready.nready--;
pthread_mutex_unlock(&nready.mutex);
The statements above are common condition variable usage. You can check
POSIX Threads Programming and Condition Variables for more about condition variables.
Why can't use mutex only? You can poll a mutex lock again and again. Obviously, it is CPU time consuming and may be hugely affect system performance. Instead, you want the consume to wait in sleep when there is no more items in buff, and to be awaken when producer puts new item in buff. The condition variable is acting as this role here. When there is no items (nready.nready==0), pthread_cond_wait() function puts the current thread into sleep, save the precious cpu time. When new items are arriving, Pthread_cond_signal() awakes the consumer.
#include <stdio.h>
#include <pthread.h>
#define MAX 10 /* maximum iterations */
int number; /* the resource */
pthread_mutex_t mu= PTHREAD_MUTEX_INITIALIZER; /* to protect the resource*/
/*
Condition variable to signal consumer that a new number is available for
consumption.
*/
pthread_cond_t sig_consumer= PTHREAD_COND_INITIALIZER;
/*
Condition variable to signal the producer that
(a) the new number has been consumed,
(b) generate another one.
*/
pthread_cond_t sig_producer= PTHREAD_COND_INITIALIZER;
void *consumer(void *dummy)
{
int printed= 0;
printf("Consumer : \"Hello I am consumer #%ld. Ready to consume numbers"
" now\"\n", pthread_self());
while (1)
{
pthread_mutex_lock(&mu);
/* Signal the producer that the consumer is ready. */
pthread_cond_signal(&sig_producer);
/* Wait for a new number. */
pthread_cond_wait(&sig_consumer, &mu);
/* Consume (print) the number. */
printf("Consumer : %d\n", number);
/* Unlock the mutex. */
pthread_mutex_unlock(&mu);
/*
If the MAX number was the last consumed number, the consumer should
stop.
*/
if (number == MAX)
{
printf("Consumer done.. !!\n");
break;
}
}
}
/**
#func producer
This function is responsible for incrementing the number and signalling the
consumer.
*/
void *producer(void *dummy)
{
printf("Producer : \"Hello I am producer #%ld. Ready to produce numbers"
" now\"\n", pthread_self());
while (1)
{
pthread_mutex_lock(&mu);
number ++;
printf("Producer : %d\n", number);
/*
Signal the consumer that a new number has been generated for its
consumption.
*/
pthread_cond_signal(&sig_consumer);
/*
Now wait for consumer to confirm. Note, expect no confirmation for
consumption of MAX from consumer.
*/
if (number != MAX)
pthread_cond_wait(&sig_producer, &mu);
/* Unlock the mutex. */
pthread_mutex_unlock(&mu);
/* Stop if MAX has been produced. */
if (number == MAX)
{
printf("Producer done.. !!\n");
break;
}
}
}
void main()
{
int rc, i;
pthread_t t[2];
number= 0;
/* Create consumer & producer threads. */
if ((rc= pthread_create(&t[0], NULL, consumer, NULL)))
printf("Error creating the consumer thread..\n");
if ((rc= pthread_create(&t[1], NULL, producer, NULL)))
printf("Error creating the producer thread..\n");
/* Wait for consumer/producer to exit. */
for (i= 0; i < 2; i ++)
pthread_join(t[i], NULL);
printf("Done..\n");
}
Question: If the consumer thread is started before the producer thread, then the program will provide the expected result, but if the producer is started first, then the consumer will start consuming from number 2 onwards; the consumer is unable consume number 1. How to correct the program even though the producer thread starts first, without introducing any additional variable or sleep?
The problem with pthread_cond_t is its name. Despite being nominally a "condition", it has no state... in particular, it does not remember that it has been signalled at all -- if you thought it might count how many times it has been signalled, you will be disappointed (for that you need a semaphore). To put that another way, if there is no pthread waiting on a condition when it is signalled, the signal has no effect and is forgotten.
The "condition" is better thought of as a "wait queue", where pthreads wait for some state to be updated. So usually you have some state, protected by the mutex. If the state is not as required for the pthread to continue, then the pthread waits on the "condition". When the state is updated, the "condition" can be signalled. When the waiter wakes up, it must check the state, and decide whether everything is now ready to continue.
Where there are two or more pthreads waiting, the standard allows pthread_cond_signal() to wake up one, two or more, or all, the waiters. The mutex ensures that the waiters' access to the state is serialised, but the waiter cannot (in general, and for this reason in particular) assume that the state is unchanged since the signal. So, it is common to write the waiter:
pthread_mutex_lock(&mutex) ;
....
while(...what we need to continue...)
pthread_cond_wait(&cond, &mutex) ;
....
pthread_mutex_unlock(&mutex) ;
Which reflects the importance of the state, and how little the "condition" contributes.
I got the below code from this website:
https://computing.llnl.gov/tutorials/pthreads/#Abstract
This simple example code demonstrates the use of several Pthread
condition variable routines. The main routine creates three threads.
Two of the threads perform work and update a "count" variable. The
third thread waits until the count variable reaches a specified value.
My question is- how does the below code ensure that one of the two worker threads doesn't lock on the mutex before the watcher thread locks on it? If this was to happen, the watcher thread would be locked out and pthread_cond_wait(&count_threshold_cv, &count_mutex) would never get called?
I am under the assumption pthread_create() actually begins the thread too. Is the only reason this works because the pthread_create() for the watcher thread begins before the pthread_create() for the two worker threads?! Surely this is not cast-iron and the scheduling could cause a worker thread to begin before the watcher thread? Even the compiler could potentially re-order these lines of code?
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#define NUM_THREADS 3
#define TCOUNT 10
#define COUNT_LIMIT 12
int count = 0;
int thread_ids[3] = {0,1,2};
pthread_mutex_t count_mutex;
pthread_cond_t count_threshold_cv;
void *inc_count(void *t)
{
int i;
long my_id = (long)t;
for (i=0; i<TCOUNT; i++) {
pthread_mutex_lock(&count_mutex);
count++;
/*
Check the value of count and signal waiting thread when condition is
reached. Note that this occurs while mutex is locked.
*/
if (count == COUNT_LIMIT) {
pthread_cond_signal(&count_threshold_cv);
printf("inc_count(): thread %ld, count = %d Threshold reached.\n",
my_id, count);
}
printf("inc_count(): thread %ld, count = %d, unlocking mutex\n",
my_id, count);
pthread_mutex_unlock(&count_mutex);
/* Do some "work" so threads can alternate on mutex lock */
sleep(1);
}
pthread_exit(NULL);
}
void *watch_count(void *t)
{
long my_id = (long)t;
printf("Starting watch_count(): thread %ld\n", my_id);
/*
Lock mutex and wait for signal. Note that the pthread_cond_wait
routine will automatically and atomically unlock mutex while it waits.
Also, note that if COUNT_LIMIT is reached before this routine is run by
the waiting thread, the loop will be skipped to prevent pthread_cond_wait
from never returning.
*/
pthread_mutex_lock(&count_mutex);
while (count<COUNT_LIMIT) {
pthread_cond_wait(&count_threshold_cv, &count_mutex);
printf("watch_count(): thread %ld Condition signal received.\n", my_id);
count += 125;
printf("watch_count(): thread %ld count now = %d.\n", my_id, count);
}
pthread_mutex_unlock(&count_mutex);
pthread_exit(NULL);
}
int main (int argc, char *argv[])
{
int i, rc;
long t1=1, t2=2, t3=3;
pthread_t threads[3];
pthread_attr_t attr;
/* Initialize mutex and condition variable objects */
pthread_mutex_init(&count_mutex, NULL);
pthread_cond_init (&count_threshold_cv, NULL);
/* For portability, explicitly create threads in a joinable state */
pthread_attr_init(&attr);
pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);
pthread_create(&threads[0], &attr, watch_count, (void *)t1);
pthread_create(&threads[1], &attr, inc_count, (void *)t2);
pthread_create(&threads[2], &attr, inc_count, (void *)t3);
/* Wait for all threads to complete */
for (i=0; i<NUM_THREADS; i++) {
pthread_join(threads[i], NULL);
}
printf ("Main(): Waited on %d threads. Done.\n", NUM_THREADS);
/* Clean up and exit */
pthread_attr_destroy(&attr);
pthread_mutex_destroy(&count_mutex);
pthread_cond_destroy(&count_threshold_cv);
pthread_exit(NULL);
}
My question is- how does the below code ensure that one of the two worker threads doesn't lock on the >mutex before the watcher thread locks on it?
The code doesn't need to ensure that. It doesn't depend on the watcher thread calling pthread_cond_wait().
The watcher thread checks count<COUNT_LIMIT, this is the actual condition the thread care about - or rather the inverse, when count >= COUNT_LIMIT - the watcher thread knows that the other threads are done.
The pthread condition variable used in pthread_cond_wait() is just needed in case the threads are not done, so the watcher thread can be put to sleep and woken up to check the condition it cares about.
That said, the example looks a tad silly, it's not quite clear what watcher thread wants to achieve by doing count += 125;
the comment in your code explains that you do not have to worry about that:
Also, note that if COUNT_LIMIT is reached before this routine is run by
the waiting thread, the loop will be skipped to prevent pthread_cond_wait
from never returning.
in fact, if you notice, the while loop is run only if COUNT_LIMIT is not already reached by count. If that is the case, the pthread_cond_signal is not called at all.
I have this code as an example where two threads are created and then it looks like a pthread_cond_wait() is used to suspend that thread until it is ready to work again by the use of pthread_cond_signal(). My question is what if multiple threads are waiting at the same time? How will executing pthread_cond_signal() pick the correct thread to wake up? Is there a way to pick a specific thread to awake? Lets say i have a producer thread that puts customer orders into seperate queues where each queue is managed by a thread. If two consumer threads are suspend with wait() because they have nothing in their queue but then the producer thread only inserts an order into ONE of the consumer queues, how the heck do we differentiate? If this is not possible then what OTHER methods can i use to accomplish what i want?
Here is an example code because stackoverflow likes code... not that relevant:
Example
#define _MULTI_THREADED
#include <pthread.h>
#include <stdio.h>
#include "check.h"
/* For safe condition variable usage, must use a boolean predicate and */
/* a mutex with the condition. */
int workToDo = 0;
pthread_cond_t cond = PTHREAD_COND_INITIALIZER;
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
#define NTHREADS 2
void *threadfunc(void *parm)
{
int rc;
while (1) {
/* Usually worker threads will loop on these operations */
rc = pthread_mutex_lock(&mutex);
checkResults("pthread_mutex_lock()\n", rc);
while (!workToDo) {
printf("Thread blocked\n");
rc = pthread_cond_wait(&cond, &mutex);
checkResults("pthread_cond_wait()\n", rc);
}
printf("Thread awake, finish work!\n");
/* Under protection of the lock, complete or remove the work */
/* from whatever worker queue we have. Here it is simply a flag */
workToDo = 0;
rc = pthread_mutex_unlock(&mutex);
checkResults("pthread_mutex_lock()\n", rc);
}
return NULL;
}
int main(int argc, char **argv)
{
int rc=0;
int i;
pthread_t threadid[NTHREADS];
printf("Enter Testcase - %s\n", argv[0]);
printf("Create %d threads\n", NTHREADS);
for(i=0; i<NTHREADS; ++i) {
rc = pthread_create(&threadid[i], NULL, threadfunc, NULL);
checkResults("pthread_create()\n", rc);
}
sleep(5); /* Sleep is not a very robust way to serialize threads */
for(i=0; i<5; ++i) {
printf("Wake up a worker, work to do...\n");
rc = pthread_mutex_lock(&mutex);
checkResults("pthread_mutex_lock()\n", rc);
/* In the real world, all the threads might be busy, and */
/* we would add work to a queue instead of simply using a flag */
/* In that case the boolean predicate might be some boolean */
/* statement like: if (the-queue-contains-work) */
if (workToDo) {
printf("Work already present, likely threads are busy\n");
}
workToDo = 1;
rc = pthread_cond_signal(&cond);
checkResults("pthread_cond_broadcast()\n", rc);
rc = pthread_mutex_unlock(&mutex);
checkResults("pthread_mutex_unlock()\n", rc);
sleep(5); /* Sleep is not a very robust way to serialize threads */
}
printf("Main completed\n");
exit(0);
return 0;
}
Output:
Enter Testcase - QP0WTEST/TPCOS0
Create 2 threads
Thread blocked
Thread blocked
Wake up a worker, work to do...
Thread awake, finish work!
Thread blocked
Wake up a worker, work to do...
Thread awake, finish work!
Thread blocked
Wake up a worker, work to do...
Thread awake, finish work!
Thread blocked
Wake up a worker, work to do...
Thread awake, finish work!
Thread blocked
Wake up a worker, work to do...
Thread awake, finish work!
Thread blocked
Main completed
In practical terms, only one thread is awakened and you can't control which one it is.
(pthread_cond_signal wakes up at least one thread waiting on the given condition variable, and the thread chosen is determined by scheduling policy.)
In your case, you need to reconsider what the "condition" represented by your condition variable (condvar) means.
If the condvar truly means "a producer has added an item to one of several queues, each of which has a dedicated consumer," then you should pthread_cond_broadcast to awaken each queue's consumer and let the awakened threads figure out if there is work to do. Alternatively, you might recast the condition as "a producer has added an item to this queue, which has a dedicated consumer," and use one condvar per queue.