I'm writing a producer-consumer thread program in C. Everything in my program is working perfectly with one major exception. When I have more than one consumer thread, which is pretty much always, only the first consumer thread will actually terminate. I've tried absolutely everything that I can think of, but the problem persists. Here's my code, with the guts of it stripped out, so that you can see just the part that is relevant.
I can see from my output that both of the termination condition variables become zero, which is of course why the first consumer thread terminates. But why don't the other consumer threads also terminate?
Thank you!
sem_t full, empty, mutex;
int threads;
int to_consume;
FILE* inputfp[5];
FILE* outputfp = NULL;
char in[BUF];
void* p(void* inpFile) {
while (fscanf(inpFile, FMTSTRING, in) > 0) {
sem_wait(&empty);
sem_wait(&mutex);
// production code here
to_consume++;
sem_post(&mutex);
sem_post(&full);
}
fclose (inpFile);
sem_wait(&mutex);
threads--;
sem_post(&mutex);
return NULL;
}
void* c() {
int continuing = 1;
while (continuing) {
sem_wait(&full);
sem_wait(&mutex);
//consumption code here
to_consume--;
fprintf("%d %d\n", threads, to_consume); //these both go to zero by the end
if ( (threads <= 0) && (to_consume <= 0) ) {
continuing = 0;
}
sem_post(&mutex);
sem_post(&empty);
}
return NULL;
}
int main (int argc, char* argv[]) {
int i;
int con_threads;
con_threads = 3;
to_consume = 0;
pthread_t *pr_thread[argc-2];
pthread_t *con_thread[2];
sem_init(&full, 0, 0);
sem_init(&empty, 0, 50);
sem_init(&mutex, 0, 1);
for (i = 0; i < (argc-2); i++) {
pr_thread[i] = (pthread_t *) malloc(sizeof(pthread_t));
inputfp[i] = fopen(argv[i+1], "r");
int rc = pthread_create (pr_thread[i], NULL, p, inputfp[i]);
sem_wait(&mutex);
threads++;
sem_post(&mutex);
}
outputfp = fopen(argv[(argc-1)], "wb");
for (i = 0; i con_threads 3; i++) {
con_thread[i] = (pthread_t *) malloc(sizeof(pthread_t));
int rc = pthread_create (con_thread[i], NULL, c, NULL);
}
for (i = 0; i < (argc - 2); i++) {
pthread_join(*pr_thread[i], 0);
free(pr_thread[i]);
}
for (i = 0; i con_threads 3; i++) {
fprintf(stderr, "About to close consumer thread %d.\n", i);
pthread_join(*res_thread[i], 0);
fprintf(stderr, "Consumer thread %d closed successfully.\n", i);
free(res_thread[i]);
}
printf ("About to close the output file.\n");
/* Close the output file */
fclose (outputfp);
return EXIT_SUCCESS;
}
I think your problem is that you don't post full again when the first consumer detects that there are no threads left, so the second consumer is waiting on full but the signal will never arrive. You may need a count of the consumers, though for a first pass (proof of concept), you can leave full with a post that is never read.
Related
I tried to simulate the producer consumer problem using bounder buffer in C and threads.
Mutex and semaphores are also used.
The expected output is to display the state of buffer every time an item is produced or consumed.
Buffer size is fixed as 10. Initially buffer items are all -1. When a producer produces an item into it, the item replaces -1.
Item in 0th index is 0 ,1st index is 1 and so on.....which doesn't matter.
The program asks the number of producers and consumer we want to create.
The production is working fine....but not consumption.
Segmentation fault arises in Thread 1.I am not sure what Thread 1 is.
I tried to debug using GDB many times....without no hope.
// Producer consumer.
#include <pthread.h>
#include <stdio.h>
#include <semaphore.h>
#include <stdlib.h>
#include <unistd.h>
#define TRUE 1
int buff_size=10,i;
int buffer[25];
pthread_mutex_t mutex;
sem_t full, empty;
int counter = 0;
int consume_count=0; //No of Consumers created
int produce_count=0; //No of Producers created
void initializeData()
{
sem_init(&full, 0, 0);
sem_init(&empty, 0, buff_size);
pthread_mutex_init(&mutex, NULL);
}
int insert_item(int counter)
{
if (counter < buff_size) {
buffer[counter] = counter;
counter++;
//produce_count++;
return 0;
}
else {
printf("\n[BUFFER FULL!]");
return -1;
}
}
int remove_item()
{
printf("\n[GOING TO REMOVE AN ITEM]\n");
if (buffer[counter-1] != -1) {
buffer[counter-1] = -1;
counter--;
//consume_count++; // Commented out...
return 0;
}
else {
printf("\n[EMPTY]\n");
return -1;
}
}
void *producer(void *arg)
{
int RET = 0;
while( TRUE ) {
sem_wait(&empty);
pthread_mutex_lock(&mutex);
RET = insert_item(counter);
if (RET){
printf("\nProducer Sleeping...zzZZ\n");
sleep(2);
}
pthread_mutex_unlock(&mutex);
sem_post(&full);
if(!RET)
printf("\n[ INSERTED ]\n" );
printf("\n");
for(i=0; i < buff_size ;i++)
printf("[%d] ",buffer[i]);
printf("\n");
sleep(3);
} // end of while...
}
void *consumer(void *arg)
{
int RET = 0;
while( TRUE ) {
sem_wait(&full);
pthread_mutex_lock(&mutex);
RET = remove_item(buffer);
if (RET){
printf("\nConsumer Sleeping\n");
sleep(3);
}
pthread_mutex_unlock(&mutex);
sem_post(&empty);
if(!RET) {
printf("\nConsumed\n");
printf("\n");
}
for(i=0 ; i < buff_size ; i++)
printf("%4d",buffer[i]);
printf("\n");
sleep(2);
} //end of while...
}
void main()
{
int produce, consume;
pthread_t *prod;//thread ID
pthread_t *cons;//thread ID
printf("\nEnter the no of producers: ");
scanf("%d",&produce);
printf("\nEnter the no of consumers: ");
scanf("%d",&consume);
putchar('\n');
for (i=0; i < buff_size; i++)
buffer[i] = -1;
for (i=0; i < buff_size; i++)
printf("[%d] ", buffer[i]);
printf("\n");
initializeData();
for (i = 0; i < produce; i++)
{
pthread_create(&prod[i], NULL, producer, NULL);
produce_count++;
}
for (i = 0; i < consume; i++)
{
pthread_create(&cons[i], NULL, consumer, NULL);
consume_count++;
printf("AAAAA");
}
/*for (i = 0; i < produce; i++)
pthread_join(producer, NULL);
for (i = 0; i < consume; i++)
pthread_join(consumer, NULL);*/
printf("\n===============\n[ PRODUCED: %d ]", produce_count);
printf("\n[ CONSUMED: %d ]\n==============", consume_count);
}
pthread_create(&prod[i], NULL, producer, NULL);
In this call pthread_create will create new thread and try to return thread id in prod[i]
But:
pthread_t *prod;//thread ID
pthread_t *cons;//thread ID
These are uninitiaised pointers, if you want to collect threadids in them, you need to allocate memory to them with malloc like:
prod = malloc(sizeof(pthread_t) * produce);
cons = malloc(sizeof(pthread_t) * consume);
Otherwise pthread_create will be storing threadid in invalid memory leading to segment fault
I am writing a multi-threaded application that reads a file and seeks for a word in chunks of it a thread has in memory.
A thread needs to asynchronously close other threads looking for that word if it is first to find it.
The problem is when a word is found and other threads are being closed the program does not terminate (in 6 out of 10 executions). I have checked in gdb that one thread does not exit. It happens even when I do not call waitforthreads(n_threads).
// [...]
FILE* f;
pthread_mutex_t mutex;
pthread_t* threads;
int n_threads;
int allread;
// [...]
int main(int argc, char* argv[]) {
// [...]
threads = (pthread_t*) calloc(n_threads, sizeof(pthread_t));
pthread_mutex_init(&mutex, NULL);
runthreads(f, word, n_threads, n_records);
waitforthreads(n_threads);
pthread_mutex_destroy(&mutex);
// [...]
}
void runthreads(FILE* f, char* w, int n_threads, int n_records) {
struct targs_t args = {w, n_records};
for (int i=0; i<n_threads; i++)
pthread_create(&threads[i], NULL, findword, (void*) &args);
}
void waitforthreads(int N) {
for (int i=0; i<N; i++)
if(pthread_join(threads[i], NULL))
exit_(6);
}
void* findword(void* arg) {
pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL);
pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, NULL);
struct targs_t* args = (struct targs_t*) arg;
int max_length = args->n_records * sizeof(record_t);
record_t* records = malloc(max_length);
int found = 0;
while (!allread && !found) {
pthread_mutex_lock(&mutex);
// allread is being set in the function below
// if the whole file has been read
readRecords((char*) records, args->n_records, f);
pthread_mutex_unlock(&mutex);
for (int i=0; i<args->n_records; i++)
if (strlen(records[i].text) == 0) break;
else if (strstr(records[i].text, args->word) != NULL) {
notifyfound(pthread_self(), records[i].id);
found = 1;
break;
}
}
free(records);
return NULL;
}
void notifyfound(pthread_t tid, int id) {
printf("Found: %d (%ld)\n", id, (long) tid);
for (int i=0; i<n_threads; i++)
if (threads[i] && !pthread_equal(threads[i], tid)) {
printf(" closing %ld\n", (long) threads[i]);
pthread_cancel(threads[i]);
}
printf(" finished closing\n");
}
This has to do with cancellation points, although the specifics are hard to come by since you haven't shared a minimal example. My diagnosis is either
6/10 times you have at least one thread waiting for a mutex, and other one in readRecords, which will cancel and not free the mutex. Setup cancellation handlers with pthread_cleanup_push and pthread_cleanup_pop which will free your mutex, and read the manual for pthread_cancel. See related pthread_cleanup_push causes Syntax error for some references.
Some of your threads are not detecting the cancellation - try using pthread_testcancel to setup a guaranteed cancellation point.
Here is some code that fixes these sorts of problems, by adding a cancellation check and mutex cleanup.
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
FILE* f;
pthread_mutex_t mutex;
pthread_t* threads;
int n_threads = 3;
int allread;
long int count = 0;
int *thread_ids;
int global_quit = 0;
#define MAX 99999
void waitforthreads(int N) {
printf("waiting for %d threads\n", N);
for (int i=0; i<N; i++)
{
printf("thread %d | %d\n", i, threads[i]);
if(pthread_join(threads[i], NULL))
{
printf("problem\n");
exit(6);
}
}
printf("done.\n");
}
void notifyfound(pthread_t tid, int count) {
printf("%d | %d got big number\n", count, pthread_self());
for (int i=0; i<n_threads; i++)
if (threads[i] && !pthread_equal(threads[i], tid)) {
printf(" closing '%ld'\n", (long) threads[i]);
pthread_cancel(threads[i]);
}
global_quit = 1;
printf(" finished closing\n");
}
void waiting_thread_cleanup(void *arg)
{
pthread_mutex_unlock((pthread_mutex_t *)arg);
}
void* do_thing(void* arg) {
pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, NULL);
pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, NULL);
int* id = (int *)arg;
int quit = 0;
while (!allread) {
pthread_mutex_lock(&mutex);
pthread_cleanup_push(waiting_thread_cleanup, (void *)&mutex); /* must be paired with pop. */
if(count++==MAX)
{
notifyfound(pthread_self(), *id);
quit=1;
}
else if(count % 10000 == 0)
printf("[%d] - %d\n", *id, count);
pthread_testcancel(); /* required to allow for the cancel to ever be 'detected' other functions are sufficient as well. */
pthread_mutex_unlock(&mutex);
pthread_cleanup_pop(1); /* if this isn't here, this will occassionally hand because the mutex isn't freed. */
if(quit==1)
{
printf("%d | %d quitting\n", *id, pthread_self());
break;
}
}
return NULL;
}
void runthreads(FILE* f, int n_threads) {
for (int i=0; i<n_threads; i++)
pthread_create(&threads[i], NULL, do_thing, &(thread_ids[i]));
}
int main(int argc, char* argv[]) {
threads = (pthread_t*) calloc(n_threads, sizeof(pthread_t));
thread_ids = (int*) calloc(n_threads, sizeof(int));
for(int i=0;i<n_threads;i++)
thread_ids[i] = i;
pthread_mutex_init(&mutex, NULL);
runthreads(f, n_threads);
waitforthreads(n_threads);
pthread_mutex_destroy(&mutex);
}
I'm working with 1 Producer and N Consumers where the number of consumers is taken from prompt argument.
The problem I'm facing is: My Producer thread is ending BEFORE the consumers are over. So the Producer does NOT give the sign to consumer and it's getting stuck forever...
My Code:
void* producerFunc(void* arg)
{
while(n_insertions < N_PRODUCTS)
{
sem_wait(&sem_postAvaliable);
sem_wait(&mutex);
x = produce_item();
insert_buffer(x);
sem_post(&mutex);
sem_post(&sem_posTaken);
}
pthread_exit(NULL);
}
void* consumerFunc(void* arg)
{
struct sConsumer* cons = (struct sConsumer*) arg;
while(n_consumed < N_PRODUCTS)
{
sem_wait(&sem_posTaken);//Here is where thread consumer get stuck.
//Because producer thread is finished and cant give a post in this semaphore.
sem_wait(&mutex);
remove_buffer();
sem_post(&mutex);
sem_post(&sem_postAvaliable);
}
pthread_exit(NULL);
}
int main(int argc, char const *argv[])
{
if(argc > 1)
{
int i, numConsumers = atoi(argv[1]);
pthread_t tConsumers[numConsumers], producer;
sem_init(&sem_posTaken, 0, 0);
sem_init(&sem_postAvaliable, 0, Buffer_Size);
sem_init(&mutex, 0, 1);
for(i=0; i<numConsumers; i++)
{
struct sConsumer* cons = (struct sConsumer*) malloc(sizeof(*cons));
cons->id = i;
pthread_create(&tConsumers[i], NULL, consumerFunc, (void*) cons);
}
pthread_create(&producer, NULL, producerFunc, NULL);
for(i=0; i<numConsumers; i++)
{
pthread_join(tConsumers[i], NULL);
}
pthread_join(producer, NULL);
}
else
printf("Por favor. Informe o nĂºmero de consumidores\n");
return 0;
}
I tested with only 2 consumers and already found this problem. I'd like some idea/tip how may I fix it. Maybe the while loop ?
The entire implementation of the critical section using semaphore is what i tried to implement.
The entire code using semaphores is shown:
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <semaphore.h>
int buf[256];
int in = 0;
int out = 0;
sem_t full;
sem_t empty;
sem_t mutex;
int buf_size;
int counter = 0;
void *producer(void *arg)
{
int i, item, *index;
index = (int *)arg;
for (i = 0;; i++)
{
item = 1000 + i;
sem_wait(&empty);
sem_wait(&mutex);
buf[in] = item;
in = (in + 1) % (*index);
counter++;
printf("\n%d [P%d] ", item, *index);
sem_post(&mutex);
sem_post(&full);
/* if (i % 5 == 0)
sleep(1); */
}
}
void *consumer(void *arg)
{
int i, item, *index;
index = (int *)arg;
for (i = 0;;i++)
{
sem_wait(&full);
sem_wait(&mutex);
item = buf[out];
out = (out + 1) % (*index);
counter--;
printf("\n%d [C%d] ", item, *index);
sem_post(&mutex);
sem_post(&empty);
/* if (i % 5 == 0)
sleep(1); */
}
}
int main()
{
int produce, consume;
int i;
printf("\nThe Buffer Size:");
scanf("%d", &buf_size);
printf("\nThe Producer:");
scanf("%d", &produce);
printf("\nThe Consumer:");
scanf("%d", &consume);
pthread_t prod, cons;
void* exit_status;
sem_init(&full, 0, 0);
sem_init(&empty, 0, buf_size);
sem_init(&mutex, 0, 1);
for (i = 0; i < produce; i++)
{
pthread_create(&prod, NULL, producer, &i);
}
for (i = 0; i < consume; i++)
{
pthread_create(&cons, NULL, consumer, &i);
}
pthread_join(prod, &exit_status);
pthread_join(cons, &exit_status);
// pthread_exit(NULL);
return 0;
}
The code compiles without error but the output is nothing but the inputs that it takes and then thit just shows 1000 and p1 1001 p2 and values like that
You create your first producer and then immediately wait for it to terminate with join(), so blocking your main thread. That producer runs in its for loop, pushing the queue until the 'empty' semaphore has no more units and the producer then blocks.
The main and one producer thread are now both stuck. The other producers, and all the consumers, are never started. No consumers means nothing to pop from the queue, so the whole lot is deadlocked.
This is not untypical when using join().
In my main function, I spawn j threads which all compute the same task in parallel -- and then I want to wait for them to finish before exiting.
int main(...) {
// ...
int threads = 6;
pthread_t* thread = malloc(sizeof(pthread_t)*threads);
for(i = 0; i < threads; i++) {
struct thread_param *tp;
tp = malloc(sizeof(*tp));
// ...
int ret = pthread_create(&thread[i], NULL, &control, (void*)tp);
if(ret != 0) {
printf ("Create pthread error!\n");
exit (1);
}
}
for (j = 0; j < threads; j++) {
printf("JOINING THREAD: %i\n", j);
pthread_join( &thread[j], NULL);
}
exit(0);
}
However, nothing waits. Main just exits without ever completing the threaded tasks. Am I missing something?
hey, try pthread_join( thread[j], NULL); i think the problem is with types. I checked docs:
int pthread_join(pthread_t thread, void **value_ptr);
and thread is p_thread*, and thread[j] is p_thread, while &thread[j] is p_thread*, which is invalid. There just might internal error happen.
Edit: Yeah, I am very positive with that, pthread_t is basically int, so pthread_t* is accepted, it is just invalid thread handle, so pthread_join fails internally.