C pthread allow only four threads to execute function - c

Question

Here is a problem, say I need to execute a function x times which does some taks, but only four threads can be executing it at any given time. So thread A,B,C,D can start task 0,1,2,3 respectively. However, task four can't start until one of the threads completed, so say if thread A completes, then the next task can be executed by one of the free threads. This should repeat x times, where x is the number of times the function needs to be called.
So I've used semaphores and join the pthread after it completes to ensure it completes. However, sometimes the main function finishes executing before some of the threads complete, and valgrind is complaining that my pthread_create is leaking memory. I think the way I'm doing is incorrect or is a naive approach, so any guidance or example code to fix this will be most appreciated! Here is my code:
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <time.h>
#include <pthread.h>
#include <semaphore.h>
sem_t s;
typedef struct Data Data;
struct Data {
pthread_t* a;
int index;
int j;
};
void* someFunction(void* arg){
/* Only at most num_threads should be here at once; */
sem_wait(&s);
Data* d = arg;
printf("Successfully completed task %d with thread %d\n", d->index, d->j);
sleep(2);
pthread_t* z = d->a;
free(d);
pthread_join(*z, NULL);
sem_post(&s);
return 0;
}
int main(void){
int num_task = 15; // i need to call someFunction() 9000 times
int num_threads = 4;
int j = 0;
sem_init(&s, 0, num_threads);
pthread_t thread_ids[num_threads];
for (int i = 0; i < num_task; i ++){
/*NEED TO COMPLETE num_tasks using four threads;
4 threads can run someFunction() at the same time; so one all four are currently executing someFunction(), other threads can't enter until one has completed. */
if (j == num_threads){
j = 0; // j goes 0 1 2 3 0 1 2 3 ...
}
Data* a = malloc(sizeof(Data));
a->a = thread_ids + j;
a->index = i;
a->j = j;
sem_wait(&s);
pthread_create(thread_ids + j, NULL, someFunction, a);
sem_post(&s);
j ++;
}
return 0;
}
Thank you so much

Answer 1

Having threads wait for each other usually gets messy quickly, and you're likely to end up in situations where a thread tries to join itself, or is never joined.
The most reliable way to have at most four threads running is to only create four threads.
Instead of creating threads as needed, you let each thread (potentially) perform more than one task.
You can separate the "task" concept from the "thread" concept:
Make a queue of tasks for the threads to perform.
Create four threads.
Each thread takes a task from the queue and performs it, repeating until the queue is empty.
Wait for the threads to finish in main.
The only thing that needs synchronising is the removal of a task from the queue, which is very simple.
(If the tasks are not independent, you need more complex plumbing.)
Pseudocode (I have invented some names as I'm not overly familiar with pthreads):
typedef struct Task
{
/* whatever */
};
/* Very simplistic queue structure. */
typedef struct Queue
{
mutex lock;
int head;
Task tasks[num_tasks];
};
/* Return front of queue; NULL if empty. */
Task* dequeue(Queue* q)
{
Task* t = NULL;
lock_mutex(q->lock);
if (q->head < num_tasks)
{
t = &q->tasks[q->head];
q->head++;
}
unlock_mutex(q->lock);
return t;
}
/* The thread function is completely unaware of any multithreading
and can be used in a single-threaded program while debugging. */
void* process(void* arg)
{
Queue* queue = (Queue*) arg;
for (;;)
{
Task* t = dequeue(queue);
if (!t)
{
/* Done. */
return NULL;
}
/* Perform task t */
}
}
/* main is very simple - set up tasks, launch threads, wait for threads.
No signalling, no memory allocation. */
int main(void)
{
pthread threads[num_threads];
Queue q;
q.head = 0;
/* Fill in q.tasks... */
/* Initialise q.lock... */
for (int ti = 0; ti < num_threads; ti++)
{
pthread_create(threads + ti, NULL, process, &q);
}
for (int ti = 0; ti < num_threads; ti++)
{
/* join the thread */
}
return 0;
}

Answer 2

Your code starts four threads at a time and waits until they are finished. However, your main loop only creates the threads, it does not for them to exit.
After you created a thread your OS will schedul it whenever it wants.
That means you have to join the last four threads you created after your for loop. So they have a chance to finish their work and free their memory.
Regards