I'm trying to implement a thread-safe queue based on a fixed array. The queue holds an array of void pointers.
pthread_t a;
pthread_t b;
Queue *q;
Queue* queue_new(int size)
{
// malloc stuff
}
void queue_put(Queue* q, void* item)
{
pthread_mutex_lock(&(q->lock));
// details on array queue managment
pthread_mutex_unlock(&(q->lock));
}
void* queue_get(Queue* q)
{
pthread_mutex_lock(&(q->lock));
// more details ...
return q->queue[old_front];
pthread_mutex_unlock(&(q->lock));
}
void *func_a (void *ptr)
{
void *ptr1 = malloc(sizeof(int));
*((int*)ptr1) = 5;
queue_put(q, ptr1);
void *ptr2 = malloc(sizeof(int));
*((int*)ptr2) = 4;
queue_put(q, ptr2);
return NULL;
}
void *func_b (void *ptr)
{
void *ptr3 = malloc(sizeof(int));
*((int*)ptr3) = 7;
queue_put(q, ptr3);
queue_get(q); // critical part !
return NULL;
}
int main ()
{
q = queue_new(3);
pthread_create(&a, NULL, func_a, NULL);
pthread_create(&b, NULL, func_b, NULL);
pthread_join(a, NULL);
pthread_join(b, NULL);
queue_print(q);
return 0;
}
I think this a pretty straight-forward approach. Unfortunately, the program freezes. However, when I remove the queue_get(q); in func_b it works just fine. I think this must be a deadlock of some kind. Any ideas? The non-threadsafe version of the Queue has been tested and works just fine. The code is hidden for clarity. Any ideas?
You'll want to move the unlock-line in queue_get above the return, as of now it is never reached. So the lock is never released.
pthread_mutex_unlock(&(q->lock));
return q->queue[old_front];
or, what you'll probably want, to avoid touching it outside a lock:
void * ret = q->queue[old_front];
pthread_mutex_unlock(&(q->lock));
return ret;
(From a more stylistic point: your value-allocating would be much 'cleaner' this way:
int * ptr1 = malloc(sizeof(*ptr1));
*ptr1 = 5;
queue_put(q, ptr1);
note the lack of casts needed)
I believe the problem lies within queue_get. You are returning before the mutex is unlocked. Try storing the return value into a temperary variable, unlock the mutex, then return the value.
void* queue_get(Queue* q)
{
void* temp;
pthread_mutex_lock(&(q->lock));
// more details ...
temp = q->queue[old_front];
pthread_mutex_unlock(&(q->lock));
return temp;
}
You are returning before you unlock the mutex in queue_get:
return q->queue[old_front];
pthread_mutex_unlock(&(q->lock));
This should be:
void *retValue = q->queue[old_front];
pthread_mutex_unlock(&(q->lock));
return retValue;
Related
i need to create a thread pool it,it works but in function do_work that the function pthread_create calls i have problem in the free (memory leak) just when calling pthread_exit()
*in function create threadpool i just initilaize the struct and call
function do work *
void* do_work(void* p)
{
threadpool* pool = (threadpool*)p;
work_t* work;
while(1)
{
pthread_mutex_lock(&pool->qlock);
if(pool->shutdown == 1)
{
pthread_mutex_unlock(&pool->qlock);
//pthread_exit(EXIT_SUCCESS);// here is the free problem when deleting it all good
return NULL;
}
while(!pool->qsize)
{
if(pthread_cond_wait(&pool->q_not_empty,&pool->qlock))
perror("pthread_cond_wait\n");
if(pool->shutdown)
break;
}
//Check if the system is shutting down
if(pool->shutdown == 1)
{
pthread_mutex_unlock(&pool->qlock);
//pthread_exit(EXIT_SUCCESS);y
return NULL;
}
work = pool->qhead; //set the cur variable.
pool->qsize--; //decriment the size.
if(pool->qsize == 0) {
pool->qhead = NULL;
pool->qtail = NULL;
}
else {
pool->qhead = work->next;
}
if(pool->qsize == 0 && ! pool->shutdown) {
//the q is empty again, now signal that its empty.
pthread_cond_signal(&(pool->q_empty));
}
pthread_mutex_unlock(&(pool->qlock));
(work->routine) (work->arg); //actually do work.
free(work);
}
}
The pattern looks like a fairly standard thread pool:
typedef struct {
pthread_mutex_t qlock;
pthread_cond_t q_not_empty;
volatile work_t *qhead;
volatile work_t *qtail;
size_t qsize; /* Not needed */
volatile int shutdown;
} threadpool;
Properly indenting OP's code would make it more readable.
However, the implementation looks odd. I would expect
void *do_work(void *poolptr)
{
threadpool *const pool = poolptr;
work_t *work;
pthread_mutex_lock(&(pool->qlock));
while (!(pool->shutdown)) {
if (!(pool->qhead)) {
/* Queue empty */
pthread_cond_wait(&(pool->q_not_empty), &(pool->qlock));
continue;
}
work = pool->qhead;
pool->qhead = work->next;
if (!pool->qhead)
pool->qtail = NULL;
work->next = NULL;
pthread_unlock(&(pool->qlock));
work->process(work);
pthread_lock(&(pool->qlock));
}
pthread_mutex_unlock(&(pool->qlock));
return (void *)0;
}
and the code that appends a new work item to the queue to be
void append(threadpool *pool, work_t *work)
{
work->next = NULL;
pthread_mutex_lock(&(pool->qlock));
if (pool->qtail) {
pool->qtail->next = work;
pool->qtail = work;
} else {
pool->qhead = work;
pool->qtail = work;
}
pthread_cond_signal(&(pool->q_not_empty));
pthread_mutex_unlock(&(pool->qlock));
}
It is difficult to say where OP's implementation leaks memory. The most likely candidate is OP's arg member in each work_t, if it is dynamically allocated.
My implementation above passes the entire work_t to the routine function, which I renamed to process. It is responsible for freeing the work structure, too. The minimal definition for the work structure is
typedef struct work_t {
struct work_t *next;
void (*process)(struct work_t *);
/* Optional other fields */
} work_t;
Other possible causes for the memory leaks is if the qsize member is not properly updated everywhere. Because there is no use for it, really, I just omit it altogether.
The simpler the code, the easier it is to avoid bugs.
I'm looking for most convenient/best/smartest way for multiple memory deallocation. Not quite sure if that is best explanation out there, but here is an example that will show what i want to do:
void func()
{
int* ptr1 = malloc(1);
if(ptr1 == NULL)
{
//...
}
int* ptr2 = malloc(1);
if(ptr2 == NULL)
{
//...
free(ptr1);
}
int* ptr3 = malloc(1);
if(ptr3 == NULL)
{
//...
free(ptr2);
free(ptr1);
}
//code...
}
Only thing that comes up to my mind is array filled with flags, if the flag is raised particular memory has to be freed. Is there any other more convenient way to do this? You can imagine how many times i need to repeat free() if there is need for more malloc()-ing.
What you posted is a common practice for error handling and resource release in a function, you acquire several resources and in case any error occurred, you need to release those that have been acquired prior, there is nothing wrong, just do it one by one.
void func(void) {
void *ptr1 = NULL;
void *ptr2 = NULL;
void *ptr3 = NULL;
ptr1 = malloc(SIZE);
if (!ptr1) goto end;
ptr2 = malloc(SIZE);
if (!ptr2) goto end;
ptr3 = malloc(SIZE);
if (!ptr3) goto end;
// do your work with the buffers
end:
free(ptr1);
free(ptr2);
free(ptr3);
}
You can use an array of pointers and keep count of the number of mallocs done. And use a common free function to free them all. Like,
void func()
{
char* ptr[10];
int n = 0, i;
for(i = 0; i < 10; i++)
ptr[i] = NULL;
ptr[n] = malloc(1);
if(ptr[n] == NULL)
{
//...
}
n++;
ptr[n] = malloc(1);
if(ptr[n] == NULL)
{
//...
custom_free(ptr1, n);
}
n++;
ptr[n] = malloc(1);
if(ptr[n] == NULL)
{
//...
custom_free(ptr, n);
}
n++;
//code...
}
And the custom_free() can be something like,
void custom_free(char* ptr[], int n)
{
int i;
for(i = 0; i <= n; i++)
free(ptr[i]);
}
An alternative way is to allocate all needed memory in one big chunk,
and treat parts of that as p0,p1,p2:
void worker(void)
{
#define N_ELEM 123
int *work_mem;
int *p0,*p1,*p2;
work_mem = malloc ( 3* N_ELEM * sizeof *work_mem);
if (!work_mem) { OMG(); return; }
p0 = work_mem, p1= work_mem + N_ELEM, p2 = work_mem + 2 * N_ELEM;
/* do useful stuff here with work_mem
** , possibly via p0,p1,p2
*/
free(work_mem);
}
I am writing a unit test that involves running multiple threads and I ran into a memory access issue that I can't seem to understand.
Here is the original (pseudo) code:
void thread_func(void * err) {
/*foo will return an allocated error_type if error occurs else NULL*/
err = (void *) foo(...)
pthread_exit(NULL);
}
void unit_test() {
int i = 0;
err_type *err_array[10];
pthread_t threads[10];
for (i = 0; i < 10; i++) {
pthread_create(&(threads[i]), NULL, thread_func, (void *) err_array[i]);
}
for(i = 0; i < 10; i++) {
pthread_join(threads[i], NULL);
ASSERT_NULL(err_array[i]);
}
}
What I am confused about is that all the threads will return NULL (checked with a gdb), however err_array[1] and err_array[5] will be NOT NULL. And instead of a valid err_type, they will contain garbage. In the case of err_array[1] will contain a string of the unit_test file path, and err_array[5] will contain a bunch of access out of bounds addresses.
A work-around I've found is to use a global err_array, and pass in the index for each element to the thread. As well as initializing all the elements of the array to NULL.
My question is why does the above 2 methods work, and not the original code?
The err variable is local to thread_func. It goes out of scope when thread_func returns. You need to pass the thread a pointer to the thing you want it to modify, not the current value of the thing you want it to modify.
So:
void thread_func(void ** err) {
/*foo will return an allocated error_type if error occurs else NULL*/
*err = (void *) foo(...)
pthread_exit(NULL);
}
And:
pthread_create(&(threads[i]), NULL, thread_func, (void **) &err_array[i]);
I'm trying to create a thread library. MyThreadCreate returns an object of type MyThread. E.g.:
struct MyThread
{
ucontext_t Context;
ThreadStatus Status;
int ThreadId;
};
typedef struct MyThread MyThread;
void * MyThreadCreate(void(*start_funct)(void *), void *args)
{
MyThread temp;
char *stack;
stack = (char *)malloc(8192*sizeof(char));
ucontext_t tempContext;
if (getcontext(&tempContext) == -1)
temp->ThreadId = 0;
tempContext.uc_stack.ss_sp = stack;
tempContext.uc_stack.ss_size = 8192*sizeof(char);
tempContext.uc_link = NULL;
makecontext(&tempContext,(void(*)(void))start_funct,1, args);
temp->Context = tempContext;
temp->Status = READY;
temp->ParentThread = currentThread;
temp->ThreadId = NEW_THREAD_ID++;
return temp;
}
In my client.c, I call as follows.
MyThread T;
T = (MyThread)MyThreadCreate(t0, (void *)n2);
MyThreadJoin(T);
In MyThreadJoin, when I check the value of threadId of T, i get a random value.
void MyThreadJoin(MyThread thread); //defintion for MyThreadJoin
Updated part:
When I try to return an object named MyThread, i get a segmentation fault immediately after I call MyThreadCreate.
Also, note that I'm including a header file(I cant change that) which contains the following definition.
typedef void *MyThread
So, the code works only when I return void * for MyThreadCreate and not when MyThread. But even though the code works fine, I'm not able to get the threadId in this case.
Can some1 tell me where I've gone wrong.
When I tried to keep the return value of MyTHreadCreate as MyThread, it throws segmentation fault. So I made it into void * and i'm able to get the object and then use it to call a MyThreadJoin, but there I get a junk value for MyThreadId. What is that I'm missing.
The problem boils down to:
void * MyThreadCreate(...)
{
MyThread temp;
// manipulate temp
return temp;
}
Outside code is only expecting a void * on the stack so only the first sizeof(void *) bytes of temp are preserved. Try to access anything beyond that and you're getting random garbage on the stack.
Instead, your function should return MyThread like this:
MyThread MyThreadCreate(...)
{
MyThread temp;
// manipulate temp
return temp;
}
Or, if you must return a void * to interface with some other code, then you must allocate the return value on the heap:
void * MyThreadCreate(...)
{
MyThread *temp = malloc(sizeof(MyThread));
// manipulate temp
return temp;
}
But that is a memory leak waiting to happen, so I would return the struct by value if possible.
I have a worker thread processing a queue of work items. I just implemented a second worker that process the items which were inserted in worker1. However, I came across some Invalid reads while using Valgrind.
I'm assuming this is because struct foo that I pass to worker2() is freed at some point in the main thread. Essentially struct foo is a struct that constantly gets updated (malloc/free), however, I'd like worker2 to insert some missing items into foo.
My question is: is it possible for worker2 to stop processing as soon as struct foo is NULL? and start again when create_foo() is called? I'm not sure what would be the best approach to insert the missing items into foo with a thread? Any feedback is appreciated.
//canonical form
//producer
void push_into_queue(char *item)
{
pthread_mutex_lock(&queueMutex);
if (workQueue.full) { // full }
else
{
add_item_into_queue(item);
pthread_cond_signal(&queueSignalPush);
}
pthread_mutex_unlock(&queueMutex);
}
}
// consumer1
void *worker1(void *arg)
{
while (true) {
pthread_mutex_lock(&queueMutex);
while (workQueue.empty)
pthread_cond_wait(&queueSignalPush, &queueMutex);
item = workQueue.front; // pop from queue
add_item_into_list(item);
pthread_cond_broadcast(&queueSignalPop);
pthread_mutex_unlock(&queueMutex);
}
return NULL;
}
pthread_create(&thread1, NULL, (void *) &worker, NULL);
// consumer2
void *worker2(void *arg)
{
my_struct *foo = (my_struct *) arg;
while (true) {
pthread_mutex_lock(&queueMutex);
while (list.empty)
pthread_cond_wait(&queueSignalPop, &queueMutex);
for (i = 0; i < list.size; i++)
insert_item_into_foo(list[i].item, foo);
pthread_cond_broadcast(&queueSignalPop);
pthread_mutex_unlock(&queueMutex);
}
return NULL;
}
void create_foo()
{
my_struct *foo = calloc(10, sizeof(my_struct));
pthread_create(&thread2, NULL, (void *) &worker2, foo);
}
void free_foo()
{
pthread_mutex_lock(&queueMutex);
int i;
for (i=0; i<5; i++)
free(foo[i].list->string);
free(foo[i].list);
free(foo);
pthread_mutex_unlock(&queueMutex);
}
You did not define any terminating condition for both worker1 and worker2. I suppose that the eol of foo could be considered as such. This means that both workers must monitor the existence of foo by owning a reference to it (ie. a foo **).
void *worker2(void *arg)
{
my_struct **foo = (my_struct **) arg;
while(true) {
pthread_mutex_lock(&queueMutex);
while (list.empty)
pthread_cond_wait(&queueSignalPop, &queueMutex);
if (NULL == *foo)
break;
for (i = 0; i < list.size; i++)
insert_item_into_foo(list[i].item, *foo);
pthread_cond_broadcast(&queueSignalPop);
pthread_mutex_unlock(&queueMutex);
}
free(foo);
return NULL;
}
void create_foo()
{
my_struct *foo = calloc(10, sizeof(my_struct ));
my_struct **foo_ptr = malloc(1, sizeof(my_struct *));
*foo_ptr = foo;
pthread_create(&thread2, NULL, (void *) &worker2, foo_ptr);
// more work with foo
}
Note that somehow foo must be assigned to a different variable so as to be reachable in free_foo (your code supposes this fact without explicitely showing it - hence my comment at the end of create_foo).
With the code above, each instance of worker2 owns a pointer to rely on for its whole lifetime, and which it must take care of before exiting.
Update:
Perhaps a better solution consists in passing a struct to thread2, which contains the foo pointer, as well as a flag indicating if that pointer is still valid. You may add any other piece of information needed by the thread in the struct ad lib.
struct th2_data {
enum {RUNNING, TERMINATING} state;
my_struct *foo;
};
Then allocate a instance of that struct, initialize it as {RUNNING, foo}, and pass it to thread2. Keep a copy of its address somewhere to be able to signal the TERMINATING state to thread2. Indeed, as you asked in your comments, you would have to replace the if (NULL == *foo) test in thread2 by if (foo.state == TERMINATING).
Make foo global and add some pointer check in the loop.
Next time when you call create_foo, it will restart the thread.
my_struct *foo = NULL;
// consumer2
void *worker2(void *arg)
{
while (true) {
if ( fool == NULL )
return;
pthread_mutex_lock(&queueMutex);
while (list.empty)
pthread_cond_wait(&queueSignalPop, &queueMutex);
for (i = 0; i < list.size; i++)
insert_item_into_foo(list[i].item, foo);
pthread_cond_broadcast(&queueSignalPop);
pthread_mutex_unlock(&queueMutex);
}
return NULL;
}