pthread queue in c - c

I am new to using threads, and I thought a good excercise would be to write a queue that works with threads. However, something is wrong, I suspect that the consumer threads try to access the same data or something. I use mutexes but I probably has misunderstood something...
I get this error (about every third run):
*** glibc detected *** ./t_queue_test: double free or corruption (fasttop): 0x0000000002114610 ***
The code is rather long so I've posted it on pastebin, but if thats incorrect, I can paste it here.
t_queue.h -> http://pastebin.com/2KYmujeE
t_queue.c -> http://pastebin.com/1wZPMwDB
t_queue_test.c -> http://pastebin.com/QKCTQWaf
I thinks the error occures in function 'get_q', and I've marked it in the code.
Thanks for any pointers or suggestions. I've digged around stackoverflow for similar questions, and I will dig some more! Valgrind doesn't show anything yet either.

Just a note for later in case pastebin goes away; the faulty code is this;
if(q->rear != NULL && q->front != NULL)
{
node_n = q->front;
*d = node_n->data;
q->front = node_n->next;
free(node_n);
}
It cleans up the front quite ok, however if the last element is removed, rear also needs to be updated to reflect the queue being empty. For example, this will do it;
if(q->rear != NULL && q->front != NULL)
{
node_n = q->front;
*d = node_n->data;
q->front = node_n->next;
if(q->front == NULL)
q->rear = NULL;
free(node_n);
}

Related

Popping stack doesn't work right? (Stack from two queues)

I've been stewing over this one for a while, and I can't quite seem to figure out why at execution, when once the following function executes, and the test code in main calls to see if the stack is empty, for some reason it isn't. I can't seem to figure out the exact cause, though I have a feeling it has something to do with the "remove" function not deleting the last node, but for some reason, I can't figure out how to fix it.
TYPE listStackPop (struct Stack* stack)
{
/* FIXME: You will write this function */
assert (stack != NULL);
assert (!listQueueIsEmpty (stack->q1));
return listQueueRemoveFront (stack->q1);
}
---later calls to listQueueRemoveFront---
TYPE listQueueRemoveFront (struct Queue* queue)
{
/* FIXME: You will write this function */
assert (queue != 0);
assert (!listQueueIsEmpty (queue));
struct Link* toDelete;
toDelete = queue->head->next;
if (toDelete == queue->tail) {
queue->tail = queue->head;
}
else {
queue->head->next = toDelete->next;
}
int retVal = toDelete->value;
return retVal;
}
--For clarity, TYPE is defined as int--
I've tried going from simply return toDelete->value to copying toDelete's value to an int, and passing that int to be returned, as I thought maybe toDelete was being removed prematurely or something, but that isn't the case.
Unfortunately, google doesn't really have much info on this either. So far anyway.
Here is the full code on pastebin, in case anyone is interested to read it in full: https://pastebin.com/cDvdHmTu
I had expected it to pass, but for some reason, it failed even though all the other test cases passed.
results:
-------------------------------------------------
---- Testing stack from queue implementation ----
-------------------------------------------------
stack init...
stackIsEmpty == 1: PASSED
pushing 4, 5, -300...
stackIsEmpty == 0: PASSED
popping; val == -300: PASSED
popping; val == 5: PASSED
top val == 4 : PASSED
popping; val == 4: PASSED
stackIsEmpty == 1: FAILED
pushing 0-9...
top val == 9 : PASSED
C:\Users\Zedri\source\repos\Stack From Queues\Debug\Stack From Queues.exe (process 8928) exited with code 0.
Press any key to close this window . . .
Compiler/IDE Used: Visual Studio 2019
-~-~-~Edit 1~-~-~-
removed malloc call to the toDelete pointer
~-~-~-Edit 2-~-~-~
Fixed code on pastebin. Issue resolved.
Never mind, I simply needed to move queue->head->next = toDelete->next; outside of the else block, and remove the else block entirely.

How do we handle freeing a BST when malloc fails in the middle of our recursive build?

I've done some looking around and can't really find a good source that even addresses the idea.
First: It's well known that we should always check if malloc() and realloc() return null. This is commonly done in some way similar to:
Word* temp;
if ((temp = (Word*)malloc(sizeof(Word))) == NULL) {
fprintf(stderr, "unable to malloc for node.\n");
exit(EXIT_FAILURE);
}
However, we also generally build binary search trees in a recursive manner, like so:
void buildTree(Word** tree, const char* input) {
//See if we have found the spot to insert the node.
//Do this by checking for NULL
if (!(*tree)) {
*tree = createNode(input);
return;
}
//else, move left or right accordingly.
if (strcmp(input, (*tree)->data) < 0)
buildTree(&(*tree)->left, input);
else
buildTree(&(*tree)->right, input);
return;
}
So, what do we do if we start working with massive data sets and malloc() fails to allocate memory in the middle of that recursive buildTree function? I've tried a number of things from keeping track of a "global error" flag and a "global head" node pointer and it just seems to be more and more messy the more I try. Examples working with building BSTs rarely seem to give any thought to malloc() failing, so they aren't really helpful in this regard.
I can logically see that one answer is "Don't use recursion and return the head of the tree each time." and while I can see why that would work, I'm an undergraduate TA and one of the things we use BSTs to teach is recursion. So, saying "don't use recursion" to my students when we are TEACHING recursion would be self-defeating.
Any thoughts, suggestions, or links would be greatly appreciated.
We usually use a return error and let the caller free it, after all it could very well free other non critical resources and try to insert the node again.
#define BUILD_OK 0
#define BUILD_FAILED 1
int buildTree(Word** tree, const char* input) {
int res;
//See if we have found the spot to insert the node.
//Do this by checking for NULL
if (!(*tree)) {
if (!(*tree = createNode(input)))
return BUILD_FAILED;
//Maybe other checks
return BUILD_OK;
}
//else, move left or right accordingly.
if (strcmp(input, (*tree)->data) < 0)
res = buildTree(&(*tree)->left, input);
else
res = buildTree(&(*tree)->right, input);
return res;
}

Whether code is read from top to bottom

I am creating a program in c which is based on a linked list, where every node (of struct) holds an integer and a pointer to the next node.
I use dynamic allocation (malloc) and deallocation (free) as new nodes are added and old nodes are deleted.
when a node is deleted a function named delete is called.
I discovered that the program crashes sometimes when this delete-function is called and I KNOW that its something with the pointers in the method but I dont know WHERE in the code (row number) and WHY this happends.
I am used to high-level languages such as Java and I am used to encircle the problem by putting print-syntax at certain places in the method just to reveal WHERE it crashes.
I thought I could do the same with c and with pointer because to my knowledge I beleive the code is read from top to bottom that is 1, 2, 3, 4, and so on. (maybe interrupt handlers behave another way?)
So in this function named delete I have gone so far by putting this printf() at the very beginning of the delete-function - and all the same the program crashes.
So my Question - is it really possible that its some syntax in the delete-function (when I loop pointers for instance) that causes the crash WHEN not even the printf() is printing?
Am I wrong when I believe that the program is executed from to to bottom - that is 1, 2, 3 ....
You can se my printf-function in the very beginning of delete-function
And by the way - how could I solve this problem when I get this cryptic crash message from windows? See the bitmap!!
Greatful for answers!!!
int delete(int data) {
printf("IN THE BEGINNING OF DELETE!!!");
int result = 0;
if (queueref.last != NULL) {
node *curr_ptr;
node *prev_ptr;
node *temp_ptr;
if (queueref.first->data == data) {
temp_ptr = queueref.first;
queueref.first = queueref.first->next;
destroy_node(temp_ptr);
result = 1;
if (queueref.first == NULL) {
queueref.last = NULL;
puts("queue is now empty!!!");
}
} else {
prev_ptr = queueref.first;
curr_ptr = queueref.first->next;
printf("prev_ptr: %d\n", prev_ptr);
printf("curr_ptr: %d\n", curr_ptr);
while(curr_ptr != NULL) {
if (curr_ptr->data == data) {
result = 1;
if (curr_ptr->next != NULL) {
temp_ptr = curr_ptr;
destroy_node(temp_ptr);
prev_ptr->next = curr_ptr->next;
} else {
temp_ptr = curr_ptr;
queueref.last = prev_ptr;
prev_ptr->next = NULL;
destroy_node(temp_ptr);
}
}
curr_ptr = curr_ptr->next;
prev_ptr = prev_ptr->next;
}
}
}
return result;
}
Common mistake, here's the deal. This
printf("IN THE BEGINNING OF DELETE!!!");
needs to be
printf("IN THE BEGINNING OF DELETE!!!\n");
^^ note the newline
The reason is because stdio does not flush stdout until it sees a newline. If you add that newline, you should see the printf when the code enters the function. Without it, the program could crash, the stdout buffer would not have been flushed and would not see the printf.
Your code seems to have lots of implementation flaws. As a general advice I would recommend using some standard well-tested queue support library and static code analyzers (in this case you would even find dynamic analyzer valgrind very helpful, I guess).
For example, if implementation of destroy_node(ptr) is equivalent to free(ptr), then your code suffers from referencing destroyed data (or ,in other words, garbage) in this code snippet:
while(curr_ptr != NULL) {
if (curr_ptr->data == data) {
result = 1;
if (curr_ptr->next != NULL) {
temp_ptr = curr_ptr;
destroy_node(temp_ptr);
prev_ptr->next = curr_ptr->next; //<- curr_ptr is still in stack
//or register, but curr->next
//is garbage
// what if curr_ptr is first node? did you forget to update queueref.first?
} else {
temp_ptr = curr_ptr;
queueref.last = prev_ptr;
prev_ptr->next = NULL;
destroy_node(temp_ptr);
}
// if you you need to destroy only one node - you can leave the loop here with break;
}
curr_ptr = curr_ptr->next; /// assigning garbage again if node is found
prev_ptr = prev_ptr->next;
The reason why using destroyed data can work in * most * (if I can say that, basically this is unpredictable) cases is that the chances that this memory can be reused by other part of program for dynamically allocated data can vary on timings and code flow.
PS
Regarding cryptic messages in the Windows box - when program crashes OS basically generates crashdump and prints registers (and dumps some relevant memory parts). Registers and memory dumps can show the place of crash and immediate register/stack values but you have to now memory map and assembler output to understand it. Crashdump can be loaded to debugger (WinDbg) together with unstripped binary to check stactrace and values of local variables at the moment of crash. All these I described very very briefly, you could find tons of books / guides searching for "windows crash or crashdump analysis"

Lock-free queue

Also I am doing a c implementation and currently have the structure of the queue:
typedef struct queueelem {
queuedata_t data;
struct queueelem *next;
} queueelem_t;
typedef struct queue {
int capacity;
int size;
queueelem_t *head;
queueelem_t *tail;
} queue_t;
queue_t *
queue_init(int capacity)
{
queue_t *q = (queue_t *) malloc(sizeof(queue_t));
q->head = q->tail = NULL;
q->size = 0;
q->capacity = capacity;
return q;
}
int CompareAndExchange (void **a, void *comparand,void *new) {
int success = 0;
pthread_mutex_lock(&CE_MUTEX);
if ((*a) != comparand) {
(*a) = new;
//return TRUE
success = 1;
}
pthread_mutex_unlock(&CE_MUTEX);
//return FALSE
return success;
}
But not sure How to continue, with queue and dequeue functions...
How would the code look like?
Sometime ago, I've found a nice solution to this problem. I believe that it the smallest found so far.
The repository has a example of how use it to create N threads (readers and writers) and make then share a single seat.
I made some benchmarks, on the test example and got the following results (in million ops/sec) :
By buffer size
By number of threads
Notice how the number of threads do not change the throughput.
I think this is the ultimate solution to this problem. It works and is incredible fast and simple. Even with hundreds of threads and a queue of a single position. It can be used as a pipeline beween threads, allocating space inside the queue.
The repository has some early versions written in C# and pascal. Im working to make something more complete polished to show its real powers.
I hope some of you can validate the work or help with some ideas. Or at least, can you break it?
Your pseudo-code can (and most likely does) suffer from the ABA problem, as only the pointer is checked, and not an accompanying unique stamp, you'll find this paper of use in that regard and as a general guide to lock-free queue implementation, with its pitfalls.
When dealing with lock free programing, its also a good idea to read up on Herb Sutter's works, as He gives good, insightful explanations to whats required, why its required and its potential weak points (though beware that some of his older publications/articles where found to contain some hidden/unforseen problems).
and also the recent boost'con talk about this subject :
https://github.com/boostcon/2011_presentations/raw/master/wed/lockfree_2011_slides.pdf
(Leaving this here for now, but see edit.)
Do you know a implementation of lock free queue in C?
I wrote lockless queue recently (http://www.ideone.com/l2QRp). I can't actually guarantee it works correctly, but I can't find any bugs and I've used it in a couple of single threaded programs without any problems, so there's nothing too obvious wrong with it.
Trivial usage example:
queue_t queue;
int val = 42;
queue_init(&queue,sizeof val);
queue_put(&queue,&val);
val = 0;
queue_pop(&queue,&val);
printf("%i\n",val); // 42
queue_destroy(&queue);
Edit:
As #Alexey Kukanov pointed out, queue_pop can fail if tmp is popped,freed,allocated again, and put again between checking for null and swapping:
if(!tmp->next) return errno = ENODATA;
/* can fail here */
} while(!sync_swap(q->head,tmp,tmp->next));
I'm not yet sure how to fix this, but I'll (hopefully) update this once I figure it out. For now, disregard this.
You may try this library it is built in c native. lfqueue
For Example
int* int_data;
lfqueue_t my_queue;
if (lfqueue_init(&my_queue) == -1)
return -1;
/** Wrap This scope in other threads **/
int_data = (int*) malloc(sizeof(int));
assert(int_data != NULL);
*int_data = i++;
/*Enqueue*/
while (lfqueue_enq(&my_queue, int_data) == -1) {
printf("ENQ Full ?\n");
}
/** Wrap This scope in other threads **/
/*Dequeue*/
while ( (int_data = lfqueue_deq(&my_queue)) == NULL) {
printf("DEQ EMPTY ..\n");
}
// printf("%d\n", *(int*) int_data );
free(int_data);
/** End **/
lfqueue_destroy(&my_queue);

How to use free on a handle inside a list?-> C -> windows API

I have a list in C that is something like this:
typedef struct _node
{
int number;
DWORD threadID;
HANDLE threadH;
struct *_node next;
} *node;
And you have somthing like this:
node new_node = malloc(sizeof(node));
As you may have guessed out, this list will store information for threads, including their handlers and Id's. Still I am having trouble when I try to do this:
free(new_node);
Everytime I try to do this I encounter an unexpected error, VS saying that there was a data corruption. I've pinned down as much as possible and I found that the problem resides when I try to use free the handle.
I've searched on MSDN how to do this but the only thing I can find is the function that closes the thread (which is not intended here, since I want the thread to run, just deleting it's record from the list).
The question is: how I am supposed to free an handle from the memory? (Considering that this is only a copy of the value of the handle, the active handle is not being deleted).
EDIT: This is the function to insert nodes from the list:
int insereVisitanteLista(node* lista, DWORD threadID, HANDLE threadH, int num_visitante)
{
node visitanteAnterior;
node novoVisitante = (node)malloc(sizeof(node));
if(novoVisitante == NULL)
return 0;
novoVisitante->threadID = threadID;
novoVisitante->threadH = threadH;
novoVisitante->number = num_visitante;
novoVisitante->next = NULL;
if(*lista == NULL)
{
*lista = novoVisitante;
return 1;
}
visitanteAnterior = *lista;
while(visitanteAnterior->next != NULL)
visitanteAnterior = visitanteAnterior->next;
visitanteAnterior->next =novoVisitante;
return 1;
}
And this is the function to delete nodes:
int removeVisitanteLista(node * lista, DWORD threadID)
{
node visitanteAnterior = NULL, visitanteActual;
if(*lista == NULL)
return 0;
visitanteActual = *lista;
if((*lista)->threadID == threadID)
{
*lista = visitanteActual->next;
visitanteActual->next = NULL;
free(visitanteActual);
return 1;
}
while(visitanteActual != NULL && visitanteActual->threadID != threadID)
{
visitanteAnterior = visitanteActual;
visitanteActual = visitanteActual->next;
}
if (visitanteActual == NULL)
return 0;
visitanteAnterior->next = visitanteActual->next;
free(visitanteActual);
return 1;
}
What exactly is a node that you are trying to free? Is this a pointer to a struct _node? If yes, have you allocated it previously? If no, free is not needed, otherwise you have to check if node is not NULL and make sure you do not free it multiple times. It is hard to guess what you are doing and where is an error without a minimal working example reproducing the problem. The only thing I can suggest is to read about memory management in C. This resource might help.
UPDATE:
node in your code is a pointer to _node. So sizeof (node) is a size of a pointer, which is either 4 or 8 bytes (depending on architecture). So you allocate 8 bytes, for example, but assume you have a pointer to the structure which is much larger. As a result, you corrupt memory, and behavior of the program becomes undefined. So changing node novoVisitante = (node)malloc(sizeof(node)) to node novoVisitante = (node)malloc(sizeof(_node)) should fix the problem.
You haven't shown us the context of your call to free() so I need to speculate a little but my first concern is that you didn't mention removing the node from the list before deleting it.
Start by unlinking the node by modifying the next field of the previous (or head) node. If you still get the error, then you have corrupted memory somehow by writing past the end of one of your allocated memory structures or something similar.
Also, I assume node is a pointer. You really haven't provided much information about what you're doing.

Resources