I am writing a program to store multiple typedef "Item" instances in a queue (using this queue implementation) and I am running into a problem getting the data back from the queue.
Here is the relevant code, minus the file open/close lines (the code takes an input file and stores individual values into the instance):
Sample file:
1 2 10 20 30
------------
/* create new item */
Item *newItem = malloc(sizeof(Item));
/* string tokenizer */
...
/* set item variables */
newItem->value1 = strtol(tokens[0], NULL, 10); //should contain 1
newItem->value2 = strtol(tokens[1], NULL, 10); //should contain 2
newItem->value3 = strtol(tokens[2], NULL, 10); //should contain 10
newItem->value4 = strtol(tokens[3], NULL, 10); //should contain 20
newItem->value5 = strtol(tokens[4], NULL, 10); //should contain 30
/* add to item queue */
queue_push_tail(itemQueue, &newItem);
/* add second item with different values, same method as above */
...
/* check queue values */
if(!queue_is_empty(itemQueue)) {
Item *itemHead = queue_peek_head(itemQueue); //this should differ...
printf("Head: %d %d\n", itemHead->value1, itemHead->value5);
Item *itemTail = queue_peek_tail(processQueue); //...from this
printf("Tail: %d %d\n", itemTail->value1, itemTail->value5);
}
How can I then access one of these items to view a variable? I thought I could use something like queue_peek_head(itemQueue)->value1 to see the first variable in the item (in the above example, 1 since that was stored in the first newItem.value1 in the queue), but that doesn't work for some reason.
Like #WhozCraig said in the previous comment, queue_peek_head() is supposed to return 'void *' which you should cast to 'Item *'.
Another issue with your code - 'newItem' is an on-stack variable, you can keep in a queue only as long as you are within the call stack of the function that pushed newItem to the queue. A safer practice would be to allocate newItem before pushing to a queue and free it somewhere else when you don't need it to be queued anymore.
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 2 years ago.
Improve this question
I'm working on a C project for an embedded target running a Linux distribution (build with Yocto). I'm new to Linux embedded world and I have to build a data logging system.
I'm already learning how to use threads, and I'm thinking on the project organization.
Here is what I imagine :
multiple threads to collect data from different interfaces, CAN bus, I2C... (different sample rate)
one thread with a sample rate of 200ms to populate csv files
one thread with a sample rate of 3 seconds to send data with http request
Threads will stop on CAN info or external event
I don't know what is the best way to organize this project. I see two ways, in the first a startup program create each thread and wait in a while loop with event watching to stop them. The second way is a startup program execute others binaries as thread.
In the two ways I don't know how share data between threads.
Can you share me your experience ?
Thank you
EDIT :
First, thanks a lot to #Glärbo, for your explanations. It's really helpful to learn multi threading mechanic.
I've tested it with success.
For future readers I've drawn diagrams to illustrate #Glärbo answer.
main thread
productor-sensor thread
datalogger thread
I would do it simpler, using a simple multiple producers, single consumer approach.
Let's assume each data item can be described using a single numerical value:
struct value {
struct value *next; /* Forming a singly-linked list of data items */
struct sensor *from; /* Identifies which sensor value this is */
struct timespec when; /* Time of sensor reading in UTC */
double value; /* Numerical value */
};
I would use two lists of values: one for sensor readings received but not stored, and one for unused value buckets. This way you don't need to dynamically allocate or free value buckets, unless you want to (by manipulating the unused list).
Both lists are protected by a mutex. Since the unused list may be empty, we need a condition variable (that is signaled on whenever a new unused value is added to it) so that threads can wait for one to become available. The received list similarly needs a condition variable, so that if it happens to be empty when the consumer (data storer) wants them, it can wait for at least one to appear.
static pthread_mutex_t unused_lock = PTHREAD_MUTEX_INITIALIZER;
static pthread_cond_t unused_wait = PTHREAD_COND_INITIALIZER;
static struct value *unused_list = NULL;
static pthread_mutex_t received_lock = PTHREAD_MUTEX_INITIALIZER;
static pthread_cond_t received_wait = PTHREAD_COND_INITIALIZER;
static struct value *received_list = NULL;
For the unused list, we need three helpers: one to create new unused value items from scratch (which you call initially to create say two or three value items per sensor, plus a few), and later on, if you think you need them (say, if you add new sensors run time):
int unused_create(void)
{
struct value *v;
v = malloc(sizeof *v);
if (!v)
return ENOMEM;
v->from = NULL;
pthread_mutex_lock(&unused_lock);
v->next = unused_list;
unused_list = v;
pthread_cond_signal(&unused_wait);
pthread_mutex_unlock(&unused_lock);
return 0;
}
The other two are needed to get and put value items from/back to the list:
struct value *unused_get(void)
{
struct value *v;
pthread_mutex_lock(&unused_lock);
while (!unused_list)
pthread_cond_wait(&unused_wait, &unused_lock);
v = unused_list;
unused_list = unused_list->next;
pthread_mutex_unlock(&unused_lock);
v->from = NULL;
return v;
}
void unused_put(struct value *v)
{
v->from = NULL;
pthread_mutex_lock(&unused_lock);
v->next = unused_list;
unused_list = v;
pthread_cond_signal(&unused_wait);
pthread_mutex_unlock(&unused_lock);
}
The idea above is that when the from member is NULL, the item is unused (as it is not from any sensor). Technically, we don't need to clear it to NULL at every stage, but I like to be thorough: it's not like setting it is a costly operation.
Sensor-accessing producers take the sensor reading, get the current time using e.g. clock_gettime(CLOCK_REALTIME, ×pec), and then use unused_get() to grab a new unused item. (The order is important, because unused_get() may take some time, if there are no free items.) Then, they fill in the fields, and call the following received_put() to prepend the reading to the list:
void received_put(struct value *v)
{
pthread_mutex_lock(&received_lock);
v->next = received_list;
received_list = v;
pthread_mutex_signal(&received_wait);
pthread_mutex_unlock(&received_lock);
}
There is only one thread that periodically collects all received sensor readings, and stores them. It can keep a set of most recent readings, and send those periodically. Instead of calling some received_get() repeatedly until there are no more received values not handled yet, we should use a function that returns the whole list of them:
struct value *received_getall(void)
{
struct value *v;
pthread_mutex_lock(&received_lock);
while (!received_list)
pthread_cond_wait(&received_wait, &received_lock);
v = received_list;
received_list = NULL;
pthread_mutex_unlock(&received_lock);
return v;
}
The consumer thread, storing/sending the summaries and readings, should obtain the whole list, then handle them one by one. After each item has been processed, they should be added to the unused list. In other words, something like
struct value *all, v;
while (1) {
all = receive_getall();
while (all) {
v = all;
all = all->next;
v->next = NULL;
/* Store/summarize value item v */
unused_put(v);
}
}
As you can see, while the consumer thread is handling the sensor value items, the sensor threads can add new readings for the next round, as long as there are enough free value item buckets to use.
Of course, you can also allocate lots of values at one malloc() call, but then you must somehow remember which pool of values each value belongs to to free them. So:
struct owner {
size_t size; /* Number of value's */
size_t used; /* Number of value's not freed yet */
struct value value[];
};
struct value {
struct value *next; /* Forming a singly-linked list of data items */
struct owner *owner; /* Part of which value array, NULL if standalone */
struct sensor *from; /* Identifies which sensor value this is */
struct timespec when; /* Time of sensor reading in UTC */
double value; /* Numerical value */
};
int unused_add_array(const size_t size)
{
struct owner *o;
struct value *v;
size_t i;
o = malloc(sizeof (struct owner) + size * sizeof (struct value));
if (!o)
return ENOMEM;
o->size = size;
o->used = used;
i = size - 1;
pthread_mutex_lock(&unused_lock);
o->value[i].next = unused_list;
while (i-->0)
o->value[i].next = o->value + i + 1;
unused_list = o->value[0];
pthread_cond_broadcast(&unused_wait);
pthread_mutex_unlock(&unused_lock);
return 0;
}
/* Instead of unused_put(), call unused_free() to discard a value */
void unused_free(struct value *v)
{
pthread_mutex_lock(&unused_lock);
v->from = NULL;
if (v->owner) {
if (v->owner->used > 1) {
v->owner->used--;
return;
}
v->owner->size = 0;
v->owner->used = 0;
free(v->owner);
return;
}
free(v);
return;
}
The reason unused_free() uses unused_lock is that we must be sure that no other thread is accessing the bucket when we free it. Otherwise, we can have a race window, where the other thread may use the value after we free()d it.
Remember that the Linux C library, like most other C libraries, does not return dynamically allocated memory to the operating system at free(); memory is only returned if it is large enough to matter. (Currently on x86 and x86-64, the glibc limit is about 132,000 bytes or so; anything smaller is left in the process heap, and used to satisfy future malloc()/calloc()/realloc() calls.)
The contents of the struct sensor are up to you, but personally, I'd put at least
struct sensor {
pthread_t worker;
int connfd; /* Device or socket descriptor */
const char *name; /* Some kind of identifier, perhaps header in CSV */
const char *units; /* Optional, could be useful */
};
plus possibly sensor reading interval (in, say, milliseconds) in it.
In practice, because there is only one consumer thread, I'd use the main thread for it.
I'm new to C, so the answer might be obvious but I just can't get my head around it.
I'm trying to create C++ vector kind of structure in C. I use Windows EnumWindows function to loop through all the windows. In the callback function I create new instance of windowHandle -structure for each window. For some reason however it doesn't seem to create new instance but rather just overwrites the old one. Or maybe it creates new instance, but when I give value to attribute windowHandle->title, it applies that change to every instance of windowHandle?
Callback function:
BOOL CALLBACK delegate(HWND wnd, LPARAM param){
if (filter(wnd)){ //<- just to make sure it's kind of window that we want.
char windowName[256] = {};
GetWindowText(wnd, windowName, sizeof(windowName));
windowHandle *handle = malloc(sizeof(windowHandle)); //<- create new instance of windowHandle for each window
handle->hWND = wnd;
handle->title = windowName;
insert((windowArray*) param, handle); //<- insert each windowHandle in our 'vector'
}
// but return true here so that we iterate all windows
return TRUE;
}
Call to EnumWindows function:
windowArray* array = array_init(20);
EnumWindows(&delegate, (LPARAM) array); //<- not quite sure what that '&'-sign does?
Structures:
typedef struct windowHandle{
HWND hWND;
char* title;
} windowHandle;
typedef struct windowArray{
int count;
int capacity;
int objectSize;
windowHandle* windows;
} windowArray;
And lastly insert:
int insert(windowArray* array, void* handle){
int index = (array->count * array->objectSize);
if(index > 0){
printf("\n%s %s\n", "first element's title is: ", array->windows->title); //<- prints to see if new handle overwrite the previous ones
}
printf("%s %s %s %p %s %d\n", "Inserted element ", ((windowHandle*)handle)->title, " with pointer ", handle, " on index ", index);
fflush(stdout); //<- prints info about first & current window
memcpy(array->windows + (index), (windowHandle*)handle, array->objectSize);
array->count++;
}
Output during insert calls:
Inserted element joo - Java - nativeWindowCapture/src/NativeWindowHookImp.c - Eclipse SDK with pointer 0000000000FB1B90 on index 0
first element's title is: Komentokehote
Inserted element Komentokehote with pointer 0000000000FB1BB0 on index 16
first element's title is: C struct instance overwrites previous instance - Stack Overflow - Mozilla Firefox
Inserted element C struct instance overwrites previous instance - Stack Overflow - Mozilla Firefox with pointer 0000000000FB1BD0 on index 32
first element's title is: JNI compiler kutsu – Muistio
Inserted element JNI compiler kutsu – Muistio with pointer 0000000000FB1BF0 on index 48
first element's title is: Skype™?
Inserted element Skype™? with pointer 0000000000FB1C10 on index 64
first element's title is: eclipse
Inserted element eclipse with pointer 0000000000FB1C30 on index 80
first element's title is: src
Inserted element src with pointer 0000000000FBCCE0 on index 96
all added
and then if I print the full content of windowArray->windows, I get the following results:
Getting value src , at index 0 and pointer 0000000000FBBB80
Getting value src , at index 16 and pointer 0000000000FBBC80
Getting value src , at index 32 and pointer 0000000000FBBD80
Getting value src , at index 48 and pointer 0000000000FBBE80
Getting value src , at index 64 and pointer 0000000000FBBF80
Getting value src , at index 80 and pointer 0000000000FBC080
Getting value src , at index 96 and pointer 0000000000FBC180
P.S. I would also like to know why windowArray's windows-attribute's pointers are different from those of *handle-objects created in delegate? That doesn't seem to be very efficient memorywise?
You set handle->title to point to windowName. But as soon as you exit this scope, windowName will cease to exist. So your structure contains a pointer to an array that no longer exists.
char windowName[256] = {};
GetWindowText(wnd, windowName, sizeof(windowName));
windowHandle *handle = malloc(sizeof(windowHandle)); //<- create new instance of windowHandle for each window
handle->hWND = wnd;
handle->title = windowName;
}
// when we get here, windowName ceases to exist
// so why did we store a pointer to it?
After you exit the scope, it is an error to follow the title pointer, since it no longer points to any object that is still alive. Perhaps change title from char * to char[256].
This question has been asked multiple times, but I've done (from what I can tell) everything that's been mentioned here. Basically, I'm getting 1 character at a time from a TCP socket and I'm building a dynamically growing string with the one character. I can do looping prints and see that the string grows and grows, and then when it gets to 20 characters long, the program crashes.
while(FilterAmount != 0)
{
g_io_channel_read_chars (source,(gchar *) ScanLine,1,&BytesRead,&GlibError);
printf("Scanline: %s FilterAmount: %ld\n", ScanLine, FilterAmount);
//the filters are delimited by \n, munch these off, reset important variables, save the last filter which is complete
if(ScanLine[0] == FilterTerminator[0]) {
//if the Filter Name actually has a filter in it
if(FilterName != NULL){
FilterArray = FilterName; //save off the filter name
printf("This is the filter name just added: %s\n", FilterName);
FilterArray++; //increment the pointer to point to the next memory location.
FilterAmount--; //update how many filters we have left
FilterNameCount = 0; //reset how many characters each filter name is
free(FilterName);
free(FilterTmp);
}
}
else {
printf("else!\n");
//keep track of the string length of the filter
FilterNameCount++;
//allocate more memory in the string used to store the filter name + null terminating character
FilterTmp = (gchar*)realloc(FilterName, FilterNameCount*sizeof(char) + 1);
if(FilterTmp == NULL)
{
free(FilterName);
printf("Error reallocating memory for the filter name temporary variable!");
return 1;
}
FilterName = FilterTmp;
printf("filter name: %s\n", FilterName);
//concat the character to the end of the string where space was just made for it.
strcat(FilterName, ScanLine);
}
}
}
This section of code loops and loops whenever we have a non "\n" character in a buffer we're reading data into. The program crashes when allocating the 21st character's location every single time. Here are the pertinent declarations:
static gchar *FilterName = NULL, *FilterTmp = NULL;
static gchar ScanLine[9640];
I am trying to implement a queue in C (using this implementation) where, given an input file with some number of lines each containing 5 values, a struct Item is created for that line and the values are assigned to the struct variables value1...value5. The struct is then placed in a queue, so the first line of the file will be the first struct in the queue, and the last line will be the last struct.
However, I am getting some weird behaviour when trying to read values from the structs in the queue. The end of the code segment below prints value1 and value5 as a debug check to make sure the values are correct. It should print struct1->value1 and struct1->value5 for both head and tail on the first iteration, and then struct1->value1 and struct1->value5 for head and struct2->value1 and struct2->value5 for tail on the second iteration, but instead it prints strange large values for both head and tail. (plus, it seems to forget about the first struct altogether.) What am I doing wrong here?
Sample file:
1 0 10 1 3
2 0 10 10 1
------------
/* read input file */
/* while the current line of the file is not null */
...
/* create new item */
Item *newItem = malloc(sizeof(Item));
/* string tokenizer */
/* adds tokens from current line to tokens[i] */
...
/* set item variables */
newItem->value1 = strtol(tokens[0], NULL, 10); //should contain e.g. 1
newItem->value2 = strtol(tokens[1], NULL, 10); //should contain e.g. 0
newItem->value3 = strtol(tokens[2], NULL, 10); //should contain e.g. 10
newItem->value4 = strtol(tokens[3], NULL, 10); //should contain e.g. 1
newItem->value5 = strtol(tokens[4], NULL, 10); //should contain e.g. 3
/* add to item queue */
queue_push_tail(itemQueue, &newItem);
/* check queue values */
if(!queue_is_empty(itemQueue)) { //after two lines,
Item *itemHead = queue_peek_head(itemQueue); //this should differ...
printf("Head: %d %d\n", itemHead->value1, itemHead->value5);
Item *itemTail = queue_peek_tail(processQueue); //...from this
printf("Tail: %d %d\n", itemTail->value1, itemTail->value5);
}
Expected output:
Head: 1 3 //when first line is read
Tail: 1 3
Head: 1 3 //when second line is read
Tail: 2 1
Actual output:
Head: 146752 -4196581 //when first line is read
Tail: 146752 -4196581
Head: 146792 -4196581 //when second line is read
Tail: 146792 -4196581
This:
queue_push_tail(itemQueue, &newItem);
doesn't look right, it doesn't match how you peek items from the queue. If the queue is storing pointers, hand it newItem, not the address of newItem.
I need to make some operations on a list of files with particular extension (*.bob), all stored in the same directory. The files are image frames, and their name format is frame_XXXX.bob. I don't know the number of frames a priori, and I need to make sure I process them in order (from frame 0 to last one). I read the content of the folder with struct dirent *readdir(DIR *dirp), but since it doesn't guarantee files will be read in alphabetical order (even though it always seems to), I want to put them into a single linked list, and then sort them before processing further.
I save the head of the list before populating it to a pointer filesListStart, then read the entire folder content, adding each entry to the list if it has ".bob" extension. This all works great when I have up to 100 frames, but for some reason breaks down above that - the value of what pointer filesListStart points at doesn't contain the filename of the first entry in the list anymore. The code doesn't use any numerals, so I don't know what would be the significance of going over 100 elements.
I wrote out memory address of filesListStart before I start populating the list, and after, and they are the same, but values they show at magically change. When I set filesListStart it points at object with field fileName equals to "frame_0000.bob" (which is as expected), but after populating the list the name it points at becomes "e_0102.bob".
The list structure is defined as
// List structure
struct FilesList {
char *fileName ;
struct FilesList *next ;
} ;
The code in question is:
DIR *moviesDir ;
moviesDir = opendir("movies") ;
if(moviesDir == NULL)
{
printf("Make Movie failed to open directory containing bob frames\n") ;
return ;
}
struct dirent *dirContent ;
// Get first .bob frame name from the directory
dirContent = readdir(moviesDir) ;
// isBobFile(dirContent) returns 1 if entry has ".bob" extension and 0 otherwise
while( !isBobFile(dirContent) )
{
dirContent = readdir(moviesDir) ;
}
struct FilesList *filesList = (struct FilesList*)
malloc( sizeof(struct FilesList) ) ;
// Initialize the list start at that first found .bob frame
filesList->fileName = dirContent->d_name;
// And save the head of the list
struct FilesList *filesListStart = filesList ;
printf("FilesListStart: %s\n", filesListStart->fileName) ;
printf("Address is: %p\n", filesListStart) ;
// For all other bob frames
while( (dirContent = readdir(moviesDir) ) != NULL )
{
if( isBobFile(dirContent) )
{
struct FilesList *temporaryNode = (struct FilesList*)
malloc( sizeof(struct FilesList) );
temporaryNode->fileName = dirContent->d_name ;
filesList->next = temporaryNode ;
filesList = temporaryNode ;
}
}
// Set the 'next' pointer of the last element in list to NULL
filesList->next = NULL ;
// close stream to directory with .bob frames
closedir(moviesDir) ;
// Check what FilesListStart points at
printf("FilesListStart: %s\n", filesListStart->fileName) ;
printf("Address is: %p\n", filesListStart) ;
// Rest of the code
You should be making a copy of dirContent->d_name rather than using the actual value.
The runtime libraries are free to change the contents of that dirent structure whenever you call readdir and, if all you've stored is it's address, the underlying memory may change. From the POSIX man-pages:
The pointer returned by readdir() points to data which may be overwritten by another call to readdir() on the same directory stream.
In other words, replace the lines:
filesList->fileName = dirContent->d_name;
temporaryNode->fileName = dirContent->d_name ;
with:
filesList->fileName = strdup (dirContent->d_name);
temporaryNode->fileName = strdup (dirContent->d_name);
assuming you have a strdup-like function and, if not, you can get one cheap here.
If it's only changing after 100 calls, it's probably the runtime trying to be a bit more intelligent but even it can't store an infinite number so it probably sets a reasonable limit.
Just remember to free all those char pointers before you free the linked list nodes (somewhere in that "rest of code" section presumably).