I am implementing FIFO in C. One thread is writing in FIFO and other is reading from it.
#define BUFFER_LENGTH 1000
struct Frame
{
char data[1024];
unsigned int data_len;
struct Frame* frame;
};
struct Frame * get_from_fifo ()
{
if (!fifo_length)
{
first = last = NULL;
return NULL;
}
struct Frame* frame = first;
first = first->frame;
fifo_length--;
return frame;
}
int add_to_fifo (const char* data, unsigned int frame_size)
{
if (fifo_length >= BUFFER_LENGTH)
{
ast_log(LOG_ERROR, "Buffer full\n");
return SURESH_ERROR;
}
struct Frame* frame = malloc(sizeof (struct Frame));
frame->data_len = frame_size;
memcpy(frame->data, data, frame_size);
if (last)
{
last->frame = frame;
last = frame;
}
if (!first)
{
first = last = frame;
}
fifo_length++;
return SURESH_SUCCESS;
}
how can I prevent functions *add_to_fifo* and *get_from_fifo* to be called at the same time by different threads. i.e. *get_from_fifo* should only be called when the other thread is not executing *add_to_fifo* and vice verca.
As you are implementing FIFO stack the only really concurrent operation you have is changing the stack size (fifo_length).
You are adding entries to the tail of the stack and removing entries from the head of the stack so these two operation will never interfere with each other. So the only part you will need to worry about is changing the stack size (fifo_length), I would put it into separate function synchronised by mutex or flag (as mentioned by "Joey" above) and call it from both add_to_fifo() and get_from_fifo() functions.
You need to use a mutex (mutual exclusion) variable. The pthread library has everything you will need. Here's a good place to start looking at the available functions:
http://pubs.opengroup.org/onlinepubs/009695399/basedefs/pthread.h.html
You'll need to init a mutex variable that each thread will have access to:
http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_mutex_init.html
Then your threads will need to lock it when they need access to shared memory, and then unlock it when they are done using the shared memory:
http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_mutex_lock.html
Here's a simple example:
http://publib.boulder.ibm.com/infocenter/iseries/v5r3/index.jsp?topic=%2Frzahw%2Frzahwe18rx.htm
Good luck!
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 2 years ago.
Improve this question
I'm working on a C project for an embedded target running a Linux distribution (build with Yocto). I'm new to Linux embedded world and I have to build a data logging system.
I'm already learning how to use threads, and I'm thinking on the project organization.
Here is what I imagine :
multiple threads to collect data from different interfaces, CAN bus, I2C... (different sample rate)
one thread with a sample rate of 200ms to populate csv files
one thread with a sample rate of 3 seconds to send data with http request
Threads will stop on CAN info or external event
I don't know what is the best way to organize this project. I see two ways, in the first a startup program create each thread and wait in a while loop with event watching to stop them. The second way is a startup program execute others binaries as thread.
In the two ways I don't know how share data between threads.
Can you share me your experience ?
Thank you
EDIT :
First, thanks a lot to #Glärbo, for your explanations. It's really helpful to learn multi threading mechanic.
I've tested it with success.
For future readers I've drawn diagrams to illustrate #Glärbo answer.
main thread
productor-sensor thread
datalogger thread
I would do it simpler, using a simple multiple producers, single consumer approach.
Let's assume each data item can be described using a single numerical value:
struct value {
struct value *next; /* Forming a singly-linked list of data items */
struct sensor *from; /* Identifies which sensor value this is */
struct timespec when; /* Time of sensor reading in UTC */
double value; /* Numerical value */
};
I would use two lists of values: one for sensor readings received but not stored, and one for unused value buckets. This way you don't need to dynamically allocate or free value buckets, unless you want to (by manipulating the unused list).
Both lists are protected by a mutex. Since the unused list may be empty, we need a condition variable (that is signaled on whenever a new unused value is added to it) so that threads can wait for one to become available. The received list similarly needs a condition variable, so that if it happens to be empty when the consumer (data storer) wants them, it can wait for at least one to appear.
static pthread_mutex_t unused_lock = PTHREAD_MUTEX_INITIALIZER;
static pthread_cond_t unused_wait = PTHREAD_COND_INITIALIZER;
static struct value *unused_list = NULL;
static pthread_mutex_t received_lock = PTHREAD_MUTEX_INITIALIZER;
static pthread_cond_t received_wait = PTHREAD_COND_INITIALIZER;
static struct value *received_list = NULL;
For the unused list, we need three helpers: one to create new unused value items from scratch (which you call initially to create say two or three value items per sensor, plus a few), and later on, if you think you need them (say, if you add new sensors run time):
int unused_create(void)
{
struct value *v;
v = malloc(sizeof *v);
if (!v)
return ENOMEM;
v->from = NULL;
pthread_mutex_lock(&unused_lock);
v->next = unused_list;
unused_list = v;
pthread_cond_signal(&unused_wait);
pthread_mutex_unlock(&unused_lock);
return 0;
}
The other two are needed to get and put value items from/back to the list:
struct value *unused_get(void)
{
struct value *v;
pthread_mutex_lock(&unused_lock);
while (!unused_list)
pthread_cond_wait(&unused_wait, &unused_lock);
v = unused_list;
unused_list = unused_list->next;
pthread_mutex_unlock(&unused_lock);
v->from = NULL;
return v;
}
void unused_put(struct value *v)
{
v->from = NULL;
pthread_mutex_lock(&unused_lock);
v->next = unused_list;
unused_list = v;
pthread_cond_signal(&unused_wait);
pthread_mutex_unlock(&unused_lock);
}
The idea above is that when the from member is NULL, the item is unused (as it is not from any sensor). Technically, we don't need to clear it to NULL at every stage, but I like to be thorough: it's not like setting it is a costly operation.
Sensor-accessing producers take the sensor reading, get the current time using e.g. clock_gettime(CLOCK_REALTIME, ×pec), and then use unused_get() to grab a new unused item. (The order is important, because unused_get() may take some time, if there are no free items.) Then, they fill in the fields, and call the following received_put() to prepend the reading to the list:
void received_put(struct value *v)
{
pthread_mutex_lock(&received_lock);
v->next = received_list;
received_list = v;
pthread_mutex_signal(&received_wait);
pthread_mutex_unlock(&received_lock);
}
There is only one thread that periodically collects all received sensor readings, and stores them. It can keep a set of most recent readings, and send those periodically. Instead of calling some received_get() repeatedly until there are no more received values not handled yet, we should use a function that returns the whole list of them:
struct value *received_getall(void)
{
struct value *v;
pthread_mutex_lock(&received_lock);
while (!received_list)
pthread_cond_wait(&received_wait, &received_lock);
v = received_list;
received_list = NULL;
pthread_mutex_unlock(&received_lock);
return v;
}
The consumer thread, storing/sending the summaries and readings, should obtain the whole list, then handle them one by one. After each item has been processed, they should be added to the unused list. In other words, something like
struct value *all, v;
while (1) {
all = receive_getall();
while (all) {
v = all;
all = all->next;
v->next = NULL;
/* Store/summarize value item v */
unused_put(v);
}
}
As you can see, while the consumer thread is handling the sensor value items, the sensor threads can add new readings for the next round, as long as there are enough free value item buckets to use.
Of course, you can also allocate lots of values at one malloc() call, but then you must somehow remember which pool of values each value belongs to to free them. So:
struct owner {
size_t size; /* Number of value's */
size_t used; /* Number of value's not freed yet */
struct value value[];
};
struct value {
struct value *next; /* Forming a singly-linked list of data items */
struct owner *owner; /* Part of which value array, NULL if standalone */
struct sensor *from; /* Identifies which sensor value this is */
struct timespec when; /* Time of sensor reading in UTC */
double value; /* Numerical value */
};
int unused_add_array(const size_t size)
{
struct owner *o;
struct value *v;
size_t i;
o = malloc(sizeof (struct owner) + size * sizeof (struct value));
if (!o)
return ENOMEM;
o->size = size;
o->used = used;
i = size - 1;
pthread_mutex_lock(&unused_lock);
o->value[i].next = unused_list;
while (i-->0)
o->value[i].next = o->value + i + 1;
unused_list = o->value[0];
pthread_cond_broadcast(&unused_wait);
pthread_mutex_unlock(&unused_lock);
return 0;
}
/* Instead of unused_put(), call unused_free() to discard a value */
void unused_free(struct value *v)
{
pthread_mutex_lock(&unused_lock);
v->from = NULL;
if (v->owner) {
if (v->owner->used > 1) {
v->owner->used--;
return;
}
v->owner->size = 0;
v->owner->used = 0;
free(v->owner);
return;
}
free(v);
return;
}
The reason unused_free() uses unused_lock is that we must be sure that no other thread is accessing the bucket when we free it. Otherwise, we can have a race window, where the other thread may use the value after we free()d it.
Remember that the Linux C library, like most other C libraries, does not return dynamically allocated memory to the operating system at free(); memory is only returned if it is large enough to matter. (Currently on x86 and x86-64, the glibc limit is about 132,000 bytes or so; anything smaller is left in the process heap, and used to satisfy future malloc()/calloc()/realloc() calls.)
The contents of the struct sensor are up to you, but personally, I'd put at least
struct sensor {
pthread_t worker;
int connfd; /* Device or socket descriptor */
const char *name; /* Some kind of identifier, perhaps header in CSV */
const char *units; /* Optional, could be useful */
};
plus possibly sensor reading interval (in, say, milliseconds) in it.
In practice, because there is only one consumer thread, I'd use the main thread for it.
The following function doesn't work. pin_thread_function sometimes receive garbage instead of the struct data. What is wrong? I know that is some basic scope related problem, but I can't explain.
typedef void (*state_callback_t)(int state);
struct pin_thread_params
{
char pin[3 + 1];
state_callback_t callback_onchange;
};
extern int pin_monitor_create(const char * pin,
state_callback_t callback_onchange)
{
int ret;
unsigned long int thread;
struct pin_thread_params params;
//Setup struct
strcpy(params.pin, "123");
params.callback_onchange = callback_onchange;
//Create thread with struc as parameter
ret = pthread_create(&thread, NULL, pin_thread_function, ¶ms);
if (!ret)
{
ret = pthread_detach(thread);
}
return ret;
}
static void * pin_thread_function(void * context)
{
struct pin_thread_params params;
memcpy(¶ms, context, sizeof(params));
//Sometimes the correct string, most time garbage
printf("Started monitoring %s", params.pin);
}
When params is malloc'ed before pthread_create, everything works fine, like this:
...
struct pin_thread_params * params;
//Setup struct with malloc
params = malloc(sizeof(struct pin_thread_params));
strcpy(params->pin, "123");
params->callback_onchange = callback_onchange;
...
struct pin_thread_params params is statically allocated and the address of it is not safe to use once the scope of it is over (i.e. after pin_monitor_create has returned). It may happen that sometimes the thread execution starts before the pin_monitor_create has exited and you see the valid data in params. However, the dynamically allocated memory is from heap and will be usable until you free it, so you always get valid data in params within pin_thread_function .
I'm not particularly knowledgeable about pthreads (can't just comment quite yet), but you are passing a pointer to stack allocated memory to the thread which will eventually be clobbered by proceeding function calls.
Can anyone tell me why my shared memory data structure (implemented using sys/shm.h) is not being read correctly by pthreads? This is an edited version of my question, with a reduced amount of code. Hopefully its easier to navigate.
Initially, the structure being referenced is created in shared memory space, so two different applications can read and write to it. The aim: to have one application update the shared structure, and the other read it using pthreads. So far everything things are working to an extent. Both applications can read and write to the shared memory, except the pthreads. they don't seem to pick up the modified shared structure?
An overview of the code is below. It is based on a basic runtime system, however, it is not overly complicated. The function executed within the pthreads is:
void* do_work(void *p)
The shared structure is:
typedef struct WL_CTRL_T
Currently all i am trying do is print out the elements of the array. Initially all elements are set to true. Halfway through the execution, using GDB to halt the process, i update the structure from outside, using the other application, by changing elements 0 and 1 to false, then continue to the process. At this i also print out the state of the array from each application via the sequential code, and the print out is correct. However, when the threads are set off, they print the original state of the array, all true...
The structure contains an array of structs, where the active bool field is read by the pthread
I have tried many ways to try and correct this problem, but no joy.
Any advice appreciated, thanks :-)
/*controller api.h*/
typedef struct WL_CTRL_T
{
int targetNumThreads;
int sizeBuf;
int numEntries;
int nextIdx;
thread_state_control_t volatile thread_state_control[THREAD_NUM];
mon_entry_t buffer[];
} wl_ctrl_t;
typedef struct THREADPOOL_T
{
int num_threads;
int qsize;
pthread_t *threads;
todo_t *qhead;
todo_t *qtail;
pthread_mutex_t qlock;
pthread_cond_t q_not_empty;
pthread_cond_t q_empty;
int shutdown;
int dont_accept;
}threadpool_t;
typedef struct TODO_T
{
void (*routine) (void*);
void * arg;
int lock;
struct todo_t* next;
} todo_t;
The function assigned to the pthread
/********************************************************************
*
* do_work:
*
* this is the reusable thread, assigned work via the dispatch
* function.
*
********************************************************************/
void* do_work(void *p)
{
int c = 0;
thread_args_t *thread_args = (thread_args_t*)p;
threadpool_t *pool = thread_args->threadpool;
todo_t* workload;
wl_ctrl_t volatile *wcc = thread_args->wl_ctrl;
while(1)
{
pool->qsize = pool->qsize;
/* while work que is empty, spinlock */
while( pool->qsize == 0)
{
if(c<1)
printf("thread: %d spin-lock \n", thread_args->thread_id);
c++;
}
/* update the threadpool, minus current workload */
workload = pool->qhead;
pool->qsize--;
if(pool->qsize == 0)
{
pool->qhead = NULL;
pool->qtail = NULL;
}
else
{
pool->qhead = workload->next;
}
/* execute workload */
(workload->routine) (workload->arg);
free(workload);
/* check this threads wait state */
printf("In thread: %d\n",wcc->thread_state_control[thread_args->thread_id].active);
}
}
I want to use getaddrinfo_a function. Is this method thread safe?
In the man page example given uses a global list for resolving the hostnames.
If I manipulate that list in user space then is it safe?
Pseudo-code as follows:
static struct gaicb **reqs =NULL; // global list of hostname to resolve.
addToList() {
ret =
getaddrinfo_a(
GAI_NOWAIT,
&reqs[nreqs_base],
nreqs - nreqs_base,
NULL ); // enque hostname queue.
}
//another thread method
dequeu_list( int i ) {
struct gaicb * result = reqs[i] ;
reqs[i] = NULL;
}
Yes, see in the source code:
...
int
getaddrinfo_a (int mode, struct gaicb *list[], int ent, struct sigevent *sig)
{
...
no acess to list
...
/* Request the mutex. */
pthread_mutex_lock (&__gai_requests_mutex);
/* Now we can enqueue all requests. Since we already acquired the
mutex the enqueue function need not do this. */
for (cnt = 0; cnt < ent; ++cnt)
if (list[cnt] != NULL)
{
...
It acquires a mutex before accessing list.
Anyway it's similar to getaddrinfo which is required to be thread-safe:
The freeaddrinfo() and getaddrinfo() functions shall be thread-safe.
I want to make every element in an array of structure thread safe by using mutex lock for accessing each element of array.
This is my structure:
typedef struct {
void *value;
void *key;
uint32_t value_length;
uint32_t key_length;
uint64_t access_count;
void *next;
pthread_mutex_t *mutex;
} lruc_item;
I have an array of this structure, and want to use mutex locks in order to make structure elements thread safe.
I tried using the lock on one of the array element in a function and then intensionally didn't unlock it, just to ensure that my locks are working fine, but the strange thing was that there was no deadlock and the 2nd function accessing the same array element was able to access it.
Can some one please guide me on how to use mutexes to lock every element in a structure array (so as to make each element of the struture thread safe).
sample code to explain my point:
/** FUNCTION THAT CREATES ELEMENTS OF THE STRUCTURE **/
lruc_item *create_item(lruc *cache) {
lruc_item *item = NULL;
item = (lruc_item *) calloc(sizeof(lruc_item), 1);
item->mutex = (pthread_mutex_t *) malloc(sizeof(pthread_mutex_t));
if(pthread_mutex_init(item->mutex, NULL)) {
perror("LRU Cache unable to initialise mutex for page");
return NULL;
}
}
return item;
}
set()
{
item = create_item(cache);
pthread_mutex_lock(item->mutex);
item->value = value;
item->key = key;
item->value_length = value_length;
item->key_length = key_length;
item->access_count = ++cache->access_count;
pthread_mutex_unlock(item->mutex); /** (LINE P) tried commenting out this to check proper working of mutex(deadlock expected if the same "item" is accessed in another function) **/
}
get(lruc_item *item)
{
pthread_mutex_lock(item->mutex); /** deadlock doesn't occur when "LINE P" is commented out**/
*value = item->value;
item->access_count = ++cache->access_count;
pthread_mutex_unlock(item->mutex);
}
It's important to note that a mutex only locks out code from other threads. If you tried to execute WaitForMultipleObjects with the same mutex in the same thread it wouldn't block. I'm assuming Windows, because you haven't detailed that.
But, if you provide more detail, maybe we can pin-point where the issue really is.
Now, assuming again Windows, if you want to make accesses to the individual elements "thread-safe", you might want to consider the InterlockedExchange-class of functions instead of a mutex. For example:
InterlockExchange(&s.value_length, newValue);
or
InterlockedExchange64(&s.access_count, new64Value);
or
InterlockedExchangePointer(&s.value, newPointer);
If what you want to do is make sure multiple element accesses to the structure, as a transaction, is thread-safe, then mutex can do that for you. Mutex is useful across process boundaries. If you are only dealing within a single process, a critical section might be a better idea.