I'm writing a kernel module, that uses a module-wide hashmap to store connections. I want to release all these connections, when the module is unloaded, delete them from the hashmap and then delete the whole map.
I defined the hashmap:
#define CONNECTIONS_HASH_BITS 10
static DEFINE_HASHTABLE(connection_hashtable, CONNECTIONS_HASH_BITS);
add entries with
hash_add_rcu(connection_hashtable, &con->t_hash,
oat_hash(&con->key, sizeof(struct hash_key)));
and finally want to delete all entries:
struct connection *result = NULL;
struct hlist_node *node;
unsigned int i;
hash_for_each_rcu(connection_hashtable, i, node, result, t_hash)
{
hash_del_rcu(node);
}
My questions:
Can I delete in the for loop hash_for_each_rcu?
How do I make this threadsafe?
Do I need to call something like free_hash for hashmap? (My guess here is no as it is an array and no kalloc was called, but I'm not that good with c)
Bonus: Do you have a good/easy tutorial on RCU in the Linux Kernel?
Thank you
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 2 years ago.
Improve this question
I'm working on a C project for an embedded target running a Linux distribution (build with Yocto). I'm new to Linux embedded world and I have to build a data logging system.
I'm already learning how to use threads, and I'm thinking on the project organization.
Here is what I imagine :
multiple threads to collect data from different interfaces, CAN bus, I2C... (different sample rate)
one thread with a sample rate of 200ms to populate csv files
one thread with a sample rate of 3 seconds to send data with http request
Threads will stop on CAN info or external event
I don't know what is the best way to organize this project. I see two ways, in the first a startup program create each thread and wait in a while loop with event watching to stop them. The second way is a startup program execute others binaries as thread.
In the two ways I don't know how share data between threads.
Can you share me your experience ?
Thank you
EDIT :
First, thanks a lot to #Glärbo, for your explanations. It's really helpful to learn multi threading mechanic.
I've tested it with success.
For future readers I've drawn diagrams to illustrate #Glärbo answer.
main thread
productor-sensor thread
datalogger thread
I would do it simpler, using a simple multiple producers, single consumer approach.
Let's assume each data item can be described using a single numerical value:
struct value {
struct value *next; /* Forming a singly-linked list of data items */
struct sensor *from; /* Identifies which sensor value this is */
struct timespec when; /* Time of sensor reading in UTC */
double value; /* Numerical value */
};
I would use two lists of values: one for sensor readings received but not stored, and one for unused value buckets. This way you don't need to dynamically allocate or free value buckets, unless you want to (by manipulating the unused list).
Both lists are protected by a mutex. Since the unused list may be empty, we need a condition variable (that is signaled on whenever a new unused value is added to it) so that threads can wait for one to become available. The received list similarly needs a condition variable, so that if it happens to be empty when the consumer (data storer) wants them, it can wait for at least one to appear.
static pthread_mutex_t unused_lock = PTHREAD_MUTEX_INITIALIZER;
static pthread_cond_t unused_wait = PTHREAD_COND_INITIALIZER;
static struct value *unused_list = NULL;
static pthread_mutex_t received_lock = PTHREAD_MUTEX_INITIALIZER;
static pthread_cond_t received_wait = PTHREAD_COND_INITIALIZER;
static struct value *received_list = NULL;
For the unused list, we need three helpers: one to create new unused value items from scratch (which you call initially to create say two or three value items per sensor, plus a few), and later on, if you think you need them (say, if you add new sensors run time):
int unused_create(void)
{
struct value *v;
v = malloc(sizeof *v);
if (!v)
return ENOMEM;
v->from = NULL;
pthread_mutex_lock(&unused_lock);
v->next = unused_list;
unused_list = v;
pthread_cond_signal(&unused_wait);
pthread_mutex_unlock(&unused_lock);
return 0;
}
The other two are needed to get and put value items from/back to the list:
struct value *unused_get(void)
{
struct value *v;
pthread_mutex_lock(&unused_lock);
while (!unused_list)
pthread_cond_wait(&unused_wait, &unused_lock);
v = unused_list;
unused_list = unused_list->next;
pthread_mutex_unlock(&unused_lock);
v->from = NULL;
return v;
}
void unused_put(struct value *v)
{
v->from = NULL;
pthread_mutex_lock(&unused_lock);
v->next = unused_list;
unused_list = v;
pthread_cond_signal(&unused_wait);
pthread_mutex_unlock(&unused_lock);
}
The idea above is that when the from member is NULL, the item is unused (as it is not from any sensor). Technically, we don't need to clear it to NULL at every stage, but I like to be thorough: it's not like setting it is a costly operation.
Sensor-accessing producers take the sensor reading, get the current time using e.g. clock_gettime(CLOCK_REALTIME, ×pec), and then use unused_get() to grab a new unused item. (The order is important, because unused_get() may take some time, if there are no free items.) Then, they fill in the fields, and call the following received_put() to prepend the reading to the list:
void received_put(struct value *v)
{
pthread_mutex_lock(&received_lock);
v->next = received_list;
received_list = v;
pthread_mutex_signal(&received_wait);
pthread_mutex_unlock(&received_lock);
}
There is only one thread that periodically collects all received sensor readings, and stores them. It can keep a set of most recent readings, and send those periodically. Instead of calling some received_get() repeatedly until there are no more received values not handled yet, we should use a function that returns the whole list of them:
struct value *received_getall(void)
{
struct value *v;
pthread_mutex_lock(&received_lock);
while (!received_list)
pthread_cond_wait(&received_wait, &received_lock);
v = received_list;
received_list = NULL;
pthread_mutex_unlock(&received_lock);
return v;
}
The consumer thread, storing/sending the summaries and readings, should obtain the whole list, then handle them one by one. After each item has been processed, they should be added to the unused list. In other words, something like
struct value *all, v;
while (1) {
all = receive_getall();
while (all) {
v = all;
all = all->next;
v->next = NULL;
/* Store/summarize value item v */
unused_put(v);
}
}
As you can see, while the consumer thread is handling the sensor value items, the sensor threads can add new readings for the next round, as long as there are enough free value item buckets to use.
Of course, you can also allocate lots of values at one malloc() call, but then you must somehow remember which pool of values each value belongs to to free them. So:
struct owner {
size_t size; /* Number of value's */
size_t used; /* Number of value's not freed yet */
struct value value[];
};
struct value {
struct value *next; /* Forming a singly-linked list of data items */
struct owner *owner; /* Part of which value array, NULL if standalone */
struct sensor *from; /* Identifies which sensor value this is */
struct timespec when; /* Time of sensor reading in UTC */
double value; /* Numerical value */
};
int unused_add_array(const size_t size)
{
struct owner *o;
struct value *v;
size_t i;
o = malloc(sizeof (struct owner) + size * sizeof (struct value));
if (!o)
return ENOMEM;
o->size = size;
o->used = used;
i = size - 1;
pthread_mutex_lock(&unused_lock);
o->value[i].next = unused_list;
while (i-->0)
o->value[i].next = o->value + i + 1;
unused_list = o->value[0];
pthread_cond_broadcast(&unused_wait);
pthread_mutex_unlock(&unused_lock);
return 0;
}
/* Instead of unused_put(), call unused_free() to discard a value */
void unused_free(struct value *v)
{
pthread_mutex_lock(&unused_lock);
v->from = NULL;
if (v->owner) {
if (v->owner->used > 1) {
v->owner->used--;
return;
}
v->owner->size = 0;
v->owner->used = 0;
free(v->owner);
return;
}
free(v);
return;
}
The reason unused_free() uses unused_lock is that we must be sure that no other thread is accessing the bucket when we free it. Otherwise, we can have a race window, where the other thread may use the value after we free()d it.
Remember that the Linux C library, like most other C libraries, does not return dynamically allocated memory to the operating system at free(); memory is only returned if it is large enough to matter. (Currently on x86 and x86-64, the glibc limit is about 132,000 bytes or so; anything smaller is left in the process heap, and used to satisfy future malloc()/calloc()/realloc() calls.)
The contents of the struct sensor are up to you, but personally, I'd put at least
struct sensor {
pthread_t worker;
int connfd; /* Device or socket descriptor */
const char *name; /* Some kind of identifier, perhaps header in CSV */
const char *units; /* Optional, could be useful */
};
plus possibly sensor reading interval (in, say, milliseconds) in it.
In practice, because there is only one consumer thread, I'd use the main thread for it.
I am trying to develop the cache simulator using fifo algorithm. I understand how the fifo algorithm works, however I have problems with understanding how to implement it. I am provided with template for developing the cache. I wonder about the good way to implement the algorithm.
#include <stdlib.h>
extern int opt_assoc, opt_block, opt_capacity, opt_repl, opt_verbose;
typedef struct {
int set; // set ID: 0 ~ (#sets - 1)
int blk; // block ID: 0 ~ (#blocks - 1)
bool miss; // whether this access is a miss
bool evict; // whether eviction happens
bool is_victim_dirty; // if evicted, whether the evicted block is dirty
} cache_state_t;
cache_state_t cache(unsigned int addr, char type)
{
// type: either 'l' or 's'
cs.miss = true;
static bool first_time = true;
if (first_time ) {
first_time = false;
}
cache_state_t cs;
return cs;
}
Your code is incorrect and incomplete, I think you're just starting to program in C. Use ANSI C programming language (the book) to learn how to implement a doubly linked list or a circular queue, and then go from there.
To give more hints cache_state_t will be a node in doubly linked list or a circular queue, once you have filled the cache the next request if its a miss will result in removing the head node of cache and putting the new cache node in the tail.
FIFO cache replacement is normally given in the class assignments, very rarely these strategies are implemented commercially, LRU is the horizon of commercial cache management solutions.
Ideally this should be a comment but then I don't have commenting privileges.
I am trying to create a simple top utility xv6. To do this, I have created a system call that will allow me to to access the kernel space. I have followed many guides on how to create system calls and my code compiles without issues.
My problem exists when I try running top in qemu. I will get a trap 14 error whenever I try to access my struct array whether it be in kernel or user space.
To break things down a little, I have a top.c file:
...
int main(){
struct uproc table[MAX];//where MAX is defined as 10
if(!getprocs(MAX, table))
printf("YAY");
...
in sysproc.c:
...
int
sys_getprocs(int max, struct uproc table[]){
return gettable(table);
}
and then in procs.c
....
int gettable(struct uproc table[]){
struct uproc u;
struct proc *p;
int i = 0;
aquire(&ptable.lock);
for(p->state.proc; p < &ptable.proc[NPROC];p++){
if(//p->state is valid){
u.state = p->state;
...
table[i] = u;//where I get the trap 14 error
}
}
}
Again, my assumption is that when I pass table around from user to kernel it's getting corrupted, but with that said I'm not sure how I could properly pass it.
All xv6 sys_* functions are parameterless.
Read the argint, argstr, and argptr functions in order to figure our how to pass parameters to the kernel from user mode,
Since I will use the Linux's built-in crypto API for different purposes, I've been reading the sources carefully.
While attempting to understand what's going on, I have been struck by the instance handling part of the code.
The code is C, nevertheless this part is clearly object-oriented. Thus, when a user needs to perform some encryption, it will ask for the allocation of a transform instance which will involve the use of a particular algorithm in a particular encryption mode. The pair (algorithm, mode) will be handled this way in the crypto API :
The "pure" algorithm is stored in a struct crypto_alg structure.
The mode is defined through a struct crypto_template which says how to initialize a specific struct crypto_instance
Here are the definitions of the structures crypto_instance and crypto_spawn
struct crypto_instance {
struct crypto_alg alg;
/*
* The alg struct will be filled according to the template 'alloc' method
*/
struct crypto_template *tmpl;
struct hlist_node list;
void *__ctx[] CRYPTO_MINALIGN_ATTR;
};
struct crypto_spawn {
struct list_head list; // Embedded list_head to list the spawns
struct crypto_alg *alg; // Ptr to the underlying algorithm
struct crypto_instance *inst;
const struct crypto_type *frontend;
u32 mask;
}
Then, to me it looks like there is a kind of hierarchy : the crypto_spawn is here to manage the combination of an encryption mode defined by a crypto_instance and a pure encryption algorithm defined by a crypto_alg. In terms of model object, we can see that crypto_instance inherits crypto_alg and it is totally reasonable to consider "cascading" templates over a given algorithm.
The thing I don't understand is why the crypto_spawn and the crypto_instance structures are not merged together. This question has even more struck me when I met the instance allocation code :
struct crypto_instance *crypto_alloc_instance(const char *name,
struct crypto_alg *alg)
{
struct crypto_instance *inst;
struct crypto_spawn *spawn;
int err;
inst = crypto_alloc_instance2(name, alg, 0);
if (IS_ERR(inst))
goto out;
spawn = crypto_instance_ctx(inst);
/*
* This line expands to :
* spawn = inst->__ctx;
*/
err = crypto_init_spawn(spawn, alg, inst,
CRYPTO_ALG_TYPE_MASK | CRYPTO_ALG_ASYNC);
if (err)
goto err_free_inst;
return inst;
err_free_inst:
kfree(inst);
inst = ERR_PTR(err);
out:
return inst;
}
The crypto_alloc_instance2 just allocates the physical memory to hold the structure and fills the inst->alg structure's name fields :
void *crypto_alloc_instance2(const char *name, struct crypto_alg *alg,
unsigned int head)
{
struct crypto_instance *inst;
char *p;
int err;
p = kzalloc(head + sizeof(*inst) + sizeof(struct crypto_spawn),
GFP_KERNEL);
if (!p)
return ERR_PTR(-ENOMEM);
inst = (void *)(p + head);
/* Names editing + error checking ... */
return p;
}
As you can see, a spawn is "physically" bound to an instance, so why are they defined separately. I find it very troubling when then trying to understand how the whole thing is mixed and handled.
At the moment, the only reason that came to my mind is that it allows the API to opacify the underlying algorithm from the crypto_instance object. But since the last bytes of the structure will easily give the spawn through the spawn = inst->__ctx instruction, it is not very very opaque.
As a recall after this bunch of code and comments, the question is :
What reasons lead the developers to make this "separation" between the instance and the spawn structure ?
Thanks in advance for any enlightenment !
Note : I added the tag cryptography because I thought that developers interested in this tag have probably already taken a look into the Linux Kernel crypto part.
I want to make every element in an array of structure thread safe by using mutex lock for accessing each element of array.
This is my structure:
typedef struct {
void *value;
void *key;
uint32_t value_length;
uint32_t key_length;
uint64_t access_count;
void *next;
pthread_mutex_t *mutex;
} lruc_item;
I have an array of this structure, and want to use mutex locks in order to make structure elements thread safe.
I tried using the lock on one of the array element in a function and then intensionally didn't unlock it, just to ensure that my locks are working fine, but the strange thing was that there was no deadlock and the 2nd function accessing the same array element was able to access it.
Can some one please guide me on how to use mutexes to lock every element in a structure array (so as to make each element of the struture thread safe).
sample code to explain my point:
/** FUNCTION THAT CREATES ELEMENTS OF THE STRUCTURE **/
lruc_item *create_item(lruc *cache) {
lruc_item *item = NULL;
item = (lruc_item *) calloc(sizeof(lruc_item), 1);
item->mutex = (pthread_mutex_t *) malloc(sizeof(pthread_mutex_t));
if(pthread_mutex_init(item->mutex, NULL)) {
perror("LRU Cache unable to initialise mutex for page");
return NULL;
}
}
return item;
}
set()
{
item = create_item(cache);
pthread_mutex_lock(item->mutex);
item->value = value;
item->key = key;
item->value_length = value_length;
item->key_length = key_length;
item->access_count = ++cache->access_count;
pthread_mutex_unlock(item->mutex); /** (LINE P) tried commenting out this to check proper working of mutex(deadlock expected if the same "item" is accessed in another function) **/
}
get(lruc_item *item)
{
pthread_mutex_lock(item->mutex); /** deadlock doesn't occur when "LINE P" is commented out**/
*value = item->value;
item->access_count = ++cache->access_count;
pthread_mutex_unlock(item->mutex);
}
It's important to note that a mutex only locks out code from other threads. If you tried to execute WaitForMultipleObjects with the same mutex in the same thread it wouldn't block. I'm assuming Windows, because you haven't detailed that.
But, if you provide more detail, maybe we can pin-point where the issue really is.
Now, assuming again Windows, if you want to make accesses to the individual elements "thread-safe", you might want to consider the InterlockedExchange-class of functions instead of a mutex. For example:
InterlockExchange(&s.value_length, newValue);
or
InterlockedExchange64(&s.access_count, new64Value);
or
InterlockedExchangePointer(&s.value, newPointer);
If what you want to do is make sure multiple element accesses to the structure, as a transaction, is thread-safe, then mutex can do that for you. Mutex is useful across process boundaries. If you are only dealing within a single process, a critical section might be a better idea.