choosing between relaxed and acquire/release memory ordering in atomics - c

Question

I am implementing a very simple logic. Although, I am having a tough time figuring out the memory ordering to choose.
I have a simple struct like:
typedef unsigned int bitmap;
const size_t bits_count = sizeof(bitmap) * 8;
struct Container {
_Atomic(bitmap) reader_bitmap, writer_bitmap;
int cs[bits_count];
};
cs[bits_count] is the main deal here. Multiple producers will add data to it and multiple consumers will get data from it.
To synchronise this, I have 2 bitmaps, reader_bitmap and writer_bitmap. Both of them are initially 0. What I basically do is when I want to store a value to the array, I check the bitmap if it contains any unset bit and get that bit's position. Then I set the bit atomically and using this bit position as the index, I store data in cs[bit_posn]. Then to tell readers that data is available for reading, the same bit is set in reader_bitmap.
When some thread wants to get a value from this array, it will try to unset any one bit at bit_posn from reader_bitmap and get cs[bit_posn] and then it will set the same bit in writer_bitmap to tell the produces that this position is vacant for them to write.
Here is the code for the same (minimal reproducible example, I may have missed some edge cases):
#include <stdio.h>
#include <stdatomic.h>
#include <stdbool.h>
#define UNSET_BIT(bitmap, n) (bitmap & ~((typeof(bitmap))1 << n))
#define SET_BIT(bitmap, n) (bitmap | ((typeof(bitmap))1 << n))
typedef unsigned int bitmap;
const size_t bits_count = sizeof(bitmap) * 8;
struct Container {
_Atomic(bitmap) reader_bitmap, writer_bitmap;
int cs[bits_count];
};
bool store(struct Container *cont, int data) {
bitmap old_val, new_val;
int first_unset_bit;
old_val = atomic_load_explicit(&cont->writer_bitmap, memory_order_relaxed);
do {
first_unset_bit = __builtin_ffs(~old_val) - 1;
if (first_unset_bit == -1) return false;
new_val = SET_BIT(old_val, first_unset_bit);
} while(!atomic_compare_exchange_weak_explicit(
&cont->writer_bitmap,
&old_val,
new_val,
memory_order_relaxed,
memory_order_relaxed
));
cont->cs[first_unset_bit] = data;
old_val = atomic_load_explicit(&cont->reader_bitmap, memory_order_relaxed);
do {
new_val = SET_BIT(old_val, first_unset_bit);
} while(!atomic_compare_exchange_weak_explicit(
&cont->reader_bitmap,
&old_val,
new_val,
memory_order_relaxed,
memory_order_relaxed
));
return true;
}
int load(struct Container *cont, bool *did_load_fail) {
bitmap old_val, new_val;
int first_set_bit;
int retval;
*did_load_fail = false;
old_val = atomic_load_explicit(&cont->reader_bitmap, memory_order_relaxed);
do {
first_set_bit = __builtin_ffs(old_val) - 1;
if (first_set_bit == -1) {
*did_load_fail = true;
return -1;
}
new_val = UNSET_BIT(old_val, first_set_bit);
} while(!atomic_compare_exchange_weak_explicit(
&cont->reader_bitmap,
&old_val,
new_val,
memory_order_relaxed,
memory_order_relaxed
));
retval = cont->cs[first_set_bit];
old_val = atomic_load_explicit(&cont->writer_bitmap, memory_order_relaxed);
do {
new_val = UNSET_BIT(old_val, first_set_bit);
} while(!atomic_compare_exchange_weak_explicit(
&cont->writer_bitmap,
&old_val,
new_val,
memory_order_relaxed,
memory_order_relaxed
));
return retval;
}
int main(int argc, char *argv[])
{
return 0;
}
I tried to think about memory orderings here but it seems so vague to me. I need help understanding this in actual technical terms understanding what possible cases can actually happen.
I will try to give a general description on what my theory is. It maybe totally nonsense because I am really not sure how all this is actually working behind the scenes which is why I need help.
Let's say I chose to use release ordering everywhere like I did above. In store(), first I set the writer_bitmap's bit at bit_posn and then I set reader_bitmap at bit_posn.
Is it possible for some thread to "see" this bit_posn not set in writer_bitmap but set in reader_bitmap?
Let's say it was possible (actually want to know if it is or not), and then that thread called load(), and bit_posn was found set in reader_bitmap but that thread was still unaware that same bit is also set in writer_bitmap.
It would unset that in reader_bitmap and then try to unset the same in writer_bitmap. As per this, not knowing about writer_bitmap didn't matter as at last it was synchronized anyways.
Another case that I think of is writer_bitmap has a bit set but other thread doesn't know about it? (I guess this is not possible and is handled by atomics).
I know the question is pretty long. If it's unclear anywhere, please ask in the comments.