Send message through Ring (Circular) Buffer between Threads (in C) - c

I need to send a message from Main thread to my Created Thread using WinAPI and Ring Buffer.
I defined structures and wrote functions for my Ring buffer.
Ring Buffer - it contains head, tail, size and pointer to the structure Descriptor which has length of Data and data itself. As I need to send 2 parameters to CreateThread function, I created the third structure ThreadParams to keep 2 parameters.
I want to leave this structures how they are now, not changeable.
typedef struct _Descriptor
{
uint32_t dataLen;
void * data;
} Descriptor;
typedef struct _ringBuffer
{
Descriptor *bufferData;
int head;
int tail;
int size;
} ringBuffer;
typedef struct _ThreadParams
{
void * ptr1;
void * ptr2;
} ThreadParams;
There are my realisations of Ring Buffer functions:
void bufferFree(ringBuffer *buffer)
{
free(buffer->bufferData);
}
void ringInitialization(ringBuffer *buffer, int size)
{
buffer->size = size;
buffer->head = 0;
buffer->tail = 0;
buffer->bufferData = (Descriptor*)malloc(sizeof(Descriptor) * size);
}
int pushBack(ringBuffer *buffer, void * data) // fill buffer
{
buffer->bufferData[buffer->tail++] = *(Descriptor*)data;
if (buffer->tail == buffer->size)
{
buffer->tail = 0;
}
return 0;
}
int popFront(ringBuffer *buffer)
{
if (buffer->head != buffer->tail)
{
buffer->head++;
if (buffer->head == buffer->size)
{
buffer->head = 0;
}
}
return 0;
}
My main: I checked that I can send a few bytes (the memory is shared between threads), now I need to send a big message (> BUFF_SIZE) though Ring Buffer what I'm trying to do in while() cycle. Here is the question: how should I do it? My thing doesn't work because I catch an exception in printf() function (memory acces violation).
#include <stdio.h>
#include <stdlib.h>
#include <conio.h>
#include <windows.h>
#include <strsafe.h>
#include <stdint.h>
#define RING_SIZE 256
#define BUFFER_SIZE 1024
DWORD WINAPI HandleSendThread(LPVOID params);
uint8_t * getPointer(uint8_t *buffer, uint32_t index)
{
uint8_t * ptr = ((uint8_t*)buffer) + index * BUFFER_SIZE;
return ptr;
}
int main(int argc, char * argv[])
{
//Descriptor * ringData = (Descriptor *)malloc(sizeof(Descriptor) * RING_SIZE);
ringBuffer ring;
ringInitialization(&ring, RING_SIZE);
void * packetBuffer = malloc(BUFFER_SIZE * RING_SIZE);
uint8_t * currentBuffer = getPointer(packetBuffer, 0);
uint8_t * str = "Mr. and Mrs. Dursley, of number four, Privet Drive, were proud to say that they were perfectly normal, thank you very much. They were the last people you'd expect to be involved in anything strange or mysterious, because they just didn't hold with such nonsense. Mr.Dursley was the director of a firm called Grunnings, which made drills.He was a big, beefy man with hardly any neck, although he did have a very large mustache.Mrs.Dursley was thin and blonde and had nearly twice the usual amount of neck, which came in very useful as she spent so much of her time craning over garden fences, spying on the neighbors.The Dursleys had a small son called Dudley and in their opinion there was no finer boy anywhere.";
strcpy(currentBuffer, str);
ring.bufferData[0].data = currentBuffer;
ring.bufferData[0].dataLen = strlen(str);
int currentSize = 0;
int ringSize = RING_SIZE;
while(ring.bufferData[0].data != '\0')
{
for (int i = currentSize; i < ringSize; i + RING_SIZE)
{
pushBack(&ring, currentBuffer);
printf("h = %s, tail = %s, dataBuffer = %s\n", (char*)ring.head, (char*)ring.tail, (char*)ring.bufferData[i].data);
}
currentSize = ringSize;
ringSize = 2 * ringSize;
popFront(&ring);
}
ThreadParams params = { &ring, packetBuffer };
HANDLE MessageThread = 0;
MessageThread = CreateThread(NULL, 0, HandleSendThread, &params, 0, NULL);
if (MessageThread == NULL)
{
ExitProcess(MessageThread);
}
WaitForSingleObject(MessageThread, INFINITE);
CloseHandle(MessageThread);
system("pause");
return 0;
}
And my CreateThread function:
DWORD WINAPI HandleSendThread(LPVOID params)
{
ringBuffer * ring = ((ThreadParams*)params)->ptr1;
void * buffer = ((ThreadParams*)params)->ptr2;
//ring->bufferData[0].dataLen = sizeof(buffer) + sizeof(ring->bufferData[0])*1024;
printf("Shared memory check: ringBuffer data = \"%s\", \nlength = %d\n", (char*)ring->bufferData[0].data, ring->bufferData[0].dataLen);
return 0;
}

Your most immediate problem is the inconsistency between the code in pushBack(), which expects data to point to a Descriptor, and the code in your main function, which passes in a pointer to a string instead.
If you had declared pushBack() properly, i.e.,
void pushBack(ringBuffer *buffer, Descriptor * data)
{
buffer->bufferData[buffer->tail++] = *data;
if (buffer->tail == buffer->size)
{
buffer->tail = 0;
}
}
Then the compiler would have been able to warn you about the discrepancy.
You also have an infinite loop here:
for (int i = currentSize; i < ringSize; i + RING_SIZE)
You probably meant
for (int i = currentSize; i < ringSize; i += RING_SIZE)
... although it still doesn't look to me like it will do anything sensible. Nor do I understand the purpose of the outer loop, which compares a pointer to a character.

Found a solution
int main(int argc, char * argv[])
{
ringBuffer ring;
ringInitialization(&ring, RING_SIZE);
void * packetBuffer = malloc(BUFFER_SIZE * RING_SIZE);
Descriptor temp = { 0 };
uint8_t * currentBuffer = getPointer(packetBuffer, 0);
uint8_t * str = "Mr. and Mrs. Dursley, of number four, Privet Drive, were proud to say that they were perfectly normal, thank you very much. They were the last people you'd expect to be involved in anything strange or mysterious, because they just didn't hold with such nonsense. Mr.Dursley was the director of a firm called Grunnings, which made drills.He was a big, beefy man with hardly any neck, although he did have a very large mustache.Mrs.Dursley was thin and blonde and had nearly twice the usual amount of neck, which came in very useful as she spent so much of her time craning over garden fences, spying on the neighbors.The Dursleys had a small son called Dudley and in their opinion there was no finer boy anywhere.";
strcpy(currentBuffer, str);
temp.dataLen = strlen(str);
temp.data = currentBuffer;
pushBack(&ring, &temp);
ThreadParams params = { &ring, packetBuffer };
HANDLE MessageThread = 0;
MessageThread = CreateThread(NULL, 0, HandleSendThread, &params, 0, NULL);
if (MessageThread == NULL)
{
ExitProcess(MessageThread);
}
WaitForSingleObject(MessageThread, INFINITE);
CloseHandle(MessageThread);
system("pause");
return 0;
}
DWORD WINAPI HandleSendThread(LPVOID params)
{
ringBuffer * ring = ((ThreadParams*)params)->ptr1;
void * buffer = ((ThreadParams*)params)->ptr2;
Descriptor * temp = &ring->bufferData[ring->head];
for (int i = 0; i < temp->dataLen; i++)
{
printf("%c", ((char*)temp->data)[i]);
}
printf("\n");
return 0;
}

Related

How to realloc across multiple threads in C?

I have created a list of an undefined size using malloc which keeps track of threads and their data. However, if I realloc while the threads are still running, they are no longer able to save their data as the struct's memory location has changed. My project is constantly adding/subtracting threads so I need realloc.
Code:
#include <windows.h>
#include <stdio.h>
typedef struct option_data {
char* contract[3];
} option_data;
typedef struct thread_data {
char* thread;
} thread_data;
DWORD WINAPI optionSymbol(void* dat) {
option_data* data = (option_data*)dat;
data->contract[0] = 6;
data->contract[1] = 7;
data->contract[2] = 5;
return 0;
}
int create_thread(void* data, void* dat2) {
HANDLE thread = CreateThread(NULL, 0, optionSymbol, data, 0, NULL);
thread_data* t_data = (thread_data*)dat2;
t_data->thread = thread;
return 0;
}
void reallocation(lista, sizeOfList)
{
char** list = lista;
char* listThingy = realloc(*list, sizeOfList * sizeof * list);
if (listThingy == NULL)
free(listThingy);
else
*list = listThingy;
}
int getChains(void)
{
option_data* optionDataList = malloc(sizeof(option_data));
thread_data* threadDataList = malloc(sizeof(thread_data));
create_thread(&optionDataList[0], &threadDataList[0]);
//reallocation(&optionDataList, 2)
//reallocation(*threadDataList, 2) <-- The code returns unpredictably when un-noting these two lines.
create_thread(&optionDataList[0], &threadDataList[0]);
WaitForSingleObject(threadDataList[0].thread, INFINITE);
CloseHandle(threadDataList[0].thread);
printf("%i", optionDataList[0].contract[0]);
return 0;
}
int main()
{
getChains();
return 0;
}
How would I realloc without changing the original memory location of the structs or send the new memory location to the threads?

C: Multi producer, multi consumer bounded queue

I try (better tried) to implement a circular buffer with the following interface:
ring_buffer *ring_buffer_create(int capacity, int element_size);
void ring_buffer_destroy(ring_buffer *buffer)
const void *ring_buffer_read_acquire(ring_buffer *buffer, ring_buffer_loc *loc);
void ring_buffer_read_finish(ring_buffer *buffer, ring_buffer_loc loc);
void *ring_buffer_write_acquire(ring_buffer *buffer, ring_buffer_loc *loc);
void ring_buffer_write_finish(ring_buffer *buffer, ring_buffer_loc loc);
It should be possible to read / write multiple elements concurrently (and even in parallel). E.g.:
ring_buffer *buffer = ring_buffer_create(10, sizeof(int));
/* Write a single element */
ring_buffer_loc loc0;
int *i0 = ring_buffer_write_acquire(buffer, &loc);
*i0 = 42; // this could be a big data structure and way more expensive
ring_buffer_write_finish(buffer, loc0);
/* Write "concurrently" */
ring_buffer_loc loc1, loc2;
int *i1 = ring_buffer_write_acquire(buffer, &loc);
int *i2 = ring_buffer_write_acquire(buffer, &loc);
*i1 = 1729;
*i2 = 314;
ring_buffer_write_finish(buffer, loc1);
ring_buffer_write_finish(buffer, loc2);
All "acquire"-functions should be blocking until the operation is possible.
So far, so good. I thought this is simple and so I started with a clean implementation which is based on mutex. But soon I could see that this was far too slow for my use-case (100'000 writes and reads per second), so I switched over to spin-locks etc.
My implementation became quite messy and at some point (now), I started to think about why not something "simple" like this with the desired interface already exists? Probably, it is anyway not a great idea to re-implement something like this.
Maybe someone knows an implementation which has such an interface and which is blocking if the operation is not possible? I was looking quite long in the internet, but I could not find a good match for my problem. Maybe my desired interface is just "bad" or "wrong"?
Nevertheless, I add my current code. It basically assigns each "cell" (=value) a state which can be NONE (not set; the cell is basically empty), WRITING (someone acquired the cell to write data), READING (someone acquired the cell to read) and SET (the cell has a value which could be read). Each cell has a spin-lock which is used to update the cell state.
It then works like this:
When someone acquires a read and the current cell has the state "SET", then the value can be read (new state is READING) and the read index is increased. In all other cases a conditional variable is used to wait until an element is available. When an element read is finished, the cell state is changed to NONE and if any writers are waiting, a conditional variable signal is sent.
The same is true if a cell write is acquires. The only difference is that the cell needs the state "NONE" to be used and possible readers are signaled if there are any.
For some reasons the code sometimes locks and so I had to add a "dirty" timeout to my conditional variable. I would already be super happy if this could be solved, because the "timeout" basically makes the code polling (which is relatively ugly) and at the same time many context switches are done. Maybe someone sees the bug? The "new" code also has the disadvantage that it sometimes is really slow which is like a killer for my application. I attached the "old" and the "new" code (the changed lines are marked).
Thank you for helping me:)!
#include <stdio.h>
#include <stdlib.h>
#include <stdatomic.h>
#include <stdbool.h>
#include <string.h>
#include <time.h>
#include <assert.h>
#include <pthread.h>
#include <errno.h>
#include <unistd.h>
typedef int ring_buffer_loc;
enum t_ring_buffer_cell_state
{
NONE = 0,
WRITING = 1,
READING = 2,
SET = 3
};
typedef struct {
char *buffer; // data
atomic_int_fast8_t *states; // state per cell
pthread_spinlock_t *locks; // lock per cell
int capacity;
int element_size;
pthread_spinlock_t read_i_lock;
int read_i;
pthread_spinlock_t write_i_lock;
int write_i;
pthread_spinlock_t waiting_readers_lock;
int waiting_readers;
pthread_spinlock_t waiting_writers_lock;
int waiting_writers;
pthread_mutex_t value_written_lock;
pthread_mutex_t value_read_lock;
pthread_cond_t value_written;
pthread_cond_t value_read;
} ring_buffer;
ring_buffer *ring_buffer_create(int capacity, int element_size)
{
ring_buffer *res = calloc(1, sizeof(ring_buffer));
res->buffer = calloc(capacity, element_size);
res->states = calloc(capacity, sizeof(*res->states));
res->locks = malloc(capacity * sizeof(*res->locks));
for (int i = 0; i < capacity; ++i) {
pthread_spin_init(&res->locks[i], PTHREAD_PROCESS_PRIVATE);
}
pthread_spin_init(&res->write_i_lock, PTHREAD_PROCESS_PRIVATE);
pthread_spin_init(&res->read_i_lock, PTHREAD_PROCESS_PRIVATE);
pthread_spin_init(&res->waiting_readers_lock, PTHREAD_PROCESS_PRIVATE);
pthread_spin_init(&res->waiting_writers_lock, PTHREAD_PROCESS_PRIVATE);
res->capacity = capacity;
res->element_size = element_size;
return res;
}
void ring_buffer_destroy(ring_buffer *buffer)
{
free(buffer->buffer);
free(buffer->states);
free(buffer);
}
static inline void ring_buffer_inc_index(ring_buffer *buffer, int *index)
{
*index = (*index + 1) % buffer->capacity;
}
void timespec_now_plus_ms(struct timespec *result, long ms_to_add)
{
const int one_second_us = 1000 * 1000 * 1000;
timespec_get(result, TIME_UTC);
const long nsec = result->tv_nsec + ms_to_add * 1000 * 1000;
result->tv_sec += nsec / one_second_us;
result->tv_nsec += nsec % one_second_us;
}
const void *ring_buffer_read_acquire(ring_buffer *buffer, ring_buffer_loc *loc)
{
bool is_waiting = false;
start:
pthread_spin_lock(&buffer->read_i_lock);
const int read_i = buffer->read_i;
pthread_spinlock_t *cell_lock = &buffer->locks[read_i];
pthread_spin_lock(cell_lock);
const int state = buffer->states[read_i];
if (state == NONE || state == WRITING || state == READING) {
if (!is_waiting) {
is_waiting = true;
pthread_spin_lock(&buffer->waiting_readers_lock);
++buffer->waiting_readers;
pthread_mutex_lock(&buffer->value_written_lock);
pthread_spin_unlock(&buffer->waiting_readers_lock);
} else {
pthread_mutex_lock(&buffer->value_written_lock);
}
pthread_spin_unlock(cell_lock);
pthread_spin_unlock(&buffer->read_i_lock);
// "new" code:
// struct timespec ts;
// do {
// timespec_now_plus_ms(&ts, 50);
// } while (pthread_cond_timedwait(&buffer->value_written, &buffer->value_written_lock, &ts) == ETIMEDOUT && buffer->states[read_i] == state);
// pthread_mutex_unlock(&buffer->value_written_lock);
// "old" code (which hangs quite often):
pthread_cond_wait(&buffer->value_written, &buffer->value_written_lock);
pthread_mutex_unlock(&buffer->value_written_lock);
goto start;
} else if (state == SET) {
if (is_waiting) {
pthread_spin_lock(&buffer->waiting_readers_lock);
--buffer->waiting_readers;
assert(buffer->waiting_readers >= 0);
pthread_spin_unlock(&buffer->waiting_readers_lock);
}
buffer->states[read_i] = READING;
ring_buffer_inc_index(buffer, &buffer->read_i);
pthread_spin_unlock(&buffer->read_i_lock);
pthread_spin_unlock(cell_lock);
*loc = read_i;
return &buffer->buffer[read_i * buffer->element_size];
} else {
printf("unknown state!\n");
exit(1);
}
}
void ring_buffer_read_finish(ring_buffer *buffer, ring_buffer_loc loc)
{
pthread_spinlock_t *cell_lock = &buffer->locks[loc];
pthread_spin_lock(cell_lock);
buffer->states[loc] = NONE;
pthread_spin_unlock(cell_lock);
pthread_spin_lock(&buffer->waiting_writers_lock);
if (buffer->waiting_writers > 0) {
pthread_cond_signal(&buffer->value_read);
}
pthread_spin_unlock(&buffer->waiting_writers_lock);
}
void *ring_buffer_write_acquire(ring_buffer *buffer, ring_buffer_loc *loc)
{
bool is_waiting = false;
start:
pthread_spin_lock(&buffer->write_i_lock);
const int write_i = buffer->write_i;
pthread_spinlock_t *cell_lock = &buffer->locks[write_i];
pthread_spin_lock(cell_lock);
const int state = buffer->states[write_i];
if (state == SET || state == READING || state == WRITING) {
if (!is_waiting) {
is_waiting = true;
pthread_spin_lock(&buffer->waiting_writers_lock);
++buffer->waiting_writers;
pthread_mutex_lock(&buffer->value_read_lock);
pthread_spin_unlock(&buffer->waiting_writers_lock);
} else {
pthread_mutex_lock(&buffer->value_read_lock);
}
pthread_spin_unlock(cell_lock);
pthread_spin_unlock(&buffer->write_i_lock);
// "new" code:
// struct timespec ts;
// do {
// timespec_now_plus_ms(&ts, 5);
// } while (pthread_cond_timedwait(&buffer->value_read, &buffer->value_read_lock, &ts) == ETIMEDOUT && buffer->states[write_i] == state);
// pthread_mutex_unlock(&buffer->value_read_lock);
// "old" code (which hangs quite often):
pthread_cond_wait(&buffer->value_read, &buffer->value_read_lock);
pthread_mutex_unlock(&buffer->value_read_lock);
goto start;
} else if (state == NONE) {
if (is_waiting) {
pthread_spin_lock(&buffer->waiting_writers_lock);
--buffer->waiting_writers;
assert(buffer->waiting_writers >= 0);
pthread_spin_unlock(&buffer->waiting_writers_lock);
}
buffer->states[write_i] = WRITING;
ring_buffer_inc_index(buffer, &buffer->write_i);
pthread_spin_unlock(&buffer->write_i_lock);
pthread_spin_unlock(cell_lock);
*loc = write_i;
return &buffer->buffer[write_i * buffer->element_size];
} else {
printf("unknown state!\n");
exit(1);
}
}
void ring_buffer_write_finish(ring_buffer *buffer, ring_buffer_loc loc)
{
pthread_spinlock_t *cell_lock = &buffer->locks[loc];
pthread_spin_lock(cell_lock);
buffer->states[loc] = SET;
pthread_spin_unlock(cell_lock);
pthread_spin_lock(&buffer->waiting_readers_lock);
if (buffer->waiting_readers > 0) {
pthread_cond_signal(&buffer->value_written);
}
pthread_spin_unlock(&buffer->waiting_readers_lock);
}
/* just for debugging */
void ring_buffer_dump(const ring_buffer *buffer)
{
printf("RingBuffer\n");
printf(" Capacity: %d\n", buffer->capacity);
printf(" Element size: %d\n", buffer->element_size);
printf(" Read index: %d\n", buffer->read_i);
printf(" Write index: %d\n", buffer->write_i);
printf(" Cells:\n");
for (int i = 0; i < buffer->capacity; ++i) {
printf(" [%d]: STATE = ", i);
switch (buffer->states[i]) {
case NONE:
printf("NONE");
break;
case WRITING:
printf("WRITING");
break;
case READING:
printf("READING");
break;
case SET:
printf("SET");
break;
}
printf("\n");
}
printf("\n");
}
/*
* Test run
*/
struct write_read_n_conf {
ring_buffer *buffer;
int n;
};
static void *producer_thread(void *arg)
{
struct write_read_n_conf conf = *(struct write_read_n_conf *)arg;
for (int i = 0; i < conf.n; ++i) {
ring_buffer_loc loc;
int *value = ring_buffer_write_acquire(conf.buffer, &loc);
*value = i;
ring_buffer_write_finish(conf.buffer, loc);
if (i % 1000 == 0) {
printf("%d / %d\n", i, conf.n);
}
}
return NULL;
}
static void *consumer_thread(void *arg)
{
struct write_read_n_conf conf = *(struct write_read_n_conf *)arg;
int tmp;
bool ok = true;
for (int i = 0; i < conf.n; ++i) {
ring_buffer_loc loc;
const int *value = ring_buffer_read_acquire(conf.buffer, &loc);
tmp = *value;
ring_buffer_read_finish(conf.buffer, loc);
ok = ok && (tmp == i);
}
printf("ok = %d\n", ok);
return (void *)ok;
}
void write_read_n_parallel(int n)
{
ring_buffer *buffer = ring_buffer_create(50, sizeof(int));
struct write_read_n_conf conf = {
.buffer = buffer,
.n = n
};
pthread_t consumer;
pthread_t producer;
pthread_create(&consumer, NULL, consumer_thread, &conf);
pthread_create(&producer, NULL, producer_thread, &conf);
pthread_join(producer, NULL);
void *res;
pthread_join(consumer, &res); // hacky way to pass a bool: res == NULL means false, and otherwise true
assert(res != NULL);
}
int main() {
write_read_n_parallel(10000000);
}

Data exchange between Server and Client with Ring Buffer (Circular Buffer) in C

I wrote client-server chat for multiple clients (multiplexing i/o with non-blocking sockets).
Now I have fixed bufferSize (for example, length = 64).
I want to make data exchange between client and server with Ring Buffer. I also wrote a few functions (buffer inisialisation, popFront and pushBack) to write into a buffer.
How is it possible to realize this idea? For example, I suppose I need 2 structures: the first structure has bufferlength and pointer to data in the other buffer, the second buffer has data. But now I have no idea what I should do with it. Could you give me an advice and maybe show with code? Thanks.
Here are my ringBuffer structures and function (I suppose the function bufferSize is wrong):
struct ringBuffer
{
int *bufferData;
int head;
int tail;
int size;
};
void bufferFree(struct ringBuffer *buffer)
{
free(buffer->bufferData);
}
void bufferInitialization(struct ringBuffer *buffer, int size)
{
buffer->size = size;
buffer->head = 0;
buffer->tail = 0;
buffer->bufferData = (int*)malloc(sizeof(int) * size);
}
int pushBack(struct ringBuffer *buffer, int data)
{
buffer->bufferData[buffer->tail++] = data;
if (buffer->tail == buffer->size)
{
buffer->tail = 0;
}
return 0;
}
int popFront(struct ringBuffer *buffer)
{
if (buffer->head != buffer->tail)
{
buffer->head++;
if (buffer->head == buffer->size)
{
buffer->head = 0;
}
}
return 0;
}
int bufferSize(struct ringBuffer *buffer)
{
if (buffer->head >= buffer->tail)
{
return (buffer->head - buffer->tail);
}
else
{
return buffer->size - ((buffer->size - buffer->tail) + buffer->head);
}
/*for (int i = buffer->head; buffer->head < buffer->tail; i++)
{
printf("head[%d] and tail[%d] --> bufferData = %d", buffer->head, buffer->tail, buffer->bufferData);
}*/
return 0;
}
Basically, you need two things:
A shared ring of pointers between your clients and server.
A shared pool of buffers between your clients and server.
There are a variety of flavors for the ring buffer: lockless, multi-consumer, multi-producer etc.
Have a look at DPDK's ring library as an example or lockless rings. Here is a detailed description of algorithms:
http://dpdk.org/doc/guides/prog_guide/ring_lib.html
And here is the code:
http://dpdk.org/browse/dpdk/tree/lib/librte_ring

Memory comparison causes system halt

I am working on a kernel module and I need to compare two buffers to find out if they are equivalent. I am using the memcmp function defined in the Linux kernel to do so. My first buffer is like this:
cache_buffer = (unsigned char *)vmalloc(4097);
cache_buffer[4096] = '/0';
The second buffer is from a page using the page_address() function.
page = bio_page(bio);
kmap(page);
write_buffer = (char *)page_address(page);
kunmap(page);
I have printed the contents of both buffers before hand and not only to they print correctly, but they also have the same content. So next, I do this:
result = memcmp(write_buffer, cache_buffer, 2048); // only comparing up to 2048 positions
This causes the kernel to freeze up and I cannot figure out why. I checked the implementation of memcmp and saw nothing that would cause the freeze. Can anyone suggest a cause?
Here is the memcmp implementation:
int memcmp(const void *cs, const void *ct, size_t count)
{
const unsigned char *su1, *su2;
int res = 0;
for (su1 = cs, su2 = ct; 0 < count; ++su1, ++su2, count--)
if ((res = *su1 - *su2) != 0)
break;
return res;
}
EDIT: The function causing the freeze is memcmp. When I commented it out, everything worked. Also, when I did I memcmp as follows
memcmp(write_buffer, write_buffer, 2048); //comparing two write_buffers
Everything worked as well. Only when I throw the cache_buffer into the mix is when I get the error. Also, above is a simplification of my actual code. Here is the entire function:
static int compare_data(sector_t location, struct bio * bio, struct cache_c * dmc)
{
struct dm_io_region where;
unsigned long bits;
int segno;
struct bio_vec * bvec;
struct page * page;
unsigned char * cache_data;
char * temp_data;
char * write_data;
int result, length, i;
cache_data = (unsigned char *)vmalloc((dmc->block_size * 512) + 1);
where.bdev = dmc->cache_dev->bdev;
where.count = dmc->block_size;
where.sector = location << dmc->block_shift;
printk(KERN_DEBUG "place: %llu\n", where.sector);
dm_io_sync_vm(1, &where, READ, cache_data, &bits, dmc);
length = 0;
bio_for_each_segment(bvec, bio, segno)
{
if(segno == 0)
{
page = bio_page(bio);
kmap(page);
write_data = (char *)page_address(page);
//kunmap(page);
length += bvec->bv_len;
}
else
{
page = bio_page(bio);
kmap(page);
temp_data = strcat(write_data, (char *)page_address(page));
//kunmap(page);
write_data = temp_data;
length += bvec->bv_len;
}
}
printk(KERN_INFO "length: %u\n", length);
cache_data[dmc->block_size * 512] = '\0';
for(i = 0; i < 2048; i++)
{
printk("%c", write_data[i]);
}
printk("\n");
for(i = 0; i < 2048; i++)
{
printk("%c", cache_data[i]);
}
printk("\n");
result = memcmp(write_data, cache_data, length);
return result;
}
EDIT #2: Sorry guys. The problem was not memcmp. It was the result of memcmp. When ever it returned a positive or negative number, the function that called my function would play with some pointers, one of which was uninitialized. I don't know why I didn't realize it before. Thanks for trying to help though!
I'm no kernel expert, but I would assume you need to keep this memory mapped while doing the comparison? In other words, don't call kunmap until after the memcmp is complete. I would presume that calling it before will result in write_buffer pointing to a page which is no longer mapped.
Taking your code in the other question, here is a rough attempt at incremental. Still needs some cleanup, I'm sure:
static int compare_data(sector_t location, struct bio * bio, struct cache_c * dmc)
{
struct dm_io_region where;
unsigned long bits;
int segno;
struct bio_vec * bvec;
struct page * page;
unsigned char * cache_data;
char * temp_data;
char * write_data;
int length, i;
int result = 0;
size_t position = 0;
size_t max_size = (dmc->block_size * 512) + 1;
cache_data = (unsigned char *)vmalloc(max_size);
where.bdev = dmc->cache_dev->bdev;
where.count = dmc->block_size;
where.sector = location << dmc->block_shift;
printk(KERN_DEBUG "place: %llu\n", where.sector);
dm_io_sync_vm(1, &where, READ, cache_data, &bits, dmc);
bio_for_each_segment(bvec, bio, segno)
{
// Map the page into memory
page = bio_page(bio);
write_data = (char *)kmap(page);
length = bvec->bv_len;
// Make sure we don't go past the end
if(position >= max_size)
break;
if(position + length > max_size)
length = max_size - position;
// Compare the data
result = memcmp(write_data, cache_data + position, length);
position += length;
kunmap(page);
// If the memory is not equal, bail out now and return the result
if(result != 0)
break;
}
cache_data[dmc->block_size * 512] = '\0';
return result;
}

Producer/consumer with bounded buffer

Could someone check my code and tell me if I am on the right track.. It seems like I am a bit lost.. if you see my errors, please let me know them..
What I am trying to do is to solve bounded buffer using my own semaphores as well as GCD.
Thanks in advance..
sema.c
void procure( Semaphore *semaphore ) {
pthread_mutex_lock(semaphore->mutex1);
while(semaphore->value <= 0)
pthread_cond_wait(&semaphore->condition, semaphore->mutex1);
semaphore->value--;
pthread_mutex_unlock(semaphore->mutex1);
}
void vacate( Semaphore *semaphore ) {
pthread_mutex_lock(semaphore->mutex1);
semaphore->value++;
pthread_cond_signal(&semaphore->condition);
pthread_mutex_unlock(semaphore->mutex1);
}
void init ( Semaphore *semaphore ){
semaphore->value = 1;
pthread_mutex_t myMutex;
semaphore->mutex1 = &myMutex;
pthread_mutex_init( semaphore->mutex1, NULL);
}
void destroy ( Semaphore *semaphore ) {
pthread_mutex_destroy(semaphore->mutex1);
}
and main.c
struct variables {
Semaphore *sem;
};
struct variables vars;
void constructer (int *buffer, int *in, int *out) {
init(vars.sem);
}
void deconstructer () {
destroy(vars.sem);
}
int rand_num_gen() {
uint_fast16_t buffer;
int file;
int *rand;
file = open("/dev/random", O_RDONLY);
while( 1 ) {
read(file, &buffer, sizeof(buffer));
printf("16 bit number: %hu\n", buffer );
*rand = (int) buffer;
close(file);
break;
}
return *rand;
}
void put_buffer( int* buffer, int* in, int* out ) {
buffer[*in] = rand_num_gen(); // produce
procure(vars.sem); // wait here
*in = (*in + 1) % BUF_SIZE;
vacate(vars.sem);
}
void get_buffer( int* buffer, int* in, int* out ) {
int value;
procure(vars.sem);
value = buffer[*out];
vacate(vars.sem);
*out = (*out + 1) % BUF_SIZE;
}
int main (void) {
int *in, *out, *buffer;
constructer(buffer, in, out);
dispatch_queue_t producer, consumer;
producer = dispatch_queue_create("put_buffer", NULL);
consumer = dispatch_queue_create("get_buffer", NULL);
dispatch_async(producer,
^{
int i;
do
{
put_buffer( buffer, in, out );
dispatch_async(consumer,
^{
get_buffer( buffer, in, out );
if (i == RUN_LENGTH) exit(EXIT_SUCCESS);
});
}
while (i < RUN_LENGTH);
});
dispatch_main();
deconstructer();
exit (0);
}
Your code has a bug. In the init function you assign the address of a local variable to semaphore->mutex1, and when the function returns this address will be invalid. Later you still use this address, so this leads to undefined behavior.
You must either allocate the memory for the mutex directly in the semaphore (without a pointer) or allocate the memory via malloc.
Update:
Your program has so many bugs that you should definitely pick an easier topic to learn the basic concepts about memory management, how to allocate, use and reference a buffer, do proper error handling, etc. Here is a slightly edited version of your code. It still won't work, but probably has some ideas that you should follow.
#include <limits.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
void procure(Semaphore *semaphore) {
pthread_mutex_lock(semaphore->mutex1);
while (semaphore->value <= 0)
pthread_cond_wait(&semaphore->condition, semaphore->mutex1);
semaphore->value--;
pthread_mutex_unlock(semaphore->mutex1);
}
void vacate(Semaphore *semaphore) {
pthread_mutex_lock(semaphore->mutex1);
semaphore->value++;
pthread_cond_signal(&semaphore->condition);
pthread_mutex_unlock(semaphore->mutex1);
}
struct variables {
mutex_t sem_mutex;
Semaphore sem;
};
struct variables vars;
void constructor(int *buffer, int *in, int *out) {
vars.sem.value = 1;
vars.sem.mutex1 = &vars.sem_mutex;
pthread_mutex_init(vars.sem.mutex1, NULL);
}
void deconstructor() {
pthread_mutex_destroy(&semaphore->mutex1);
}
int rand_num_gen() {
const char *randomfile = "/dev/random";
unsigned char buffer[2]; // Changed: always treat files as byte sequences.
FILE *f = fopen(randomfile, "rb");
// Changed: using stdio instead of raw POSIX file access,
// since the API is much simpler; you don't have to care
// about interrupting signals or partial reads.
if (f == NULL) { // Added: error handling
fprintf(stderr, "E: cannot open %s\n", randomfile);
exit(EXIT_FAILURE);
}
if (fread(buffer, 1, 2, f) != 2) { // Added: error handling
fprintf(stderr, "E: cannot read from %s\n", randomfile);
exit(EXIT_FAILURE);
}
fclose(f);
int number = (buffer[0] << CHAR_BIT) | buffer[1];
// Changed: be independent of the endianness of the system.
// This doesn't matter for random number generators but is
// still an important coding style.
printf("DEBUG: random number: %x\n", (unsigned int) number);
return number;
}
void put_buffer( int* buffer, int* in, int* out ) {
buffer[*in] = rand_num_gen(); // produce
procure(&vars.sem); // wait here
*in = (*in + 1) % BUF_SIZE;
vacate(&vars.sem);
}
void get_buffer( int* buffer, int* in, int* out ) {
int value;
procure(&vars.sem);
value = buffer[*out];
vacate(&vars.sem);
*out = (*out + 1) % BUF_SIZE;
}
int main (void) {
int inindex = 0, outindex = 0;
int buffer[BUF_SIZE];
constructor(buffer, &inindex, &outindex);
// Changed: provided an actual buffer and actual variables
// for the indices into the buffer.
dispatch_queue_t producer, consumer;
producer = dispatch_queue_create("put_buffer", NULL);
consumer = dispatch_queue_create("get_buffer", NULL);
dispatch_async(producer, ^{
int i;
do {
put_buffer(buffer, &inindex, &outindex);
dispatch_async(consumer, ^{
get_buffer(buffer, &inindex, &outindex);
if (i == RUN_LENGTH) exit(EXIT_SUCCESS);
});
} while (i < RUN_LENGTH);
});
dispatch_main();
deconstructor();
exit (0);
}
As I said, I didn't catch all the bugs.

Resources