Shared memory to read an modify arrays - c

I am starting to learn basics of OS and I have no experience at all at C and pointers. I am doing something wrong in my code but I don't know how to fix it.I am trying to populate an array first process initializes the array the second updates the values randomly. my issues are:
the first process writes to shared memory then exits I want it to actually read the array after it is updated
I think I am storing the array in a wrong way
I need a way to tell each process when the other one is done to keep the communication live until the first process decide it is over. below are my codes:
First Process
int main()
{
key_t key;
int shm_id;
int *shm_ptr;
if ((key = ftok("/tmp", 'y')) == -1)
{
perror("ftok() failed");
exit(EXIT_FAILURE);
}
shm_id = shmget(key, SIZE*sizeof(int), IPC_CREAT | IPC_EXCL | 0600 );
if (shm_id == -1)
{
perror("Failed to get memory");
exit(EXIT_FAILURE);
}
shm_ptr = (int *) shmat ( shm_id, NULL, 0);
if (shm_ptr == (int *) -1)
{
perror( "shmat failed" ) ;
exit( EXIT_FAILURE ) ;
}
printf("Array Initialized as : \n");
for(int i=0; i<SIZE-1; i++)
{
shm_ptr[i]=-1;
}
// for(int i = 0; i< 30; i++)
// {
// printf(" %d", shm_ptr[i]);
// }
shmdt(shm_ptr);
// shmctl(shm_id,IPC_RMID, NULL);
return 0;
}
Second Process
int main()
{
key_t key;
int sum = 0;
int shm_id;
int *shm_ptr;
key = ftok("/tmp", 'y');
shm_id = shmget( key, 0 , 0600 );
if (shm_id == -1)
{
perror("Failed to get memory");
exit(EXIT_FAILURE);
}
shm_ptr = (int*) shmat(shm_id,NULL,0);
printf("Client array \n");
srand(time(NULL));
for(int i = 0; i< 30; i++)
{
shm_ptr[i] = rand() % 20 + 1;
}
for(int i = 0; i< 30; i++)
{
printf(" %d", shm_ptr[i]);
}
shmdt(shm_ptr);
shmctl(shm_id,IPC_RMID, NULL);
return 0;
}

There is a lack of process synchronization in your program. Essentially, your second process may try to obtain a shared memory segment which doesn't exist at this point in time. Since, you are working across two executables, procuring PID is not an option hence, we cannot use signals. But, we can make it work with shared memory segments alone.
Create an additional shared memory segment such as data_ready. You want to use IPC_CREAT on both but, not IPC_EXCL. And have your first process set it to one once your array has been initialized. Since, we have used IPC_CREAT, neither of the process will overwrite over the data if the process doesn't exist and if it doesn't exist, the first process will anyway initialize it to 0. You can in fact, remove IPC_EXCL in your first process if you're employing this.
Have your first process set data_ready to 1 once it has initialized the array and block your second process using a command such as
```while(data_ready==0);````
This is rather a crude method for doing this. You can of course, do this much better with threads, semaphores, message queues etc. But, for the time being, this should get the job done.

Related

Using pthread_join() on multiple threads giving unexpected behavior

I am learning how to use threads in C and have run into a problem when creating the threads. I am making a program that takes in 2 or more file names as command line arguments, counts the number of bytes in each file in their own thread, and then outputs the name of the largest file. When I use pthread_join() directly after creating a thread, the program runs as intended. However, I know this isn't how threads should be used because it defeats the purpose. When I use pthread_join() in a for loop after creating all the threads, then the program does not work correctly. Could anyone tell me what I am doing wrong? All help is appreciated. Here is my main function.
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER; //mutex for changing max_bytes and max_name
int max_bytes = 0;
char max_name[100];
struct arg_struct{ //struct to hold args to pass the threads
int fd;
char name[100];
};
int main(int argc, char* argv[])
{
if(argc < 3){ //checks for correct number of arguments passed
perror("Wrong number of arguments");
return EXIT_FAILURE;
}
int arg_num = argc - 1; //holds number of arguments passed
pthread_t threadid[arg_num]; //array of thread IDs
struct arg_struct args;
for(int i = 0; i < arg_num; i++){
args.fd = open(argv[i+1], O_RDONLY);
memcpy(args.name, argv[i+1], sizeof(args.name)); //copies file name into arg_struct
int thread_err = pthread_create(&threadid[i], NULL, count_bytes, (void*)&args); //create thread by calling count_bytes and passing it a struct of args
//pthread_join(threadid[i], NULL);
if(thread_err != 0){
perror("pthread_create failed");
return EXIT_FAILURE;
}
}
for(int i = 0; i < arg_num; i++){
pthread_join(threadid[i], NULL);
}
printf("%s is the largest of the submitted files\n", max_name);
return 0;
}
This is the function that the threads are running.
void *count_bytes(void* arguments)
{
struct arg_struct *args = (struct arg_struct*)arguments; //casting arguments back to struct from void*
int fd = args -> fd;
char name[100];
memcpy(name, args -> name, sizeof(name)); //copies file name into name from args.name
int bytes = 0;
int size = 10;
char* buffer = (char*) malloc(size);
if(buffer == NULL){
perror("malloc failed");
exit(EXIT_FAILURE);
}
int buffer_count = 0;
for(int i = 0; i < size; i++){
buffer[i] = '\0'; //sets all elements to '\0' to determine end of file later
}
int read_return = read(fd, &buffer[buffer_count], 1);
if(read_return == -1){
perror("reading failed");
exit(EXIT_FAILURE);
}
while(buffer[buffer_count] != '\0'){
bytes++;
buffer_count++;
buffer[buffer_count] = '\0'; //sets previous element to '\0' to determine end of file later
if(buffer_count >= size){
buffer_count = 0; //buffer will hold up to 10 elements and then go back to the beginning
}
read_return = read(fd, &buffer[buffer_count], 1);
if(read_return == -1){
perror("reading failed");
exit(EXIT_FAILURE);
}
}
printf("%s has %d bytes\n", name, bytes);
pthread_mutex_lock(&mutex);
if(bytes > max_bytes){
max_bytes = bytes;
memcpy(max_name, name, sizeof(max_name));
}
//locks mutex to avoid race condition
//then sets bytes to max_bytes if it is later than max_bytes
//then locks mutex to allow another thread to have access
pthread_mutex_unlock(&mutex);
return NULL;
}
If it is of any use, these are the two outputs produced when it is running correctly
./a.out another buffered_readword.c
another has 8 bytes
buffered_readword.c has 3747 bytes
buffered_readword.c is the largest of the submitted files
And not correctly
./a.out another buffered_readword.c
buffered_readword.c has 1867 bytes
buffered_readword.c has 1881 bytes
buffered_readword.c is the largest of the submitted files
The problem is that there is only one args structure. After pthread_create is called the new thread may not run immediately. By the time the threads run it is likely that they will both see the same args values. Calling pthread_join inside the thread creation loop "fixes" that because it ensures each thread finishes before args is updated to the next value.
To fix properly pass a different args to each thread. Illustrative code to do that:
struct arg_struct args[arg_num];
for(int i = 0; i < arg_num; i++){
args[i].fd = open(argv[i+1], O_RDONLY);
memcpy(args[i].name, argv[i+1], sizeof(args[i].name));
int thread_err = pthread_create(&threadid[i], NULL, count_bytes, &args[i]);
....

C Reader Writer Program, One reader isnt reading all the data

I am working on a reader/writer program where there is one writer to n readers. I am having an issue where if multiple readers are in, like the screenshot posted below, then the entire message from shared memory isnt displayed.
Output:
Enter a Message: Test
Reader1: Test
Reader2: Test
Writer: test test
Reader1: test
Reader2: test test
Writer:
Readers:
I have tried to add a count variable because I assume that the writers turn is being flagged before all readers have the ability to print and its making the writer then exit the nested while() in the writer and stop the readers from printing.
Any suggestions on to make the readers both print, whether it be a flag or some sort of count? Attached below are also screenshots of the writer and reader loops.
Reader:
int main() {
DataShared data;
data.turn = 0;
signal(SIGINT, sigHandler);
//generates key
key = ftok("mkey",65);
//returns an identifier in mId
if ((mId = shmget(key, SIZE, IPC_CREAT|S_IRUSR|S_IWUSR)) < 0){
perror("shared memory error");
exit(1);
}
// shmat to attach to shared memory
if((mPtr = shmat(mId, 0, 0)) == (void*) -1) {
perror("Can't attach\n");
exit(1);
}
while(1) {
// request critical section
while(!data.turn && data.count == 0) {
//not time for the reader, check if token is changed.
memcpy(&data, mPtr, sizeof(DataShared));
}
data.count++;
// enter critical section
usleep(1);
fprintf(stderr, "Read from memory: %s\n", data.message);
usleep(1);
// leave critical section
data.count--;
while(data.count > 0){
;
}
data.turn = 0;
memcpy(mPtr, &data, sizeof(DataShared));
};
return 0;
}
Writer:
int main() {
DataShared data;
data.turn = 0;
data.count = 0;
signal(SIGINT, sigHandler);
key = ftok("mkey",65);
if((shmId = shmget(key, SIZE, IPC_CREAT|S_IRUSR|S_IWUSR)) < 0 ) {
perror("Error creating shared memory\n");
exit(1);
}
if((shmPtr = shmat(shmId, 0, 0)) == (void*) -1) {
perror("Can't attach\n");
exit(1);
}
while(1) {
while (data.turn) {
memcpy(&data, shmPtr, sizeof(DataShared));
}
// enter critical section
printf("Enter a message: \n" );
fgets(data.message, 1024, stdin);
// leave critical section
printf("Message written to memory: %s\n", data.message);
data.turn = 1;
memcpy(shmPtr, &data, sizeof(DataShared));
};
return 0;
}
This may not be the explanation of your observation, but what you do is fishy.
You have multiple processes and the OS schedules each process.
First, there is no guarantee that all readers will read the message. It is very well possible that one reader finishes, sets the flag to 0 and copies the data back to shared memory before another reader had a chance to read the data.
Then your data.count. It starts with the local variable data of the writer. there you do not initialize data.count so it has an indeterminate value. In the readers you set it to 0 but it will be overwritten with the value from shared memory (the indeterminate value). You do a ++, later a -- and then wait for it to become 0. How would that ever become zero? That reader could wait forever.

Accessing a shared memory buffer in another process

I am trying to solve the producer consumer problem using mutexes and a shared buffer, but am having trouble accessing values in my shared buffer struct, specifically the char array. When I invoke the producer.c file in one terminal and print the values (the input is a txt file of the alphabet) using
printf("%c", newBuff->bytes[newBuff->rear]);
the chars do appear as normal, however when I do the same thing in consumer.c, but with
printf("%c", newBuff->bytes[newBuff->front]);
the values appear blank. The newBuff->front value is zero, so it should print the letter a. When I access other values in my struct in consumer.c like front, count, or rear they are correct. Share memory creation as well as attachment also works properly so I believe the issue is either I am not storing the char values properly in the array or I am trying to access them incorrectly. In the code below I placed the printf in the loop for producer.c and then outside the loop for consumer.c so I know for a fact a value is present before the consumer starts extracting data.
Consumer.c
typedef struct buffer{
pthread_mutex_t lock;
pthread_cond_t shout;
int front;
int rear;
int count;
int endOfFile;
char bytes[1024];
} buffer;
int main(int argc, char const *argv[]) {
int i=0;
FILE *file = fopen(argv[1], "w");
if (argc != 2){
printf("You must enter in a file name\n");
}
int shmid, swapCount=0;
char swapBytes[] = "";
char path[] = "~";
key_t key = ftok(path, 7);
buffer* newBuff;
if ((shmid = shmget(key, SIZE, 0666 | IPC_CREAT | IPC_EXCL)) != -1) {
newBuff = (buffer*) shmat(shmid, 0, 0);
printf("successful creation\n");
newBuff->front = 0;
newBuff->count = 0;
newBuff->endOfFile = 0;
pthread_mutexattr_t attr;
pthread_condattr_t condAttr;
pthread_mutexattr_init(&attr);
pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
pthread_mutex_init(&newBuff->lock, &attr);
pthread_condattr_init(&condAttr);
pthread_condattr_setpshared(&condAttr, PTHREAD_PROCESS_SHARED);
pthread_cond_init(&newBuff->shout, &condAttr);
} //shared memory creation
else if ((shmid = shmget(key, 0, 0)) != -1){
printf("%d\n", shmid);
printf("successful attachment\n" );
newBuff = (buffer*) shmat(shmid, 0, 0);
printf("%c\n", newBuff->count);
}
else{
printf("oops\n");
exit(0);
}
pthread_mutex_lock(&newBuff->lock);
printf("%c\n", newBuff->bytes[newBuff->front]);
while (newBuff->endOfFile != 1)
{
while (newBuff->count == 0){
pthread_cond_signal(&newBuff->shout);
pthread_cond_wait(&newBuff->shout, &newBuff->lock);
}
newBuff->front = ((newBuff->front + 1)%SIZE);
newBuff->count--;
}
pthread_mutex_unlock(&newBuff->lock);
shmdt(&newBuff);
//pthread_mutexattr_destroy(&attr);
//pthread_condattr_destroy(&condAttr);*/
return 0;
}
Producer.c
typedef struct buffer{
pthread_mutex_t lock;
pthread_cond_t shout;
int front;
int rear;
int count;
int endOfFile;
char bytes[1024];
} buffer;
int main(int argc, char const *argv[]) {
FILE *file = fopen(argv[1], "r");
if (argc != 2){
printf("You must enter in a file dumbass\n");
}
int shmid;
char path[] = "~";
key_t key = ftok(path, 7);
buffer* newBuff;
printf("dfasdfasdf\n");
if ((shmid = shmget(key, SIZE, 0666 | IPC_CREAT | IPC_EXCL)) != -1) {
newBuff = (buffer*) shmat(shmid, 0, 0);
printf("successful creation\n");
newBuff->front = 0;
newBuff->count = 0;
newBuff->endOfFile=0;
pthread_mutexattr_t attr;
pthread_condattr_t condAttr;
pthread_mutexattr_init(&attr);
pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
pthread_mutex_init(&newBuff->lock, &attr);
pthread_condattr_init(&condAttr);
pthread_condattr_setpshared(&condAttr, PTHREAD_PROCESS_SHARED);
pthread_cond_init(&newBuff->shout, &condAttr);
} //shared memory creation
else if ((shmid = shmget(key, 0, 0)) != -1){
printf("successful attachment\n" );
newBuff = (buffer*) shmat(shmid, 0, 0);
}
else{
printf("oops\n");
exit(0);
}
printf("%d\n", shmid);
pthread_mutex_lock(&newBuff->lock);
while (fscanf(file, "%c", &newBuff->bytes[newBuff->rear]) != EOF) //read in file
{
printf("%c\n", newBuff->bytes[newBuff->rear]);
while (newBuff->count >= SIZE){ //buffer is full
//("%c\n", newBuff->bytes[newBuff->rear]);
pthread_cond_signal(&newBuff->shout);
pthread_cond_wait(&newBuff->shout, &newBuff->lock);
}
//printf("%c\n", newBuff->bytes[newBuff->rear]);
newBuff->rear = ((newBuff->front + 1)%SIZE);
newBuff->count++;
}
newBuff->endOfFile = 1;
pthread_cond_signal(&newBuff->shout);
pthread_mutex_unlock(&newBuff->lock);
shmdt(&newBuff);
//pthread_mutexattr_destroy(&attr);
//pthread_condattr_destroy(&condAttr);
return 0;
}
There are several difficulties with your code, some already addressed in comments:
ftok() requires the path passed to it to designate an existing file, but the path you are passing does not.
You request less shared memory than you actually need: only the size of the buffer content, not of a whole struct buffer. Because the amount of shared memory actually allocated will be rounded up to a multiple of the page size, this may end up being ok, but you should ensure that it will be ok by requesting the amount you actually need.
System V shared memory segments have kernel persistence, so once created, they will continue to exist until they are explicitly removed or the system is rebooted. You never remove yours. You also initialize its contents only when you first create it. Unless you manually delete it between runs, therefore, you'll use old data -- with the end-of-file indicator set, for instance -- on the second and subsequent runs. I suggest having the consumer schedule it for removal.
The consumer prints only one byte of data from the buffer, and it does so before verifying that there is anything to read.
After adding a byte to the buffer, the producer does not update the available byte count until after signaling the consumer. At best, this is wasteful, because the consumer will not see the change in count until the next time (if any) it wakes.
The producer updates the rear index of the buffer incorrectly, based on the current front value instead of on the current rear value. The data will therefore not be written into the correct places in the buffer array.
Once the producer sets the endOfFile flag, the consumer ignores all but one of any remaining unread bytes.
If the producer leaves the count zero when it finishes, the consumer will deadlock.
I find that modified versions of your programs addressing all of these issues successfully and accurately communicate data through shared memory.
Update:
Also,
The way in which consumer and / or producer initializes the mutex and condition variable is not itself safe. It is possible for whichever process attempts the shmget() second (or third, or ...) to access those objects before the first finishes initializing them. More generally, once a shared memory segment is attached, there is no inherent memory barrier involved in writing to it. To address these issues, the natural companion to SysV shared memory is SysV semaphores.

Client server program with shared memory and semaphores in C

I don't really understand how shared memory works and I am trying to write a server-client program in which the server and client talk to each other using shared memory and semaphores.
Shared memory structure:
typedef struct shared_mem{
int board[BOARD_SIZE * BOARD_SIZE];
int goal;
int client;
int direction;
sem_t sem_server;
}shared_mem;
shared_mem *msg;
Server:
int shmid;
key_t key=ftok("2048_client.c", 42);
if(key == -1) {
printf("ftok failed");
return -1;
}
shared_mem *shm;
if ((shmid = shmget(key, sizeof(msg), IPC_CREAT|0600)) < 0) {
perror("shmget");
exit(1);
}
if ((shm = shmat(shmid, NULL, 0)) == (char *) -1) {
perror("shmat");
exit(1);
}
msg=shm;
int direction = -1;
srand(time(NULL));
//SERVER SETS VALUES FOR SHARED MEMORY STRUCTURE
sem_wait(&msg->sem_server);
// setup the board
initialize();
// the board starts with 2 pieces
create_game_piece();
printf("pieces created");
create_game_piece();
msg->client=0;
int i;
for (i = 0; i < BOARD_SIZE * BOARD_SIZE; i++)
msg->board[i] = board[i];
sem_post(&msg->sem_server);
// game loop
while (1) {
//CLIENT READS AND CHANGES VALUES
//SERVER READS VALUES CHANGED BY CLIENT
if (!move_board(direction))
continue;
sem_wait(&msg->sem_server);
moves++;
direction=msg->direction;
if (check_win()) {
print_board(-1);
printf("congratulations! you've won in %d moves\r\n", moves);
return 0;
}
create_game_piece();
if (!has_moves_left()) {
print_board(-1);
printf("you lose! try again\r\n");
//sleep(1);
return 1;
}
sem_post(&msg->sem_server);
}
Client:
int shmid;
key_t key=ftok("2048_client.c", 42);
if(key == -1) {
printf("ftok failed");
return -1;
}
shared_mem *shm;
msg=(shared_mem *)malloc(sizeof(shared_mem));
if ((shmid = shmget(key, sizeof(msg), IPC_CREAT|0600)) < 0) {
perror("shmget");
exit(1);
}
if ((shm = shmat(shmid, NULL, 0)) == (char *) -1) {
perror("shmat");
exit(1);
}
atexit(on_exit2);
system("stty raw");
srand(time(NULL));
while (1) {
// CLIENT READS VALUES CHANGED BY SERVER AND CHANGES THEM
sem_wait(&msg->sem_server);
print_board(direction);
direction = keypress();
msg->direction=direction;
sem_post(&msg->sem_server);
}
Could someone please tell me how to coordinate the way the client and server access the shared memory so that it works according to the comments in the code?
SHM_UNLOCK does not do what you think. It tries to lock pages in memory not prevent access. You need a semaphore to control access to the shared memory. See semctl, semget, etc., and employ them in both the client and server for mutually exclusive access.
There are a lot of things wrong with your code.
You did not initialize the semaphore at all. For this operation where the
semaphore is located in shared memory, make sure to set the pshared argument to nonzero.
You cannot do two-way synchronization this way with only one semaphore
(otherwise you do a sem_post, that is immediately sem_waited by the same
process, hence that thread may continue, the other one doesn't get to run).
You malloc'ed msg in the client, and never pushed it to the shared memory,
thus the shared memory is still not initialized.
What do you think system("stty raw") is going to accomplish, other than opening a seriously big can of security issues?
Obviously it wasn't runnable code. I've fixed quite a few small things, but gave up after half an hour because there's just too much wrong with it. Things like comparing a char* to a shared_mem* is wrong. Enable -Wall and -Wextra on your compiler and fix all warnings you get. Your code is full of it and unfortunately all indicate serious issues.
From a design perspective, there are even more issues.
Apparently you are using a 1-D array 'board' that is global in the server. This is not proper for two reasons:
The board is always 2-dimensional, why not use a 2D array?
You use it as a global variable, accessible by all functions. This is "javascript"-style programming. Please make the board local to main and pass it to the various functions.

threads have the same id

I learn threads. I have read that thread terminates after it is out of a function (that is passed as parameter to pthread_create function).
So I create threads in the loop, they are executed and afterwards they are terminated.
(sorry for some long code)
But when I call a function pthread_create, new threads get the same ids. Why?
struct data {
FILE *f;
};
void *read_line_of_file(void *gdata) {
pthread_mutex_lock(&g_count_mutex); // only one thread can work with file,
//doing so we block other threads from accessing it
data *ldata = (data *) gdata;
char line[80];
int ret_val =fscanf(ldata->f,"%s",line);
pthread_mutex_unlock(&g_count_mutex); // allow other threads to access it
if (ret_val != EOF)
printf("%s %lu\n ", line, pthread_self());
// some time consuming operations, while they are being executed by one thread,
// other threads are not influenced by it (if there are executed on different cores)
volatile int a=8;
for (int i=0;i <10000;i++ )
for (int i=0;i <10000;i++ ) {
a=a/7+i;
}
if (ret_val == EOF) // here thread ends
pthread_exit((void *)1);
pthread_exit((void *)0);
}
int main() {
int kNumber_of_threads=3, val=0;
pthread_t threads[kNumber_of_threads];
int ret_val_from_thread=0;
data mydata;
mydata.f = fopen("data.txt","r");
if ( mydata.f == NULL) {
printf("file is not found\n");
return 0;
}
for( ; val != 1 ;) {
// THIS IS THAT PLACE, IDs are the same (according to the number of processes),
// I expected them to be changing..
for(int i=0; i<kNumber_of_threads; i++) {
pthread_create(&threads[i],NULL,read_line_of_file, &mydata);
}
for(int i=0; i<kNumber_of_threads; i++) {
pthread_join(threads[i], (void **) &ret_val_from_thread);
if (ret_val_from_thread != 0)
val = ret_val_from_thread;
}
printf(" next %d\n",val);
}
printf("work is finished\n");
fclose(mydata.f);
return 0;
}
as result, I see that id of threads are not being changed:
I wonder, are new threads really created?
Thanks in advance!
Thread IDs are only guaranteed to be different among currently running threads. If you destroy a thread and create a new one, it may well be created with a previously used thread ID.

Resources