I'm trying to build a basic "game" server in C, I stock each file descriptor of an accepted socket in an int *, each of these are stocked in int ** clients. I then create a thread and pass clients as its argument.
Here's the code:
game_server_threaded.c
#define PLAYERS 4
int ** clients;
/* Setting up server... */
clients = malloc(sizeof(int *) * PLAYERS);
while (1)
{
for (int i = 0; i < PLAYERS; i++) {
clients[i] = malloc(sizeof(int));
*clients[i] = accept(server_sock,
(struct sockaddr *)&client_address, &size);
printf("s%d: %d\n", i, *clients[i]);
}
pthread_t th;
if (pthread_create(&th, NULL, play_game, clients) < 0)
die("create thread");
}
void * play_game(void * s)
void * play_game(void * s)
{
int ** clients = (int **) s;
srand (time(NULL));
int k = rand() % MAX_K;
int tries = 3;
int turn = 0;
for (int i = 0; i < PLAYERS; i++)
{
printf("s%d: %d\n", i, *clients[i]);
}
/* Game mechanics... */
}
However, the output is very weird. In the thread, all *clients[i] are set, except for *clients[0].
Here it is:
s0: 4
s1: 5
s2: 6
s3: 7
s0: 0
s1: 5
s2: 6
s3: 7
I've tested this extensively with different casts and such. I've also created a version which sends this struct to the thread:
struct player_threads {
char * msg;
char c;
int ** clients;
};
In the thread, msg and c are intact but the first element of clients is still 0!
I managed to get it working with a simple int * clients, but still, I can't comprehend this behaviour, could someone explain it to me?
Thank you.
EDIT****
The file from which the above code comes from is relatively short, it can be found here if you want to reproduce it.
https://pastebin.com/61nRKvQf
The created threads are sharing data that they expected to remain stable but is actually being modified outside the thread.
Specifically, the memory pointed to by clients, pointing to an "array" of int * is allocated before any threads are created. It is being filled in with a new set of int * before each new thread is created and is then passed to the new thread in the final argument of the pthread_create call. When the new thread examines the "array" of int *, it may see a mixture of the original elements at the time of the pthread_create call plus some elements that have since been overwritten by the main thread. Also, because assignment to an object of pointer type is not guaranteed to be atomic, it is also possible for the thread to access a partially updated (and therefore invalid) pointer value.
In the example given, the first four lines are printed by the main thread by this code:
for (int i = 0; i < PLAYERS; i++) {
clients[i] = malloc(sizeof(int));
*clients[i] = accept(server_sock,
(struct sockaddr *)&client_address, &size);
printf("s%d: %d\n", i, *clients[i]);
}
Output:
s0: 4
s1: 5
s2: 6
s3: 7
The main thread then creates a new thread:
pthread_t th;
if (pthread_create(&th, NULL, play_game, clients) < 0)
die("create thread");
The main thread then repeats the above for loop. The first thing that does is overwrite clients[0] with a new pointer value from malloc(sizeof(int)) and then it blocks on the accept call.
Meanwhile, the thread starts and prints stuff from the clients array, but clients[0] may have already been overwritten.
Output:
(At this point, the main thread has overwritten client[0]. The contents of client[0][0] are indeterminate at this point, but happens to contain 0 here.) ...
s0: 0
(At this point, the main thread is blocked on the call to accept, so clients[1] through to clients[3] still have the same values they had at the time of the call to pcreate_thread.) ...
s1: 5
s2: 6
s3: 7
To correct the problem, a new clients block needs to be allocated for each thread. That can be done by moving the allocation of clients into the loop that creates the new threads:
#define PLAYERS 4
int ** clients;
/* Setting up server... */
while (1)
{
clients = malloc(sizeof(int *) * PLAYERS);
for (int i = 0; i < PLAYERS; i++) {
clients[i] = malloc(sizeof(int));
*clients[i] = accept(server_sock,
(struct sockaddr *)&client_address, &size);
printf("s%d: %d\n", i, *clients[i]);
}
pthread_t th;
if (pthread_create(&th, NULL, play_game, clients) < 0)
die("create thread");
}
As an aside, there is unnecessary indirection in the use of int **clients since all that is really being passed in this example is one int value per player (containing the file descriptor of the accepted socket connection for that player). That could be simplified as follows:
Main thread
#define PLAYERS 4
int * clients;
/* Setting up server... */
while (1)
{
clients = malloc(sizeof(int) * PLAYERS);
for (int i = 0; i < PLAYERS; i++) {
clients[i] = accept(server_sock,
(struct sockaddr *)&client_address, &size);
printf("s%d: %d\n", i, clients[i]);
}
pthread_t th;
if (pthread_create(&th, NULL, play_game, clients) < 0)
die("create thread");
}
Game thread
void * play_game(void * s)
{
int * clients = s;
srand (time(NULL));
int k = rand() % MAX_K;
int tries = 3;
int turn = 0;
for (int i = 0; i < PLAYERS; i++)
{
printf("s%d: %d\n", i, clients[i]);
}
/* Game mechanics... */
}
For the version that passes a pointer to a struct player_threads to each thread, with the clients member type simplified to int *:
struct player_threads
struct player_threads {
char * msg;
char c;
int * clients;
};
Main thread
#define PLAYERS 4
struct player_threads * players;
/* Setting up server... */
while (1)
{
players = malloc(sizeof(*players));
players->clients = malloc(sizeof(int) * PLAYERS);
players->msg = "Hello";
players->c = 'X';
for (int i = 0; i < PLAYERS; i++) {
players->clients[i] = accept(server_sock,
(struct sockaddr *)&client_address, &size);
printf("s%d: %d\n", i, players->clients[i]);
}
pthread_t th;
if (pthread_create(&th, NULL, play_game, players) < 0)
die("create thread");
}
Game thread
void * play_game(void * s)
{
struct player_threads * players = s;
srand (time(NULL));
int k = rand() % MAX_K;
int tries = 3;
int turn = 0;
for (int i = 0; i < PLAYERS; i++)
{
printf("s%d: %d\n", i, players->clients[i]);
}
/* Game mechanics... */
}
As another aside, it would be better to replace the calls to srand and rand with a call to rand_r because the rand call might not be thread-safe:
unsigned int seed = time(NULL);
int k = rand_r(&seed) % MAX_K;
seed needs to be in storage local to the thread. In the above, it is on the thread's stack, but it may be better to provide storage for it as a member of struct player_threads:
struct player_threads {
char * msg;
char c;
unsigned int seed;
int * clients;
};
players->seed = time(NULL);
int k = rand_r(&players->seed) % MAX_K;
Related
I am learning how to use threads in C and have run into a problem when creating the threads. I am making a program that takes in 2 or more file names as command line arguments, counts the number of bytes in each file in their own thread, and then outputs the name of the largest file. When I use pthread_join() directly after creating a thread, the program runs as intended. However, I know this isn't how threads should be used because it defeats the purpose. When I use pthread_join() in a for loop after creating all the threads, then the program does not work correctly. Could anyone tell me what I am doing wrong? All help is appreciated. Here is my main function.
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER; //mutex for changing max_bytes and max_name
int max_bytes = 0;
char max_name[100];
struct arg_struct{ //struct to hold args to pass the threads
int fd;
char name[100];
};
int main(int argc, char* argv[])
{
if(argc < 3){ //checks for correct number of arguments passed
perror("Wrong number of arguments");
return EXIT_FAILURE;
}
int arg_num = argc - 1; //holds number of arguments passed
pthread_t threadid[arg_num]; //array of thread IDs
struct arg_struct args;
for(int i = 0; i < arg_num; i++){
args.fd = open(argv[i+1], O_RDONLY);
memcpy(args.name, argv[i+1], sizeof(args.name)); //copies file name into arg_struct
int thread_err = pthread_create(&threadid[i], NULL, count_bytes, (void*)&args); //create thread by calling count_bytes and passing it a struct of args
//pthread_join(threadid[i], NULL);
if(thread_err != 0){
perror("pthread_create failed");
return EXIT_FAILURE;
}
}
for(int i = 0; i < arg_num; i++){
pthread_join(threadid[i], NULL);
}
printf("%s is the largest of the submitted files\n", max_name);
return 0;
}
This is the function that the threads are running.
void *count_bytes(void* arguments)
{
struct arg_struct *args = (struct arg_struct*)arguments; //casting arguments back to struct from void*
int fd = args -> fd;
char name[100];
memcpy(name, args -> name, sizeof(name)); //copies file name into name from args.name
int bytes = 0;
int size = 10;
char* buffer = (char*) malloc(size);
if(buffer == NULL){
perror("malloc failed");
exit(EXIT_FAILURE);
}
int buffer_count = 0;
for(int i = 0; i < size; i++){
buffer[i] = '\0'; //sets all elements to '\0' to determine end of file later
}
int read_return = read(fd, &buffer[buffer_count], 1);
if(read_return == -1){
perror("reading failed");
exit(EXIT_FAILURE);
}
while(buffer[buffer_count] != '\0'){
bytes++;
buffer_count++;
buffer[buffer_count] = '\0'; //sets previous element to '\0' to determine end of file later
if(buffer_count >= size){
buffer_count = 0; //buffer will hold up to 10 elements and then go back to the beginning
}
read_return = read(fd, &buffer[buffer_count], 1);
if(read_return == -1){
perror("reading failed");
exit(EXIT_FAILURE);
}
}
printf("%s has %d bytes\n", name, bytes);
pthread_mutex_lock(&mutex);
if(bytes > max_bytes){
max_bytes = bytes;
memcpy(max_name, name, sizeof(max_name));
}
//locks mutex to avoid race condition
//then sets bytes to max_bytes if it is later than max_bytes
//then locks mutex to allow another thread to have access
pthread_mutex_unlock(&mutex);
return NULL;
}
If it is of any use, these are the two outputs produced when it is running correctly
./a.out another buffered_readword.c
another has 8 bytes
buffered_readword.c has 3747 bytes
buffered_readword.c is the largest of the submitted files
And not correctly
./a.out another buffered_readword.c
buffered_readword.c has 1867 bytes
buffered_readword.c has 1881 bytes
buffered_readword.c is the largest of the submitted files
The problem is that there is only one args structure. After pthread_create is called the new thread may not run immediately. By the time the threads run it is likely that they will both see the same args values. Calling pthread_join inside the thread creation loop "fixes" that because it ensures each thread finishes before args is updated to the next value.
To fix properly pass a different args to each thread. Illustrative code to do that:
struct arg_struct args[arg_num];
for(int i = 0; i < arg_num; i++){
args[i].fd = open(argv[i+1], O_RDONLY);
memcpy(args[i].name, argv[i+1], sizeof(args[i].name));
int thread_err = pthread_create(&threadid[i], NULL, count_bytes, &args[i]);
....
I am trying to read a file with aio.h byte by byte using aio_read with a number of threads. But I don't know if I am on the right track since there are not so many stuff to read on the Internet.
I have just created a worker function to pass it to my threads. And also as an argument to pass to the thread, I created a struct called thread_arguments and I pass a few necessary arguments in it, which will be needed for aiocb such as offset, file_path to open, buffer size, and priority.
I can read a file with one thread from start to end successfully. But when it comes to reading a file as chunks from within a few threads, I couldn't make it. And I am not even sure if I can do that with aio->reqprio without using semaphores or mutexes. (Trying to open a file from within a few threads at the same time?)
How can I read a few number of bytes from within a few threads asynchronously?
Let's say the file contains "foobarquax" and we have three threads using aio library.
Then first one should read "foo",
the second should read "bar" and
the last one should read "quax" asynchronously.
You can see screenshots of issues regarding running it with multiple threads on here
#include <stdio.h>
#include <pthread.h>
#include <stdlib.h>
#include <aio.h>
#include <string.h>
#include <fcntl.h> // open -> file descriptor O_RDONLY, O_WRONLY, O_RDWR
#include <errno.h>
#include <unistd.h>
typedef struct thread_args {
char *source_path;
char *destination_path;
long int buffer_size;
long int buffer_size_last; // buffer size for the last thread in case there is a remainder.
long int offset;
int priority;
} t_args;
void *worker(void *thread_args) {
t_args *arguments = (t_args *) thread_args;
struct aiocb *aiocbRead;
aiocbRead = calloc(1, sizeof(struct aiocb));
aiocbRead->aio_fildes = open(arguments->source_path, O_RDONLY);
if (aiocbRead->aio_fildes == -1) {
printf("Error");
}
printf("opened on descriptor %d\n", aiocbRead->aio_fildes);
aiocbRead->aio_buf = malloc(sizeof(arguments->buffer_size));
aiocbRead->aio_offset = arguments->offset;
aiocbRead->aio_nbytes = (size_t) arguments->buffer_size;
aiocbRead->aio_reqprio = arguments->priority;
int s = aio_read(aiocbRead);
if (s == -1) {
printf("There was an error");
}
while (aio_error(aiocbRead) == EINPROGRESS) {}
printf("Bytes read %ld", aio_return(aiocbRead));
close(aiocbRead->aio_fildes);
for (int i = 0; i < arguments->buffer_size ; ++i) {
printf("%c\n", (*((char *) aiocbRead->aio_buf + i)));
}
}
// Returns a random alphabetic character
char getrandomChar() {
int letterTypeFlag = (rand() % 2);
if (letterTypeFlag)
return (char) ('a' + (rand() % 26));
else
return (char) ('A' + (rand() % 26));
}
void createRandomFile(char *source, int numberofBytes) {
FILE *fp = fopen(source, "w");
for (int i = 0; i < numberofBytes; i++) {
fprintf(fp, "%c", getrandomChar());
}
fclose(fp);
}
int main(int argc, char *argv[]) {
char *source_path = argv[1];
char *destination_path = argv[2];
long int nthreads = strtol(argv[3], NULL, 10);
// Set the seed.
srand(time(NULL));
// Get random number of bytes to write to create the random file.
int numberofBytes = 10 + (rand() % 100000001);
// Create a random filled file at the source path.
createRandomFile(source_path, 100);
// Calculate the payload for each thread.
long int payload = 100 / nthreads;
long int payloadLast = payload + 100 % nthreads;
// Create a thread argument to pass to pthread.
t_args *thread_arguments = (t_args *) malloc(nthreads * sizeof(t_args));
for (int l = 0; l < nthreads; ++l) {
// Set arguments in the struct.
(&thread_arguments)[l]->source_path = source_path;
(&thread_arguments)[l]->destination_path = destination_path;
(&thread_arguments)[l]->buffer_size = payload;
(&thread_arguments)[l]->buffer_size_last = payloadLast;
(&thread_arguments)[l]->offset = l * payload;
(&thread_arguments)[l]->priority = l;
}
pthread_t tID[nthreads];
// Create pthreads.
for (int i = 0; i < nthreads; ++i) {
pthread_create(&tID[i], NULL, worker, (void *) &thread_arguments[i]);
}
// Wait for pthreads to be done.
for (int j = 0; j < nthreads; ++j) {
pthread_join(tID[j], NULL);
}
free(thread_arguments);
return 0;
}
This code is reading succesfully if I just call it from one thread but doesn't work if I use it for more than one threads which is what I want.
Can't seem to make any progress on this C assignment, hopefully somebody immediately spots the problem: Goal is to implement a thread pool using a POSIX message queue. The threads are created at start and then receive task after task from the message queue. The threads are expected to receive a 16 byte struct of type task_t that contains the function as well as a pointer to the arguments it should be executed with.
I get a segmentation fault as soon as the first task is being worked on by a thread, i.e. when executing function foo which can be accessed in threadPoolFun as task_ptr->func and is supposed to be run with argument task_ptr->this:
void * threadPoolFun(void * arg) {
task_t * task_ptr;
int size = attr.mq_msgsize; //size = 16
char buf[size];
while (1) {
int ret_val = mq_receive(task_queue, buf, size, NULL);
if (ret_val == -1) {
perror("Error receiving message: ");
}
task_ptr = (task_t *) buf;
if (task_ptr->func == NULL) {
break;
}
task_ptr->func(task_ptr->this);
}
return NULL;
}
Segmentation fault:
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x000000000040086a in foo (arg=0x7ffdd71fdfd0) at all_in_one.c:37
37 td->retVal = (double) td->tid;
[Current thread is 1 (Thread 0x7fec2d544700 (LWP 9853))]
(gdb) list
32 fooTask_t; // type for this tasks
33
34 //definition of task: write tid to retVal
35 void foo(void * arg) {
36 fooTask_t *td = (fooTask_t *) arg;
37 td->retVal = (double) td->tid;
38 }
39
40 // initialize task
41 void fooInit(fooTask_t * t) {
(gdb) backtrace
#0 0x000000000040086a in foo (arg=0x7ffdd71fdfd0) at all_in_one.c:37
#1 0x0000000000400b8c in threadPoolFun (arg=0x0) at all_in_one.c:118
#2 0x00007fec2f1196ba in start_thread (arg=0x7fec2d544700)
at pthread_create.c:333
#3 0x00007fec2ee4f41d in clone ()
Full code:
#include <mqueue.h>
#include <pthread.h>
#include <stdio.h>
//number of threads
#define NTH 4
//number of tasks
#define NTASKS 10
void * threadPoolFun(void * arg);
typedef void( * taskfun_t)(void * );
//minimal task data structure: func to be executed with args this, abstraction from actual task
typedef struct minTaskDataStruct {
void * this;
taskfun_t func;
}
task_t;
//data structure for actual task
typedef struct fooDataStruct {
// mandatory entries
void * this; // pointer to this structure
taskfun_t func; // function with signature: void foo(void *)
// data for individual task instances
long tid; // task id
double retVal; // return value
}
fooTask_t; // type for this tasks
//definition of task: write tid to retVal
void foo(void * arg) {
fooTask_t *td = (fooTask_t *) arg;
td->retVal = (double) td->tid;
}
// initialize task
void fooInit(fooTask_t * t) {
t-> this = t; // set this pointer
t-> func = foo; // set task function
}
//data structure for threads
pthread_t th[NTH];
//data structure for task queue attributes
static struct mq_attr attr;
//task queue
mqd_t task_queue;
//create task structs and store them in array td
fooTask_t td[NTASKS];
int main() {
printf("Setting up tasks\n");
for (int i = 0; i < NTASKS; i++) {
fooTask_t task;
task.tid = i;
fooInit(&task);
td[i] = task;
}
// set attributes
attr.mq_flags = 0; /* Flags: 0 or O_NONBLOCK */
attr.mq_maxmsg = 10; /* Max. # of messages on queue */
attr.mq_msgsize = 16; /* Max. message size (bytes) */
printf("Opening task queue\n");
// set up task queue
task_queue = mq_open("/my_task_queue_mq", O_CREAT | O_RDWR, 0700, & attr);
//create threads with default attributes, pass threadpool function
//threads will run as long as func passed to them is not NULL
for (long i = 0; i < NTH; i++) {
printf("Creating thread %ld\n", i);
pthread_create( & th[i], NULL, threadPoolFun, NULL);
}
//send tasks to queue, tasks to be consumed by threads, only send first 16 bytes
for (int i = 0; i < NTASKS; i++) {
printf("Sending task %d\n", i);
mq_send(task_queue, (const char * ) &td[i], sizeof(task_t), 0);
}
//send dummy tasks with func==NULL to terminate threads
for (int i = 0; i < NTASKS; i++) {
printf("Sending dummy task %d\n", i);
task_t dummy_task;
dummy_task.this = & dummy_task;
dummy_task.func = NULL;
mq_send(task_queue, (const char * ) & dummy_task, sizeof(task_t), 0);
}
//verify task execution
int sum1 = 0;
int sum2 = 0;
for (int i = 0; i < NTASKS; i++) {
sum1 += td[i].retVal;
sum2 += i;
}
if (sum1 == sum2) {
printf("Success: Sum1 %d equals Sum2 %d", sum1, sum2);
} else {
printf("Fail: Sum1 %d does not Sum2 %d", sum1, sum2);
}
return 0;
}
//threadPoolFun function definition
void * threadPoolFun(void * arg) {
task_t * task_ptr;
int size = attr.mq_msgsize; //size = 16
char buf[size];
while (1) {
int ret_val = mq_receive(task_queue, buf, size, NULL);
if (ret_val == -1) {
perror("Error receiving message: ");
}
task_ptr = (task_t *) buf;
if (task_ptr->func == NULL) {
break;
}
task_ptr->func(task_ptr->this);
}
return NULL;
}
Any help is appreciated, many thanks already in advance!
Lawrence
The problem is that you use addresses to objects on the stack after they are no longer there. You have this:
for (int i = 0; i < NTASKS; i++) {
fooTask_t task;
task.tid = i;
fooInit(&task);
td[i] = task;
}
&task is such a pointer. This is easy to fix, since you already have a place where you store the task. Here is better code:
for (int i = 0; i < NTASKS; i++) {
fooTask_t task;
task.tid = i;
td[i] = task;
fooInit(&td[i]);
}
The second place where this happens is here:
for (int i = 0; i < NTASKS; i++) {
printf("Sending dummy task %d\n", i);
task_t dummy_task;
dummy_task.this = & dummy_task;
dummy_task.func = NULL;
mq_send(task_queue, (const char * ) & dummy_task, sizeof(task_t), 0);
}
Since the dummy tasks don't need to be individually saved anywhere, you can move out the declaration of dummy_task, so it will live during the whole execution of main:
task_t dummy_task;
dummy_task.this = & dummy_task;
dummy_task.func = NULL;
//send dummy tasks with func==NULL to terminate threads
for (int i = 0; i < NTASKS; i++) {
fprintf(stderr, "Sending dummy task %d\n", i);
mq_send(task_queue, (const char * ) & dummy_task, sizeof(task_t), 0);
}
After this, there are still synchronization problems with your algorithm, but that is a separate problem, which I leave to you.
I changed the printing from stdout to stderr, since the former is buffered, and got lost when the program crashed.
This is for an Operating Systems programming assignment. I'm attempting to read n number of files, use threads to search each file for a number of occurrences for a specific character.
./mycount j new.txt some.txt here.txt hello.txt
The output for my test code as is should be:
argumentCount: 6
threadCount: 4
pthread_create() for thread 0 returns: 0
Thread 1
pthread_create() for thread 1 returns: 0
Thread 2
pthread_create() for thread 2 returns: 0
Thread 3
pthread_create() for thread 3 returns: 0
Thread 4
However each execution of mycount is different, with the last thread usually not executing/printing. Either that or they'll print sporadically, in tandem, etc.
I'm attempting to utilize a mutex to ensure the integrity of my data but I'm not sure what's really happening behind the scenes.
How do I ensure that everything finishes the same way each time? The last thread always returns 0, but it sometimes won't execute the function I give it completely.
Code:
//GLOBALS
int occurrences = 0;
//PROTOTYPES
void *scanFile( void *filePtr );
//Initialize space for mutex.
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
//Receive arguments from .exe call
void main ( int argumentCount, char *argumentVariables[] )
{
//Exit if argumentCount is > 50.
if (argumentCount > 50)
{
perror("Too many arguments. Enter less than 50.\n");
exit(EXIT_FAILURE);
}
printf("argumentCount: %d \n", argumentCount);
//Instantiate variables.
//i - iterator
//*newCommand - Used to hold string value of first command entered.
//*newVector - Used to hold string of the rest of the commands. Is a vector.
int i;
char *searchCharacter;
char *newVector[argumentCount];
//Iterate through command line arguments and split them.
for (i = 0; i < argumentCount; i++)
{
searchCharacter = argumentVariables[1];
if (i < argumentCount - 1)
{
newVector[i] = argumentVariables[1 + i];
}
else
{
newVector[i] = NULL;
}
}
//Exit if newest command is NULL.
if (searchCharacter == NULL)
{
perror("No character entered!\n");
exit(EXIT_FAILURE);
}
int threads = argumentCount - 2;
printf("threadCount: %d \n", threads);
pthread_t * thread = malloc(sizeof(pthread_t)*threads);
for (int i = 0; i < threads; i++)
{
int ret;
char *message = "Thread";
ret = pthread_create(&thread[i], NULL, scanFile, (void*) message);
if (ret != 0)
{
printf("Error - pthread_create() return code: %d\n", ret);
exit(EXIT_FAILURE);
}
printf("pthread_create() for thread %d returns: %d\n", i, ret);
}
exit(EXIT_SUCCESS);
}
void *scanFile( void *filePtr )
{
pthread_mutex_lock( &mutex );
char *message;
message = (char *) filePtr;
occurrences += 1;
printf("%s %d\n", message, occurrences);
pthread_mutex_unlock( &mutex );
}
Found the solution thanks to user2864740 and Ken Thomases.
Added:
for (int j = 0; j < threads; j++)
{
//Join the threads so all the data is good to go.
pthread_join(thread[j], NULL);
}
Correction:
for (int i = 0; i < threads; i++)
{
request[i].file = argumentVariables[i + 2];
request[i].character = searchCharacter;
//Create the thread. Any output from the integer newThread is an error code.
newThread = pthread_create(&thread[i], NULL, *scanFile, &request[i]);
if (newThread != 0)
{
printf("Error - pthread_create() return code: %d\n", newThread);
exit(EXIT_FAILURE);
}
}
for (int j = 0; j < threads; j++)
{
//Join the threads so all the data is good to go.
pthread_join(thread[j], NULL);
}
I am trying to implement a code to practice synchronization, so might not be best design or approach but goal is as below
Main thread
Creates a payload of 100 integers and waits for any thread to be available
When it gets signal from a thread its available - it unlocks the payload for copying and proceeds to create another payload
Worker thread
on creation of it makes itself available for data processing and sends signal that its available
Tries to lock the data payload from main thread and copy it to local array
( observing bug here - not able to access data properly)
Turn off the sign of available
( unable to turn off available state to off)
Keep processing data through local copy
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <stdbool.h>
#define WORKERS 2
#define ARRAY_ELEMENTS 100
#define MAX 1000
pthread_mutex_t mutex_bucket1 = PTHREAD_MUTEX_INITIALIZER;
pthread_mutex_t mutex_signal = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t cond_go = PTHREAD_COND_INITIALIZER;
pthread_cond_t cond_busy = PTHREAD_COND_INITIALIZER;
static int value = 0;
bool available = false;
void *worker_thread(void *pbucket)
{
sleep(5);
while(1)
{
unsigned int count = 0;
int local_array[ARRAY_ELEMENTS];
int *ptbucket = (int*)pbucket;
setbuf(stdout, NULL);
pthread_mutex_lock(&mutex_signal);
printf(" -------------- \n chainging state to available \n --------- ");
available = true;
printf(" -------------- \n from thread sending go signal \n --------- ");
pthread_cond_signal(&cond_go);
pthread_mutex_unlock(&mutex_signal);
pthread_mutex_lock(&mutex_bucket1);
printf(" -------------- \n data part locked in thread for copying \n --------- ");
while(count < ARRAY_ELEMENTS)
{
printf(" %d - \n", ptbucket[count]); /***incorrect values***/
local_array[count] = ptbucket[count];
count++;
}
pthread_mutex_unlock(&mutex_bucket1);
/*Never able to acquire mutex_signal and change state to not available*/ **BUG**
pthread_mutex_lock(&mutex_signal);
printf(" -------------- \n chainging state to not available \n --------- ");
available = false;
pthread_mutex_unlock(&mutex_signal);
count = 0;
while(count < ARRAY_ELEMENTS)
{
printf(" %d - \n", local_array[count]);
count++;
}
printf(" -------------- \n about to sleep for 5secs \n --------- ");
sleep(5);
}
}
int main(void)
{
pthread_t thread_id[WORKERS];
unsigned int* pbucket1 = (int*) malloc(sizeof(int) * ARRAY_ELEMENTS);
unsigned int* pbucket;
for(int i = 0; i < WORKERS - 1; i++)
{
pthread_create(&thread_id[i], NULL, worker_thread, (void *) pbucket);
}
for(int i = 0; i < MAX; i++)
{
unsigned int count = 0;
pbucket = pbucket1;
// Make the payload ready
pthread_mutex_lock(&mutex_bucket1);
printf(" -------------- creating data payload --------- \n");
while(count < ARRAY_ELEMENTS)
{
pbucket1[count] = i;
i++;
count++;
}
printf(" -------------- \n waiting for go signal \n --------- ");
while(!available)
{
pthread_cond_wait(&cond_go, &mutex_signal);
}
pthread_mutex_unlock(&mutex_bucket1);
/*I believe after we unlock variable "available" can be mutexed
again by other thread but seems thinking is flawed */
printf(" -------------- \n Main thread sleep for 3 seconds \n --------- ");
sleep(3);
}
for(int i = 0; i < WORKERS; i++)
{
pthread_join(thread_id[i], NULL);
}
return 0;
}
I think some of your idea is backwards; It shouldn't be the main context that is waiting, it should be the worker threads waiting for data ...
The job of the main thread should be to keep populating the payload and waking one thread at a time to process it.
So here's some scribbled code that is a little more sensible, I think:
/**
file: answer.c
compile: gcc -o answer answer.c -pthread
usage: answer [numThreads] [numElements]
**/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <pthread.h>
#define STATE_WAIT 1
#define STATE_READY 2
void *routine(void*);
typedef struct _shared_t {
pthread_mutex_t m;
pthread_cond_t c;
unsigned char state;
int *payload;
size_t numElements;
pthread_t *threads;
size_t numThreads;
} shared_t;
static inline void shared_init(shared_t *shared, size_t numThreads, size_t numElements) {
memset(shared, 0, sizeof(shared_t));
pthread_mutex_init(&shared->m, NULL);
pthread_cond_init(&shared->c, NULL);
shared->state = STATE_WAIT;
shared->numThreads = numThreads;
shared->numElements = numElements;
{
int it = 0;
shared->threads = (pthread_t*) calloc(shared->numThreads, sizeof(pthread_t));
while (it < shared->numThreads) {
if (pthread_create(&shared->threads[it], NULL, routine, shared) != 0) {
break;
}
it++;
}
}
}
static inline void shared_populate(shared_t *shared) {
if (pthread_mutex_lock(&shared->m) != 0) {
return;
}
shared->payload = (int*) calloc(shared->numElements, sizeof(int));
{
int it = 0,
end = shared->numElements;
while (it < end) {
shared->payload[it] = rand();
it++;
}
}
shared->state = STATE_READY;
pthread_cond_signal(&shared->c);
pthread_mutex_unlock(&shared->m);
}
static inline void shared_cleanup(shared_t *shared) {
int it = 0,
end = shared->numThreads;
while (it < end) {
pthread_join(shared->threads[it], NULL);
}
pthread_mutex_destroy(&shared->m);
pthread_cond_destroy(&shared->c);
free(shared->threads);
}
void* routine(void *arg) {
shared_t *shared = (shared_t*) arg;
int *payload;
do {
if (pthread_mutex_lock(&shared->m) != 0) {
break;
}
while (shared->state == STATE_WAIT) {
pthread_cond_wait(&shared->c, &shared->m);
}
payload = shared->payload;
shared->state = STATE_WAIT;
pthread_mutex_unlock(&shared->m);
if (payload) {
int it = 0,
end = shared->numElements;
while (it < end) {
printf("Thread #%ld got payload %p(%d)=%d\n",
pthread_self(), payload, it, payload[it]);
it++;
}
free(payload);
}
} while(1);
pthread_exit(NULL);
}
int main(int argc, char *argv[]) {
shared_t shared;
int numThreads = argc > 1 ? atoi(argv[1]) : 1;
int numElements = argc > 2 ? atoi(argv[2]) : 100;
shared_init(&shared, numThreads, numElements);
do {
shared_populate(&shared);
} while (1);
shared_cleanup(&shared);
return 0;
}
Obviously, the code above is not very tolerant of errors, and is not easy to shutdown cleanly ... it's illustration only.
Let's first look at main so that we know what the flow of the main program is going to be:
int main(int argc, char *argv[]) {
shared_t shared;
int numThreads = argc > 1 ? atoi(argv[1]) : 1;
int numElements = argc > 2 ? atoi(argv[2]) : 100;
shared_init(&shared, numThreads, numElements);
do {
shared_populate(&shared);
} while (1);
shared_cleanup(&shared);
return 0;
}
It keeps a shared_t on the stack:
typedef struct _shared_t {
pthread_mutex_t m;
pthread_cond_t c;
unsigned char state;
int *payload;
size_t numElements;
pthread_t *threads;
size_t numThreads;
} shared_t;
Mostly self explanatory, mutex, condition and state are required for synchronization.
First of all the shared_t must be initialized with mutex, condition, state and threads using the provided options:
static inline void shared_init(shared_t *shared, size_t numThreads, size_t numElements) {
memset(shared, 0, sizeof(shared_t));
pthread_mutex_init(&shared->m, NULL);
pthread_cond_init(&shared->c, NULL);
shared->state = STATE_WAIT;
shared->numThreads = numThreads;
shared->numElements = numElements;
{
int it = 0;
shared->threads = (pthread_t*) calloc(shared->numThreads, sizeof(pthread_t));
while (it < shared->numThreads) {
if (pthread_create(&shared->threads[it], NULL, routine, shared) != 0) {
break;
}
it++;
}
}
}
When the worker threads are created by this routine, they are forced into a waiting state.
The first call to shared_populate in the loop awakens the first thread after setting the payload to some random numbers:
static inline void shared_populate(shared_t *shared) {
if (pthread_mutex_lock(&shared->m) != 0) {
return;
}
shared->payload = (int*) calloc(shared->numElements, sizeof(int));
{
int it = 0,
end = shared->numElements;
while (it < end) {
shared->payload[it] = rand();
it++;
}
}
shared->state = STATE_READY;
pthread_cond_signal(&shared->c);
pthread_mutex_unlock(&shared->m);
}
Note the use of pthread_cond_signal over pthread_cond_broadcast, because we only want to wake the first thread.
void* routine(void *arg) {
shared_t *shared = (shared_t*) arg;
int *payload;
do {
if (pthread_mutex_lock(&shared->m) != 0) {
break;
}
while (shared->state == STATE_WAIT) {
pthread_cond_wait(&shared->c, &shared->m);
}
payload = shared->payload;
shared->state = STATE_WAIT;
pthread_mutex_unlock(&shared->m);
if (payload) {
int it = 0,
end = shared->numElements;
while (it < end) {
printf("Thread #%ld got payload %p(%d)=%d\n",
pthread_self(), payload, it, payload[it]);
it++;
}
free(payload);
}
} while(1);
pthread_exit(NULL);
}
So we wake up in routine at the call to pthread_cond_wait, the state has changed, so we break out of the loop, we save the pointer to the payload, reset the state to WAIT, and release the mutex.
At this point main can repopulate the payload and awaken the next thread, meanwhile the current worker thread can process, and then free the payload.
Some advice:
Always use as few mutex and condition variables as possible (KISS)
Research the atomic nature of condition variables
Always follow the basic rules regarding acquisition and release of mutex and signaling of condition variables:
If you locked it, unlock it.
Only ever wait for something: predicated wait loops are absolutely required, all the time.
If you can't reproduce what I done, then take the code and try to expand upon it; The first thing you need to do is be able to shutdown the process gracefully (enter shared_cleanup), maybe you need a variable sized payload, or some other requirement not mentioned in the original question.
Note about printf ... appending to a stream is not guaranteed to be atomic, it so happens that most of the time on *nix it is ... since we are just doing show and tell, we don't need to care about that ... ordinarily, do not rely on atomicity for any stream operations ...