PThreads.Consumer-Producer! printing in different files - c

i m trying to implement the producer - consumer with threads.In my code below is working well except of the point where i try to print into 2 files.(one for prod and one for cons).I use fclose(stdout) . when i delete it inside producer then the print in producer file is ok but consumers no.when i deleted at all then all of the data printed into the consumer file.My question is what i should try to make the prints correctly( i need consumer to print in cons.out.txt and producer in prod_out.txt) .Also i didnt include check for threads to make it simple but it works well.
#include "head.h"
circular_buffer cb;
pthread_mutex_t lock;
pthread_cond_t cond;
char * data[6]; //is the data taken from cmd(argv[0-6]) first one is the program the other are parameters
int main(int argc , char *argv[]) {
validate(argv,argc); //if command arguments are valid go on
for(int i=0; i < argc; i++){
data[i]=argv[i]; //copy argv array to data array
}
cb_init(&cb,10,sizeof(int)); //initialization of circular buffer for 10 nodes + size of int each node
pthread_t cons, prod;
void *consumer(), *producer();
int rc = pthread_mutex_init(&lock, NULL);
rc = pthread_cond_init(&cond, NULL);
int id[]={1,2}; //prod id is 1 and cons 2
rc = pthread_create(&prod, NULL, producer, &id[0]);
rc = pthread_create(&cons, NULL, consumer, &id[1]);
rc=pthread_join(prod, NULL);
rc=pthread_join(cons, NULL);
rc = pthread_mutex_destroy(&lock);
rc = pthread_cond_destroy(&cond);
cb_free(&cb); //free allocated memory which is alocated in head
return 1;
}
void *consumer(void * id){
int thread_id= *((int*)id); //cast to (int)
int file;
FILE * fp2 = fopen("./cons_out.txt","wb"); //creates new file with "cons_out.txt" name
int i =10;
while(i > 0){
i--;
pthread_mutex_lock(&lock);
while(isEmpty(&cb)){
int rc = pthread_cond_wait(&cond,&lock);
}//end while
int item=0; // item that will be consumed
cb_pop_front(&cb,&item);
file = open("./cons_out.txt", O_APPEND | O_WRONLY, 0777 );//open file as new
if( file == -1 ) {//fail to open file
perror("./cons_out.txt");
return EXIT_FAILURE;
}
fclose(stdout);
dup2( file, STDOUT_FILENO); //change redirections
printf("Consumer %d : %d\n",thread_id,item); //print output to file
pthread_cond_signal(&cond);
pthread_mutex_unlock(&lock);
} //end while
return NULL;
} //end consumer()
void *producer(void * id){
int thread_id= *((int*)id); //cast to (*int)
int file1;
FILE * fp1 = fopen("./prod_in.txt","wb"); //creates new file with "prod_in.txt" name
for(int i=0; i<10; i++){ //it produce 10 numbers
pthread_mutex_lock(&lock);
while(isFull(&cb)){
int rc = pthread_cond_wait(&cond,&lock);
}//end while
int seed=1;
int item=rand(); // item of this producer
cb_push_back(&cb,&item);
file1 = open("./prod_in.txt", O_APPEND | O_WRONLY, 0777 );//open file as new
if( file1 == -1 ) {//fail to open file
perror("./prod_in.txt");
return EXIT_FAILURE;
}
fclose(stdout);
dup2( file1, STDOUT_FILENO); //change output
printf("Producer %i : %i\n",thread_id,item); //print output to file
pthread_cond_signal(&cond);
pthread_mutex_unlock(&lock);
} //end for
return NULL;
} //end Producer();

There is only one open file table for your whole process, shared by all threads in it. There is one entry in that table for the standard output, and your producer and consumer are contending for it. It's not clear why.
I'm inclined to think that the immediate reason for your troubles is that fclose()ing the stdout stream releases the stream-level resources, especially the buffer and state-tracking details. open()ing a file descriptor does not provide those, nor does dup2()ing a file descriptor into a FILE object representing a closed stream.
But it baffles me why you are making it so hard. Why in the world muck with duping file descriptors? Your producer and consumer already fopen() the wanted output files, thereby each obtaining its own stream connected with the appropriate file (fp1 and fp2). It seems to me that all you need to do, then, is to write to those streams via fprintf(), etc., instead of going to a lot of trouble to be able to use printf() instead.
That would be more efficient, too, since you oughtn't to need to keep opening and closing the files. Open each once, at the beginning, as you already do, and close each once, at the end, which you presently fail to do.

Related

Critical Section Synchronization C

Trying to implement the critical section of this program to actually swap between both threads correctly as stated later in the description.
I am trying to do a problem for my Operating Systems course. The problem is that I need to input two files, one of each are put into their separate threads, they will read through the file till they hit a line with the number "0". Then the other thread is supposed to run by the same rules.
This program is supposed to take two file input and figure out the message by concatenating both of the inputs from the files in a specific order and then printing out the output after it has deciphered it.
The inputs of these two files as shown below
Person1 Person2
--------- ----------
t 0
0 h
i 0
s 0
0 i
0 s
0 a
t 0
e 0
0 s
t 0
the above inputs should result in a string with this output
Example: “thisisatest”
Currently what is going wrong with the assignment is that it is not swapping correctly between the two threads and sitting in infinite loops.
Like I said above I am trying to solve this assignment by use of Mutexes and Pthreads
Below is the current implementation of my code
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <pthread.h>
static pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
static char *charArray[1000];
void *threadPerson1(void *value){
FILE *fPTR;
char buffer[2];
char *fileName = "Person1.txt";
fPTR = fopen(fileName, "r");
if (fPTR == NULL){
printf("was unable to open: %s\n", fileName);
return NULL;
}
while(1){
//Entering the critical section
pthread_mutex_lock(&mutex);
fscanf(fPTR, "%s", buffer);
printf("This is Person1: %s\n", buffer);
if(buffer != "0"){
charArray[count] = buffer;
count++;
}
if(buffer == "0"){
pthread_mutex_unlock(&mutex);
}
//exiting the critical section
}
}
void *threadPerson2(void *value){
FILE *fPTR;
char buffer[2];
char *fileName = "Person2.txt";
fPTR = fopen(fileName, "r");
if (fPTR == NULL){
printf("was unable to open: %s\n", fileName);
return NULL;
}
while(1){
//entering the critical section
pthread_mutex_lock(&mutex);
fscanf(fPTR, "%s", buffer);
printf("This is Person2: %s\n", buffer);
if(buffer != "0"){
charArray[count] = buffer;
count++;
}
if(feof(fPTR)){
printf("read end of file of: Person2\n");
fclose(fPTR);
return NULL;
}
if(buffer == "0"){
pthread_mutex_unlock(&mutex);
}
//exiting the critical section
}
}
int main(int argc, char **argv){
pthread_t thread1;
pthread_t thread2;
pthread_create(&thread1, NULL, threadPerson1, NULL);
pthread_create(&thread2, NULL, threadPerson2, NULL);
pthread_join(thread1, NULL);
pthread_join(thread2, NULL);
for(int x = 0; x < 1000; x++){
if(charArray[x] == NULL){
printf("\n");
break;
}else{
printf("%s", charArray[x]);
}
}
return 0;
}
At least two things are incorrect in your program. First of all, if one thread releases the mutex, you are not guaranteed that the scheduler will allow the other thread will run, the releasing thread may very well go on and reacquire the mutex right away. Use a condition variable, read its manpages. Also, here are some examples, in particular 4-8: Multithreaded programming guide
Two, when you reach end of file, you need to release the mutex to clean up. Easiest way to do it is what we call RAII in C++, i.e. use a resource handle that releases the mutex when the handle object goes out of scope. You can do similar things in C e.g. by registering a cleanup function, or a 'goto' to end of function with cleanup code called from there.

Accessing a shared memory buffer in another process

I am trying to solve the producer consumer problem using mutexes and a shared buffer, but am having trouble accessing values in my shared buffer struct, specifically the char array. When I invoke the producer.c file in one terminal and print the values (the input is a txt file of the alphabet) using
printf("%c", newBuff->bytes[newBuff->rear]);
the chars do appear as normal, however when I do the same thing in consumer.c, but with
printf("%c", newBuff->bytes[newBuff->front]);
the values appear blank. The newBuff->front value is zero, so it should print the letter a. When I access other values in my struct in consumer.c like front, count, or rear they are correct. Share memory creation as well as attachment also works properly so I believe the issue is either I am not storing the char values properly in the array or I am trying to access them incorrectly. In the code below I placed the printf in the loop for producer.c and then outside the loop for consumer.c so I know for a fact a value is present before the consumer starts extracting data.
Consumer.c
typedef struct buffer{
pthread_mutex_t lock;
pthread_cond_t shout;
int front;
int rear;
int count;
int endOfFile;
char bytes[1024];
} buffer;
int main(int argc, char const *argv[]) {
int i=0;
FILE *file = fopen(argv[1], "w");
if (argc != 2){
printf("You must enter in a file name\n");
}
int shmid, swapCount=0;
char swapBytes[] = "";
char path[] = "~";
key_t key = ftok(path, 7);
buffer* newBuff;
if ((shmid = shmget(key, SIZE, 0666 | IPC_CREAT | IPC_EXCL)) != -1) {
newBuff = (buffer*) shmat(shmid, 0, 0);
printf("successful creation\n");
newBuff->front = 0;
newBuff->count = 0;
newBuff->endOfFile = 0;
pthread_mutexattr_t attr;
pthread_condattr_t condAttr;
pthread_mutexattr_init(&attr);
pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
pthread_mutex_init(&newBuff->lock, &attr);
pthread_condattr_init(&condAttr);
pthread_condattr_setpshared(&condAttr, PTHREAD_PROCESS_SHARED);
pthread_cond_init(&newBuff->shout, &condAttr);
} //shared memory creation
else if ((shmid = shmget(key, 0, 0)) != -1){
printf("%d\n", shmid);
printf("successful attachment\n" );
newBuff = (buffer*) shmat(shmid, 0, 0);
printf("%c\n", newBuff->count);
}
else{
printf("oops\n");
exit(0);
}
pthread_mutex_lock(&newBuff->lock);
printf("%c\n", newBuff->bytes[newBuff->front]);
while (newBuff->endOfFile != 1)
{
while (newBuff->count == 0){
pthread_cond_signal(&newBuff->shout);
pthread_cond_wait(&newBuff->shout, &newBuff->lock);
}
newBuff->front = ((newBuff->front + 1)%SIZE);
newBuff->count--;
}
pthread_mutex_unlock(&newBuff->lock);
shmdt(&newBuff);
//pthread_mutexattr_destroy(&attr);
//pthread_condattr_destroy(&condAttr);*/
return 0;
}
Producer.c
typedef struct buffer{
pthread_mutex_t lock;
pthread_cond_t shout;
int front;
int rear;
int count;
int endOfFile;
char bytes[1024];
} buffer;
int main(int argc, char const *argv[]) {
FILE *file = fopen(argv[1], "r");
if (argc != 2){
printf("You must enter in a file dumbass\n");
}
int shmid;
char path[] = "~";
key_t key = ftok(path, 7);
buffer* newBuff;
printf("dfasdfasdf\n");
if ((shmid = shmget(key, SIZE, 0666 | IPC_CREAT | IPC_EXCL)) != -1) {
newBuff = (buffer*) shmat(shmid, 0, 0);
printf("successful creation\n");
newBuff->front = 0;
newBuff->count = 0;
newBuff->endOfFile=0;
pthread_mutexattr_t attr;
pthread_condattr_t condAttr;
pthread_mutexattr_init(&attr);
pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
pthread_mutex_init(&newBuff->lock, &attr);
pthread_condattr_init(&condAttr);
pthread_condattr_setpshared(&condAttr, PTHREAD_PROCESS_SHARED);
pthread_cond_init(&newBuff->shout, &condAttr);
} //shared memory creation
else if ((shmid = shmget(key, 0, 0)) != -1){
printf("successful attachment\n" );
newBuff = (buffer*) shmat(shmid, 0, 0);
}
else{
printf("oops\n");
exit(0);
}
printf("%d\n", shmid);
pthread_mutex_lock(&newBuff->lock);
while (fscanf(file, "%c", &newBuff->bytes[newBuff->rear]) != EOF) //read in file
{
printf("%c\n", newBuff->bytes[newBuff->rear]);
while (newBuff->count >= SIZE){ //buffer is full
//("%c\n", newBuff->bytes[newBuff->rear]);
pthread_cond_signal(&newBuff->shout);
pthread_cond_wait(&newBuff->shout, &newBuff->lock);
}
//printf("%c\n", newBuff->bytes[newBuff->rear]);
newBuff->rear = ((newBuff->front + 1)%SIZE);
newBuff->count++;
}
newBuff->endOfFile = 1;
pthread_cond_signal(&newBuff->shout);
pthread_mutex_unlock(&newBuff->lock);
shmdt(&newBuff);
//pthread_mutexattr_destroy(&attr);
//pthread_condattr_destroy(&condAttr);
return 0;
}
There are several difficulties with your code, some already addressed in comments:
ftok() requires the path passed to it to designate an existing file, but the path you are passing does not.
You request less shared memory than you actually need: only the size of the buffer content, not of a whole struct buffer. Because the amount of shared memory actually allocated will be rounded up to a multiple of the page size, this may end up being ok, but you should ensure that it will be ok by requesting the amount you actually need.
System V shared memory segments have kernel persistence, so once created, they will continue to exist until they are explicitly removed or the system is rebooted. You never remove yours. You also initialize its contents only when you first create it. Unless you manually delete it between runs, therefore, you'll use old data -- with the end-of-file indicator set, for instance -- on the second and subsequent runs. I suggest having the consumer schedule it for removal.
The consumer prints only one byte of data from the buffer, and it does so before verifying that there is anything to read.
After adding a byte to the buffer, the producer does not update the available byte count until after signaling the consumer. At best, this is wasteful, because the consumer will not see the change in count until the next time (if any) it wakes.
The producer updates the rear index of the buffer incorrectly, based on the current front value instead of on the current rear value. The data will therefore not be written into the correct places in the buffer array.
Once the producer sets the endOfFile flag, the consumer ignores all but one of any remaining unread bytes.
If the producer leaves the count zero when it finishes, the consumer will deadlock.
I find that modified versions of your programs addressing all of these issues successfully and accurately communicate data through shared memory.
Update:
Also,
The way in which consumer and / or producer initializes the mutex and condition variable is not itself safe. It is possible for whichever process attempts the shmget() second (or third, or ...) to access those objects before the first finishes initializing them. More generally, once a shared memory segment is attached, there is no inherent memory barrier involved in writing to it. To address these issues, the natural companion to SysV shared memory is SysV semaphores.

Simple C pthread test program hangs during execution

I'm new to using the pthread library in C and I have an assignment for my class to write a simple program using them. The basic description of the program is it takes 1 or more input files containing website names and 1 output file name. I then need to create 1 thread per input file to read in the website names and push them onto a queue. Then I need to create a couple of threads to pull those names off of the queue, find their IP Address, and then write that information out to the output file. The command line arguments are expected as follows:
./multi-lookup [one or more input files] [single output file name]
My issue is this. Whenever I run the program with only 1 thread to push information to the output file then everything works properly. When I make it two threads then the program hangs and none of my testing "printf" statements are even printed. My best guess is that deadlock is occurring somehow and that I'm not using my mutexes properly but I can't figure out how to fix it. Please help!
If you need any information that I'm not providing then just let me know. Sorry for the lack of comments in the code.
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <pthread.h>
#include "util.h"
#include "queue.h"
#define STRING_SIZE 1025
#define INPUTFS "%1024s"
#define USAGE "<inputFilePath> <outputFilePath>"
#define NUM_RESOLVERS 2
queue q;
pthread_mutex_t locks[2];
int requestors_finished;
void* requestors(void* input_file);
void* resolvers(void* output_file);
int main(int argc, char* argv[])
{
FILE* inputfp = NULL;
FILE* outputfp = NULL;
char errorstr[STRING_SIZE];
pthread_t requestor_threads[argc - 2];
pthread_t resolver_threads[NUM_RESOLVERS];
int return_code;
requestors_finished = 0;
if(queue_init(&q, 10) == QUEUE_FAILURE)
fprintf(stderr, "Error: queue_init failed!\n");
if(argc < 3)
{
fprintf(stderr, "Not enough arguments: %d\n", (argc - 1));
fprintf(stderr, "Usage:\n %s %s\n", argv[0], USAGE);
return 1;
}
pthread_mutex_init(&locks[0], NULL);
pthread_mutex_init(&locks[1], NULL);
int i;
for(i = 0; i < (argc - 2); i++)
{
inputfp = fopen(argv[i+1], "r");
if(!inputfp)
{
sprintf(errorstr, "Error Opening Input File: %s", argv[i]);
perror(errorstr);
break;
}
return_code = pthread_create(&(requestor_threads[i]), NULL, requestors, inputfp);
if(return_code)
{
printf("ERROR: return code from pthread_create() is %d\n", return_code);
exit(1);
}
}
outputfp = fopen(argv[i+1], "w");
if(!outputfp)
{
sprintf(errorstr, "Errord opening Output File: %s", argv[i+1]);
perror(errorstr);
exit(1);
}
for(i = 0; i < NUM_RESOLVERS; i++)
{
return_code = pthread_create(&(resolver_threads[i]), NULL, resolvers, outputfp);
if(return_code)
{
printf("ERROR: return code from pthread_create() is %d\n", return_code);
exit(1);
}
}
for(i = 0; i < (argc - 2); i++)
pthread_join(requestor_threads[i], NULL);
requestors_finished = 1;
for(i = 0; i < NUM_RESOLVERS; i++)
pthread_join(resolver_threads[i], NULL);
pthread_mutex_destroy(&locks[0]);
pthread_mutex_destroy(&locks[1]);
return 0;
}
void* requestors(void* input_file)
{
char* hostname = (char*) malloc(STRING_SIZE);
FILE* input = input_file;
while(fscanf(input, INPUTFS, hostname) > 0)
{
while(queue_is_full(&q))
usleep((rand()%100));
if(!queue_is_full(&q))
{
pthread_mutex_lock(&locks[0]);
if(queue_push(&q, (void*)hostname) == QUEUE_FAILURE)
fprintf(stderr, "Error: queue_push failed on %s\n", hostname);
pthread_mutex_unlock(&locks[0]);
}
hostname = (char*) malloc(STRING_SIZE);
}
printf("%d\n", queue_is_full(&q));
free(hostname);
fclose(input);
pthread_exit(NULL);
}
void* resolvers(void* output_file)
{
char* hostname;
char ipstr[INET6_ADDRSTRLEN];
FILE* output = output_file;
int is_empty = queue_is_empty(&q);
//while(!queue_is_empty(&q) && !requestors_finished)
while((!requestors_finished) || (!is_empty))
{
while(is_empty)
usleep((rand()%100));
pthread_mutex_lock(&locks[0]);
hostname = (char*) queue_pop(&q);
pthread_mutex_unlock(&locks[0]);
if(dnslookup(hostname, ipstr, sizeof(ipstr)) == UTIL_FAILURE)
{
fprintf(stderr, "DNSlookup error: %s\n", hostname);
strncpy(ipstr, "", sizeof(ipstr));
}
pthread_mutex_lock(&locks[1]);
fprintf(output, "%s,%s\n", hostname, ipstr);
pthread_mutex_unlock(&locks[1]);
free(hostname);
is_empty = queue_is_empty(&q);
}
pthread_exit(NULL);
}
Although I'm not familiar with your "queue.h" library, you need to pay attention to the following:
When you check whether your queue is empty you are not acquiring the mutex, meaning that the following scenario might happen:
Some requestors thread checks for emptiness (let's call it thread1) and just before it executes pthread_mutex_lock(&locks[0]); (and after if(!queue_is_full(&q)) ) thread1 gets contex switched
Other requestors threads fill the queue up and when out thread1 finally gets hold of the mutex if will try to insert to the full queue. Now if your queue implementation crashes when one tries to insert more elements into an already full queue thread1 will never unlock the mutex and you'll have a deadlock.
Another scenario:
Some resolver thread runs first requestors_finished is initially 0 so (!requestors_finished) || (!is_empty) is initially true.
But because the queue is still empty is_empty is true.
This thread will reach while(is_empty) usleep((rand()%100)); and sleep forever, because you pthread_join this thread your program will never terminate because this value is never updated in the loop.
The general idea to remember is that when you access some resource that is not atomic and might be accessed by other threads you need to make sure you're the only one performing actions on this resource.
Using a mutex is OK but you should consider that you cannot anticipate when will a context switch occur, so if you want to chech e.g whether the queue is empty you should do this while having the mutex locked and not unlock it until you're finished with it otherwise there's no guarantee that it'll stay empty when the next line executes.
You might also want to consider reading more about the consumer producer problem.
To help you know (and control) when the consumers (resolver) threads should run and when the producer threads produce you should consider using conditional variables.
Some misc. stuff:
pthread_t requestor_threads[argc - 2]; is using VLA and not in a good way - think what will happen if I give no parameters to your program. Either decide on some maximum and define it or create it dynamically after having checked the validity of the input.
IMHO the requestors threads should open the file themselves
There might be some more problems but start by fixing those.

File operations using POSIX threads

I am learning the concept of multithreading and i encountered a problem using semaphore mutexes.
Here is my code snippet:
void *thread1_funct(void * myfileptr)
{
static int count;
printf("\nThread1 ID:%u\n",(unsigned int) pthread_self());
printf("\nThread1 function, file pointer received:0x%x\n", (FILE*)myfileptr);
for (;;)
{
count++;
sem_wait(&mutex);
fprintf(myfileptr, "%d\n", count);
sem_post(&mutex);
}
return NULL;
}
void *thread2_funct(void *myfileptr)
{
static int count=0;
printf("\nThread2 ID:%u\n",(unsigned int) pthread_self());
printf("\nThread2 function, file pointer received:0x%x\n", (FILE*)myfileptr);
for (;;)
{sem_wait(&mutex);
fscanf(myfileptr, "%d\n", &count);
printf("\n......%d......\n", count);
sem_post(&mutex);
}
return NULL;
}
The two threads i have created. One will write dfata to a file and the other will read the latest data.
Here is my main method:
int main(int argc, char **argv)
{
FILE *fileptr = fopen(*(argv+1), "a+");
sem_init(&mutex, 0x00, 0x01);
if ( (thread1_ret = pthread_create(&thread1, NULL, thread1_funct, (void*)fileptr)) == 0)
printf("\nThread1 created successfully....\n");
else
printf("\nFailed to create Thread1\n");
if ( (thread2_ret = pthread_create(&thread2, NULL, thread2_funct, (void*)fileptr)) == 0)
printf("\nThread2 created successfully....\n");
else
printf("\nFailed to create Thread2\n");
pthread_join(thread1, NULL);
pthread_join(thread2, NULL);
fclose(fileptr);
sem_destroy(&mutex);
pthread_exit(NULL);
return 0;
}
The expected output is :
........1........
........2........
........3........
and so on...... till the program is interrupted manually.
But my output is all 0s:
........0.......
........0.......
........0.......
........0.......
and so on....
Please help me. Where am i going wrong?
Thread 1 writes to the file, and advances the file pointer - to the end of the file. Thread 2 reads from the file pointer, which is pointing at the end of the file, and so you get nothing.
You could use fseek, or rewind in thread 2 to get your data.

Debug recursive thread call in C

I have been trying to debug my code whenever I had free-time for the past day and a half and I don't know what is wrong with my code. When I add the close() function to a recursive call, the program gives me an invalid pointer. But when I remove the close() function call the program runs fine, except it does not do what it is supposed to do, which is:
add up all the file sizes in a user
input directory
open sub-directories, if any, and add
up all the files inside the
sub-directory
Instead, it adds up all the file sizes in the input directory and is able to open the last sub-directory and add the files within that directory to the total file size count.
I am trying to do this with threads. The main() function creates one main thread from the user input directory and runs opendirectory() off the bat.
/*
* Iterates through given directory
*/
void *opendirectory(void *t)
{
pthread_mutex_lock(&dirlock);
DIR *dpntr;
struct dirent *dentry;
char new_directory[512], dir = t;
printf("OPENING DIRECTORY ... %s\n", t);
/* Checks if given directory can be opened */
if((dpntr = opendir(t)) == NULL) {
printf("DIRECTORY FAILED ...%s\n",t);
perror("ERROR -- COULD NOT OPEN DIR");
pthread_exit(NULL);
}
printf("DIRECTORY OPENED: %s\n", t);
/* Read each file in current directory */
while ((dentry = readdir(dpntr)) != NULL ) {
/* Ignore special directories */
if(strcmp(dentry -> d_name, ".") == 0 || strcmp(dentry -> d_name, "..") == 0) {
continue;
} else {
compilelist( t, dentry->d_name );
}
}
pthread_mutex_unlock(&dirlock);
/* Checks if directory can be closed */
if(closedir(dpntr) < 0)
printf("ERROR CLOSING %s.\n", t);
}
This is the function that will determine if a new thread should be created and is supposed to run recursively.
/*
* Determines if current file is a directory
* Creates a new thread if true
*/
void compilelist (const char* dirname, const char *filename)
{
pthread_mutex_lock(&filelock);
struct stat statdata;
char *filepathname, *dpntr;
/* Allocate memory for filepathname */
if((filepathname = (char *) malloc(sizeof(char) * strlen(dirname))) == NULL)
{
printf("CANNOT ALLOCATE MEMORY FOR FILE PATH NAME.");
pthread_exit(NULL);
}
/* Concats directory name with file name */
if(dirname[strlen(dirname) -1] == '/')
{
pthread_mutex_lock(&pathlock);
sprintf(filepathname, "%s%s", dirname, filename);
pthread_mutex_unlock(&pathlock);
}else
{
pthread_mutex_lock(&pathlock);
sprintf(filepathname, "%s/%s", dirname, filename);
pthread_mutex_unlock(&pathlock);
}
lstat(filepathname, &statdata);
/* Calls print_statdata() if current item is a file */
if(!(S_ISDIR(statdata.st_mode)))
{
printf("FILE: %s\n", filepathname);
if(!stat( filepathname, &statdata))
{
print_statdata( filename, &statdata );
}
else {
fprintf (stderr, "GETTING STAT FOR %s", filepathname);
perror( "ERROR IN STATDATA WHILE GETTING STAT");
}
}
/* Recursive call to opendirectory() */
else {
pthread_mutex_lock(&dircountlock);
dirCount++;
pthread_mutex_unlock(&dircountlock);
dpntr = filepathname;
free(filepathname);
printf("SUB-DIRECTORY THREAD: %s\nTHREAD ID NUMBER: %d\n", dpntr, dirCount);
pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);
pthread_create(&threads[dirCount-1], &attr, opendirectory, (void *)dpntr);
}
pthread_mutex_unlock(&filelock);
}
Here is the main()
/*
* Main function prompts user for a directory
*/
int main(int argc, char *argv[])
{
int i;
char *dPtr;
// pthread_attr_t attr;
printf("ENTER A DIRECTORY:\n\t");
scanf("%s", directory);
dPtr = directory;
/* Initialize mutex and condition variable objects */
pthread_mutex_init(&mutex, NULL);
pthread_mutex_init(&filelock, NULL);
pthread_mutex_init(&dirlock, NULL);
pthread_mutex_init(&dircountlock, NULL);
pthread_cond_init (&count_threshold_cv, NULL);
/* For portability, explicitly create threads in a joinable state */
pthread_attr_init(&attr);
pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);
pthread_create(&threads[0], &attr, opendirectory, (void *)dPtr);
/* Wait for all threads to complete */
for (i = 0; i < dirCount; i++) {
pthread_join(threads[i], NULL);
}
printf("TOTAL DIRECTORY SIZE: %d\n", dirSize);
/* Clean up and exit */
pthread_attr_destroy(&attr);
pthread_mutex_destroy(&mutex);
pthread_mutex_destroy(&filelock);
pthread_mutex_destroy(&dirlock);
pthread_mutex_destroy(&dircountlock);
pthread_cond_destroy(&count_threshold_cv);
pthread_exit (NULL);
}
And the global variables ...
pthread_mutex_t mutex;
pthread_mutex_t dirlock;
pthread_mutex_t filelock;
pthread_mutex_t dircountlock;
pthread_mutex_t threadlock;
pthread_cond_t count_threshold_cv;
pthread_attr_t attr;
pthread_t threads[128]; // handles up to 128 threads (i.e. 128 directories, change accordingly)
char directory[512];
int dirSize = 0;
int dirCount = 1; // user's input directory
I feel that the pthread_create() called at the bottom of the compilelist() function is not working properly. The threads[] refers to a global array of threads that has a default size of 20, assuming that there will be no more than 20 total directories. dirCount starts off at 1 because of the user's input directory and increases as new directories are encountered.
Your code:
dpntr = opendir(t)
...
if(closedir(t) < 0)
should be:
if(closedir(dpntr) < 0)
Here I found 2 problems of your code:
As wrang-wrang metioned, closedir(t) leads segfault.
"char filepathname[512];" of compilelist() is a local memory buffer, but you pass it to your thread (opendirectory) and use it continuously. You should use copying or dynamic-allocation instead.
Effo Upd#2009nov17:
After fixing above 2 points, it works fine on my FC9 x86_64 so far. Btw: threads number 20 is really not enough.
First problem:
whenever I had free-time for the past day and a half
Don't do that, your brain isn't built for it. Allocate a time, tell your workmates/wife-and-kids that, if they bother you during this time, there will be gunshots and police involvement :-)
Other problems: no idea (hence the community wiki).

Resources