I have written the following function to implement two threads in POSIX. Inside each thread, I want to append a line to a test file named "demo.txt" each time the thread runs. But nothing is being written in the file.
My sample code fragment:
#include<stdio.h>
#include<pthread.h>
#include<stdlib.h>
FILE *fp;
void * threadFunc1(void * arg)
{
int i;
for(i=1;;i++)
{
printf("%s\n",(char*)arg);
fprintf(fp,"Looping %d time in threadfunc1",i);
sleep(1);
}
}
void * threadFunc2(void * arg)
{
int i;
for(i=1;;i++)
{
printf("%s\n",(char*)arg);
fprintf(fp,"Looping %d time in threadfunc2",i);
sleep(1);
}
}
int main(void)
{
pthread_t thread1;
pthread_t thread2;
fp=fopen("demo.txt","a");
char * message1 = "i am thread 1";
char * message2 = "i am thread 2";
pthread_create(&thread1,NULL,threadFunc1,(void*)message1 );
pthread_create(&thread2,NULL,threadFunc2,(void*)message2 );
pthread_join(thread1,NULL);
pthread_join(thread2,NULL);
fclose(fp);
return 0;
}
Why there isn't anything being written in the file demo.txt? What will I need to correct if anything wrong?
In general, such scenarios necessitate the need for synchronization mechanisms. The approach depends on your architecture and requirement.
You might need to implement mutex or semaphore.
Another possible approach for synchronization could be such that every thread treats the file as shared memory using a memory mapped file.
If your scenario demands more reads than writes, then the best synchronization mechanism can be based on reader-writer-lock as it allows more concurrent reads.
Another approach can be having mechanism to ensure that at any point of time, the threads write different parts of file and do not compete for the same location in file.
You need to add fflush(fp) after fprintf(fp, ...). This forces a write buffered data.
fflush() not needed if you add \n (new line) character at end of your line.
Related
I am working on this problem: take from the command line a letter and the name of same files, count the occurrence of the char in each file, using one thread per file, and and print the total occurrences.
This is my code:
typedef struct _CharFile{
char c;
char *fileName;
} CharFile;
pthread_mutex_t count = PTHREAD_MUTEX_INITIALIZER;
int sum = 0;
void *CountFile(void *threadarg);
int main(int argc, const char * argv[]) {
pthread_t threads[argc-2];
int chck, t;
CharFile cf;
for ( t=0 ; t<argc-2 ; t++ ){
cf.c = argv[1][0];
cf.fileName = (char *)argv[t + 2];
chck = pthread_create(&threads[t], NULL, CountFile, (void *) &cf);
if (chck){
printf("ERROR; return code from pthread_create() is %d\n", chck);
exit(-1);
}
}
printf("%lld occurrences of the letter %c in %lld threads\n", (long long)sum, argv[1][0], (long long)argc-2);
return 0;
}
void *CountFile(void *threadarg){
FILE *in;
CharFile *cf;
char c;
int counter = 0;
cf = (CharFile *) threadarg;
in = fopen(cf->fileName, "r");
if (in == NULL){
perror("Error opening the file!\n");
pthread_exit(NULL);
}
while (fscanf(in, "%c", &c) != EOF){
if(c == cf->c){
counter += 1;
}
}
fclose(in);
pthread_mutex_lock(&count);
sum += counter;
pthread_mutex_unlock(&count);
pthread_exit(NULL);
}
I don't get any error in the file opening or in the thread creations, but my output is always 0 as total occurrences. I also tried to print the counter in the threads and I got every time the same numbers in all the threads, even if my input files are different. Am I using the mutex wrongly or is there something else wrong?
This is one of my outputs:
61 occurrences of e in this thread
0 occurrences of the letter e in 3 threads
61 occurrences of e in this thread
61 occurrences of e in this thread
Program ended with exit code: 9
There are several threading issues at play here.
1) The main thread will continue asynchronously to the newly spawned threads. Given the code, it is very likely that the main thread will complete and exit before the CountFile threads have completed. On Linux, when the main thread returns, the C runtime will perform a exit_group system call which will terminate all threads.
You'll need to add some check to ensure the CountFile threads have finished the relevant section of work. In this example, look at using pthread_join() in the main thread.
2) The 'cf' storage in the main thread is a stack local variable, which is passed by pointer to each thread. However, since it is the same storage, several types of failures could occur. a) the workunit may be updated by the main thread while a worker thread is accessing it. b) the same workunit is sent to multiple/all threads.
You could solve this several ways: 'cf' could be an array of CharFile for each thread. Or 'cf' could be dynamically allocated for each thread. The former is a bit more performance and memory efficient, but the later might be better structurally. Particularly that main thread is giving addresses in its local stack space to another thread.
3) Once item #1 is addressed and the threads exited before the main thread printf, the mutex usage would be ok. But it might be better to put pthread_mutex_locks around the main thread access of 'sum' anyway. It may not be necessary given this code, but future code refactors might change that.
So I have a very high data acquisition rate of 16MB/s. I am reading 4MB of data into a buffer from a device file and then processing it. However, this method of writing then reading was to slow for the project. I would like to implement a double buffer in C.
In order to simplify my idea of the double buffer I decided not to include reading from a device file for simplicity. What I have created is a C program that spawns two separate threads readThread and writeThread. I made readThread call my swap function that swaps the pointers of the buffers.
This implementation is terrible because I am using shared memory outside of the Mutex. I am actually slightly embarrassed to post it, but it will at least give you an idea of what I am trying to do. However, I can not seem to come up with a practical way of reading and writing to separate buffers at the same time then calling a swap once both threads finished writing and reading.
Can someone please tell me if its possible to implement double buffering and give me an idea of how to use signals to control when the threads read and write?
Note that readToBuff (dumb name I know) and writeToBuff aren't actually doing anything at present they have blank functions.
Here is my code:
#include <stdlib.h>
#include <stdio.h>
#include <pthread.h>
pthread_t writeThread;
pthread_t readThread;
pthread_mutex_t buffer_mutex;
char buff1[4], buff2[4];
struct mutex_shared {
int stillReading, stillWriting, run_not_over;
char *writeBuff, *readBuff;
} SHARED;
void *writeToBuff(void *idk) {
while(!SHARED.run_not_over) {
SHARED.stillWriting = 1;
for(int i = 0; i < 4; i++) {
}
SHARED.stillWriting = 0;
while(SHARED.stillReading){};
}
printf("hello from write\n");
return NULL;
}
void *readToBuff(void *idk) {
while(!SHARED.run_not_over) {
SHARED.stillReading = 1;
for(int i = 0; i < 4; i++) {
}
while(SHARED.stillWriting){};
swap(writeThread,readThread);
}
printf("hello from read");
return NULL;
}
void swap(char **a, char **b){
pthread_mutex_lock(&buffer_mutex);
printf("in swap\n");
char *temp = *a;
*a = *b;
*b = temp;
SHARED.stillReading = 0;
//SHARED.stillWriting = 0;
pthread_mutex_unlock(&buffer_mutex);
}
int main() {
SHARED.writeBuff = buff1;
SHARED.readBuff = buff2;
printf("buff1 address %p\n", (void*) &buff1);
printf("buff2 address %p\n", (void*) &buff2);
printf("writeBuff address its pointing to %p\n", SHARED.writeBuff);
printf("readBuff address its pointing to %p\n", SHARED.readBuff);
swap(&SHARED.writeBuff,&SHARED.readBuff);
printf("writeBuff address its pointing to %p\n", SHARED.writeBuff);
printf("readBuff address its pointing to %p\n", SHARED.readBuff);
pthread_mutex_init(&buffer_mutex,NULL);
printf("Creating Write Thread\n");
if (pthread_create(&writeThread, NULL, writeToBuff, NULL)) {
printf("failed to create thread\n");
return 1;
}
printf("Thread created\n");
printf("Creating Read Thread\n");
if(pthread_create(&readThread, NULL, readToBuff, NULL)) {
printf("failed to create thread\n");
return 1;
}
printf("Thread created\n");
pthread_join(writeThread, NULL);
pthread_join(readThread, NULL);
exit(0);
}
Using a pair of semaphores seems like it would be easier. Each thread has it's own semaphore to indicate that a buffer is ready to be read into or written from, and each thread has it's own index into a circular array of structures, each containing a pointer to buffer and buffer size. For double buffering, the circular array only contains two structures.
The initial state sets the read thread's semaphore count to 2, the read index to the first buffer, the write threads semaphore count to 0, and the write index to the first buffer. The write thread is then created which will immediately wait on its semaphore.
The read thread waits for non-zero semaphore count (sem_wait) on its semaphore, reads into a buffer, sets the buffer size, increments the write threads semaphore count (sem_post) and "advances" it's index to the circular array of structures.
The write thread waits for non-zero semaphore count (sem_wait) on its semaphore, writes from a buffer (using the size set by read thread), increments the read threads semaphore count (sem_post) and "advances" it's index to the circular array of structures.
When reading is complete, the read thread sets a structure's buffer size to zero to indicate the end of the read chain, then waits for the write thread to "return" all buffers.
The circular array of structures could include more than just 2 structures, allowing for more nesting of data.
I've had to use something similar for high speed data capture, but in this case, the input stream was faster than a single hard drive, so two hard drives were used, and the output alternated between two write threads. One write thread operated on the "even" buffers, the other on the "odd" buffers.
In the case of Windows, with it's WaitForMultipleObjects() (something that just about every operating system other than Posix has), each thread can use a mutex and a semaphore, along with its own linked list based message queue. The mutex controls queue ownership for queue updates, the semaphore indicates number of items pending on a queue. For retrieving a message, a single atomic WaitForMultipleObjects() waits for a mutex and a non-zero semaphore count, and when both have occurred, decrements the semaphore count and unblocks the thread. A message sender, just needs a WaitForObject() on the mutex to update another threads message queue, then posts (releases) the threads semaphore and releases the mutex. This eliminates any priority issues between threads.
See the following code:
I am having some library file like libexample.so
And in my code i am using some function X(inputbuffer , outputbuffer)present in library
/*
Assume for each thread there are corresponding input buffers and output buffers
example if there are 50 threads 50 input buffers and out buffers are available
i.e input buffers are unique to each other so as output buffers....
*/
void* threadFunX(void *num)
{
Mutex lock();
int ret = X(inputBuffer,outputbuffer,length);
if(ret)
{
//store the output buffer in afile
}
else
error occured.....
Mutex unlock();
pthread_exit(NULL);
}
int main()
{
pthread_t thread1[100];
for(i=0;i<100;i++){
p1 = pthread_create(&thread1[i],NULL,threadFunX,(void *)i);
if(p1)
printf("Error occurred while creating thread.....\n");
//pthread_join( thread1[i], NULL);
}
for(i=0;i<100;i++)
pthread_join( thread1[i], NULL);
}
My question:
If we are not using Mutex we are getting output buffer corrupted.... for this reason we are using mutex....
Is there any alternative method other than mutex....
I want to use this library function efficiently.....
My main moto is I want to reduce running time taken by mutex method also...
for example...if my code with mutex is running with 0.2 seconds..... I want better alternative method which takes lesser time than mutex with thread parallel programming...
Thanks in advance...
Check the library function X. Are you sure it is creating a new OutputBuffer everytime you are calling the function. Because as per your saying that "without mutex outputbuffer is getting corrupted", it seems to me that only one outputbuffer is created and it is returning the same to each thread.
I have just had a really good use for multithreading. As such.... I have to learn multithreading. I have a very simple program:
void *listenloop(void *arg){
while (1){
Sleep(2000);
puts("testing 123\n");
}
return NULL;
}
int main(){
pthread_t listener;
pthread_create(&listener,NULL,listenloop,"foo");
char testinput[200];
while(1){
puts("Scanning: ");
scanf("%s",testinput);
puts("\n\n");
printf("You typed: %s: ",testinput);
}
}
The theory is that it waits for user input, echos it, all while periodically printing.
None to my surprise, actually (and presumably obviously to my betters in the matter) the output is "messed up."
Now I can think of several ways around this problem, but no actual solutions. How should something of this nature be implemented? Can it just be done by manipulating the output of the program after it is displayed to the user?
Thanks!
So just wrap the prints in pthread_mutex_lock/unlocks with a single pthread_mutex_t and you should be fine.
http://linux.die.net/man/3/pthread_mutex_lock
pthread_mutex_t = PTHREAD_MUTEX_INITIALIZER;
void *listenloop(void *arg){
while (1){
Sleep(2000);
pthread_mutex_lock(&mutex);
puts("testing 123\n");
pthread_mutex_unlock(&mutex);
}
return NULL;
}
int main(){
pthread_t listener;
pthread_create(&listener,NULL,listenloop,"foo");
char testinput[200];
while(1){
pthread_mutex_lock(&mutex);
puts("Scanning: ");
pthread_mutex_unlock(&mutex);
scanf("%s",testinput);
pthread_mutex_lock(&mutex);
puts("\n\n");
printf("You typed: %s: ",testinput);
pthread_mutex_unlock(&mutex);
}
}
In your original code, there should be no "messed up" output (caused by threading, anyway) in that code as only the one thread (the main one) is doing any output.
The only thing the other thread does is infinitely loop with a delay of some sort.
Now that you've updated the question to output from the other thread then, yes, it is possible for the output to intermix.
There are several ways around this, two spring to mind immediately.
Have all output go through a series of functions which mutex-protect the output stream, such as mutexed_printf()/mutexed_puts() (which you'll need to provide) (a).
Do all output from one of the threads, meaning the other will have to send data to it via some means (inter-thread communications like a queue) - that way all output can be properly mixed, such as on newline boundaries.
(a) Also keep in mind that, if you want to mutex protect the output stream for the user input operation, you'll probably want to protect the puts/scanf atomically so that the testing output doesn't mess up your input (by outputting messages after the prompt but before/during your input). That won't be possible with a mutexed_puts() function, you'll need an expanded mutexed_prompt_and_input() one.
I'm writing a program that simply demonstrates writing and reading from a bounded buffer as it outputs what value it expected and what value actually was read. When I define N as a low value, the program executes as expected. However, when I increase the value, I start to see unexpected results. From what I understand, I am creating data races by using two threads.
By looking at the output, I think I've narrowed it down to about three examples of data races as follows:
One thread writes to the buffer while another thread reads.
Two threads simultaneously write to the buffer.
Two threads simultaneously read from the buffer.
Below is my code. The formatting is really strange for the #include statements, so I left them out.
#define BUFFSIZE 10000
#define N 10000
int Buffer[BUFFSIZE];
int numOccupied = 0; //# items currently in the buffer
int firstOccupied = 0; //where first/next value or item is to be found or placed
//Adds a given value to the next position in the buffer
void buffadd(int value)
{
Buffer[firstOccupied + numOccupied++] = value;
}
int buffrem()
{
numOccupied--;
return(Buffer[firstOccupied++]);
}
void *tcode1(void *empty)
{
int i;
//write N values into the buffer
for(i=0; i<N; i++)
buffadd(i);
}
void *tcode2(void *empty)
{
int i, val;
//Read N values from the buffer, checking the value read with what is expected for testing
for(i=0; i<N; i++)
{
val = buffrem();
if(val != i)
printf("tcode2: removed %d, expected %d\n", val, i);
}
}
main()
{
pthread_t tcb1, tcb2;
pthread_create(&tcb1, NULL, tcode1, NULL);
pthread_create(&tcb2, NULL, tcode2, NULL);
pthread_join(tcb1, NULL);
pthread_join(tcb2, NULL);
}
So here are my questions.
Where (in my code) do these data races occur?
How do I fix them?
Use a mutex to synchronize access to your shared data structure. You will need the following:
pthread_mutex_t mutex;
pthread_mutex_init(&mutex, NULL);
pthread_mutex_lock(&mutex);
pthread_mutex_unlock(&mutex);
pthread_mutex_destroy(&mutex);
As a basic principle, you lock the mutex before reading/writing to the data structure shared between threads, and unlock it afterwards. In this case, the Buffer plus metadata numOccupied and firstOccupied are your shared data structure you need to protect. So in buffadd() and buffrem(), lock the mutex at the beginning, unlock it at the end. And in main(), initialize the mutex before starting the threads, and destroy it after joining.
You have races because both threads access the same global vars numOccupied and firstOccupied. You need to synchronize access to the variables with some form of locking. For example, you could use a semaphore or mutex to lock access to the shared global state while doing add / remove operations.