Multiple processes write to the same file (C) - c

There is a program (Ubuntu 12.04 LTS, a single-core processor):
#include <stdio.h>
#include <unistd.h>
#include <signal.h>
#include <fcntl.h>
#include <sys/types.h>
int main(){
mode_t mode = S_IRUSR | S_IWUSR;
int i = 0, fd, pid;
unsigned char pi1 = 0x33, pi2 = 0x34;
if((fd = open("res", O_WRONLY | O_CREAT | O_TRUNC, mode)) < 0){
perror("open error");
exit(1);
}
if((pid = fork()) < 0){
perror("fork error");
exit(1);
}
if(pid == 0) {
if(write(fd, &pi2, 1) != 1){
perror("write error");
exit(1);
}
}else{
if(write(fd, &pi1, 1) != 1){
perror("write error");
exit(1);
}
}
close(fd);
return 0;
}
The idea is to open the file for writing, then going fork. The position at which there will be a record total for both processes. The strange thing is that if you run the program, it's output to a file "res" is not constant: I have infuriated then 34 then 4 then 3. The question is why such a conclusion? (After all, if the position is shared, then the conclusion must be either 34 or 43.).
In my suspicion, the process is interrupted in the function write, when he found a position in which to write.

When you spawn multiple processes with fork() there is no way to tell in which order they will be executed. It's up to the operating systems scheduler to decide.
So having multiple processes write to the same file is a recipe for disaster.
Regarding the question why sometimes one of the two numbers gets omitted: write first writes the data and then it increments the file pointer. I think it could be possible that the thread control changes in exactly that moment so that the second thread writes before the file position was updated. So it overwrites the data the other process just wrote.

Here's what I think is happening.
You open the file and you fork
The parent gets to run first (similar stuff can happen if the child runs first)
The parent writes 3 and exits
The parent was the controlling process for the terminal so the kernel sends a SIGHUP to all members of the foreground group
The default action of SIGHUP is to terminate the process so the child silently dies
A simple way to test this is to add sleep:
sleep(5); /* Somewhere in the child, right before you write. */
You'll see the child process dies instantaneously: the write is never performed.
Another way to test this is to ignore the SIGHUP, before you fork:
sigignore(SIGHUP); /* Define _XOPEN_SOURCE 500 before including signal.h. */
/* You can also use signal(SIGHUP, SIG_IGN); */
You'll see the process now writes both digits to the file.
The overwriting hypothesis is unlikely. After fork, both processes share a link to the same file descriptor in a system-wide table which also contains the file offset.

I run your program several times,and the result is "34" or "43".
So i have write a shell script
#!/bin/bash
for i in {1..500}
do
./your_program
for line in $(cat res)
do
echo "$line"
done
done
,and run your program 500 times. As we can see,it get '3' or '4' some times( aboat 20 times in 500)。
How we can explain this?
The answer is that:
when we fork() a child process,the child share the same file description and file state structure(which has the current file offset).
In normal,a process get offset=0 first,and write the first byte,and the offset=1;the other process get offset=1,and it will write the second byte.
But some times, if the parent process get offset=0 from the file state structure ,and child get offset=0 at the same time,a process write the first byte,and the other overwrite the first byte. The result will be "3" or "4" (depends on whether the parent write first or child ). Because they both write the first byte of the file.
Fork and offset,see this

The standard says
Write requests of {PIPE_BUF} bytes or less shall not be interleaved with data from other processes doing writes on the same pipe.
Link - http://pubs.opengroup.org/onlinepubs/009696699/functions/write.html
The atomicity of write is only guaranteed in case of writing to a pipe for less than equal to PIPE_BUF and it is not specified for a regular file and we cannot assume that.
So in this case race condition is happening and that results in incorrect data for some runs.
(In my system also it is happening after say few thousand runs).
I think you should think of using mutex/semaphore/any other locking primitive to solve this.

Are you sure you need fork() exactly? fork() creates different process with different memory space (file descriptors and so on). Maybe pthreads would suit you? In case with pthreads you'll share the same fd for all processes. But anyways, you should really think about using mutexes in your project.

Related

How does pipe works in c?

So I have 2 questions about pipes in c :
1:
When i fork a process after creating a pipe in order to make the parent write to the pipe and the child read from it, How it's synchronized ? : the parent always send data before the child attempts to read that data although they are running concurrently?
why i don't fall on the case where the child start reading before the parent try to send its data ?
2:
This is about file descriptors of the pipe, referring to the code below how can parent-process close the pipe-output file-descriptor while the child doesn't access to that file yet ? , assuming the parent starts first.
Any help will be appreciated, Thank you !
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/wait.h>
#include <stdlib.h>
#define BUFFER_SIZE 256
int main(int argc , char*argv[])
{
pid_t worker_pid ;
int descriptor[2];
unsigned char bufferR[256] , bufferW[256];
/***************************** create pipe ********************/
puts("pipe creation");
if (pipe(descriptor) !=0)
{
fprintf(stderr,"error in the pipe");
exit(1);
}
/***************************** create processes ********************/
puts("now fork processes");
worker_pid = fork(); // fork process
if (worker_pid == -1 ) // error forking processes
{
fprintf(stderr,"error in fork");
exit(2);
}
/*********************** communicate processes ********************/
if (worker_pid == 0) // in the child process : the reader
{
close(descriptor[1]); // close input file descriptor , ok because parent finished writing
read(descriptor[0],bufferR,BUFFER_SIZE);
printf("i'm the child process and i read : %s \n",bufferR);
}
if (worker_pid !=0)
{
// is there any case where child attempts to read before parent writting ?
close(descriptor[0]);// close file descriptor of child before it reads ?
sprintf(bufferW,"i'm the parent process my id is %d , and i wrote this to my child",getpid());
write(descriptor[1],bufferW,BUFFER_SIZE);
wait(NULL);
}
return 0;
}
I expected there will be some cases for question 1 where the output is :
i'm the child process and i read :
because the parent doesn't wrote it's message yet
for question 2 i expected an error saying :
invalid file descriptor in the child process because the parent already closed it (assuming the parent runs always the first)
but the actual output is always :
i'm the child process and i read: i'm the parent process my id is 7589, and i wrote this to my child
When i fork a process after creating a pipe in order to make the
parent write to the pipe and the child read from it, How it's
synchronized ? : the parent always send data before the child attempts
to read that data although they are running concurrently? why i
don't fall on the case where the child start reading before the parent
try to send its data ?
Typically, it doesn't need to be synchronized, and in fact it can itself serve as a synchronization mechanism. If you perform blocking reads (the default) then they will not complete until the corresponding data have been sent, regardless of the relative order of the initial read and write calls.
The two processes do, however, need to implement an appropriate mechanism to demarcate and recognize message boundaries, so that the reader can recognize and respond appropriately to short reads. That may be as simple as ending each message with a newline, and reading messages with fgets() (through a stream obtained via fdopen()).
This is about file descriptors of the pipe, referring to the code below how can parent-process close the pipe-output file-descriptor while the child doesn't access to that file yet ? , assuming the parent starts first.
Not an issue. Once the child is forked, it has access to the underlying open file descriptions it inherited from its parent, through the same file descriptor numbers that the parent could use, but these are separate from the parent for the purpose of determining how many times each open file description is referenced. The situation is similar to that resulting from the operation of the dup() syscall. Only when all processes close all their file descriptors for a given open file description is that open file description invalidated.
This is done internally in the kernel. When you try to read from a file descriptor (being it a pipe, or socket), if the "remote side" has not sent any data, your process stalls (the call to read does not return at all), until the other side has pushed something into the internal buffers of the kernel (in your example, wrote to the pipe).
Here you can see the internal implementation of the pipes in linux:
https://github.com/torvalds/linux/blob/master/fs/pipe.c#L272
Look for variable do_wakeup.

C fifo keeps blocked

I'm currently studying multithreading with C, but there is something I don't quite understand with our named pipe excersize.
We are expected to do an implementation of file search system that finds files and adds to a buffer with one process and the second process should take filenames from threads of first one, finds the search query inside that file and returns the position to first process via pipe. I did nearly all of it but i'm confused how to do the communication between two processes.
Here is my code that does the communication:
main.c
void *controller_thread(void *arg) {
pthread_mutex_lock(&index_mutex);
int index = t_index++; /*Get an index to thread*/
pthread_mutex_unlock(&index_mutex);
char sendPipe[10];
char recvPipe[10];
int fdsend, fdrecv;
sprintf(sendPipe, "contrl%d", (index+1));
sprintf(recvPipe, "minion%d", (index+1));
mkfifo(sendPipe, 0666);
execlp("minion", "minion", sendPipe, recvPipe, (char*) NULL);
if((fdsend = open(sendPipe, O_WRONLY|O_CREAT)) < 0)
perror("Error opening pipe");
if((fdrecv = open(recvPipe, O_RDONLY)) < 0)
perror("Error opening pipe");
while(1) {
char *fileName = pop(); /*Counting semaphore from buffer*/
if(notFile(fileName))
break;
write(fdsend, fileName, strlen(fileName));
write(fdsend, search, strlen(search));
char place[10];
while(1) {
read(fdrecv, place, 10);
if(notPlace(place)) /*Only checks if all numeric*/
break;
printf("Minion %d searching %s in %s, found at %s\n", index,
search, fileName, place);
}
}
}
From the online resources I found, I think this is the way to handle the fifo inside the main. I tried to write a test minion just to make sure it works, so here it is
minion.c
int main(int argc, char **argv) {
char *recvPipe = argv[1];
char *sendPipe = argv[2];
char fileName[100];
int fdsend, fdrecv;
return 0;
fdrecv = open(recvPipe, O_RDONLY);
mkfifo(sendPipe, 0666);
fdsend = open(sendPipe, O_WRONLY|O_CREAT);
while(1) {
read(fdrecv, fileName, 100);
write(fdsend, "12345", 6);
write(fds, "xxx", 4);
}
return 0;
}
When I run this way, the threads get blocked and prints no response if I change to O_NONBLOCK to mode of open. Then it prints "Error opening pipe no such device or address" error, so I know that somehow I couldn't open the recvPipe inside minion but I don't know what is the mistake
Among the problems with your code is an apparent misunderstanding about the usage of execlp(). On success, this function does not return, so the code following it will never be executed. One ordinarily fork()s first, then performs the execlp() in the child process, being certain to make the child terminate if the execlp() fails. The parent process may need to eventually wait for the forked child, as well.
Additionally, it is strange, and probably undesirable, that each process passes the O_CREAT flag when it attempts to open the write end of a FIFO. It should be unnecessary, because each one has just created the FIFO with mkfifo(). Even in the event that mkfifo() fails or that some other process removes it before it can be opened, you do not want to open with O_CREAT because that will get you a regular file, not a FIFO.
Once you fix the execlp() issue, you will also have a race condition. The parent process relies on the child to create one of the FIFOs, but does not wait for that process to do so. You will not get the desired behavior if the parent reaches its open attempt before the child completes its mkfifo().
I suggest having the parent create both FIFOs, before creating the child process. The child and parent must cooperate by opening the both ends of one FIFO before proceeding to open both ends of the other. One's open for reading will block until the other opens the same FIFO for writing.
Or you could use ordinary (anonymous) pipes (see pipe()) instead of FIFOs. These are created open on both ends, and they are more natural for communication between processes related by inheritance.
In any event, be sure to check the return values of your function calls. Almost all of these functions can fail, and it is much better to detect and handle that up front than to sort out the tangle that may form when you assume incorrectly that every call succeeded.
Fifos need some synchronization at open time. By default open(s) are blocking, so that an open for read is blocked until some other open the same fifo for writing, and the converse (this to let peers to be synchronized for a communication). You can use O_NONBLOCK to open for reading while there is no actual open peer, but the converse is false, because opening for writing while there is no reading peer leads to an error (letting a process trying to write while there is no reader is considered as non sense).
You may read Linux Fifo manual entry for example.

C homework with pipes and forks [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
Hello everyone,
I'm quite lost in my school homework since they haven't told us much about it and I haven't done anything like that before.
The task is:
In the C language create a program that creates two processes (fork function) and connects them via pipe (the pipe function).
The first descendant redirects its' stdout into the pipe and writes (space separated) pairs of random numbers into it (function rand).
Delay the output of the numbers (i.e. by 1 second).
The first descendant has
to treat the SIGUSR1 signal (sigaction function) and in case of receiving such signal it prints a string “TERMINATED” to it's stderr and terminates.
The second descendant redirects the pipe output to it's stdin, redirects it's stdout into a file called out.txt in
the current directory and executes a binary file (execl function) for finding the greatest common divisor (the output of our previous tasks where we had to write a makefile that runs a small C program that detects if a number is prime).
The parent process waits 5 seconds and then sends SIGUSR1 to the first process (number generator). This should perform a correct termination of both processes. It waits for the sub-processes to terminate (wait function) and terminates itself.
In fact you are implementing something like this: while : ; do echo $RANDOM $RANDOM ; sleep 1; done | ./c1_task > out.txt
I'm absolutely lost in this and I have nothing so far unfortunatelly.
I don't know where to start.
Could somebody advise me something, please?
Thanks in advance!
Since I don't believe in doing people's work for them, I can't give you the "solution." I can, however, show you some of the concepts that you need to know to fulfill your assignment. I can also give you a couple of links, but if you just search for help with the concepts you don't understand, you're likely to find the information you need anyways.
Now that I've delivered a paragraph of introductory information, I'm going to work you through some of the concepts you need to understand to solve this problem.
I may fill in some missing information if I get (and feel like it's worth spending) the time necessary to turn this into a pseudo-tutorial. :)
The information provided may be simplified, a little vague, or otherwise open to improvement. Feel free to let me know if you, dear reader, spot a problem.
First Concept: fork()-ing
What is it? fork() makes it easy to do multiple things simultaneously by duplicating (much of) the current process into another process. (Actually, it is similar to asexual reproduction.)
For instance, the child process (this is the new process that was created by making the fork() system call) inherits open file descriptors (this is an important point!), has its own copy of variables that the parent process (has/had), etc.
Example: Here's an example program that illustrates a thing or two. Note the wait(). It makes the parent, the process that called fork(), wait to continue executing the rest of the program until a child has terminated. Without wait(NULL), we can't guarantee that the parent's printf statement will run after the child's printf statement.
#include <stdio.h> //the usual, perror
#include <stdlib.h> //exit
#include <sys/types.h> //wait() / pid_t
#include <sys/wait.h> //wait()
#include <unistd.h> // fork()
int main () {
pid_t cpid;
//create our child.
//fork() returns -1 if the fork failed, otherwise it returns
// the pid of the child to the parent,
// and 0 to the child
cpid = fork();
//Both the child process and parent process executed the
//"cpid =" assignment.
//However, they both modified their own version of cpid.
//From now on, everything is run by both the child and parent.
//the fork failed; there is no child so we're done.
if (cpid < 0) {
perror("During attempted fork");
exit(EXIT_FAILURE);
}
//Though the if statement will be checked by both,
//cpid will equal 0 only in the child process.
if (cpid == 0) {
//This will be executed by the child.
printf("Hello. I'm your child.\n");
//Now that we've let Pops know that we're alive...
exit(EXIT_SUCCESS);
}
else if (cpid > 0) {
//wait for our child to terminate.
//I dare you to comment this out & run the program a few times.
//Does the parent ever print before the child?
wait(NULL);
printf("I proudly parented 1 child.\n");
}
return 0;
}
Other: You can see another example here.
Second concept: Pipes
What is a pipe? A pipe is a method for interprocess communication. Basically, it has one end that data can be put in (write() is one way to do it) and one end that data can be gotten out of (using read).
Pipes are created using the pipe() system call. It returns -1 on error. It's only argument is the address of an array of two ints, which we'll call pipe_fds.
If the call succeeded, the first element in pipe_fds contains the file descriptor that is used to read from the pipe; the second element contains the file descriptor used to write to the pipe.
You can write to the pipe with write() and read from the pipe with read(). (More info about using pipes can be found at various places on the internet.
Here's an example:
#include <stdio.h> //the usual, perror
#include <stdlib.h> //exit
#include <sys/types.h> //wait() / pid_t
#include <sys/wait.h> //wait()
#include <unistd.h> // fork(), pipe()
#define BUFLEN 256 //must be greater than one
int main () {
int pipe_fds[2],
pipe_ret;
pid_t cpid;
//Let's create a pipe.
//Note that we do this *before* forking so that our forked child
// has access to the pipe's file descriptors, pipe_fds.
pipe_ret = pipe(pipe_fds);
//we couldn't create our pipe
if (pipe_ret == -1) {
perror("Pipe Creation");
exit(EXIT_FAILURE);
}
//create our child.
cpid = fork();
//the fork failed; there is no child so we're done.
if (cpid < 0) {
perror("During attempted fork");
exit(EXIT_FAILURE);
}
//Am I the child?
if (cpid == 0) {
//close the childs read end of the pipe.
//Failing to close unused pipe ends is life or death!
//(Check `man 7 pipe`)
close(pipe_fds[0]);
//Send a message through the pipe.
//NOTE: For simplicity's sake, we assume that our printing works.
// In the real world, it might not write everything, etc.
//We could use `write()`, but this way is easier.
dprintf(pipe_fds[1], "Daddy, I'm alive.\n");
//We're done writing. Close write end of the pipe.
//This is the wise thing to do, but it would get closed anyways.
close(pipe_fds[1]);
//Now that we've let Pops know that we're alive...
exit(EXIT_SUCCESS);
}
else if (cpid > 0) {
char buf[BUFLEN] = {};
int bytes_read = 0;
//close *our* write end of the pipe. Important!
//Comment this out and watch your program hang.
//Again, check out `man 7 pipe`.
close(pipe_fds[1]);
//read data from pipe until we reach EOF
while ((bytes_read = read(pipe_fds[0], buf, BUFLEN - 1)) > 0) {
//null terminate our string.
//(We could use snprintf instead...)
buf[bytes_read] = '\0';
//You can comment this out to prove to yourself that
//we're the one printing the child's message.
printf("%s", buf);
}
//close read end of pipe
close(pipe_fds[0]);
//wait for our child to terminate.
wait(NULL);
printf("I proudly parented 1 child.\n");
}
return 0;
}
As you can see, I just gave a small tutorial on two of the concepts you need to know to finish your assignment. I need some sleep, so I'll leave it at that for tonight.
Read and experiment with the examples! Notes in the comments are to help you learn.

Can a single pipe be used for 2 way communication between parent and a child?

Suppose I use pipefdn[2] and pipe() on it , can bidirectional communication be implemented using a single pipe or do you need 2 pipes ?
Though this operation results as success in some cases, but it is not a recommended way , especially in the production code. As pipe() by default dont provide any sync mechanism and moreover the read() can go for an infinite hang, if no data or read() is called before write() from other process.
Recommended way is to always use 2 pipe. pipe1[2], pipe2[2] for two way communication.
For more info please refer the following video description.
https://www.youtube.com/watch?v=8Q9CPWuRC6o&list=PLfqABt5AS4FkW5mOn2Tn9ZZLLDwA3kZUY&index=11
No sorry. Linux pipe() is unidirectional. See the man page, and also pipe(7) & fifo(7). Consider also AF_UNIX sockets, see unix(7).
Correct me if I am wrong: But I think you can. The problem is that you probably don't want to do that. First, of all create a simple program:
#include <stdio.h>
#include <sys/types.h>
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <sys/wait.h>
int pd[2];
int num = 2;
int main(){
pid_t t = fork();
/* create a child process */
if(t<0){
printf("error in fork");
exit(1);
}
/* create a pipe */
if(pipe(pd)==-1){
printf("error in pipe");
exit(3);
}
else if(t==0){
//close(pd[1]); // child close writing end
int r = read(pd[0], &num, sizeof(num));
if(r<0){
printf("error while reading");
exit(2);
}
printf("i am the child and i read %d\n",num);
// close(pd[0]);
exit(0);
}
/* parent process */
//close(pd[0]); /* parents closes its reading end
if(write(pd[1],&num,sizeof(num)<0)){
printf("error in reading");
exit(4);
}
//close(pd[1]);
/*parent wait for your child to terminate;*/
int status;
wait(&status);
printf("my child ended with status: %d\n",status);
return 0;
}
Try to play with close(). Skip it by putting it in a comment or include it. You will find out that in order this program to run the only really needed system-call close is the one before the child reads. I found here in stack overflow an answer saying that " Because the write-end is open the system waits because a potential write could occur .. " . Personally, I tried to run it without it and I discovered that it would not terminate. The other close(), although are a good practice , don't influence the execution. ( I am not sure why that happens maybe someone more experienced can help us).
Now let's examine what you asked:
I can see some problems here:
If two processes write in the same channel you may have race conditions:
They write to the same file descriptor at the same time:
What if one process reads its own writings instead of those of the process
it tries to communicate with? How you will know, where in the file you should read?
What if the one process, writes "above" the writings of the other?
Yes it can, I've done that before. I had a parent and child send each other different messages using the same 2 pipes and receive them correctly. Just make sure you're always reading from the first file descriptor and writing to the second.

Dead lock linux pipe

I want to learn how Linux pipes work! I wrote a small and easy program that use a pipe to communicate a string between parent and child process. However, the program results in a dead lock that I have not understood what is its cause.
Here is the code :
#include <sys/wait.h>
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#define SIZE 100
int
main(int argc, char *argv[])
{
int pfd[2];
int read_pipe=0, write_pipe=0;
pid_t cpid;
char buf[SIZE];
/* PIPE ***************************************
* pipe() creates a pair of file descriptors, *
* pointing to a pipe inode, and places them *
* in the array pointed to by filedes. *
* filedes[0] is for reading, *
* filedes[1] is for writing *
**********************************************/
if (pipe(pfd) == -1) {
perror("pipe");
exit(EXIT_FAILURE);
}
read_pipe=pfd[0];
write_pipe=pfd[1];
cpid = fork();
if (cpid == -1) {
perror("fork");
exit(EXIT_FAILURE);
}
if (cpid == 0) { /* Child reads from pipe */
char * hello = "I am a child process\n";
sleep(1);
// wait until there is some data in the pipe
while (read(read_pipe, buf, SIZE) > 0);
printf("Parent process has written : %s\n", buf);
write(write_pipe, hello, strlen(hello));
close(write_pipe);
close(read_pipe);
_exit(EXIT_SUCCESS);
} else { /* Parent writes argv[1] to pipe */
char * hello = "I am a parent process\n";
write(write_pipe, hello, strlen(hello));
while (read(read_pipe, buf, SIZE) > 0);
printf("Child process has written : %s\n", buf);
close(write_pipe);
close(read_pipe);
wait(NULL); /* Wait for child */
exit(EXIT_SUCCESS);
}
}
In this link you'll find the proper mannipulation of PIPEs between parent and child. Your problem here is that the communication is not being correctly set-up.
The PIPE should be used to communicate in only one direction, so one process has to close the read descriptor and the other has to close the write descriptor. Otherwise what will happen is that the call to 'read'(both on the father and the son), since it can detect that there is another process with an open write descriptor on the PIPE, will block when it finds that the PIPE is empty (not return 0), until someone writes something in it. So, both your father and your son are getting blocked on their respective read.
There are two solutions to this:
.You create two PIPEs, one for the communication in each direction, and perform the initialization as explained in the link above. Here you have to remember to close the write descriptor when you are done sending the message, so the other process' read will return, or condition the loop to the count of bytes read (not to the return of read), so you won't perform another call when you read the whole message. For example:
int bread = 0;
while(bread < desired_count)
{
bread += read(read_pipe, buf + bread, SIZE - bread);
}
.You create one PIPE as you did, and modify the flags on the read descriptor, using fcntl to also have O_NONBLOCK, so the calls to read won't block when there's no information in the PIPE. Here you need to check on the return value of the read to know you received something, and go adding up until you get the full length of the message. Also you will have find a way to synchronize the two processes so they won't read messages that are not meant for them. I don't recommend you to use this option, but you can try it if you want using condition variables.
Maybe you can tell if you see any of yout printf() outputs?
Anyway, if you want to establish a two way communication between your paent and child, yout should use two pipes, one for writing data form parent to child an the other for writing from child to parent. Furthermore, your read loops may be dangerous: if the data comes in two or more chunks the second read() overwrites the first portion (I've never seen tha happen with local pipes, but for example with sockets). And of course, yout is not automatically null terminated after read(), so just printing int with "%s" may also cause problems.
I hope that gives you some ideas to try.

Resources