I'm new in Unix systems programming and I'm struggling to understand file descriptors and pipes. Let's consider this simple code:
#include <stdlib.h>
#include <unistd.h>
#include <sys/wait.h>
#include <string.h>
int main() {
int fd[2], p;
char *m = "123456789\n", c;
pipe(fd);
p = fork();
if (p == 0) {
// child
while(read(fd[0], &c, 1) > 0) write(1, &c, 1);
}
else {
// parent
write(fd[1], m, strlen(m));
close(fd[1]);
wait(NULL);
}
exit (0);
}
When I compile and run the code, it outputs 123456789 but the process never ends unless I issue ^C. Actually, both processes appear as stopped in htop.
If the child closes fd[1] prior to read() then it seems to work OK but I don't understand why. The fd are shared between both processes and the parent closes fd[1] after writing. Why then the child doesn't get the EOF when reading?
Thank you in advance!
Well, first of all your parent process is waiting for the child to terminate in the wait(2) system call, whyle your child is blocked in the pipe to read(2) for another character. Both processes are blocked... so you need to act externally to take them off. The problem is that the child process doesn't close it's writing descriptor of the pipe (and also the parent doesn't close its reading descriptor of the pipe, but this doesn't affect here) Simply the pipe blocks any reader while at least one such writing descriptor is still open. Only when all writing descriptors are closed, the read returns 0 to the reader.
When you did the fork(2) both pipe descriptors (fd[0] and fd[1]) were dup()ed on the child process, so you have a pipe with two open file descriptors (one in the parent, one in the child) for writing, and two open descriptors (again, one in the parent, one in the child) for reading, so as one writer remains with the pipe open for writing (the child process in this case) the read made by the child still blocks. The kernel cannot detect this as an anomaly, because the child could still write on the pipe if another thread (or a signal handler) should want to.
By the way, I'm going to comment some things you made bad in your code:
first is that you consider only two cases from fork() for the parent, and for the child, but if the fork fails, it will return -1 and you'll have a parent process writing on a pipe with no reading process, so probably it should block (as I say, this is not your case, but it is an error either) You have always to check for errors from system calls, and don't assume your fork() call is never to fail (think that -1 is considered != 0 and so it falls through the parent's code). There's only one system call that you can execute without checking it for errors, and it is close(2) (although there's much controversy on this)
This same happens with read() and write(). A better solution to your problem would be to have used a larger buffer (not just one char, to reduce the number of system calls made by your program and so speed it up) and use the return value of read() as a parameter on the write() call.
Your program should (it does on my system, indeed) work with just inserting the following line:
close(fd[1]);
just before the while loop in the child code, as shown here:
#include <stdlib.h>
#include <unistd.h>
#include <sys/wait.h>
#include <string.h>
int main() {
int fd[2], p;
char *m = "123456789\n", c;
pipe(fd);
p = fork();
if (p == 0) {
// child
close(fd[1]); // <--- this close is fundamental for the pipe to work properly.
while(read(fd[0], &c, 1) > 0) write(1, &c, 1);
}
else if (p > 0) {
// parent
// another close(fd[0]); should be included here
write(fd[1], m, strlen(m));
close(fd[1]);
wait(NULL);
} else {
// include error processing for fork() here
}
exit (0);
}
If the child closes fd[1] prior to read() then it seems to work OK but I don't understand why.
That's what you need to do. There's not much more to it than that. A read from the read end of a pipe won't return 0 (signaling EOF) until the kernel is sure that nothing will ever write to the write end of that pipe again, and as long as it's still open anywhere, including the process doing the reading, it can't be sure of that.
Related
I am trying to make a program to takes a command including pipes and then executes it. This is a simplified version of it where I'm trying to pipe the ls and wc command:
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
#include<fcntl.h>
int main(){
char* arglist1[] = {"ls", NULL}; // writing process
char* arglist2[] = {"wc", NULL}; // reading process
int pipefd[2];
pid_t p1, p2;
if (pipe(pipefd) < 0) {
printf("\nPipe could not be initialized");
return 0;
}
p1 = fork();
if (p1 < 0) {
printf("\nCould not fork");
return 0;
}
if (p1 == 0) { // Child 1 executing it needs to write at the write end
close(pipefd[0]);
dup2(pipefd[1], STDOUT_FILENO);
close(pipefd[1]);
if (execvp(arglist1[0], arglist1) < 0) {
printf("\nCould not execute command 1..");
exit(0);
}
} else { // Parent executing
p2 = fork();
if (p2 < 0) {
printf("\nCould not fork");
return 0;
}
if (p2 == 0) { // Child 2 executing it needs to read at the read end
close(pipefd[1]);
dup2(pipefd[0], STDIN_FILENO);
close(pipefd[0]);
if (execvp(arglist2[0], arglist2) < 0) {
printf("\nCould not execute command 2..");
exit(0);
}
} else { // parent executing, waiting for two children
wait(NULL);
wait(NULL);
}
}
printf("\n");
return 0;
}
Although there is error handling in the program, it neither shows anything nor ends. Where is it blocking?
Your problem is that the parent doesn't close both the pipe's file descriptors, and the wc process won't die until it gets EOF on the pipe, and that won't happen until every process that has the write end of the pipe open has closed it. You need to close both ends of the pipe in the parent before waiting for the children to die.
Rule of thumb: If you
dup2()
one end of a pipe to standard input or standard output, close both of the
original file descriptors returned by
pipe()
as soon as possible.
In particular, you should close them before using any of the
exec*()
family of functions.
The rule also applies if you duplicate the descriptors with either
dup()
or
fcntl()
with F_DUPFD or F_DUPFD_CLOEXEC.
If the parent process will not communicate with any of its children via
the pipe, it must ensure that it closes both ends of the pipe early
enough (before waiting, for example) so that its children can receive
EOF indications on read (or get SIGPIPE signals or write errors on
write), rather than blocking indefinitely.
Even if the parent uses the pipe without using dup2(), it should
normally close at least one end of the pipe — it is extremely rare for
a program to read and write on both ends of a single pipe.
Note that the O_CLOEXEC option to
open(),
and the FD_CLOEXEC and F_DUPFD_CLOEXEC options to fcntl() can also factor
into this discussion.
If you use
posix_spawn()
and its extensive family of support functions (21 functions in total),
you will need to review how to close file descriptors in the spawned process
(posix_spawn_file_actions_addclose(),
etc.).
Note that using dup2(a, b) is safer than using close(b); dup(a);
for a variety of reasons.
One is that if you want to force the file descriptor to a larger than
usual number, dup2() is the only sensible way to do that.
Another is that if a is the same as b (e.g. both 0), then dup2()
handles it correctly (it doesn't close b before duplicating a)
whereas the separate close() and dup() fails horribly.
This is an unlikely, but not impossible, circumstance.
Side notes:
Error messages should be written to stderr, not stdout, and should end with a newline. They don't normally need to start with a newline.
You don't need to test the return value from the exec*() family of functions. If they succeed, they don't return; if they return, they failed. But it is important to have code after the eec*() call to trap the error.
The program should exit with a non-zero status (e.g. EXIT_FAILURE) if the exec*() function fails. Exiting with status zero reports success.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
Hello everyone,
I'm quite lost in my school homework since they haven't told us much about it and I haven't done anything like that before.
The task is:
In the C language create a program that creates two processes (fork function) and connects them via pipe (the pipe function).
The first descendant redirects its' stdout into the pipe and writes (space separated) pairs of random numbers into it (function rand).
Delay the output of the numbers (i.e. by 1 second).
The first descendant has
to treat the SIGUSR1 signal (sigaction function) and in case of receiving such signal it prints a string “TERMINATED” to it's stderr and terminates.
The second descendant redirects the pipe output to it's stdin, redirects it's stdout into a file called out.txt in
the current directory and executes a binary file (execl function) for finding the greatest common divisor (the output of our previous tasks where we had to write a makefile that runs a small C program that detects if a number is prime).
The parent process waits 5 seconds and then sends SIGUSR1 to the first process (number generator). This should perform a correct termination of both processes. It waits for the sub-processes to terminate (wait function) and terminates itself.
In fact you are implementing something like this: while : ; do echo $RANDOM $RANDOM ; sleep 1; done | ./c1_task > out.txt
I'm absolutely lost in this and I have nothing so far unfortunatelly.
I don't know where to start.
Could somebody advise me something, please?
Thanks in advance!
Since I don't believe in doing people's work for them, I can't give you the "solution." I can, however, show you some of the concepts that you need to know to fulfill your assignment. I can also give you a couple of links, but if you just search for help with the concepts you don't understand, you're likely to find the information you need anyways.
Now that I've delivered a paragraph of introductory information, I'm going to work you through some of the concepts you need to understand to solve this problem.
I may fill in some missing information if I get (and feel like it's worth spending) the time necessary to turn this into a pseudo-tutorial. :)
The information provided may be simplified, a little vague, or otherwise open to improvement. Feel free to let me know if you, dear reader, spot a problem.
First Concept: fork()-ing
What is it? fork() makes it easy to do multiple things simultaneously by duplicating (much of) the current process into another process. (Actually, it is similar to asexual reproduction.)
For instance, the child process (this is the new process that was created by making the fork() system call) inherits open file descriptors (this is an important point!), has its own copy of variables that the parent process (has/had), etc.
Example: Here's an example program that illustrates a thing or two. Note the wait(). It makes the parent, the process that called fork(), wait to continue executing the rest of the program until a child has terminated. Without wait(NULL), we can't guarantee that the parent's printf statement will run after the child's printf statement.
#include <stdio.h> //the usual, perror
#include <stdlib.h> //exit
#include <sys/types.h> //wait() / pid_t
#include <sys/wait.h> //wait()
#include <unistd.h> // fork()
int main () {
pid_t cpid;
//create our child.
//fork() returns -1 if the fork failed, otherwise it returns
// the pid of the child to the parent,
// and 0 to the child
cpid = fork();
//Both the child process and parent process executed the
//"cpid =" assignment.
//However, they both modified their own version of cpid.
//From now on, everything is run by both the child and parent.
//the fork failed; there is no child so we're done.
if (cpid < 0) {
perror("During attempted fork");
exit(EXIT_FAILURE);
}
//Though the if statement will be checked by both,
//cpid will equal 0 only in the child process.
if (cpid == 0) {
//This will be executed by the child.
printf("Hello. I'm your child.\n");
//Now that we've let Pops know that we're alive...
exit(EXIT_SUCCESS);
}
else if (cpid > 0) {
//wait for our child to terminate.
//I dare you to comment this out & run the program a few times.
//Does the parent ever print before the child?
wait(NULL);
printf("I proudly parented 1 child.\n");
}
return 0;
}
Other: You can see another example here.
Second concept: Pipes
What is a pipe? A pipe is a method for interprocess communication. Basically, it has one end that data can be put in (write() is one way to do it) and one end that data can be gotten out of (using read).
Pipes are created using the pipe() system call. It returns -1 on error. It's only argument is the address of an array of two ints, which we'll call pipe_fds.
If the call succeeded, the first element in pipe_fds contains the file descriptor that is used to read from the pipe; the second element contains the file descriptor used to write to the pipe.
You can write to the pipe with write() and read from the pipe with read(). (More info about using pipes can be found at various places on the internet.
Here's an example:
#include <stdio.h> //the usual, perror
#include <stdlib.h> //exit
#include <sys/types.h> //wait() / pid_t
#include <sys/wait.h> //wait()
#include <unistd.h> // fork(), pipe()
#define BUFLEN 256 //must be greater than one
int main () {
int pipe_fds[2],
pipe_ret;
pid_t cpid;
//Let's create a pipe.
//Note that we do this *before* forking so that our forked child
// has access to the pipe's file descriptors, pipe_fds.
pipe_ret = pipe(pipe_fds);
//we couldn't create our pipe
if (pipe_ret == -1) {
perror("Pipe Creation");
exit(EXIT_FAILURE);
}
//create our child.
cpid = fork();
//the fork failed; there is no child so we're done.
if (cpid < 0) {
perror("During attempted fork");
exit(EXIT_FAILURE);
}
//Am I the child?
if (cpid == 0) {
//close the childs read end of the pipe.
//Failing to close unused pipe ends is life or death!
//(Check `man 7 pipe`)
close(pipe_fds[0]);
//Send a message through the pipe.
//NOTE: For simplicity's sake, we assume that our printing works.
// In the real world, it might not write everything, etc.
//We could use `write()`, but this way is easier.
dprintf(pipe_fds[1], "Daddy, I'm alive.\n");
//We're done writing. Close write end of the pipe.
//This is the wise thing to do, but it would get closed anyways.
close(pipe_fds[1]);
//Now that we've let Pops know that we're alive...
exit(EXIT_SUCCESS);
}
else if (cpid > 0) {
char buf[BUFLEN] = {};
int bytes_read = 0;
//close *our* write end of the pipe. Important!
//Comment this out and watch your program hang.
//Again, check out `man 7 pipe`.
close(pipe_fds[1]);
//read data from pipe until we reach EOF
while ((bytes_read = read(pipe_fds[0], buf, BUFLEN - 1)) > 0) {
//null terminate our string.
//(We could use snprintf instead...)
buf[bytes_read] = '\0';
//You can comment this out to prove to yourself that
//we're the one printing the child's message.
printf("%s", buf);
}
//close read end of pipe
close(pipe_fds[0]);
//wait for our child to terminate.
wait(NULL);
printf("I proudly parented 1 child.\n");
}
return 0;
}
As you can see, I just gave a small tutorial on two of the concepts you need to know to finish your assignment. I need some sleep, so I'll leave it at that for tonight.
Read and experiment with the examples! Notes in the comments are to help you learn.
My program creates child process and sets pipes to communicate with it. The problem occurs when i try to read data from the pipe. Since child process has ended (i use wait to ensure that) EOF should be on the end of the data stream thus ending the read (As in the man page for pipe). But instead read just freezes and waits for more data to come.
What am i missing here?
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>
void setfd(int *in, int *out) {
dup2(out[1], 1);
dup2(in[0], 0);
}
int main(int argc, char *argv[]) {
int status;
int pipe2ch[2], pipe2pr[2];
char *newargv[] = {NULL, NULL};
newargv[0] = argv[1];
pipe(pipe2ch);
pipe(pipe2pr);
setfd(pipe2pr, pipe2ch);
int a;
if (!(a = fork())) {
setfd(pipe2ch, pipe2pr);
execve(newargv[0], newargv, NULL);
exit(1);
} else {
printf("hello!\n");
fflush(stdout);
char str;
wait(&status);
while (read(pipe2pr[0], &str, 1) > 0) {
fprintf(stderr, "%c", str);
}
exit(0);
}
}
Since child process has ended (i use wait to ensure that) EOF should be on the end of the data stream thus ending the read (As in the man page for pipe).
I'm not sure what you've read to suggest that. Or maybe it's your wording that I don't understand. EOF is not a character on the stream.
But instead read just freezes and waits for more data to come. What am i missing here?
Several things. The most important one is probably that when a process forks, the child's copies of the parent's open file descriptors refer to the same entries in the kernel's underlying table of open files as the parent's do, each of which remains open until all handles on it are closed. The child closes all its file descriptors when it exits, but both ends of both pipes remain open in the parent, so end-of-file will not be signaled to readers. Each process must close the pipe ends it doesn't use or is finished using.
Additionally, you should read() first, then wait(), for if the child process writes enough data to the pipe then it may block, and if the parent does not read until after the child exits then you'll have a deadlock.
Furthermore, I don't see any reason to dupe either pipe end onto the parent's standard streams (resulting in closing the original ones). Just manipulate the pipes via their file descriptors, as you already half do. If you want a stream interface to those, then use fdopen() to get one.
I want to learn how Linux pipes work! I wrote a small and easy program that use a pipe to communicate a string between parent and child process. However, the program results in a dead lock that I have not understood what is its cause.
Here is the code :
#include <sys/wait.h>
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#define SIZE 100
int
main(int argc, char *argv[])
{
int pfd[2];
int read_pipe=0, write_pipe=0;
pid_t cpid;
char buf[SIZE];
/* PIPE ***************************************
* pipe() creates a pair of file descriptors, *
* pointing to a pipe inode, and places them *
* in the array pointed to by filedes. *
* filedes[0] is for reading, *
* filedes[1] is for writing *
**********************************************/
if (pipe(pfd) == -1) {
perror("pipe");
exit(EXIT_FAILURE);
}
read_pipe=pfd[0];
write_pipe=pfd[1];
cpid = fork();
if (cpid == -1) {
perror("fork");
exit(EXIT_FAILURE);
}
if (cpid == 0) { /* Child reads from pipe */
char * hello = "I am a child process\n";
sleep(1);
// wait until there is some data in the pipe
while (read(read_pipe, buf, SIZE) > 0);
printf("Parent process has written : %s\n", buf);
write(write_pipe, hello, strlen(hello));
close(write_pipe);
close(read_pipe);
_exit(EXIT_SUCCESS);
} else { /* Parent writes argv[1] to pipe */
char * hello = "I am a parent process\n";
write(write_pipe, hello, strlen(hello));
while (read(read_pipe, buf, SIZE) > 0);
printf("Child process has written : %s\n", buf);
close(write_pipe);
close(read_pipe);
wait(NULL); /* Wait for child */
exit(EXIT_SUCCESS);
}
}
In this link you'll find the proper mannipulation of PIPEs between parent and child. Your problem here is that the communication is not being correctly set-up.
The PIPE should be used to communicate in only one direction, so one process has to close the read descriptor and the other has to close the write descriptor. Otherwise what will happen is that the call to 'read'(both on the father and the son), since it can detect that there is another process with an open write descriptor on the PIPE, will block when it finds that the PIPE is empty (not return 0), until someone writes something in it. So, both your father and your son are getting blocked on their respective read.
There are two solutions to this:
.You create two PIPEs, one for the communication in each direction, and perform the initialization as explained in the link above. Here you have to remember to close the write descriptor when you are done sending the message, so the other process' read will return, or condition the loop to the count of bytes read (not to the return of read), so you won't perform another call when you read the whole message. For example:
int bread = 0;
while(bread < desired_count)
{
bread += read(read_pipe, buf + bread, SIZE - bread);
}
.You create one PIPE as you did, and modify the flags on the read descriptor, using fcntl to also have O_NONBLOCK, so the calls to read won't block when there's no information in the PIPE. Here you need to check on the return value of the read to know you received something, and go adding up until you get the full length of the message. Also you will have find a way to synchronize the two processes so they won't read messages that are not meant for them. I don't recommend you to use this option, but you can try it if you want using condition variables.
Maybe you can tell if you see any of yout printf() outputs?
Anyway, if you want to establish a two way communication between your paent and child, yout should use two pipes, one for writing data form parent to child an the other for writing from child to parent. Furthermore, your read loops may be dangerous: if the data comes in two or more chunks the second read() overwrites the first portion (I've never seen tha happen with local pipes, but for example with sockets). And of course, yout is not automatically null terminated after read(), so just printing int with "%s" may also cause problems.
I hope that gives you some ideas to try.
I need some way for the parent process to communicate with each child separately.
I have some children that need to communicate with the parent separately from the other children.
Is there any way for a parent to have a private communication channel with each child?
Also can a child for example, send to the parent a struct variable?
I'm new to these kind of things so any help is appreciated. Thank you
(I'll just assume we're talking linux here)
As you probably found out, fork() itself will just duplicate the calling process, it does not handle IPC.
From fork manual:
fork() creates a new process by duplicating the calling process.
The new process, referred to as the child, is an exact duplicate of
the calling process, referred to as the parent.
The most common way to handle IPC once you forked() is to use pipes, especially if you want "a private comunication chanel with each child". Here's a typical and easy example of use, similar to the one you can find in the pipe manual (return values are not checked):
#include <sys/wait.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
int
main(int argc, char * argv[])
{
int pipefd[2];
pid_t cpid;
char buf;
pipe(pipefd); // create the pipe
cpid = fork(); // duplicate the current process
if (cpid == 0) // if I am the child then
{
close(pipefd[1]); // close the write-end of the pipe, I'm not going to use it
while (read(pipefd[0], &buf, 1) > 0) // read while EOF
write(1, &buf, 1);
write(1, "\n", 1);
close(pipefd[0]); // close the read-end of the pipe
exit(EXIT_SUCCESS);
}
else // if I am the parent then
{
close(pipefd[0]); // close the read-end of the pipe, I'm not going to use it
write(pipefd[1], argv[1], strlen(argv[1])); // send the content of argv[1] to the reader
close(pipefd[1]); // close the write-end of the pipe, thus sending EOF to the reader
wait(NULL); // wait for the child process to exit before I do the same
exit(EXIT_SUCCESS);
}
return 0;
}
The code is pretty self-explanatory:
Parent forks()
Child reads() from the pipe until EOF
Parent writes() to the pipe then closes() it
Datas have been shared, hooray!
From there you can do anything you want; just remember to check your return values and to read dup, pipe, fork, wait... manuals, they will come in handy.
There are also a bunch of other ways to share datas between processes, they migh interest you although they do not meet your "private" requirement:
shared memory "SHM", the name says it all...
sockets, they obviously work as good if used locally
FIFO files which are basically pipes with a name
or even a simple file... (I've even used SIGUSR1/2 signals to send binary datas between processes once... But I wouldn't recommend that haha.)
And probably some more that I'm not thinking about right now.
Good luck.