I am trying to create a chain of pipes for 3 processes plus the current program such that:
1 - Exec P1 and send output of P1 to input of P2
3 - Send output of P2 to input of P3
4 - Display output of P3 in stdout
5 - Pass stdin of current/main/driver program to input of P3
I am using 2 pipes. For task #5, the main program reads from stdin and writes to a pipe that P3 reads.
I have managed to get the processes communicating with each other. However, what I noticed is there is a large delay between when P1 writes and when P2 detects this write on its STDIN, i.e. P1 may have written hundreds of times before P2 detects the write and misses many of the P1 writes at the beginning. I have confirmed via print messages that P2 is in fact launched on time, however, it does not detect/read the input in time (it is a Python script looping: "for line in sys.stdin:" )
Here is my code:
int pipe1[2];
int pipe2[2];
if (pipe(pipe1) < 0 || pipe(pipe2) < 0 )
{
perror("Error: pipe");
}
pid_t procIDC = fork();
if (procIDC == 0)
{
dup2(pipe2[0], 0);
execv("procC", argv);
}
else
{
pid_t procIDB ;
procIDB = fork();
if (procIDB == 0)
{
dup2( pipe1[0], 0);
dup2( pipe2[1], 1);
if (execl("/usr/bin/python", "/usr/bin/python", "./test.py", (char *)NULL) < 0)
{
perror("execl"); return 0;
}
}
else
{
pid_t procIDA = fork();
if (procIDA ==0)
{
dup2( pipe1[1], 1);
execv("proc1", argv);
}
else
{
dup2( pipe2[1], 1);
//print any input so it sends to p3
ssize_t read;
char *inputLine = NULL;
size_t len = 0;
while ((read = getline(&inputLine, &len, stdin)) != -1)
{
printf(inputLine);
}
}
}
I noticed is there is a large delay between when P1 writes and when P2 detects this write on its STDIN
Yes. Your question is presumably "why is there a delay"?
The answer: stdio (used by most programs) uses fully-buffered output when it detects that the output is going into a pipe.
To prevent that buffering, use fflush or setvbuf.
Some additional reading here.
Actually, the fflush is obviously being done in the writing program
You didn't show us the writing program with fflush in it. If you had, we might be able to point out your mistake (fflush will help, if done correctly).
Anyway, one way to see what is happening is to run strace -p <pid-of-writer>, and observe that immediately after the writer actually executes the write(2) syscall, the reader gets its input. That would prove that it is in fact buffering in the writer that is causing the delay.
This:
setbuf(stdout, NULL);
does not disable buffering. You really need to call setvbuf (or fflush).
There was input buffering on STDIN in P2
Related
So my program needs to pipe multiple processes and read the number of bytes each process output has.
The way I implemented it, in a for loop, we have two children:
Child 1: dups output and executes the process
Child 2: reads the output and writes it for the next input
Currently, child 1 executes the process and the child 2 reads its output, but it doesn't seem to write it in the right place because in the second loop iteration it prints the output to the screen and blocks.
for (int i = 0; i < processes; i++) {
int result = socketpair(PF_LOCAL, SOCK_STREAM, 0, apipe[i]);
if (result == -1) {
error_and_exit();
}
int pid;
int pid2;
pid = fork_or_die();
// child one points to STDOUT
if (pid == FORK_CHILD) {
if (dup2(apipe[i][1], STDOUT_FILENO) == -1)
error_and_exit();
if (close(apipe[i][1]) == -1)
error_and_exit();
if (close(apipe[i][0]) == -1)
error_and_exit();
if (execlp("/bin/sh", "sh", "-c", tabCommande[i], (char *)NULL) == -1)
error_and_exit();
}
pid2 = fork_or_die();
//CHILD 2 reads the output and writes if for the next command to use
if(pid2 == FORK_CHILD){
FILE *fp;
fp = fopen("count", "a");
close(apipe[i][1]);
int count=0;
char str[4096];
count = read(apipe[i][0], str, sizeof(str)+1);
close(apipe[i][0]);
write(STDIN_FILENO, str, count);
fprintf(fp, "%d : %d \n ", i, count);
fclose(fp);
}
}
Your second child does “write(STDIN_FILENO, …); that’s not a conventional way of using standard input.
If standard input is a terminal, then the device is usually opened for reading and writing and the three standard I/O channels are created using dup() or dup2(). Thus you can read from the outputs and write to the input — but only if the streams are connected to a login terminal (window). If the input is a pipe, you can't successfully write to it, nor can you read from the output if it is a pipe. (Similarly if the input is redirected from a file or the output is redirected to a file.) This terminal setup is done by the process that creates the terminal window. It is background information explaining why writing to standard input appears on the terminal.
Anyway, that's what you're doing. You are writing to the terminal via standard input. Your minimum necessary change is to replace STDIN_FILENO with STDOUT_FILENO.
You are also going to need a loop around the reading and writing code. In general, processes generate lots of output in small chunks. The close on the input pipe will be outside the loop, of course, not between the read() and write() operations. You should check that the write() operations write all the data to the output.
You should also have the second child exit after it closes the output file. In this code, I'd probably open the file after the counting loop (or what will become the counting loop), but that's mostly a stylistic change, keeping the scope of variables to a minimum.
You will probably eventually need to handle signals like SIGPIPE (or ignore it so that the output functions return errors when the pipe is closed early). However, that's a refinement for when you have the basic code working.
Bug: you have:
count = read(apipe[i][0], str, sizeof(str)+1);
This is a request to the o/s to give you a buffer overflow — you ask it to write more data into str than str can hold. Remove the +1!
Minor note: you don’t need to check the return value from execlp() or any of that family of functions. If the call succeeds, it doesn’t return; if it returns, it failed. Your code is correct to exit after the call to execlp(), though; that's good.
You said:
I replaced STDIN_FILENO to STDOUT_FILENO in the second child but it doesn't seem to solve the issue. The output is still shown in the terminal and there's a pipe blockage after.
That observation may well be correct, but it isn't something that can be resolved by studying this code alone. The change to write to an output stream is necessary — and in the absence of any alternative information, writing to STDOUT_FILENO is better than writing to STDIN_FILENO.
That is a necessary change, but it is probably not a sufficient change. There are other changes needed too.
Did you set up the inputs and outputs for the pair of children this code creates correctly? It is very hard to know from the code shown — but given that it is not working as you intended, it's a reasonable inference that you did not get all the plumbing correct. You need to draw a diagram of how the processes are intended to operate in the larger context. At a minimum, you need to know where the standard input for each process comes from, and where its standard input goes. Sometimes, you need to worry about standard error too — most likely though, in this case, you can quietly ignore it.
This is what I think your code could look like — though the comments in it describe numerous possible variants.
#include <sys/socket.h>
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
/* The code needs these declarations and definition to compile */
extern _Noreturn void error_and_exit(void);
extern pid_t fork_or_die(void);
extern void unknown_function(void);
static ssize_t copy_bytes(int fd1, int fd2);
#define FORK_CHILD 0
int processes;
int apipe[20][2];
char *tabCommande[21];
void unknown_function(void)
{
for (int i = 0; i < processes; i++)
{
int result = socketpair(PF_LOCAL, SOCK_STREAM, 0, apipe[i]);
if (result == -1)
error_and_exit();
int pid1 = fork_or_die();
// child one points to STDOUT
if (pid1 == FORK_CHILD)
{
if (dup2(apipe[i][1], STDOUT_FILENO) == -1)
error_and_exit();
if (close(apipe[i][1]) == -1)
error_and_exit();
if (close(apipe[i][0]) == -1)
error_and_exit();
execlp("/bin/sh", "sh", "-c", tabCommande[i], (char *)NULL);
error_and_exit();
}
//CHILD 2 reads the output and writes if for the next command to use
int pid2 = fork_or_die();
if (pid2 == FORK_CHILD)
{
close(apipe[i][1]);
ssize_t count = copy_bytes(apipe[i][0], STDOUT_FILENO);
FILE *fp = fopen("count", "a");
if (fp == NULL)
error_and_exit();
/*
** Using %zd for ssize_t is a reasonable guess at a format to
** print ssize_t - but it is a guess. Alternatively, change the
** type of count to long long and use %lld. There isn't a
** documented, official (fully standardized by POSIX) conversion
** specifier for ssize_t AFAIK.
*/
fprintf(fp, "%d : %zd\n ", i, count);
fclose(fp);
exit(EXIT_SUCCESS);
}
/*
** This is crucial - the parent has all the pipes open, and the
** child processes won't get EOF until the parent closes the
** write ends of the pipes, and they won't get EOF on the inputs
** until the parent closes the read ends of the pipe.
**
** It could be avoided if the first child creates the pipe or
** socketpair and then creates the second child as a grandchild
** of the main process. That also alters the process structure
** and reduces the number of processes that the original parent
** process has to wait for. If the first child creates the
** pipe, then the apipe array of arrays becomes unnecessary;
** you can have a simple int apipe[2]; array that's local to the
** two processes. However, you may need the array of arrays so
** that you can chain the outputs of one process (pair of
** processes) to the input of the next.
*/
close(apipe[i][0]);
close(apipe[i][1]);
}
}
static ssize_t copy_bytes(int fd1, int fd2)
{
ssize_t tbytes = 0;
ssize_t rbytes;
char buffer[4096];
while ((rbytes = read(fd1, buffer, sizeof(buffer))) > 0)
{
ssize_t wbytes = write(fd2, buffer, rbytes);
if (wbytes != rbytes)
{
/*
** There are many possible ways to deal with this. If
** wbytes is negative, then the write failed, presumably
** irrecoverably. The code could break the loop, reporting
** how many bytes were written successfully to the output.
** If wbytes is zero (pretty improbable), it isn't clear
** what happened. If wbytes is positive, then you could add
** the current value to tbytes and try to write the rest in
** a loop until everything has been written or an error
** occurs. You pays your money and takes your pick.
*/
error_and_exit();
}
tbytes += wbytes;
}
if (tbytes == 0 && rbytes < 0)
tbytes = rbytes;
return tbytes;
}
You could add #include <signal.h> and signal(SIGPIPE, SIG_IGN); to the code in the second child.
I'm writing a program with two processes that communicate through a pipe. The child process reads some parameters from the parent, executes a shell script with them and returns the results to the parent process line by line.
My code worked just fine until I wrote the while(read()) part at the end of the parent process. The child would execute the shell script, read its echo's from popen() and print them to standard output.
Now I tried to write the results to the pipe as well and read them in the while() loop at the parent's end but it blocks and neither will the child process print the result to standard output. Apparently it won't even reach the point after reading the data from the pipe sent by the parent.
If I comment out the while() at the parent process, the child will print the results and return, and the program will end smoothly.
Why does the while(read()) block even if I closed the writing end of the pipe in both parent and child processes?
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/wait.h>
#include <errno.h>
#include <string.h>
#include <fcntl.h>
int read_from_file(char **directory, int *octal) {
FILE *file = fopen("input", "r");
if (file == NULL) {
perror("error opening file");
exit(1);
}
fscanf(file, "%s %d", *directory, octal);
}
int main(int argc, char *argv[]) {
char *directory = malloc(256);
int *octal = malloc(sizeof *octal);
pid_t pid;
int pfd[2];
char res[256];
if (pipe(pfd) < 0) {
perror("Error opening pipe");
return 1;
}
if ((pid = fork()) < 0)
perror("Error forking");
if (pid == 0) {
printf("client here\n");
if (read(pfd[0], directory, 256) < 0)
perror("error reading from pipe");
if (read(pfd[0], octal, sizeof(int)) < 0)
perror("error reading from pipe");
// This won't get printed:
printf("client just read from pipe\n");
// close(pfd[0]);
char command[256] = "./asd.sh ";
strcat(command, directory);
char octal_c[5];
sprintf(octal_c, " %d", *octal);
strcat(command, octal_c);
FILE *f = popen(command, "r");
while (fgets(res, 256, f) != NULL) {
printf("%s", res);
if (write(pfd[1], res, 256) < 0)
perror("Error writing res to pipe");
}
fclose(f);
close(pfd[1]);
close(pfd[0]);
fflush(stdout);
return 1;
}
read_from_file(&directory, octal);
if (write(pfd[1], directory, 256) < 0)
perror("Error writing dir to pipe");
if (write(pfd[1], octal, sizeof(int)) < 0)
perror("error writing octal to pipe");
int r;
close(pfd[1]);
while (r = read(pfd[0], res, 256)) {
if (r > 0) {
printf("%s", res);
}
}
close(pfd[0]);
while (wait(NULL) != -1 || errno != ECHILD);
}
Since the child demonstrably reaches ...
printf("client here\n");
... but seems not to reach ...
printf("client just read from pipe\n");
... we can suppose that it blocks indefinitely on one of the two read() calls between. With the right timing, that explains why the parent blocks on its own read() from the pipe. But how and why does that blocking occur?
There are at least three significant semantic errors in your program:
pipes do not work well for bidirectional communication. It is possible, for example, for a process to read back the bytes that it wrote itself and intended for a different process. If you want bidirectional communication then use two pipes. In your case, I think that would have avoided the apparent deadlock, though it would not, by itself, have made the program work correctly.
write and read do not necessarily transfer the full number of bytes requested, and short reads and writes are not considered erroneous. On success, these functions return the number of bytes transferred, and if you want to be sure to transfer a specific number of bytes then you need to run the read or write in a loop, using the return values to track progress through the buffer being transferred. Or use fread() and fwrite() instead.
Pipes convey undifferentiated streams of bytes. That is, they are not message oriented. It is not safe to assume that reads from a pipe will be paired with writes to the pipe, so that each read receives exactly the bytes written by one write. Yet your code depends on that to happen.
Here's a plausible failure scenario that could explain your observations:
The parent:
fork()s the child.
after some time performs two writes to the pipe, one from variable directory and the other from variable octal. At least the first of those is a short write.
closes its copy of the write end of the pipe.
blocks attempting to read from the pipe.
The child:
reads all the bytes written via its first read (into its copy of directory).
blocks on its second read(). It can do this despite the parent closing its copy of the write end, because the write end of the pipe is still open in the child.
You then have a deadlock. Both ends of the pipe are open in at least one process, the pipe is empty, and both processes are blocked trying to read bytes that can never arrive.
There are other possibilities that arrive at substantially the same place, too, some of them not relying on a short write.
The parent process was trying to read from the pipe before the child could have read from it and write the results to it. Using two different pipes for the two-way communication solved the problem.
Blow is an implementation of Sieve of Eratosthenes in C
#include "kernel/stat.h"
#include "kernel/types.h"
#include "user/user.h"
void cull(int p) {
int n;
while (read(0, &n, sizeof(n))) {
// n is not prime
if (n % p != 0) {
write(1, &n, sizeof(n));
}
}
}
void redirect(int k, int pd[]) {
close(k);
dup(pd[k]);
close(pd[0]);
close(pd[1]);
}
void right() {
int pd[2], p;
// read p
if (read(0, &p, sizeof(p))) {
printf("prime %d\n", p);
pipe(pd);
if (fork()) {
// parent
redirect(0, pd);
right();
} else {
redirect(1, pd);
cull(p);
}
}
}
int main(int argc, char *argv[]) {
int pd[2];
pipe(pd);
if (fork()) {
redirect(0, pd);
right();
} else {
redirect(1, pd);
for (int i = 2; i < 36; i++) {
write(1, &i, sizeof(i));
}
}
exit(0);
I am not quite following the logic here:
1). Why does redirect need to close pd1?
2). cull() is reading from the file descriptor 0, but in right() the child process will close 0. Why doesn't this cause any problem?
3). why it is necessary to use redirect here when we only want read from 0 and write to 1?
Update:
1). I found this implementation in the following blog post:
https://abcdlsj.github.io/post/mit-6.828-lab1xv6-and-unix-utilities/
2). I think the idea is borrowed from the inventor of this algorithm
3). I think the reason that the header is so is because it is implemented in a toy operating system for educational purpose.
The code is an adaptation of the paper Coroutine prime number sieve by M. Douglas McIlroy to the Xv6 operating system, a re-implementation of Version 6 Unix used for teaching. The technique is from 1968 as an experiment in co-routines via pipes. The paper explains the algorithm and rationale.
The program implements the sieve of Eratosthenes as a kind of production line, in which each worker culls multiples of one prime from a passing stream of integers, and new workers are recruited as primes are discovered.
When Unix came to be, my fascination with coroutines led me to badger its author, KenThompson, to allow writes in one process to go not only to devices but also to matching reads in another process.
...the coroutine sieve has been a standard demo for languages or systems that support interprocess communication. Implementations using Unix processes typically place the three coroutines—source, cull and sink—in distinct executable files.The fact that the whole program can be written as a single source file, in a language that supports neither concurrency nor IPC, is a tribute not only to Unix’s pipes, but also to its clean separation of program initiation into fork for duplicating address spaces and exec for initializing them.
I believe using stdin and stdout is an artifact of its origins in the early days of Unix when piping stdin and stdout between processes was first introduced. It makes a lot more sense in shell.
#!/bin/bash
source() {
seq 2 1000000
}
cull() {
while true
do
read n
(($n % $1 != 0)) && echo $n
done
}
sink() {
read p
echo $p
cull $p | sink &
}
source | sink
In C, as we'll see, it's simpler to skip the redirection and pass around pipes.
First, what's going on?
redirect is redirecting stdin and stdout to a pipe. 0 is stdin and 1 is stdout. This can be made more clear by using STDIN_FILENO and STDOUT_FILENO.
main makes a pipe.
main forks.
The child redirects stdout to the pipe.
The child streams numbers to the pipe via stdout.
The first number must be 2.
main redirects stdin to the pipe.
main calls right.
right reads the first prime, 2, from stdin which is a pipe to the number stream.
[number stream] ->(2) [right]
After the initial condition, a switcheroo happens inside right.
right makes a pipe.
right forks.
The child redirects its stdout to the pipe.
The child's stdin is still reading from the number stream.
The child calls cull.
cull reads from stdin (the number stream) and writes to stdout (right).
right redirects its stdin to the pipe, reading from cull.
right recurses.
[number stream] ->(n) [cull] ->(p) [right]
After the first call right is reading primes from cull and writing them to the real stdout. cull reads candidates from the number stream and writes primes to right.
When the number stream loop ends the process ends and closes its pipe to cull. Once all the numbers have been read from the pipe, cull to reads EOF ending its loop and its process, closing its pipe to right. right reads EOF and returns back to main which exits.
To explain redirect we need to understand redirection in C.
First, a simple one-way pipe.
int pd[2];
pipe(pd);
//parent
if (fork()) {
// Parent must close the input side else reading from pd[0] will
// continue to try to read from pd[1] even after the child closes
// their pipe.
close(pd[1]);
int p;
while( read(pd[0], &p, sizeof(p)) ) {
printf("p = %d\n", p);
}
fprintf(stderr, "parent done reading\n");
}
// child
else {
// Not strictly necessary, but the child will not be reading.
close(pd[0]);
for (int i = 2; i < 10; i++) {
write(pd[1], &i, sizeof(i));
}
// Tell the parent we're done writing to the pipe.
// The parent will get EOF on its next read. If the child
// does not close the pipe, the parent will hang waiting to read.
close(pd[1]);
fprintf(stderr, "child done writing\n");
// Pipes are closed automatically when a process exits, but
// let's simulate the child not immediately exiting to
// illustrate why it's important to explicitly close pipes.
sleep(1);
}
The parent and child share a pipe. The parent reads from one end, the child writes to the other. The child closes their write end when they're done so the parent doesn't hang trying to read. The parent closes their write end immediately so their pipe doesn't try to read from it.
Instead of passing the pipe around, redirect is redirecting the parent's half to stdin and the child's half to stdout. Let's do that in our simple example using dup2. dup2 duplicates a descriptor to another, first closing the target.
int pd[2];
pipe(pd);
if (fork()) {
// Redirect pd[0] to stdin.
dup2(pd[0], STDIN_FILENO);
// Parent still has to close its input side.
close(pd[1]);
int p;
while( read(STDIN_FILENO, &p, sizeof(p)) ) {
printf("p = %d\n", p);
}
fprintf(stderr, "parent done reading\n");
} else {
// Redirect pd[1] to stdout.
dup2(pd[1], STDOUT_FILENO);
// Close the original pd[1] so the parent doesn't try to read from it.
close(pd[1]);
for (int i = 2; i < 10; i++) {
write(STDOUT_FILENO, &i, sizeof(i));
}
// Tell the parent we're done writing.
close(STDOUT_FILENO);
fprintf(stderr, "child done writing\n");
sleep(1);
}
The final twist is dup. dup duplicates pd[k] to the lowest numbered descriptor currently not in use by the process. redirect(0, pd) closes descriptor 0 and then copies pd[0] to the lowest numbered descriptor: 0.
redirect(1, pd) closes descriptor 1 and then copies pd[1] to what it hopes is the lowest numbered descriptor: 1. If something else closed 0, redirect(1, pd) will copy pd[1] to descriptor 0 and the code will not work. This can be avoided by using dup2 which makes it explicit which file descriptor you're copying to.
// close(k) and dup(pd[k]) to k safely and explicitly.
dup2(pd[k], k);
redirect can be rewritten as:
void redirect(int k, int pd[]) {
dup2(pd[k], k);
close(pd[0]);
close(pd[1]);
}
Note that is all for a one-way pipe. cull uses bi-directional pipes, but the idea is the same.
By redirecting its pipe to stdin and stdout the program can use the pipe without having to pass the pipe around. This lets right read the first prime from the number generator and then let cull read the rest. It could also be done explicitly.
With some simpler examples in place, now we can answer the questions.
1). Why does redirect need to close pd[1]?
The parent must close the input side of its pipe, even after it's been duplicated or closed by the child, else the pipe will remain open and the parent will hang trying to read from it.
cull() is reading from the file descriptor 0, but in right() the child process will close 0. Why doesn't this cause any problem?
right closes its 0 and then copies pd[0] to 0. cull does not close 0, it closes pd[0]. cull reads from the original 0 which is the number generator.
Why it is necessary to use redirect here when we only want read from 0 and write to 1?
Because we need 0 and 1 to be different things at different times. We don't really want to read from 0 and write to 1. We want to read and write from pipes which happen to be attached to 0 and 1. The program is redirecting its pipes to 0 and 1 to demonstrate how Unix pipes and redirection works internally.
It took a dabbler like me some time to figure out how this program worked, it would have been a lot easier if I'd read the original paper and seen the shell script version first.
It can be rewritten to use explicit pipes. This avoids action at a distance, is easier to understand, and still demonstrates pipes and co-routines, but it no longer illustrates redirection.
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
void cull(int p, int read_d, int write_d) {
int n;
while (read(read_d, &n, sizeof(n))) {
if (n % p != 0) {
write(write_d, &n, sizeof(n));
}
}
}
void right(int read_d) {
int p;
if (read(read_d, &p, sizeof(p))) {
printf("prime %d\n", p);
int cull_pipe[2];
pipe(cull_pipe);
if (fork()) {
// right() reads from cull().
close(cull_pipe[1]);
right(cull_pipe[0]);
} else {
// cull() reads from the number generator and writes to right().
close(cull_pipe[0]);
cull(p, read_d, cull_pipe[1]);
close(cull_pipe[1]);
}
}
}
int main(int argc, char *argv[]) {
int pd[2];
pipe(pd);
if (fork()) {
// The first call to right() reads from the number generator.
close(pd[1]);
right(pd[0]);
} else {
close(pd[0]);
for (int i = 2; i < 6; i++) {
write(pd[1], &i, sizeof(i));
}
close(pd[1]);
}
exit(0);
}
Other notes:
The headers can be replaced with standard headers.
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
i'm trying to call an external program from my code with some arguments. As i'm trying to see how different parameters change it's output i have to run it multiple times (about 1000 times). Every time the external program runs i'm just interested in one line of its output although it is printing a lot of (for my purpose) useless stuff. The line i'm interested in is right above the special identifier("some_signal") in the output. So i thought i'll wait till this line appears and read the line above.
I tried the following:
pid_t pid = 0;
int pipefd[2];
FILE* output;
char line[256]; // pipe read buffer
char prev_line[256]; // prev. line pipe buffer
char signal[] = "some_signal\n";
int status = 0;
double obj_Val;
pipe (pipefd); //create pipe
pid = fork (); //span child process
if (pid == 0)
{
// redirect child's output to pipe
close (pipefd[0]);
dup2 (pipefd[1], STDOUT_FILENO);
dup2 (pipefd[1], STDERR_FILENO);
execl ("/some/path",
"some/path",
"some_argument", (char*) NULL);
}
else if (pid < (pid_t) 0)
{
printf("fork failed \n");
return EXIT_FAILURE;
}
else
{
// get child output to pipe
close (pipefd[1]);
output = fdopen (pipefd[0], "r");
while (fgets (line, sizeof(line), output), signal != NULL)
{
if(strcmp(line, signal) == 0)
{
obj_Val = atof (prev_line);
kill (pid, SIGTERM);
waitpid (pid, &status, WNOHANG);
kill (pid, SIGKILL);
waitpid (pid, &status, 0);
break;
}//end if
strcpy (prev_line, line);
}//end while
}
This works fine for like 100 runs or so and then one out of two errors occurs. The first one is a segmentation fault. The second one is the calling program printing out all the output of the called program (without the line containing the wanted signal) and goes into an infinite loop (my guess is, since the signal is missing the while loop won't terminate).
Maybe someone can provide a hint or a link where to look and what to look for, or preferably tell me what i'm doing wrong.
Your while condition is broken: signal != NULL is always true, as signal is an array; and because of the comma , operator the entire condition is always true.
And because of the following, you're not checking the return value of fgets, which means that nothing was read and the buffer is uninitialized.
Also prev_line is not initialized before you do atof on it for the first time.
In any case, compile with -Wall -Werror and fix the remaining errors.
I am trying to implement a client-server type of communication system using the poll function in C. The flow is as follows:
Main program forks a sub-process
Child process calls the exec function to execute some_binary
Parent and child send messages to each other alternately, each message that is sent depends on the last message received.
I tried to implement this using poll, but ran into problems because the child process buffers its output, causing my poll calls to timeout. Here's my code:
int main() {
char *buffer = (char *) malloc(1000);
int n;
pid_t pid; /* pid of child process */
int rpipe[2]; /* pipe used to read from child process */
int wpipe[2]; /* pipe used to write to child process */
pipe(rpipe);
pipe(wpipe);
pid = fork();
if (pid == (pid_t) 0)
{
/* child */
dup2(wpipe[0], STDIN_FILENO);
dup2(rpipe[1], STDOUT_FILENO);
close(wpipe[0]); close(rpipe[0]);
close(wpipe[1]); close(rpipe[1]);
if (execl("./server", "./server", (char *) NULL) == -1)
{
fprintf(stderr, "exec failed\n");
return EXIT_FAILURE;
}
return EXIT_SUCCESS;
}
else
{
/* parent */
/* close the other ends */
close(wpipe[0]);
close(rpipe[1]);
/*
poll to check if write is good to go
This poll succeeds, write goes through
*/
struct pollfd pfds[1];
pfds[0].fd = wpipe[1];
pfds[0].events = POLLIN | POLLOUT;
int pres = poll(pfds, (nfds_t) 1, 1000);
if (pres > 0)
{
if (pfds[0].revents & POLLOUT)
{
printf("Writing data...\n");
write(wpipe[1], "hello\n", 6);
}
}
/*
poll to check if there's something to read.
This poll times out because the child buffers its stdout stream.
*/
pfds[0].fd = rpipe[0];
pfds[0].events = POLLIN | POLLOUT;
pres = poll(pfds, (nfds_t) 1, 1000);
if (pres > 0)
{
if (pfds[0].revents & POLLIN)
{
printf("Reading data...\n");
int n = read(rpipe[0], buffer, 1000);
buffer[n] = '\0';
printf("child says:\n%s\n", buffer);
}
}
kill(pid, SIGTERM);
return EXIT_SUCCESS;
}
}
The server code is simply:
int main() {
char *buffer = (char *) malloc(1000);
while (scanf("%s", buffer) != EOF)
{
printf("I received %s\n", buffer);
}
return 0;
}
How do I prevent poll calls from timing out because of buffering?
EDIT:
I would like the program to work even when the execed binary is external, i.e., I have no control over the code - like a unix command, e.g., cat or ls.
There seem to be two problems in your code. "stdout" is by default buffered,
so the server should flush it explicitly:
printf("I received %s\n", buffer);
fflush(stdout);
And the main program should not register for POLLOUT when trying to read
(but you may want register for POLLERR):
pfds[0].fd = rpipe[0];
pfds[0].events = POLLIN | POLLERR;
With these modifications you get the expected output:
$ ./main
Writing data...
Reading data...
child says:
I received hello
Generally, you should also check the return value of poll(), and repeat the call if
necessary (e.g. in the case of an interrupted system call or timeout).
You need, as I answered in a related answer to a previous question by you, to implement an event loop; as it name implies, it is looping, so you should code in the parent process:
while (1) { // simplistic event loop!
int status=0;
if (waitpid(pid, &status, WNOHANG) == pid)
{ // clean up, child process has ended
handle_process_end(status);
break;
};
struct pollpfd pfd[2];
memset (&pfd, 0, sizeof(pfd)); // probably useless but dont harm
pfd[0].fd = rpipe[0];
pfd[0].events = POLL_IN;
pfd[1].fd = wpipe[1];
pfd[0].event = POLL_OUT;
#define DELAY 5000 /* 5 seconds */
if (poll(pfd, 2, DELAY)>0) {
if (pfd[0].revents & POLL_IN) {
/* read something from rpipe[0]; detect end of file;
you probably need to do some buffering, because you may
e.g. read some partial line chunk written by the child, and
you could only handle full lines. */
};
if (pfd[1].revents & POLL_OUT) {
/* write something on wpipe[1] */
};
}
fflush(NULL);
} /* end while(1) */
you cannot predict in which order the pipes are readable or writable, and this can happen many times. Of course, a lot of buffering (in the parent process) is involved, I leave the details to you.... You have no influence on the buffering in the child process (some programs detect that their output is or not a terminal with isatty).
What an event polling loop like above gives you is to avoid the deadlock situation where the child process is blocked because its stdout pipe is full, while the parent is blocked writing (to the child's stdin pipe) because the pipe is full: with an event loop, you read as soon as some data is polled readable on the input pipe (i.e. the stdout of the child process), and you write some data as soon as the output pipe is polled writable (i.e. is not full). You cannot predict in advance in which order these events "output of child is readable by parent" and "input of child is writable by parent" happen.
I recommend reading Advanced Linux Programming which has several chapters explaining these issues!
BTW my simplistic event loop is a bit wrong: if the child process terminated and some data remains in its stdout pipe, its reading is not done. You could move the waitpid test after the poll
Also, don't expect that a single write (from the child process) into a pipe would trigger one single read in the parent process. In other words, there is no notion of message length. However, POSIX knows about PIPE_MAX .... See its write documentation. Probably your buffer passed to read and write should be of PIPE_MAX size.
I repeat: you need to call poll inside your event loop and very probably poll will be called several times (because your loop will be repeated many times!), and will report readable or writable pipe ends in an unpredictable (and non-reproducible) order! A first run of your program could report "rpipe[0] readable", you read 324 bytes from it, you repeat the event loop, poll says you "wpipe[1] writable", you can write 10 bytes to it, you repeat the event loop, poll tells that "rpipe[0] readable", you read 110 bytes from it, you repeat the event loop, poll tells again "rpipe[0] readable", you read 4096 bytes from it, etc etc etc... A second run of the same program in the same environment would give different events, like: poll says that "wpipe[1] writable", you write 1000 bytes to it, you repeat the loop, poll says that "rpipe[0] readable, etc....
NB: your issue is not the buffering in the child ("client") program, which we assume you cannot change. So what matters is not the buffered data in it, but the genuine input and output (that is the only thing your parent process can observe; internal child buffering is irrelevant for the parent), i.e. the data that your child program has been able to really read(2) and write(2). And if going thru a pipe(7), such data will become poll(2)-able in the parent process (and your parent process can read or write some of it after POLL_IN or POLL_OUT in the updated revents field after poll). BTW, if you did code the child, don't forget to call fflush at appropriate places inside it.