Related
I'm trying to use named pipe in C to run a child process in the background from a path in non-blocking mode and read the output of the child.
This is my code:
int fifo_in = open("fifo_1", O_RDONLY| O_NONBLOCK);
int fifo_out = open("fifo_2", O_WRONLY| O_NONBLOCK);
dup2(fifo_in, 0);
dup2(fifo_out, 1);
char app[] = "/usr/local/bin/probemB";
char * const argsv[] = { app, "1", NULL };
if (execv(app, argsv) < 0) {
printf("execv error\n");
exit(4);
}
I will later use read function to read the child process output.
But the problem is that execv is blocking while it is reading the output from the process instead of allowing me to read.
Can someone help me to correct the above problem please ?
You're wrong in that execv is blocking.
If execv works, it will never return. It replaces your program. You need to fork a new process for execv:
if (fork() == 0)
{
// In child process, first setup the file descriptors
dup2(fifo_out, STDOUT_FILENO); // Writes to standard output will be written to the pipe
close(fifo_out); // These are not needed anymore
close(fifo_in);
// Run the program with execv...
}
else
{
// Unless there was an error, this is in the parent process
close(fifo_out);
// TODO: Read from fifo_in, which will contain the standard output of the child process
}
Another thing, you seem have two different and unconnected named pipes. You should open only one pipe, for reading in the parent process, and for writing in the child process:
int fifo_in = open("fifo_1", O_RDONLY| O_NONBLOCK);
int fifo_out = open("fifo_1", O_WRONLY| O_NONBLOCK);
But if you only want to communicate internally, you don't need named pipes. Instead use anonymous pipes as created by the pipe function.
I have a very specific problem for which I am unable to find the answer after numerous searches. I have a linux program. It's job is to launch another secondary executable (via fork() and exec()) when it receives a specific message over the network. I do not have access to modify the secondary executable.
My program prints all its TTY to stdout, and I typically launch it via ./program > output.tty The problem I have is that this second executable is very verbose. It simultaneously prints to stdout while also putting the same TTY in a log file. So my output.tty file ends up containing both output streams.
How can I set things up such that the secondary executable's TTY gets redirected to /dev/null? I can't use system() because I can't afford to wait for the child process. I need to be able to fire and forget.
Thanks.
In child process use dup2() to redirect the output to a file.
int main(int argc, const char * argv[]) {
pid_t ch;
ch = fork();
int fd;
if(ch == 0)
{
//child process
fd = open("/dev/null",O_WRONLY | O_CREAT, 0666); // open the file /dev/null
dup2(fd, 1); // replace standard output with output file
execlp("ls", "ls",".",NULL); // Excecute the command
close(fd); // Close the output file
}
//parent process
return 0;
}
In the child process, before calling exec, you need to close the standard output stream.
pid_t pid =fork();
if (pid == 0) {
close(1);
// call exec
} else if (pid > 0) {
// parent
}
I'm new to piping and have been trying to create a pair of pipes which allow a child process to write to the parent process, and the parent process to communicate back. There is 1 parent with up to 4 children. The child becomes a different program with exec.
What I have working:
Writing from the parent to the child process. When I read in the child program's stdin, it will receive what I wrote from the parent.
The aim:
To create a card game where the parent talks to each individual client (the child processes) and gives all the moves and information to them, from its stdout to the children's stdin. The individual child processes give back their moves on their stdout, read by the main parent. The moves that the game makes is fully decided by a sequence, not players. So it's a bot game.
What I am stuck with:
I'm not sure how to get it so the parent can read the child's stdout through a file stream. When I try to setup the reading from child lines, the code seems to stop working. Not even the child can read from parent (it seems to stop at the now commented out liens for setting up child to parent).
I also am unsure how to "wait" until something appears. Like, at the start the players have to send a "ready" message back to the parent to let them know they are working. Once I send the "ready" from the child, how do I "wait" indefinitely until the next message appears?
I'm not sure if I'm setting up the pipes correctly. Can someone provide guidance on how to use communication pipes and confirm my logic below?
What I gather for getting parent to write to child is:
Create the pipe first
Fork off the parent process into another process (child)
Connect the pipe's in to the parent's stdout, and close off the reading side for the parent using dup2 and close
Connect the pipe's out to the child's stdin, and close off the writing part for the child using dup2 and close
Get a file stream using fdopen() from the file descriptor and then print to that.
The child process stdin is now whatever you print to stdout from the parent.
Is this correct? I tried applying this kind of logic for child to parent but reversing it.
Connect the in pipe to the read file stream, which is where the child program writes to from its stdout.
Connect the out pipe to the read stream, where the parent reads from.
void start_child_process(Game *game, int position) {
int child2Parent[2];
int parent2Child[2];
if (pipe(parent2Child)) {
printf("PIPE FAIL!\n");
}
if (pipe(child2Parent)) {
printf("PIPE FAIL!\n");
}
pid_t pid = fork();
game->readStream[position] = fdopen(child2Parent[0], "r");
game->writeStream[position] = fdopen(parent2Child[1], "w");
if (pid) { // Parent
// Write from parent to child
close(parent2Child[0]);
dup2(fileno(game->writeStream[position]), STDOUT_FILENO);
fprintf(game->writeStream[position], "%s", "test message");
fflush(game->writeStream[position]);
close(parent2Child[1]);
// Read from child -- not working
/*dup2(child2Parent[0], STDIN_FILENO);
close(child2Parent[0]);
close(child2Parent[1]);
*/
} else {
// Setup child to read from stdin from parent
dup2(parent2Child[0], STDIN_FILENO);
close(parent2Child[1]);
// Setup writing from child to parent
/*
if (dup2(child2Parent[1], STDOUT_FILENO) == -1) {
fprintf(stderr, "dup2 in child failed\n");
} else {
fprintf(stderr, "dup2 in child successful\n");
close(child2Parent[0]);
close(child2Parent[1]);
}
*/
if ((int)execl("child", "2", "A", NULL) == -1) {
printf("Failed child process\n");
}
}
}
My child main has the following which reads it:
char string[100];
printf("reading from pipe: %s\n", fgets(string, 100, stdin));
But I'm not sure how
Also, I'm not permitted to use popen() or write(). I'm also encouraged to use file streams apparently.
I speak principally to your main question of establishing two-way communication between parent and child processes. If you would like additional answers then please pose separate questions.
You seem to have a reasonable general approach, but you do have one serious misconception / design flaw: whereas it is reasonable for multiple clients to each connect their standard streams to pipes for communicating with the parent process, you cannot connect the parent's end of all those pipes to the parent's standard streams if you want to be able to handle more than one client at a time. The parent only has one set of standard streams, after all. To support multiple clients, then, the parent process must maintain a separate pair of file descriptors and/or streams for each one, and must communicate via those instead of via its standard streams.
I am uncertain why your parent / child communication is failing when you hook up the child-to-parent direction. The process is indeed analogous to setting up the other other endpoint. Here is a working example:
parent.c:
#include <stdlib.h>
#include <stdio.h>
#include <errno.h>
#include <unistd.h>
int main() {
int child2Parent[2];
int parent2Child[2];
char buffer[256];
FILE *p2cStream;
FILE *c2pStream;
pid_t pid;
if (pipe(parent2Child) || pipe(child2Parent)) {
perror("Failed to create pipes");
exit(EXIT_FAILURE);
}
switch (pid = fork()) {
case -1: /* error */
perror("Failed to fork");
break;
case 0: /* child */
// Setup child to read from stdin from parent
close(parent2Child[1]); /* ignoring any error */
close(child2Parent[0]); /* ignoring any error */
if ((dup2(parent2Child[0], STDIN_FILENO) < 0)
|| (dup2(child2Parent[1], STDOUT_FILENO) < 0)) {
perror("Failed to duplicate file descriptors");
} else {
/* conventionally, the first program argument is the program name */
/* also, execl() returns only if it fails */
execl("child", "child", "2", "A", NULL);
perror("Failed to exec child process");
}
exit(EXIT_FAILURE);
break;
default: /* parent */
close(parent2Child[0]); /* ignoring any error */
close(child2Parent[1]); /* ignoring any error */
if (!(p2cStream = fdopen(parent2Child[1], "w"))
|| !(c2pStream = fdopen(child2Parent[0], "r"))) {
perror("Failed to open streams");
exit(EXIT_FAILURE);
}
if ((fprintf(p2cStream, "test message from parent\n") < 0)
|| fclose(p2cStream)) {
perror("Failed to write to the child");
exit(EXIT_FAILURE);
}
if (fscanf(c2pStream, "%255[^\n]", buffer) < 1) {
perror("Failed to read the child's message");
exit(EXIT_FAILURE);
}
printf("The child responds: '%s'\n", buffer); /* ignoring any error */
break;
}
return EXIT_SUCCESS;
}
child.c:
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <errno.h>
int main(int argc, char *argv[]) {
char buffer[256] = { 0 };
if (scanf("%255[^\n]", buffer) < 0) {
perror("Failed to reading input");
exit(EXIT_FAILURE);
}
/*
* If stdout is connected to the parent then we must avoid
* writing anything unexpected to it
*/
if (fprintf(stderr, "received: '%s'\n", buffer) < 0) {
perror("Failed to echo input");
exit(EXIT_FAILURE);
}
printf("Hi, Mom!\n"); /* ignoring any error */
fflush(stdout); /* ignoring any error */
return EXIT_SUCCESS;
}
In contrast to your code, do note
the attention to checking all return values that may signal an error that I care about;
the parent's use of streams other than the standard streams for communicating with the child (though with only one child, that's a convenience rather than a necessity);
the convention for execl() arguments.
Note also that as for waiting for something to "appear", I/O operations on the IPC streams you set up this way will automatically produce that effect. How you can or should make use of that, however, is certainly a different issue.
I'm having trouble with my program waiting for a child process (gzip) to finish and taking a very long time in doing so.
Before it starts waiting it closes the input stream to gzip so this should trigger it to terminate pretty quickly. I've checked the system and gzip isn't consuming any CPU or waiting on IO (to write to disk).
The very odd thing is the timing on when it stops waiting...
The program us using pthreads internally. It's processing 4 pthreads side by side. Each thread processes many units of work and for each unit of work one it kicks off a new gzip process (using fork() and execve()) to write the result. Threads hang when gzip doesn't terminate, but it suddenly does terminate when other threads close their instance.
For clarity, I'm setting up a pipeline that goes: my program(pthread) --> gzip --> file.gz
I guess this could be explained in part by CPU load. But when processes are kicked off minutes apart and the whole system ends up using only 1 core of 4 because of this locking issue, that seems unlikely.
The code to kick off gzip is below. The execPipeProcess is called such that the child writes direct to file, but reads from my program. That is:
execPipeProcess(&process, "gzip", -1, gzFileFd)
Any suggestions?
typedef struct {
int processID;
const char * command;
int stdin;
int stdout;
} ChildProcess;
void closeAndWait(ChildProcess * process) {
if (process->stdin >= 0) {
stdLog("Closing post process stdin");
if (close(process->stdin)) {
exitError(-1,errno, "Failed to close stdin for %s", process->command);
}
}
if (process->stdout >= 0) {
stdLog("Closing post process stdin");
if (close(process->stdout)) {
exitError(-1,errno, "Failed to close stdout for %s", process->command);
}
}
int status;
stdLog("waiting on post process %d", process->processID);
if (waitpid(process->processID, &status, 0) == -1) {
exitError(-1, errno, "Could not wait for %s", process->command);
}
stdLog("post process finished");
if (!WIFEXITED(status)) exitError(-1, 0, "Command did not exit properly %s", process->command);
if (WEXITSTATUS(status)) exitError(-1, 0, "Command %s returned %d not 0", process->command, WEXITSTATUS(status));
process->processID = 0;
}
void execPipeProcess(ChildProcess * process, const char* szCommand, int in, int out) {
// Expand any args
wordexp_t words;
if (wordexp (szCommand, &words, 0)) exitError(-1, 0, "Could not expand command %s\n", szCommand);
// Runs the command
char nChar;
int nResult;
if (in < 0) {
int aStdinPipe[2];
if (pipe(aStdinPipe) < 0) {
exitError(-1, errno, "allocating pipe for child input redirect failed");
}
process->stdin = aStdinPipe[PIPE_WRITE];
in = aStdinPipe[PIPE_READ];
}
else {
process->stdin = -1;
}
if (out < 0) {
int aStdoutPipe[2];
if (pipe(aStdoutPipe) < 0) {
exitError(-1, errno, "allocating pipe for child input redirect failed");
}
process->stdout = aStdoutPipe[PIPE_READ];
out = aStdoutPipe[PIPE_WRITE];
}
else {
process->stdout = -1;
}
process->processID = fork();
if (0 == process->processID) {
// child continues here
// these are for use by parent only
if (process->stdin >= 0) close(process->stdin);
if (process->stdout >= 0) close(process->stdout);
// redirect stdin
if (STDIN_FILENO != in) {
if (dup2(in, STDIN_FILENO) == -1) {
exitError(-1, errno, "redirecting stdin failed");
}
close(in);
}
// redirect stdout
if (STDOUT_FILENO != out) {
if (dup2(out, STDOUT_FILENO) == -1) {
exitError(-1, errno, "redirecting stdout failed");
}
close(out);
}
// we're done with these; they've been duplicated to STDIN and STDOUT
// run child process image
// replace this with any exec* function find easier to use ("man exec")
nResult = execvp(words.we_wordv[0], words.we_wordv);
// if we get here at all, an error occurred, but we are in the child
// process, so just exit
exitError(-1, errno, "could not run %s", szCommand);
} else if (process->processID > 0) {
wordfree(&words);
// parent continues here
// close unused file descriptors, these are for child only
close(in);
close(out);
process->command = szCommand;
} else {
exitError(-1,errno, "Failed to fork");
}
}
Child process inherits open file descriptors.
Every subsequent gzip child process inherits not only pipe file descriptors intended for communication with that particular instance but also file descriptors for pipes connected to previous child process instances.
It means that stdin pipe is still open when the main process performs close since there are some other file descriptors for the same pipe in a few child processes. Once those ones terminate the pipe is finally closed.
A quick fix is to prevent child processes from inheriting pipe file descriptors intended for the master process by setting close-on-exec flag.
Since there are multiple threads involved spawning child processes should be serialized to prevent child process from inheriting pipe fds intended for another child process.
You have not given us enough information to be sure, as the answer depends on how you use the functions presented. However, your closeAndWait() function looks a bit suspicious. It may be reasonable to suppose that that the child process in question will exit when it reaches the end of its stdin, but what is supposed to happen to data it has written or even may still write to its stdout? It is possible that your child processes hang because their standard output is blocked, and it is slow for them to recognize it.
I think this reflects a design problem. If you are capturing the child processes' output, as you seem at least to support doing, then after you close the parent's end of a child's input stream you'll want the parent to continue reading the child's output to its end, and performing whatever processing it intends to do on it. Otherwise you may lose some of it (which for a child performing gzip would mean corrupted data). You cannot do that if you make closing both streams part of the process of terminating the child.
Instead, you should to close the parent's end of the child's stdin first, continue processing its output until you reach its end, and only then try to collect the child. You can make closing the parent's end of the child's output stream part of the process of collecting that child if you like. Alternatively, if you really do want to discard any remaining output from the child, then you should drain its output stream between closing the input and closing the output.
I'm implementing piping on a simulated file system in C++ (with mostly C). It needs to run commands in the host shell but perform the piping itself on the simulated file system.
I could achieve this with the pipe(), fork(), and system() system calls, but I'd prefer to use popen() (which handles creating a pipe, forking a process, and passing a command to the shell). This may not be possible because (I think) I need to be able to write from the parent process of the pipe, read on the child process end, write the output back from the child, and finally read that output from the parent. The man page for popen() on my system says a bidirectional pipe is possible, but my code needs to run on a system with an older version supporting only unidirectional pipes.
With the separate calls above, I can open/close pipes to achieve this. Is that possible with popen()?
For a trivial example, to run ls -l | grep .txt | grep cmds I need to:
Open a pipe and process to run ls -l on the host; read its output back
Pipe the output of ls -l back to my simulator
Open a pipe and process to run grep .txt on the host on the piped output of ls -l
Pipe the output of this back to the simulator (stuck here)
Open a pipe and process to run grep cmds on the host on the piped output of grep .txt
Pipe the output of this back to the simulator and print it
man popen
From Mac OS X:
The popen() function 'opens' a
process by creating a bidirectional
pipe, forking, and invoking the shell.
Any streams opened by previous popen()
calls in the parent process are closed
in the new child process.
Historically, popen() was implemented
with a unidirectional pipe; hence,
many implementations of popen() only
allow the mode argument to specify
reading or writing, not both. Because
popen() is now implemented using a
bidirectional pipe, the mode argument
may request a bidirectional data flow.
The mode argument is a pointer to a
null-terminated string which must be
'r' for reading, 'w' for writing, or
'r+' for reading and writing.
I'd suggest writing your own function to do the piping/forking/system-ing for you. You could have the function spawn a process and return read/write file descriptors, as in...
typedef void pfunc_t (int rfd, int wfd);
pid_t pcreate(int fds[2], pfunc_t pfunc) {
/* Spawn a process from pfunc, returning it's pid. The fds array passed will
* be filled with two descriptors: fds[0] will read from the child process,
* and fds[1] will write to it.
* Similarly, the child process will receive a reading/writing fd set (in
* that same order) as arguments.
*/
pid_t pid;
int pipes[4];
/* Warning: I'm not handling possible errors in pipe/fork */
pipe(&pipes[0]); /* Parent read/child write pipe */
pipe(&pipes[2]); /* Child read/parent write pipe */
if ((pid = fork()) > 0) {
/* Parent process */
fds[0] = pipes[0];
fds[1] = pipes[3];
close(pipes[1]);
close(pipes[2]);
return pid;
} else {
close(pipes[0]);
close(pipes[3]);
pfunc(pipes[2], pipes[1]);
exit(0);
}
return -1; /* ? */
}
You can add whatever functionality you need in there.
You seem to have answered your own question. If your code needs to work on an older system that doesn't support popen opening bidirectional pipes, then you won't be able to use popen (at least not the one that's supplied).
The real question would be about the exact capabilities of the older systems in question. In particular, does their pipe support creating bidirectional pipes? If they have a pipe that can create a bidirectional pipe, but popen that doesn't, then I'd write the main stream of the code to use popen with a bidirectional pipe, and supply an implementation of popen that can use a bidirectional pipe that gets compiled in an used where needed.
If you need to support systems old enough that pipe only supports unidirectional pipes, then you're pretty much stuck with using pipe, fork, dup2, etc., on your own. I'd probably still wrap this up in a function that works almost like a modern version of popen, but instead of returning one file handle, fills in a small structure with two file handles, one for the child's stdin, the other for the child's stdout.
POSIX stipulates that the popen() call is not designed to provide bi-directional communication:
The mode argument to popen() is a string that specifies I/O mode:
If mode is r, when the child process is started, its file descriptor STDOUT_FILENO shall be the writable end of the pipe, and the file descriptor fileno(stream) in the calling process, where stream is the stream pointer returned by popen(), shall be the readable end of the pipe.
If mode is w, when the child process is started its file descriptor STDIN_FILENO shall be the readable end of the pipe, and the file descriptor fileno(stream) in the calling process, where stream is the stream pointer returned by popen(), shall be the writable end of the pipe.
If mode is any other value, the result is unspecified.
Any portable code will make no assumptions beyond that. The BSD popen() is similar to what your question describes.
Additionally, pipes are different from sockets and each pipe file descriptor is uni-directional. You would have to create two pipes, one configured for each direction.
In one of netresolve backends I'm talking to a script and therefore I need to write to its stdin and read from its stdout. The following function executes a command with stdin and stdout redirected to a pipe. You can use it and adapt it to your liking.
static bool
start_subprocess(char *const command[], int *pid, int *infd, int *outfd)
{
int p1[2], p2[2];
if (!pid || !infd || !outfd)
return false;
if (pipe(p1) == -1)
goto err_pipe1;
if (pipe(p2) == -1)
goto err_pipe2;
if ((*pid = fork()) == -1)
goto err_fork;
if (*pid) {
/* Parent process. */
*infd = p1[1];
*outfd = p2[0];
close(p1[0]);
close(p2[1]);
return true;
} else {
/* Child process. */
dup2(p1[0], 0);
dup2(p2[1], 1);
close(p1[0]);
close(p1[1]);
close(p2[0]);
close(p2[1]);
execvp(*command, command);
/* Error occured. */
fprintf(stderr, "error running %s: %s", *command, strerror(errno));
abort();
}
err_fork:
close(p2[1]);
close(p2[0]);
err_pipe2:
close(p1[1]);
close(p1[0]);
err_pipe1:
return false;
}
https://github.com/crossdistro/netresolve/blob/master/backends/exec.c#L46
(I used the same code in popen simultaneous read and write)
Here's the code (C++, but can be easily converted to C):
#include <unistd.h>
#include <cstdlib>
#include <cstdio>
#include <cstring>
#include <utility>
// Like popen(), but returns two FILE*: child's stdin and stdout, respectively.
std::pair<FILE *, FILE *> popen2(const char *__command)
{
// pipes[0]: parent writes, child reads (child's stdin)
// pipes[1]: child writes, parent reads (child's stdout)
int pipes[2][2];
pipe(pipes[0]);
pipe(pipes[1]);
if (fork() > 0)
{
// parent
close(pipes[0][0]);
close(pipes[1][1]);
return {fdopen(pipes[0][1], "w"), fdopen(pipes[1][0], "r")};
}
else
{
// child
close(pipes[0][1]);
close(pipes[1][0]);
dup2(pipes[0][0], STDIN_FILENO);
dup2(pipes[1][1], STDOUT_FILENO);
execl("/bin/sh", "/bin/sh", "-c", __command, NULL);
exit(1);
}
}
Usage:
int main()
{
auto [p_stdin, p_stdout] = popen2("cat -n");
if (p_stdin == NULL || p_stdout == NULL)
{
printf("popen2() failed\n");
return 1;
}
const char msg[] = "Hello there!";
char buf[32];
printf("I say \"%s\"\n", msg);
fwrite(msg, 1, sizeof(msg), p_stdin);
fclose(p_stdin);
fread(buf, 1, sizeof(buf), p_stdout);
fclose(p_stdout);
printf("child says \"%s\"\n", buf);
return 0;
}
Possible Output:
I say "Hello there!"
child says " 1 Hello there!"
No need to create two pipes and waste a filedescriptor in each process. Just use a socket instead. https://stackoverflow.com/a/25177958/894520