I have a problem with child processes and pipes in C.
My program creates two child processes (silblings) and also two pipes. The father reads from stdin and sends the data through the first pipe. The first child redirects the stdin to the first pipe and the stdout to the second pipe. After that the child runs gzip -cf with execlp(3). The second child reads from the second pipe and writes the data in the stdout (or in a file).
I tried it but it doesnt work with the binary data. If the first child only sends the text data through the second pipe (without using gzip) and the second child reads it, the program works fine. But if I use gzip with execlp() and the process writes binary data the program doesn't work, because I don't get all data.
Here is a part of the code:
First child:
/* Close the pipe-ends which are not needed [...] */
/* Redirect the stdin and stdout with dup2 [...] */
if(execlp(EXEC,EXEC_ARG0,EXEC_ARG1,(char*) 0) == -1) /* run execlp with 'gzip -cf' */
bail_out(EXIT_FAILURE,"Child_1: Cannot execute "EXEC_ARG0" "EXEC_ARG1"!");
Second child:
/* Close the pipe-ends which are not needed [...] */
FILE *fd = NULL;
char str[4096];
fd = fdopen(pipefd_com2[0],"rb");
(void) fread(str,sizeof(str),1,fd); /* read from pipe */
(void) fclose(fd);
FILE *fw = NULL;
if(file_arg == NULL) /* check if there was a file as argument */
fw = stdout;
else
fw = fopen(file_arg,"wb");
if(fw == NULL)
bail_out(EXIT_FAILURE,"Failed to write file!"); /* does not return */
(void) fwrite(str,strlen(str),1,fw); /* write to stdout or file */
(void) fclose(fw);
The strlen function is for C-style strings, not for binary data. Also, your parameters to fread are wrong. The second parameter is the size of each object you are reading. For arbitrary binary data, that should be 1, since each object is a byte. That will make the return value from fread the number of bytes you read, which is what you'll need to pass to fwrite as the third parameter.
In sum:
size_t bytes_read;
// ...
bytes_read = fread(str,1,sizeof(str),fd); /* read from pipe */
// ...
(void) fwrite(str,1,bytes_read,fw); /* write to stdout or file */
You should also check the return value from fread.
Related
Ok guys, there are a billion demos relating to dup, dup2, fcntl, pipe and all kinds of stuff that are wonderful when multiple processes exist. However, I have yet to see one very basic thing that I think will help explain the behavior of pipe and its relationship to standard out and in.
My goal is to simply (in the same process) reroute standard output through a pipe back to standard output directly. I have already accomplished this
with intermediate stages which redirect the pipe output to a file or write into a buffer... and then put standard output back to where it started. At that point, of course I can write the buffer back to stdout, but I don't want to do this.
Since I moved standard output to another location in the file table, I'd like to direct the output of the pipe to feed directly into the new standard output position and have it print like it normally would.
I feel like there is some kind of layer surrounding the file table that I am not understanding.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
int main() {
int pipeEnds_arr1[2];
char str1[] = "STRING TO FEED INTO PIPE \n"; // make a string array
pipe(pipeEnds_arr1);
printf("File Descriptor for pipe ends from array\nPOSITION out 0 : %d\nPOSITION in 1 : %d\n", pipeEnds_arr1[0], pipeEnds_arr1[1]);
/* now my goal is to shift the input of the pipe into the position of
* standard output, so that the print command feeds the pipe, then I
* would like to redirect the other end of the pipe to standard out.
*/
int someInt = dup(1); // duplicates stdout to next available file table position
printf ("Some Int FD: %d\n", someInt); // print out the fd for someInt just for knowing where it is
/* This is the problem area. The out end of the pipe never
* makes it back to std out, and I see no way to do so.
* Stdout should be in the file table position 5, but when
* I dup2 the output end of the pipe into this position ,
* I believe I am actually overwriting std out completely.
* But I don't want to overwrite it, i want to feed the output
* of the pipe into std out. I think I am fundamentally
* misunderstanding this issue.
*/
dup2(pipeEnds_arr1[1], 1); //put input end of pipe into std out position
//dup2(pipeEnds_arr1[0], 5); // this will not work
//and other tests I have conducted do not work
printf("File Descriptor for pipe ends from array\nPOSITION out 0 : %d\nPOSITION in 1 : %d\n", pipeEnds_arr1[0], pipeEnds_arr1[1]);
fflush(stdout);
close(pipeEnds_arr1[0]);
close(pipeEnds_arr1[1]);
return 0;
}
EDIT*********
OK, what I know is that somehow std out takes information from commands like printf and then routs it into a buffer that is then flushed to the shell.
What I believe is that there must be a way to rout the "read" or output end of the pipe to that same buffer that then gets to the shell. I have figured out how to rout the pipe output into a string, and then I can do as I please. In the example code I post below, I will first rout the pipe out to a string and then open a file and write the string to the open file descriptor of that file...
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
int main() {
/* Each pipe end array has to have 2 positions in it. The array
* position represents the two pipe ends with the 0 index
* position representing the output of the pipe (the place you want
* read your data from), and 1 index position representing the
* input file descriptor of the pipe (the place you want to write
* your data).
*/
int pipeEnds_arr1[2];
char str1[] = "Hello, we are feeding this into the pipe that we are through stdout into a pipe and then reading from the pipe and then feeding that output into a file \n"; // make a string array
/* Here we want to actually do the pipe command. We feed it the array
* with the 2 positions in it which will now hold file descriptors
* attached to the current process which allow for input and output
* through the new pipe. At this point, we don't know what the
* exact file decriptors are, but we can look at them by printing
*/
pipe(pipeEnds_arr1);
printf("File Descriptor for pipe ends from array\nPOSITION out 0 : %d\nPOSITION in 1 : %d\n", pipeEnds_arr1[0], pipeEnds_arr1[1]);
/* now my goal is to shift the input of the pipe into the position of
* standard output, so that the print command feeds the pipe, then we
* will try to read from the pipe and redirect the output to the std
* or in this test case out to a file.
*/
int someInt = dup(1); // we moved what was stdout into someInt;
/* put the write end of the pipe in the old stdout position by
* using dup2 so we will print directly into the pipe
*/
dup2(pipeEnds_arr1[1], 1);
/* this is where id like to re-rout the pipe back to stdout but
* im obviously not understanding this correctly
*/
//dup2(someInt, 3);
/* since std out has now been replaced by the pipe write end, this
* printf will print into the pipe
*/
printf("%s", str1);
/* now we read from the pipe into a new string we make */
int n;
char str2[strlen(str1)];
n = read(pipeEnds_arr1[0], str2, sizeof(str2)-1);
str2[n] = 0;
/* open a file and then write into it from the output of the pipe
* that we saved into the str2
*/
int fd = open("tmp.out", O_WRONLY | O_CREAT | O_TRUNC, 0644);
write(fd, str2, strlen(str2));
/* not sure about these last commands and their relevance */
fflush(stdout);
close(pipeEnds_arr1[0]);
close(pipeEnds_arr1[1]);
close(fd);
return 0;
}
Pipes aren't between file descriptors. They are between processes. So it doesn't make any sense to "reroute standard out through a pipe".
What you can do is modify a process's file descriptor table so that its stdout (fd 1) is the write side of a pipe. And you can modify another process's file descriptor table so that some file descriptor, perhaps even stdin (fd 0) is the read side of the same pipe. That allows you to pass data through the pipe between the two processes. (You can set up a pipe between two fds in the same process, if you want to; it's occasionally useful but watch out for deadlocking.)
stdout is not some sort of magical entity. It's just entry 1 in the fd table, and it might refer to any "file", in the Unix sense of the word, which includes regular files, devices (including the console and the pseudoterminal your shell is communicating with), sockets, pipes, FIFOs, and whatever else the operating system feels worthy of allowing streaming access to.
Normally, when the shell starts a running a command-line utility, it first clones fds 0, 1 and 2 (stdin, stdout and stderr) from its own fd 0, 1, and 2, which are normally all the same device: the console, or more commonly these days, the pseudoterminal provided by the graphical console application you are using. But you can change those assignments with, for example, shell redirection operators, shell pipe operators, and some shell-provided special files.
Finally, pipes do have small buffers in the kernel, but the key is rhe word "small" -- the buffer might hold as little as 4096 bytes. If gets full, attempts to write to the pipe will hang until space becomes available, which only happens when data is read from the other sude. That's why it is so easy to deadlock if the same process is using both sides of the pipe: if the process is hanging waiting for the pileto be emptied, it wikk neverbe able to read the pipe.
Similar questions has already been asked, but their solutions aren't helping me much
Program that read file and send it to parent process with pipe
Read/writing on a pipe, accomplishing file copying in C
I'm trying to read from a file test.txt ( which contains a single line of text), write it to a pipe, from where a child process will read from the pipe and write the contents to another file.
/* Read the contents of a file and display it using pipe */
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
#include <string.h>
void main()
{
char buffer[100];
char childbuff[100];
int fd[2], des, bytes, target;
pipe(fd);
if(fork()) {
/* parent process closes the downstream */
close(fd[0]);
/* reads the file */
des = open("test.txt", O_RDONLY);
bytes = read(des, buffer, sizeof(buffer));
/* puts data in pipe */
write(fd[1], buffer, bytes);
} else {
/* Child process closes the upstream */
close(fd[1]);
/* reads from the pipe */
read(fd[0], childbuff, sizeof(childbuff));
close(fd[0]);
/* output the received string */
printf("\nReceived string is -- %s", childbuff);
target = open("copy.txt", O_CREAT, 00777);
write(target, childbuff, (strlen(childbuff)-1));
}
}
Problem is printf() prints the string on terminal, a file named copy.txt also gets created, but nothing is getting copied to it (it seems there is a problem with the write() function )
however, if I change
write(target, childbuff, (strlen(childbuff)-1));
to
write(1, childbuff, (strlen(childbuff)-1));
string is simply getting written on my terminal.
So what possibly am I doing wrong while writing to file?
You also need O_WRONLY to write to the file:
target = open("copy.txt", O_CREAT |O_WRONLY, 00777);
Note that you can't use strlen() or %s to print it as a C-string. read(2) doesn't return a NUL terminated string.
Instead get the number of bytes read from read() and use it in write():
ssize_t num_bytes = read(fd[0], childbuff, sizeof(childbuff));
write(target, childbuff, num_bytes);
You should check the return of all system calls for failure.
i want to redirect the output of an unnamed pipe to an opened log file in c but i can't seem to make it happen, my code looks like this:
close(fildes[1]);
FILE * file = fopen(logfile, "w");
int fd = fileno(file);
if (fd == -1) {
bail_out(EXIT_FAILURE, strerror(errno));
}
/* if (dup2(fd, fildes[0]) == -1) {
bail_out(EXIT_FAILURE, strerror(errno));
} */
/* for testing purposes */
char reading_buf[3];
while(read(fildes[0], reading_buf, 3) > 0) {
write(fd, reading_buf, 1);
}
(void)wait(&status);
close(fildes[0]);
break;
the pipe gets filled, i've tested that with the while loop at the bottom. but when i comment out the dup2 call nothing gets redirected to the file. i think i don't fully understand dup2 and pipes.
Also: if i dup2 the reading end of the pipe and then close the original fildes[0], does the pipe get closed? or is it simply the FD that gets closed. not entirely should about that either..
To redirect output from a pipe to a file, somebody needs to read from the read end of the pipe and write to the file's file descriptor. It can't be done merely by duping the file descriptors.
For instance, say you have a pipe
int filedes[2];
pipe (filedes);
and a file
FILE *logfile = fopen (logfile_path, "w");
int logfd = fileno (logfile);
you could launch a child process to do the redirection using the cat command
int child = fork ();
if (child == 0) {
// close the write end of the pipe
close (filedes[1]);
// make STDOUT point to the log file
dup2 (logfd, STDOUT_FILENO);
// make STDIN point to the read end of the pipe
dup2 (filedes[0], STDIN_FILENO);
// launch cat
execlp ("cat", "cat", (char*) NULL);
// in case of error
exit (1);
}
Now, whatever is written to the write end of the pipe in the parent will be read by the child executing cat and written to the file.
// close the read end of the pipe, it's not needed in the parent
close (filedes[0]);
// write something
const char *msg = "Hello World!\n";
write (filedes[1], msg, strlen (msg));
// close the write end of the pipe, this will make the child read EOF
close (filedes[1]);
Don't forget to collect the zombie
wait (&status);
and close the file
fclose (logfile);
About your last question, when a file descriptor is duped there will be two of them pointing to the same underlying open file. So, when one of them is closed the file remains open for it can be accessed through other descriptor.
This is explained in the close man page:
If fd is the last file descriptor referring to the underlying open file
description (see open(2)), the resources associated with the open file
description are freed;
I am trying to implement multiple pipes in C like
ls - al | less | wc
I have trouble with creating the pipeline. I have a loop that is supposed to create the processes and connect them with pipes:
for(i=0;i<num_cmds;i++){
create_commands(cmds[i]);
}
My create_commands() function looks like this
void create_commands (char cmd[MAX_CMD_LENGTH]) // Command be processed
{
int pipeid[2];
pipe(pipeid);
if (childpid = fork())
{
/* this is the parent process */
dup2(pipeid[1], 1); // dup2() the write end of the pipe to standard output.
close(pipeid[1]); // close() the write end of the pipe
//parse the command
parse_command(cmd, argvector);
// execute the command
execvp(argvector[0], argvector);
close(1); // close standard output
}
else
{
/* child process */
dup2( pipeid[0], 0); // the read end of the pipe to standard input
close( pipeid[0] ); // close() the read end of the pipe
}
}
But this doesn't work, I'm getting my stdin and stdout messed up.
Could anyone please point me to what I am doing wrong?
Thank you in advance!
The popen() function executes the command specified by the string command. It creates a pipe between the calling program and the executed command, and returns a pointer to a stream that can be used to either read from or write to the pipe.
#include <stdio.h>
int main(int argc, char *argv[])
{
FILE *fp;
int status;
int PATH_MAX = 1024;
char path[PATH_MAX];
fp = popen("ls -al | less | wc", "r");
if (fp == NULL)
/* Handle error */;
while (fgets(path, PATH_MAX, fp) != NULL)
printf("%s", path);
status = pclose(fp);
if (status == -1) {
/* Error reported by pclose() */
} else {
/* Use macros described under wait() to inspect `status' in order
to determine success/failure of command executed by popen() */
}
}
You can use a preset string to be called within popen(), you can also use your argv[] arguments to be piped in you'ld like.
popen() gives you a pipe, a FIFO First In First Out stream, and popen also feeds the STDOUT back to your program.
Here's the man page for popen():
http://linux.die.net/man/3/popen
I wrote this piece of code that is supposed to redirect something written on the STDOUT by a function to the STDIN so that it can be read by another function. I cannot access these functions, so this is the only way I can use them.
mpz_fput(stdout, c) is one of these function. It just prints on the STDOUT something contained in the c data structure.
Now everything worked fine while debugging as before the following code I had a printf(); followed by a fflush(stdout); (needed to print debugging messages).
Now that I removed these two lines I noticed (using gdb) that this code stays idle on the read() function (last line of this piece of code)
char buffer[BUFSIZ] = "";
int out_pipe[2];
int in_pipe[2];
int saved_stdout;
int saved_stdin;
int errno;
// REDIRECT STDIN
saved_stdin = dup(STDIN_FILENO); /* save stdin for later */
if(errno= pipe(in_pipe) != 0 ) { /* make a pipe */
printf("\n%s",strerror(errno));
exit(1);
}
close(STDIN_FILENO);
dup2(in_pipe[0], STDIN_FILENO); /* redirect pipe to stdin */
// REDIRECT STDOUT
saved_stdout = dup(STDOUT_FILENO); /* save stdout for display later */
if(errno= pipe(out_pipe) != 0 ) { /* make a pipe */
printf("\n%s",strerror(errno));
exit(1);
}
dup2(out_pipe[1], STDOUT_FILENO); /* redirect stdout to the pipe */
close(out_pipe[1]);
mpz_fput(stdout,c); // put c on stdout
read(out_pipe[0], buffer, BUFSIZ); // read c from stdout pipe into buffer
any idea why is that?
Seems you used the blocking type. In this case, out_pipe[0] is a blocking handle. So read() blocked, and waiting for anything out from out_pipe[0].
Besides, I think there's something to do with the fflush():
For output streams, fflush() forces a write of all user-space buffered data
for the given output or update stream via the stream's underlying write
function. For input streams, fflush() discards any buffered data that has
been fetched from the underlying file, but has not been by the application.
The open status of the stream is unaffected.
In your case, you redirected pipe to STDOUT, then called fflush() to make everything in STDOUT flushed and move them to read() buffer. Then you called read() to read them out. If you didn't call fflush(), and read() buffer would be empty. Since it's a blocking handle used by read(), you can't read anything from the buffer, so it will be blocked.
This is the brief theory, I suggest you to read Linux manpage for more details.