I'm working on a fairly simple application in C. The end goal is to pipe the output from one process to in input of another in a *nix environment (yes, I am aware of the pipe() command and dup/dup2 but I'm trying to find away around using those commands). I was wondering if there is any way to connect the streams rather than using file descriptors (The systems aren't guaranteed to be POSIX compliant).
So basically I want to do something like this (pseudo-code)
pid = fork()
if pid == 0
// assign this process's stdin to the parents stdout.
stdin = parent.stdout;
exec() // launch new process that receives the parents stdout as stdin
// child stuff....
else
// parent stuff....
I know that it probably won't be as simple as just doing an assignment as above, but is there any way to do this using only streams? I tried looking around, but couldn't find anything..
Thanks!
sorry if I'm missing the point here but the whole philosophy of *nix is one program, one job. If you need a program to dump the contents of a program to the screen then you have the cat command. If the files too big and you need page breaks you pipe the output of cat to the more command:
cat myfile.txt | more
If you need to pipe between two terminal applications then you're meant to use the command line to do so:
myprog1 | myprog2
Obviously that's the philosophical approach, so if that doesn't help then can you clarify what you're trying to pipe and why you're trying to do it in process ?
Related
I understand how IPC works and how when you make a pipe before fork() the child/children can inherit the pipe for communication. However, I cant wrap my head around how I would do this for two unrelated processes? (i.e processes that are not the parent or children of each other?).
Im asking because Im trying to get one of my .c files to communicate with another .c file via pipes, but I dont know how that works explicitly in xv6. Any help/ideas would be greatly appreciated!!
There are several ways to accomplish this, for something simple, try checking out popen(). One program can spawn an unrelated one and then can then read and write to each other.
https://man7.org/linux/man-pages/man3/popen.3.html
For other solutions you should look into FIFO files or even sockets as a means of IPC.
Pipes are not able to communicate between 2 processes unless those two processes share a common ancestor, regardless of the Linux distro you use. If you are trying to allow 2 unrelated files to communicate, one of the exec() commands must be used in the child or parent process; this page may have what you're looking for.
What pipe does is to connect the standard output of one program to the standard input of another program.
Consider the cat and wc program in xv6. cat displays the content of a file or files. Let's say we have a text file test.txt with Hello World in it. cat can display its content like this:
$ cat test.txt
Hello World
wc calculates the number of lines, words, and characters in a file, files, OR stdin. So it can be used either like this(read from a file):
$ wc test.txt
1 2 12 test.txt
or this(read from stdin):
$ wc
Hello World
<Press Ctrl-D>
1 2 12
We can use pipe to connect these two commands:
$ cat test.txt | wc
1 2 12
So here cat reads the file and prints its content into stdout, but it doesn't get displayed onto the screen. Instead it gets feeded into the stdin of wc and wc counts the lines, words, and characters in it. Notice that this communication is unidirectional. It only goes from left to right, no backwards.
You can refer to the source code of these two commands to write your program. Basically all you have to do is to make sure the previous command outputs into stdout and the subsequent command reads from stdin.
I've got a cmd line app in C under Linux that has to run another process, the problem is that the child process prints a lot in a comand line and the whole app gets messy.
Is it possible to disallow child process to print anything in cmd line from parent process? It would be very helpful to for example being able to define a command that allows or disallows printing by a child process.
There's the time-honoured tradition of just redirecting the output to the bit bucket(a), along the lines of:
system("runChild >/dev/null 2>&1");
Or, if you're doing it via fork/exec, simply redirect the file handles using dup2 between the fork and exec.
It won't stop a determined child from outputting to your standard output but it will have to be very tricky to do that.
(a) I'm not usually a big fan of that, just in case something goes wrong. I'd prefer to redirect it to a real file which can be examined later if need be (and deleted eventually if not).
Read Advanced Linux Programming then syscalls(2).
On recent Linux, every executable is in ELF format (except init or systemd; play with pstree(1) or proc(5)) is running in a process started by fork(2) (or clone(2)...) and execve(2).
You might use cleverly dup2(2) with open(2) to redirect STDOUT_FILENO to /dev/null (see null(4), stdout(3), fileno(3))
I've got a cmd line app in C under Linux that has to run another process, the problem is that the child process prints a lot in a comand line
I would instead provide a way to selectively redirect the child process' output. You could use program arguments or environment variables (see getenv(3) and/or environ(7)) to provide such an option to your user.
An example of such a command program starting and redirecting subprocesses and redirecting them is your GCC compiler (see gcc(1); it runs cc1 and as(1) and ld(1)...). Consider downloading and studying its source code.
Study also -for inspiration- the source code of some shell (e.g. sash), or write your own one.
I'm on a project of creating a terminal shell from C (with Bash as a reference) and I enventually had to deal with pipes.
The way I made them worked with basic commands like ls | rev | wc -l
However my program enter a never-ending loop when I try to pipe commands that never ends like this one: base64 /dev/urandom | head -c 1000; Bash does not.
The way I created my pipeline make my program wait for base64 to end before calling head.
I don't understand when and how I am supposed to wait & execute commands anymore.
How can I reproduce Bash's behavior with such piped commands in C ? Did I make a simple mistake or should I totally rethink my system ?
Here is in pseudo-code how I do my command execution. It lacks details & security like closing the pipes but the whole idea is present.
while (command)
{
pipe(fd);
if (!fork())
{
dup2();
execve(command);
}
wait();
command = command->next;
}
The short answer is:
Get rid of the wait() for starters.
But there's more. Not that I'm an expert, but from observing bash behavior, I have learned that all components in a pipe are executed simultaneously in parallel
AFAIK, bash ( or perhaps the kernel implementation of the fifo ) will generate signals to a process when another process on the other side of the pipe has closed the pipe. 'base64 /dev/urandom' terminates because it received a signal when the head closed it's stdin.
As you have seen, base64 /dev/urandom never ends by itself.
Going forward, kick off all of the processes, creating fifos that have their stdout and stdin connected. I hope this gets you going in the right direction. There's a lot of discussion of how to use fifos out there so hopefully this is a nudge in the right direction.
It's possible to redirect everything that is written in the terminal to a process?
For example, after I started the process, if I write "command" in the terminal, this should be redirected to a pipe from my process or something like this.
Yes, it should be practical to redirect all terminal output from your program (and all of its child processes) after your program has started. Unix programs usually write to the terminal by writing to standard output (stdout). Standard output is always on the file descriptor number 1 (the C constant is STDOUT_FILENO), for all processes. You can use the dup2() system call to replace any file descriptor number with another file descriptor.
So you can e.g. create a pipe using int fds[2]; pipe(fds);. Then fds[1] will be a file descriptor number that you can use to write to the pipe. If you do dup2(fds[1], STDOUT_FILENO); then standard output will also write to the pipe. (You can close(fds[1]); afterwards since you probably don't need it, now that you can use stdout instead.)
You can also open a file for writing with fd = open("filename", O_WRONLY); and then dup2(fd, STDOUT_FILENO); so everything written to stdout goes into your file.
Note that you need to redirect stdout at the very beginning of your program before doing anything that might write to stdout.
The above trick will make standard output go to your pipe instead of the terminal. If you want the output to go to the terminal, and also get a copy of the output in a pipe of file, that's more difficult but can also be done. You need to create an internal pipe, then dup2(that_pipe, STDOUT_FILENO); so stdout writes to that pipe. Then you need to read from that pipe (probably using poll() then read()) and write everything you got to both 1) the terminal and 2) to another pipe or file that is going outside your program. So you need two pipes if you want to copy output.
The tee command does this (copy stdout to files) from the shell.
This dup2() approach is not bulletproof because a Unix terminal (even when using a GUI terminal emulator instead of a hardware console) is a device in /dev. You can type tty in a shell or use ttyname(STDOUT_FILENO) in C to see which file in /dev corresponds to the terminal that stdout is writing to. In principle, any program (under the same user account) could open the terminal device using that filename and write to it without asking for permission from any other program. You can easily try this from the shell using the write program:
echo hello world | write $(whoami) /dev/ttys123
where /dev/ttys123 is whatever you got by typing tty in some other terminal window (the name looks a bit different on different operating systems, e.g. Linux and MacOS). You should see hello world appear in that other window.
From a child process, no. You must set this up in the parent preocess, and have it propagate downwards to children (barring some kind of crazy hack).
From the shell, you can redirect.
exec >file
This will redirect standard output to file, and it will apply to all future commands run in the shell. You can make this into a function, if you like.
My program launches a helper program using fork() / execvp() and I'd like to show the helper program's output in my program's GUI. The helper's output should be shown line by line in a listview widget embedded in my program's GUI. Of course, I could just redirect the output to a file, wait until the helper has finished, and then read the whole file and show it. But that's not an optimal solution. Ideally, I'd like to show the helper's output as it is sent to stdout, i.e. line by line, while the helper is still working.
What is the suggested way of doing this?
From the top of my head, what comes to mind is the following solution but I'm not sure whether it will work at all because one process will write to the file while the other is trying to read from it:
1) Start the helper like this using execvp() after a fork():
./helper > tmpfile
2) After that, my program tries to open "tmpfile" using open() and then uses select() to wait until there's something to read from that file. Once my program has obtained a line of output, it sends it to my GUI's listview widget.
Is this how it should be done or am I totally on the wrong track here?
Thanks!
You should open a pipe and monitor the progress of the child process using select. You can also use popen if you only need a one way communication, in that case you will get the file descriptor by a call to fileno on the returned FILE*.
See:
pipe
popen
select
fileno