Communicate with child process stdout/stdin - c

Question

I am trying to communicate with a process (that itself writes to stdin and stdout to interact in a terminal with a user) and read it's stdin and write to it's stdout in C.
Hence I try to substitute a shell user programmatically. A methapohrical example: Imagine I want to use VIM in C for some reason. Then I also need to write commands (stdout) and read stuff from the editor (stdin).
Initially I thought this might be a trivial task, but it seems like there's no standard approach. int system(const char *command); just executes a command and sets the commands stdin/stdout to the one of the calling process.
Because this leads nowhere, I looked at FILE *popen(const char *command, const char *type); but the manual pages state that:
Since a pipe is by definition unidirectional, the type argument may specify only reading or writing, not both; the resulting stream is correspondingly read-only or write-only.
and its implication:
The return value from popen() is a normal standard I/O stream in all respects save that it must be closed with pclose() rather than fclose(3). Writing to such a stream writes to the standard input
of the command; the command's standard output is the same as that of the process that called popen(), unless this is altered by the command itself. Conversely, reading from a "popened" stream reads
the command's standard output, and the command's standard input is the same as that of the process that called popen().
Hence it wouldn't be completely impossible to use popen(), but it appears to me very inelegant, because I would have to parse the stdout of the calling process (the code that called popen()) in order to parse data sent from the popened command (when using popen type 'w').
Conversely, when popen is called with type 'r', I would need to write to the calling's process stdin, in order to write data to the popened command. It's not even clear to me whether both these processes receive the same data in the stdin in this case...
I just need to control stdin and stdout of a program. I mean can't there be a function like:
stdin_of_process, stdout_of_process = real_popen("/path/to/bin", "rw")
// write some data to the process stdin
write("hello", stdin_of_process)
// read the response of the process
read(stdout_of_process)
So my first question: What is the best way to implement the upper functionality?
Currently I am trying the following approach to communicate with another process:
Set up two pipes with int pipe(int fildes[2]);. One pipe to read the stdout of the process, the other pipe to write to the stdin of the process.
Fork.
Execute the process that I want to communicate with in the forked child process using int execvp(const char *file, char *const argv[]);.
Communicate with the child using the two pipes in the original process.
That's easy said bot not so trivially implemented (At least for me). I oddly managed to do so in one case, but when I tried to understand what I am doing with a simpler example, I fail. Here is my current problem:
I have two programs. The first just writes a incremented number every 100 ms to it's stdout:
#include <unistd.h>
#include <time.h>
#include <stdint.h>
#include <stdio.h>
#include <string.h>
void sleepMs(uint32_t ms) {
struct timespec ts;
ts.tv_sec = 0 + (ms / 1000);
ts.tv_nsec = 1000 * 1000 * (ms % 1000);
nanosleep(&ts, NULL);
}
int main(int argc, char *argv[]) {
long int cnt = 0;
char buf[0x10] = {0};
while (1) {
sleepMs(100);
sprintf(buf, "%ld\n", ++cnt);
if (write(STDOUT_FILENO, buf, strlen(buf)) == -1)
perror("write");
}
}
Now the second program is supposed to read the stdout of the first program (Please keep in my mind that I eventually want to read AND write with a process, so a technical correct solution to use popen() for the upper use case might be right in this specific case, because I simplified my experiments to just capture the stdout of the bottom program). I expect from the bottom program that it reads whatever data the upper program writes to stdout. But it does not read anything. Where could be the reason? (second question).
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <stdint.h>
#include <time.h>
void sleepMs(uint32_t ms) {
struct timespec ts;
ts.tv_sec = 0 + (ms / 1000);
ts.tv_nsec = 1000 * 1000 * (ms % 1000);
nanosleep(&ts, NULL);
}
int main() {
int pipe_fds[2];
int n;
char buf[0x100] = {0};
pid_t pid;
pipe(pipe_fds);
char *cmd[] = {"/path/to/program/above", NULL};
if ((pid = fork()) == 0) { /* child */
dup2(pipe_fds[1], 1); // set stdout of the process to the write end of the pipe
execvp(cmd[0], cmd); // execute the program.
fflush(stdout);
perror(cmd[0]); // only reached in case of error
exit(0);
} else if (pid == -1) { /* failed */
perror("fork");
exit(1);
} else { /* parent */
while (1) {
sleepMs(500); // Wait a bit to let the child program run a little
printf("Trying to read\n");
if ((n = read(pipe_fds[0], buf, 0x100)) >= 0) { // Try to read stdout of the child process from the read end of the pipe
buf[n] = 0; /* terminate the string */
fprintf(stderr, "Got: %s", buf); // this should print "1 2 3 4 5 6 7 8 9 10 ..."
} else {
fprintf(stderr, "read failed\n");
perror("read");
}
}
}
}

Answer 1

Here is a (C++11-flavored) complete example:
//
// Example of communication with a subprocess via stdin/stdout
// Author: Konstantin Tretyakov
// License: MIT
//
#include <ext/stdio_filebuf.h> // NB: Specific to libstdc++
#include <sys/wait.h>
#include <unistd.h>
#include <iostream>
#include <memory>
#include <exception>
// Wrapping pipe in a class makes sure they are closed when we leave scope
class cpipe {
private:
int fd[2];
public:
const inline int read_fd() const { return fd[0]; }
const inline int write_fd() const { return fd[1]; }
cpipe() { if (pipe(fd)) throw std::runtime_error("Failed to create pipe"); }
void close() { ::close(fd[0]); ::close(fd[1]); }
~cpipe() { close(); }
};
//
// Usage:
// spawn s(argv)
// s.stdin << ...
// s.stdout >> ...
// s.send_eol()
// s.wait()
//
class spawn {
private:
cpipe write_pipe;
cpipe read_pipe;
public:
int child_pid = -1;
std::unique_ptr<__gnu_cxx::stdio_filebuf<char> > write_buf = NULL;
std::unique_ptr<__gnu_cxx::stdio_filebuf<char> > read_buf = NULL;
std::ostream stdin;
std::istream stdout;
spawn(const char* const argv[], bool with_path = false, const char* const envp[] = 0): stdin(NULL), stdout(NULL) {
child_pid = fork();
if (child_pid == -1) throw std::runtime_error("Failed to start child process");
if (child_pid == 0) { // In child process
dup2(write_pipe.read_fd(), STDIN_FILENO);
dup2(read_pipe.write_fd(), STDOUT_FILENO);
write_pipe.close(); read_pipe.close();
int result;
if (with_path) {
if (envp != 0) result = execvpe(argv[0], const_cast<char* const*>(argv), const_cast<char* const*>(envp));
else result = execvp(argv[0], const_cast<char* const*>(argv));
}
else {
if (envp != 0) result = execve(argv[0], const_cast<char* const*>(argv), const_cast<char* const*>(envp));
else result = execv(argv[0], const_cast<char* const*>(argv));
}
if (result == -1) {
// Note: no point writing to stdout here, it has been redirected
std::cerr << "Error: Failed to launch program" << std::endl;
exit(1);
}
}
else {
close(write_pipe.read_fd());
close(read_pipe.write_fd());
write_buf = std::unique_ptr<__gnu_cxx::stdio_filebuf<char> >(new __gnu_cxx::stdio_filebuf<char>(write_pipe.write_fd(), std::ios::out));
read_buf = std::unique_ptr<__gnu_cxx::stdio_filebuf<char> >(new __gnu_cxx::stdio_filebuf<char>(read_pipe.read_fd(), std::ios::in));
stdin.rdbuf(write_buf.get());
stdout.rdbuf(read_buf.get());
}
}
void send_eof() { write_buf->close(); }
int wait() {
int status;
waitpid(child_pid, &status, 0);
return status;
}
};
// ---------------- Usage example -------------------- //
#include <string>
using std::string;
using std::getline;
using std::cout;
using std::endl;
int main() {
const char* const argv[] = {"/bin/cat", (const char*)0};
spawn cat(argv);
cat.stdin << "Hello" << std::endl;
string s;
getline(cat.stdout, s);
cout << "Read from program: '" << s << "'" << endl;
cat.send_eof();
cout << "Waiting to terminate..." << endl;
cout << "Status: " << cat.wait() << endl;
return 0;
}
For many practical purposes, however, the Expect library could probably be a good choice (check out the code in the example subdirectory of its source distribution).

Answer 2

You've got the right idea, and I don't have time to analyze all of your code to point out the specific problem, but I do want to point out a few things that you may have overlooked on how programs and terminals work.
The idea of a terminal as a "file" is naivé. Programs like vi use a library (ncurses) to send special control characters (and change terminal device driver settings). For example, vi puts the terminal device driver itself into a mode where it can read a character at a time, among other things.
It is very non-trivial to "control" a program like vi this way.
On your simplified experiment...
Your buffer is one byte too small. Also, be aware IO is sometimes line buffered. So, you might try making sure the newline is getting transferred (use printf instead of sprintf/strlen/write...you hooked the stdout up to your pipe already), otherwise you might not see data until a newline is hit. I don't remember pipe being line buffered, but it is worth a shot.