How can I capture another process's output using pure C? Can you provide sample code?
EDIT: let's assume Linux. I would be interested in "pretty portable" code. All I want to do is to execute a command, capture it's output and process it in some way.
There are several options, but it does somewhat depend on your platform. That said popen should work in most places, e.g.
#include <stdio.h>
FILE *stream;
stream = popen("acommand", "r");
/* use fread, fgets, etc. on stream */
pclose(stream);
Note that this has a very specific use, it creates the process by running the command acommand and attaches its standard out in a such as way as to make it accessible from your program through the stream FILE*.
If you need to connect to an existing process, or need to do richer operations, you may need to look into other facilities. Unix has various mechanisms for hooking up a processes stdout etc.
Under windows you can use the CreateProcess API to create a new process and hook up its standard output handle to what you want. Windows also supports popen.
There's no plain C way to do this that I know of though, so it's always going somewhat dependent on platform specific APis.
Based on your edits popen seems ideal, it is "pretty portable", I don't think there's a unix like OS without it, indeed it is part of the Single Unix Specification, and POSIX, and it lets you do exactly what you want, execute a process, grab its output and process it.
If you can use system pipes, simply pipe the other process's output to your C program, and in your C program, just read the standard input.
otherprocess | your_c_program
Which OS are you using? On *nix type OS if you are process is outputting to STDOUT or STDERR you can obviously use pipes
Related
I wrote a C program and in the program there are many printf() which output log information to stdout. Now I want to use multiple processes to run the program simultaneously with different arguments. And I want to redirect the output from stdout to a log file using >.
But multiple processes are running at the same time, their log information output overlap with each other, which can be confusing for future analysis.
one solution is: considering that different processes will exit at different time,modify the C program, so each log information is temporarily written into a temporal file. When the C program is about to exit. Read from the temporal file and write the content to stdout, this requires a lot of modification.
My idea is: I hope in the C program, all the printf() output can be buffered, the outputs put into stdout/redirection only when the process exits.
is it possible or not?
thanks!
This is not really possible, unless you are sure that the output is reasonably bounded (e.g. the total output is less than a few megabytes), otherwise use a logging mechanism which send to some central logger (like syslog).
On Linux and most Posix systems, the simplest way to do logging would be to use syslog(3) which is designed for logging (and is able to deal with different processes). I think this is the preferable approach.
With GNU libc, you could consider using open_memstream(3) -to write to memory, and here you need to be sure the total output is bounded- and use atexit(3) to have the memory stream written at the exit of the program into some file; you probably want to use some locking mechanism like flock(2) etc...
As commented by J.Holetzeck the simplest way is to redirect output into different files (perhaps using freopen(3), or simply in the invoking shell), and later merge these files.
I'm guessing you use Linux, or some Posix system. For Windows, I have no idea.
My program is controlling an external application on Linux, passing in input commands via a pipe to the external applications stdin, and reading output result via a pipe from the external applications stdout.
The problem is that writes to pipes are buffered by block, and not by line, and therefore delays occur before my app receives data output by the external application. The external application cannot be altered to add explicit fflush() calls.
When I set the external application to /bin/cat -n (it echoes back the input, with line numbers added), it works correctly, it seems, cat flushes after each line. The only way to force the external application to flush, is sending exit command to it; as it receives the command, it flushes, and all the answers appears on the stdout, just before exiting.
I'm pretty sure, that Unix pipes are appropiate solution for that kind of interprocess communication (pseudo server-client), but maybe I'm wrong.
(I've just copied some text from a similar question: Force another program's standard output to be unbuffered using Python)
Don't use a pipe. Use a pty instead. Pty's (pseudo-ttys) have the benefit of being line buffered if you want it, which provides you with simple framing for your data stream.
Using a PTY may be an overkill for the problem at hand (although it will work).
If the "target application" (the Delphi command-line utility) is dynamically linked, a possibly much simpler solution is to interpose (via LD_PRELOAD) a small library into the application. That library simply needs to implement isatty, and answer true (return 1) regardless of whether the output is going to a pipe or a terminal. You may wish to do that for all file descriptors, or just for STDOUT_FILENO.
Most UNIX implementations will call isatty to decide whether to do full buffering or line buffering for a given file descriptor.
Hmm, glibc doesn't. It calls __fxstat, and then only calls isatty if the status indicates that fd is going to a character device. So you'll need to interpose both __fxstat and isatty. More on library interposition here.
By default standard input and standard output are fully buffered unless they are connected to an interactive device in which cases they are line buffered [1]. Pipes are non-interactive devices. PTYs are interactive devices. "Fully buffered" means "use a chunk of memory of a certain size".
I'm sure you want line buffering. Therefore using a master/slave PTY instead of pipes should bring the controlled application into the right buffering mode automatically.
[1] see "stdin(3)" and "setbuf(3)" for details.
Why calling fflush suitably (on the write side) don't work for you?
You can use poll (or other syscalls like ppoll, pselect, select) to check availability of input on the read side.
If the external application is using <stdio.h> without calling fflush appropriately (perhaps by setbuf making it happen on newlines ....), data would remain inside its FILE* buffer without even being sent (with a write syscall) to the pipe!
An application can detect if its output is a terminal with e.g. isatty. But it should ensure that flushing happens...
As Michael Dillon suggested, using pty-s is probably the best. But it is hard (I forgot the gory details).
Could you please tell me the best way to do it? I can use popen, but it is nesessary to create a large buffer for arguments every time I need to launch my application. I can use fork + execv, but then the program writes to stdout and I cant read the output ( to display it in the text field ) Is there any other solution?
Could you please tell me the best way to do it? I can use popen, but it is nesessary to create a large buffer for arguments every time I need to launch my application.
popen() is one good standard way if you only need one way communication with the child application, like writing to its stdin or reading from stdout, but not both.
When using C one needs to be comfortable with strings. It helps a lot to use a string library for C to ease string operations, such as string concatenation in your case, because the standard C library provides only basic low-level functions for that.
I can use fork + execv, but then the program writes to stdout and I cant read the output ( to display it in the text field )
popen() gives you a FILE* pointer to the child program's stdout from which you can read its output using the standard C I/O function fread() or fscanf(). Again, the standard C library has this functionality and it pays to familiarize yourself with it.
Is there any other solution?
You can make the child program write to a file and then read that file, but in any case you need to be able to construct the command line string and read the file.
In C, how should I execute external program and get its results as if it was ran in the console?
if there is an executable called dummy, and it displays 4 digit number in command prompt when executed, I want to know how to run that executable and get the 4 digit number that it had generated. In C.
popen() handles this quite nicely. For instance if you want to call something and read the results line by line:
char buffer[140];
FILE *in;
extern FILE *popen();
if(! (in = popen(somecommand, "r"""))){
exit(1);
}
while(fgets(buff, sizeof(buff), in) != NULL){
//buff is now the output of your command, line by line, do with it what you will
}
pclose(in);
This has worked for me before, hopefully it's helpful. Make sure to include stdio in order to use this.
You can use popen() on UNIX.
This is not actually something ISO C can do on its own (by that I mean the standard itself doesn't provide this capability) - possibly the most portable solution is to simply run the program, redirecting its standard output to a file, like:
system ("myprog >myprog.out");
then use the standard ISO C fopen/fread/fclose to read that output into a variable.
This is not necessarily the best solution since that may depend on the underlying environment (and even the ability to redirect output is platform-specific) but I thought I'd add it for completeness.
There is popen() on unix as mentioned before, which gives you a FILE* to read from.
Alternatively on unix, you can use a combination of pipe(), fork(), exec(), select(), and read(), and wait() to accomplish the task in a more generalized/flexible way.
The popen library call invokes fork and pipe under the hood to do its work. Using it, you're limited to simply reading whatever the process dumps to stdout (which you could use the underlying shell to redirect). Using the lower-level functions you can do pretty much whatever you want, including reading stderr and writing stdin.
On windows, see calls like CreatePipe() and CreateProcess(), with the IO members of STARTUPINFO set to your pipes. You can get a file descriptor to do read()'s using _open_ofshandle() with the process handle. Depending on the app, you may need to read multi-threaded, or it may be okay to block.
I'm writing a utility for running programs, and I need to capture unbuffered stdout and stderr from the programs. I need to:
Capture stdout and stderr to separate files.
Output needs to not be buffered (or be line buffered).
Without modifying the source of the program being run.
The problem is, when piping output to a file, the stdout stream becomes block buffered rather than line buffered. If the program crashes, the output never gets flushed, and is blank. So I need to capture stdout without buffering (or with line buffering).
I think this can be done with pty's but I'm having difficulty finding any examples that do exactly what I want (most ignore stderr). In fact, I'm not sure I've found any pty examples in C at all; most use a higher-level interface like Python's pty and subprocess modules.
Can anyone help (with code snippets or links)? Any help would be appreciated.
EDIT: I think I've solved it. The following two links were pretty helpful.
http://publib.boulder.ibm.com/infocenter/zos/v1r10/index.jsp?topic=/com.ibm.zos.r10.bpxbd00/posixopenpt.htm
http://www.gidforums.com/t-3369.html
My code is available as a repository:
https://bitbucket.org/elliottslaughter/pty
see man 7 pty
In particular:
Unix 98 pseudo-terminals
An unused Unix 98 pseudo-terminal master is opened by calling
posix_openpt(3). (This function opens the master clone device,
/dev/ptmx; see pts(4).) After performing any program-specific initializations,
changing the ownership and permissions of the slave device
using grantpt(3), and unlocking the slave using unlockpt(3)), the corresponding
slave device can be opened by passing the name returned by
ptsname(3) in a call to open(2).
And now that you know the names of the library functions such a code will need to call, you can do two useful things:
Look up their man pages
Google for example code. Since you know what keywords to use with the search engine I suspect you will have much more luck hunting down examples.