This seems like a bit of a computing systems 101 question, but I'm stumped.
I am integrating existing code from C/C++ project A into my own project B. Both A and B will be linked into a single executable, threaded process. Project A's code makes extensive use of printf for output. This is fine, but I want also to capture that output into my own buffers. Is there a way I can read from stdout once the printf calls have written to it? I cannot fork the process or pipe. And my efforts to poll() stdout, or to dup() it, have not succeeded (I may be doing something wrong here).
You can use freopen to change the descriptor.
#include<stdio.h>
main(int argc, char** argv) {
FILE *fp = freopen("output.txt", "w", stdout);
printf("Hello\n");
fclose(fp);
}
If you run that you'll see the printf output in output.txt and nothing will go to your screen.
You can now open the file to read the data or you could even mmap it into your memory space and process it that way.
Before you printf(), you could close fd 1, and dup2() a pipe that you've created into fd 1.
Not to mention: there is now a handy U-Streams C source code library that makes redirecting stdout and stderr quite trivial. And you can even redirect them very easily to multiple destinations. And, you can create your own streams besides that can be used in exactly the same way stdout and stderr behave.
Look for the U-Streams C Library... handy indeed.
Once it's gone out, it's gone. If you want to compile it all into a single executable, you'll have to go through the code for A with a search and replace and change all those printf calls into ones to your own stream, where you can copy them and then pass them on to stdout.
Related
I want to reopen the stdin and stdout (and perhaps stderr while I'm at it) filehandles, so that future calls to printf() or putchar() or puts() will go to a file, and future calls to getc() and such will come from a file.
1) I don't want to permanently lose standard input/output/error. I may want to reuse them later in the program.
2) I don't want to open new filehandles because these filehandles would have to be either passed around a lot or global (shudder).
3) I don't want to use any open() or fork() or other system-dependent functions if I can't help it.
So basically, does it work to do this:
stdin = fopen("newin", "r");
And, if it does, how can I get the original value of stdin back? Do I have to store it in a FILE * and just get it back later?
Why use freopen()? The C89 specification has the answer in one of the endnotes for the section on <stdio.h>:
116. The primary use of the freopen function is to change the file associated with a standard
text stream (stderr,
stdin, or stdout), as those identifiers need not be
modifiable lvalues to which the value
returned by the fopen function
may be assigned.
freopen is commonly misused, e.g. stdin = freopen("newin", "r", stdin);. This is no more portable than fclose(stdin); stdin = fopen("newin", "r");. Both expressions attempt to assign to stdin, which is not guaranteed to be assignable.
The right way to use freopen is to omit the assignment: freopen("newin", "r", stdin);
I think you're looking for something like freopen()
This is a modified version of Tim Post's method; I used /dev/tty instead of /dev/stdout. I don't know why it doesn't work with stdout
(which is a link to /proc/self/fd/1):
freopen("log.txt","w",stdout);
...
...
freopen("/dev/tty","w",stdout);
By using /dev/tty the output is redirected to the terminal from where the app was launched.
Hope this info is useful.
freopen("/my/newstdin", "r", stdin);
freopen("/my/newstdout", "w", stdout);
freopen("/my/newstderr", "w", stderr);
... do your stuff
freopen("/dev/stdin", "r", stdin);
...
...
This peaks the needle on my round-peg-square-hole-o-meter, what are you trying to accomplish?
Edit:
Remember that stdin, stdout and stderr are file descriptors 0, 1 and 2 for every newly created process. freopen() should keep the same fd's, just assign new streams to them.
So, a good way to ensure that this is actually doing what you want it to do would be:
printf("Stdout is descriptor %d\n", fileno(stdout));
freopen("/tmp/newstdout", "w", stdout);
printf("Stdout is now /tmp/newstdout and hopefully still fd %d\n",
fileno(stdout));
freopen("/dev/stdout", "w", stdout);
printf("Now we put it back, hopefully its still fd %d\n",
fileno(stdout));
I believe this is the expected behavior of freopen(), as you can see, you're still only using three file descriptors (and associated streams).
This would override any shell redirection, as there would be nothing for the shell to redirect. However, its probably going to break pipes. You might want to be sure to set up a handler for SIGPIPE, in case your program finds itself on the blocking end of a pipe (not FIFO, pipe).
So, ./your_program --stdout /tmp/stdout.txt --stderr /tmp/stderr.txt should be easily accomplished with freopen() and keeping the same actual file descriptors. What I don't understand is why you'd need to put them back once changing them? Surely, if someone passed either option, they would want it to persist until the program terminated?
The os function dup2() should provide what you need (if not references to exactly what you need).
More specifically, you can dup2() the stdin file descriptor to another file descriptor, do other stuff with stdin, and then copy it back when you want.
The dup() function duplicates an open file descriptor. Specifically, it provides an alternate interface to the service provided by the fcntl() function using the F_DUPFD constant command value, with 0 for its third argument. The duplicated file descriptor shares any locks with the original.
On success, dup() returns a new file descriptor that has the following in common with the original:
Same open file (or pipe)
Same file pointer (both file descriptors share one file pointer)
Same access mode (read, write, or read/write)
freopen solves the easy part. Keeping old stdin around is not hard if you haven't read anything and if you're willing to use POSIX system calls like dup or dup2. If you're started to read from it, all bets are off.
Maybe you can tell us the context in which this problem occurs?
I'd encourage you to stick to situations where you're willing to abandon old stdin and stdout and can therefore use freopen.
And in the meantime, there's a C source code library that will do all this for you, redirecting stdout or stderr. But the cool part is that it lets you assign as many callback functions as you want to the intercepted streams, allowing you then to very easily send a single message to multiple destinations, a DB, a text file, etc.
On top of that, it makes it trivial to create new streams that look and behave the same as stdout and stderr, where you can redirect these new streams to multiple locations as well.
look for U-Streams C library on *oogle.
This is the most readily available, handy and useful way to do
freopen("dir","r",stdin);
Hello I have seen some solutions on the internet, all of them are basically creating a file, however I want to store them in an array of char. Speed is really important for me and I don't want to spend any time for working on hard drive. So popen() is not a real solution for me.
Here is a working code snippet:
char bash_cmd[256] = "ls -l";
char buffer[1000];
FILE *pipe;
int len;
pipe = popen(bash_cmd, "r");
if (NULL == pipe) {
perror("pipe");
exit(1);
}
fgets(buffer, sizeof(buffer), pipe);
len = strlen(buffer);
buffer[len-1] = '\0';
pclose(pipe);
If you would read the manpage of popen, you would notice the following:
The popen() function opens a process by creating a pipe, forking,
and invoking the shell. [...] The return value from popen() is a
normal standard I/O stream in all respects save that it must be
closed with pclose() rather than fclose(3). [...] reading from a
"popened" stream reads the command's standard output, and the
command's standard input is the same as that of the process that
called popen().
(emphasis mine)
As you can see, a call to popen results in the stdout of the command being piped into your program through an I/O stream, which has nothing to do with disk I/O at all, but rather with interprocess communication managed by the operating system.
(As a sidenote: It's generally a good idea to rely on the basic functionality of the operating system, within reason, to solve common problems. And since popen is part of POSIX.1-2001 you can rely on it to be available on all standards compliant operarting systems, even windows)
EDIT: if you want to know more, read this: http://linux.die.net/man/3/popen
Never forget Knuth's saying that "premature optimization is the root of all evil". Don't worry about performance until it matters, and then measure before doing anything. Except for very rare situations, the value of your time is much higher than the cost of the program runs.
Jon Bentley's "Writing efficient programs" (sadly out of print, in his "Programming Pearls" one chapter is a summary) is a detailed discussion on how to make programs run faster (if it is worthwhile); and only as the very last measure, to squeeze out the last possible 2% of performance (after cutting run time down by half) it recommends using changes like you propose. The cited book includes some very entertaining war stories of "performance optimizations" that were a complete waste (optimize code that isn't ever used, oprimize the code run while the operating system twiddles its thumbs, ...).
If speed is important to you, you can write your own version of popen.
It may make sense, since popen()
- creates a pipe
- forks
- executes the shell (very expensive!)
- the shell than creates a pipe, forks, executes your program
Your customized version could reduce the procedure to:
- creates a pipe
- forks
- executes your program
You could even extend popen to control the commands STDOUT, STDERR and STDIN seperately.
I wrote such a routine, see https://github.com/rockdaboot/mget/blob/master/libmget/pipe.c
It is GPL'ed.
You call mget_popen3() with FILE pointers or mget_fd_popen3() with file descriptors.
At least, it should give you an idea on how to do it.
Do you mind having more than one C programs?If you don't ,you can make use of the command line arguments. In the fist C program you can do the following
system("YourCommand | SecondProgram");
The SecondProgram will be the "executable" of the second C program you will be writing. In the second C program you can receive the output of the command YourCommand as a command line argument in the SecondProgram. For that purpose you may begin the main() of second C program as below
main(int argc,char *argv[])
The array argv will have the output of the YourCommand and argc will contain the number of elements in the array argv.
I have a program which prints (by printf) to the stdout some data and also calls to function *foo*
which also prints to the stdout some data [the way (implementation) of how printing is done from foo is unknown and I can`t see the code of foo].
I have to redirect everything from stdout to buffer or file. I tried to do it in several ways
freopen(file.txt, stdout) - only my code prints are written to the file.txt. What was printed from foo is lost.
setbuf(buffer, stdout) - only my code prints are written to the buffer. What was printed from foo is appears in the stdout.(It appears on the screen)
What can explain this behavior? How can the problem be solved?
Note:This code has to work in cross-OS( lunux/wind && mac OS).I use gcc in order compile the code and I have cygwin
It's likely that foo isn't using stdio for printing and directly calling the OS for this.
I don't know about win32, but on POSIX you could use dup2 to take care of it.
/* Before the function foo is called, make `STDOUT_FILENO` refer to `fd` */
int fd;
fd = open(...);
dup2(fd, STDOUT_FILENO);
EDIT
Much to my surprise, win32 has _dup2 but it does something else.
How do you know that foo() is printing to stdout? Have you tried redirecting standard output to a file at the shell and seeing whether the output from foo() still appears on the screen?
If the file redirection sends foo()'s output to the file, then you may have to rejig the file descriptor level, as in cnicutar's answer.
If the file redirection does not send foo()'s output to the file, then it may be writing to stderr or it may be opening and using /dev/tty or something similar. You can test for stderr by redirecting it separately from stdout:
your_program >/tmp/stdout.me 2>/tmp/stderr.me
If it is opening /dev/tty, the output will still appear on your screen.
Which platform are you on? If you can track system calls (strace on Linux, truss on Solaris, ...), then you may be able to see in that what the foo() function is doing. You can help things by writing a message before and after calling the function, and ensuring you flush the output:
printf("About to call foo()\n");
fflush(0);
foo();
printf("Returned from foo()\n");
fflush(0);
The printf/fflush calls will be visible in the trace output, so what appears between is done by foo().
What can explain this behavior?
I have seen this sort of behavior when the code you are calling into uses a different C library than yours. On Windows I used to see this sort of thing when one DLL is compiled with GCC and another with Visual C++. The implementation of stdio for these is apparently different enough such that this can be problematic.
Another is that the code you are calling is not using stdio. If you are on Unix you can use dup2 to get around this, eg. dup2(my_file_descriptor, 1). On many implementations if you have a FILE* you can say dup2(fileno(f), 1). This may not be portable.
In C, how should I execute external program and get its results as if it was ran in the console?
if there is an executable called dummy, and it displays 4 digit number in command prompt when executed, I want to know how to run that executable and get the 4 digit number that it had generated. In C.
popen() handles this quite nicely. For instance if you want to call something and read the results line by line:
char buffer[140];
FILE *in;
extern FILE *popen();
if(! (in = popen(somecommand, "r"""))){
exit(1);
}
while(fgets(buff, sizeof(buff), in) != NULL){
//buff is now the output of your command, line by line, do with it what you will
}
pclose(in);
This has worked for me before, hopefully it's helpful. Make sure to include stdio in order to use this.
You can use popen() on UNIX.
This is not actually something ISO C can do on its own (by that I mean the standard itself doesn't provide this capability) - possibly the most portable solution is to simply run the program, redirecting its standard output to a file, like:
system ("myprog >myprog.out");
then use the standard ISO C fopen/fread/fclose to read that output into a variable.
This is not necessarily the best solution since that may depend on the underlying environment (and even the ability to redirect output is platform-specific) but I thought I'd add it for completeness.
There is popen() on unix as mentioned before, which gives you a FILE* to read from.
Alternatively on unix, you can use a combination of pipe(), fork(), exec(), select(), and read(), and wait() to accomplish the task in a more generalized/flexible way.
The popen library call invokes fork and pipe under the hood to do its work. Using it, you're limited to simply reading whatever the process dumps to stdout (which you could use the underlying shell to redirect). Using the lower-level functions you can do pretty much whatever you want, including reading stderr and writing stdin.
On windows, see calls like CreatePipe() and CreateProcess(), with the IO members of STARTUPINFO set to your pipes. You can get a file descriptor to do read()'s using _open_ofshandle() with the process handle. Depending on the app, you may need to read multi-threaded, or it may be okay to block.
I want to reopen the stdin and stdout (and perhaps stderr while I'm at it) filehandles, so that future calls to printf() or putchar() or puts() will go to a file, and future calls to getc() and such will come from a file.
1) I don't want to permanently lose standard input/output/error. I may want to reuse them later in the program.
2) I don't want to open new filehandles because these filehandles would have to be either passed around a lot or global (shudder).
3) I don't want to use any open() or fork() or other system-dependent functions if I can't help it.
So basically, does it work to do this:
stdin = fopen("newin", "r");
And, if it does, how can I get the original value of stdin back? Do I have to store it in a FILE * and just get it back later?
Why use freopen()? The C89 specification has the answer in one of the endnotes for the section on <stdio.h>:
116. The primary use of the freopen function is to change the file associated with a standard
text stream (stderr,
stdin, or stdout), as those identifiers need not be
modifiable lvalues to which the value
returned by the fopen function
may be assigned.
freopen is commonly misused, e.g. stdin = freopen("newin", "r", stdin);. This is no more portable than fclose(stdin); stdin = fopen("newin", "r");. Both expressions attempt to assign to stdin, which is not guaranteed to be assignable.
The right way to use freopen is to omit the assignment: freopen("newin", "r", stdin);
I think you're looking for something like freopen()
This is a modified version of Tim Post's method; I used /dev/tty instead of /dev/stdout. I don't know why it doesn't work with stdout
(which is a link to /proc/self/fd/1):
freopen("log.txt","w",stdout);
...
...
freopen("/dev/tty","w",stdout);
By using /dev/tty the output is redirected to the terminal from where the app was launched.
Hope this info is useful.
freopen("/my/newstdin", "r", stdin);
freopen("/my/newstdout", "w", stdout);
freopen("/my/newstderr", "w", stderr);
... do your stuff
freopen("/dev/stdin", "r", stdin);
...
...
This peaks the needle on my round-peg-square-hole-o-meter, what are you trying to accomplish?
Edit:
Remember that stdin, stdout and stderr are file descriptors 0, 1 and 2 for every newly created process. freopen() should keep the same fd's, just assign new streams to them.
So, a good way to ensure that this is actually doing what you want it to do would be:
printf("Stdout is descriptor %d\n", fileno(stdout));
freopen("/tmp/newstdout", "w", stdout);
printf("Stdout is now /tmp/newstdout and hopefully still fd %d\n",
fileno(stdout));
freopen("/dev/stdout", "w", stdout);
printf("Now we put it back, hopefully its still fd %d\n",
fileno(stdout));
I believe this is the expected behavior of freopen(), as you can see, you're still only using three file descriptors (and associated streams).
This would override any shell redirection, as there would be nothing for the shell to redirect. However, its probably going to break pipes. You might want to be sure to set up a handler for SIGPIPE, in case your program finds itself on the blocking end of a pipe (not FIFO, pipe).
So, ./your_program --stdout /tmp/stdout.txt --stderr /tmp/stderr.txt should be easily accomplished with freopen() and keeping the same actual file descriptors. What I don't understand is why you'd need to put them back once changing them? Surely, if someone passed either option, they would want it to persist until the program terminated?
The os function dup2() should provide what you need (if not references to exactly what you need).
More specifically, you can dup2() the stdin file descriptor to another file descriptor, do other stuff with stdin, and then copy it back when you want.
The dup() function duplicates an open file descriptor. Specifically, it provides an alternate interface to the service provided by the fcntl() function using the F_DUPFD constant command value, with 0 for its third argument. The duplicated file descriptor shares any locks with the original.
On success, dup() returns a new file descriptor that has the following in common with the original:
Same open file (or pipe)
Same file pointer (both file descriptors share one file pointer)
Same access mode (read, write, or read/write)
freopen solves the easy part. Keeping old stdin around is not hard if you haven't read anything and if you're willing to use POSIX system calls like dup or dup2. If you're started to read from it, all bets are off.
Maybe you can tell us the context in which this problem occurs?
I'd encourage you to stick to situations where you're willing to abandon old stdin and stdout and can therefore use freopen.
And in the meantime, there's a C source code library that will do all this for you, redirecting stdout or stderr. But the cool part is that it lets you assign as many callback functions as you want to the intercepted streams, allowing you then to very easily send a single message to multiple destinations, a DB, a text file, etc.
On top of that, it makes it trivial to create new streams that look and behave the same as stdout and stderr, where you can redirect these new streams to multiple locations as well.
look for U-Streams C library on *oogle.
This is the most readily available, handy and useful way to do
freopen("dir","r",stdin);