I have a program which prints (by printf) to the stdout some data and also calls to function *foo*
which also prints to the stdout some data [the way (implementation) of how printing is done from foo is unknown and I can`t see the code of foo].
I have to redirect everything from stdout to buffer or file. I tried to do it in several ways
freopen(file.txt, stdout) - only my code prints are written to the file.txt. What was printed from foo is lost.
setbuf(buffer, stdout) - only my code prints are written to the buffer. What was printed from foo is appears in the stdout.(It appears on the screen)
What can explain this behavior? How can the problem be solved?
Note:This code has to work in cross-OS( lunux/wind && mac OS).I use gcc in order compile the code and I have cygwin
It's likely that foo isn't using stdio for printing and directly calling the OS for this.
I don't know about win32, but on POSIX you could use dup2 to take care of it.
/* Before the function foo is called, make `STDOUT_FILENO` refer to `fd` */
int fd;
fd = open(...);
dup2(fd, STDOUT_FILENO);
EDIT
Much to my surprise, win32 has _dup2 but it does something else.
How do you know that foo() is printing to stdout? Have you tried redirecting standard output to a file at the shell and seeing whether the output from foo() still appears on the screen?
If the file redirection sends foo()'s output to the file, then you may have to rejig the file descriptor level, as in cnicutar's answer.
If the file redirection does not send foo()'s output to the file, then it may be writing to stderr or it may be opening and using /dev/tty or something similar. You can test for stderr by redirecting it separately from stdout:
your_program >/tmp/stdout.me 2>/tmp/stderr.me
If it is opening /dev/tty, the output will still appear on your screen.
Which platform are you on? If you can track system calls (strace on Linux, truss on Solaris, ...), then you may be able to see in that what the foo() function is doing. You can help things by writing a message before and after calling the function, and ensuring you flush the output:
printf("About to call foo()\n");
fflush(0);
foo();
printf("Returned from foo()\n");
fflush(0);
The printf/fflush calls will be visible in the trace output, so what appears between is done by foo().
What can explain this behavior?
I have seen this sort of behavior when the code you are calling into uses a different C library than yours. On Windows I used to see this sort of thing when one DLL is compiled with GCC and another with Visual C++. The implementation of stdio for these is apparently different enough such that this can be problematic.
Another is that the code you are calling is not using stdio. If you are on Unix you can use dup2 to get around this, eg. dup2(my_file_descriptor, 1). On many implementations if you have a FILE* you can say dup2(fileno(f), 1). This may not be portable.
Related
I know already how to implement methods regarding usual freopen(), popen() or similar stdout/stdin/stderr -based redirecting mechanisms, but I wondered how should I apply the said mechanism to static (own) libraries in C? Say, I want to use a library to capture any program with printf() commands or so into a file (for instance) without letting it appear on the console - are there some things I need to acknowledge before applying simple fd dups and just calling the library in the main program? Even piping seems to be complex seeing as execing here is risky...
thanks in advance.
There's an old-timers' trick to force the entire process, regardless of what library the code comes from, to have one of the standard IO ports connected to a different filehandle. You simply close the filehandle in question, then open a new one. If you close(1), then open('some_file', 'w'), then ALL calls that would result in a write to stdout will go to some_file from that point forward.
This works because open() always uses the first file descriptor that isn't currently in use. Presuming that you haven't closed stdin (fd=0), the call to open will get a file descriptor of 1.
There are some caveats. FILE outputs that haven't flushed their buffers will have undefined behavior, but you probably won't be doing this in the middle of execution. Set it up as your process starts and you'll be golden.
This seems like a bit of a computing systems 101 question, but I'm stumped.
I am integrating existing code from C/C++ project A into my own project B. Both A and B will be linked into a single executable, threaded process. Project A's code makes extensive use of printf for output. This is fine, but I want also to capture that output into my own buffers. Is there a way I can read from stdout once the printf calls have written to it? I cannot fork the process or pipe. And my efforts to poll() stdout, or to dup() it, have not succeeded (I may be doing something wrong here).
You can use freopen to change the descriptor.
#include<stdio.h>
main(int argc, char** argv) {
FILE *fp = freopen("output.txt", "w", stdout);
printf("Hello\n");
fclose(fp);
}
If you run that you'll see the printf output in output.txt and nothing will go to your screen.
You can now open the file to read the data or you could even mmap it into your memory space and process it that way.
Before you printf(), you could close fd 1, and dup2() a pipe that you've created into fd 1.
Not to mention: there is now a handy U-Streams C source code library that makes redirecting stdout and stderr quite trivial. And you can even redirect them very easily to multiple destinations. And, you can create your own streams besides that can be used in exactly the same way stdout and stderr behave.
Look for the U-Streams C Library... handy indeed.
Once it's gone out, it's gone. If you want to compile it all into a single executable, you'll have to go through the code for A with a search and replace and change all those printf calls into ones to your own stream, where you can copy them and then pass them on to stdout.
In C, how should I execute external program and get its results as if it was ran in the console?
if there is an executable called dummy, and it displays 4 digit number in command prompt when executed, I want to know how to run that executable and get the 4 digit number that it had generated. In C.
popen() handles this quite nicely. For instance if you want to call something and read the results line by line:
char buffer[140];
FILE *in;
extern FILE *popen();
if(! (in = popen(somecommand, "r"""))){
exit(1);
}
while(fgets(buff, sizeof(buff), in) != NULL){
//buff is now the output of your command, line by line, do with it what you will
}
pclose(in);
This has worked for me before, hopefully it's helpful. Make sure to include stdio in order to use this.
You can use popen() on UNIX.
This is not actually something ISO C can do on its own (by that I mean the standard itself doesn't provide this capability) - possibly the most portable solution is to simply run the program, redirecting its standard output to a file, like:
system ("myprog >myprog.out");
then use the standard ISO C fopen/fread/fclose to read that output into a variable.
This is not necessarily the best solution since that may depend on the underlying environment (and even the ability to redirect output is platform-specific) but I thought I'd add it for completeness.
There is popen() on unix as mentioned before, which gives you a FILE* to read from.
Alternatively on unix, you can use a combination of pipe(), fork(), exec(), select(), and read(), and wait() to accomplish the task in a more generalized/flexible way.
The popen library call invokes fork and pipe under the hood to do its work. Using it, you're limited to simply reading whatever the process dumps to stdout (which you could use the underlying shell to redirect). Using the lower-level functions you can do pretty much whatever you want, including reading stderr and writing stdin.
On windows, see calls like CreatePipe() and CreateProcess(), with the IO members of STARTUPINFO set to your pipes. You can get a file descriptor to do read()'s using _open_ofshandle() with the process handle. Depending on the app, you may need to read multi-threaded, or it may be okay to block.
I've coded a program in C that sends messages to the stdout using printf and I'm having trouble redirecting the output to a file (running from bash).
I've tried:
./program argument >> program.out
./program argument > program.out
./program >> program.out argument
./program > program.out argument
In each case, the file program.out is created but it remains empty. After the execution ends the file size is 0.
If I omit the redirection when executing the program:
./program argument
Then, all messages sent to stdout using printf are shown in the terminal.
I have other C programs for which I've no problem redirecting the output this way.
Does it have to do with the program itself? with the argument passing?
Where should look for the problem?
Some details about the C program:
It does not read anything from stdin
It uses BSD Internet Domain sockets
It uses POSIX threads
It assigns a special handler function for SIGINT signal using sigaction
It sends lots of newlines to stdout (for those of you thinking I should flush)
Some code:
int main(int argc, char** argv)
{
printf("Execution started\n");
do
{
/* lots of printf here */
} while (1);
/* Code never reached */
pthread_exit(EXIT_SUCCESS);
}
Flushing after newlines only works when printing to a terminal, but not necessarily when printing to a file. A quick Google search revealed this page with further information: http://www.pixelbeat.org/programming/stdio_buffering/
See the section titled "Default Buffering modes".
You might have to add some calls to fflush(stdout), after all.
You could also set the buffer size and behavior using setvbuf.
Flushing the buffers is normally handled by the exit() function, which is usually called implicitly by a return from main(). You are ending your program by raising SIGINT, and apparently the default SIGINT handler does not flush the buffers.
Take a look at this article:
Applying Design Patterns to Simplify Signal Handling. The article is mostly C++, but there is a useful C example in the 2nd section, which shows how to use SIGINT to exit your program gracefully.
As for why the behavior of a terminal differs from a file,
take a look at Stevens' Advanced Programing in the UNIX Environment Section 5.4 on Buffering. He says that:
Most implementations default to the following types of buffering.
Standard error is always unbuffered.
All other streams are line buffered if they refer to a terminal device; otherwise, they are fully buffered.
The four platforms discussed in this book follow these conventions for standard I/O buffering: standard error is unbuffered, streams open to terminal devices are line buffered, and all other streams are fully buffered.
Has the program terminated by the time you check the contents of the redirected file? If it's still running, your output might still be buffered somewhere up the chain, so you don't see it in the file.
Apart from that, and the other answers provided so far, I think it's time to show a representative example of the problem code. There's too many esoteric possibilities.
EDIT
From the look of the sample code, if you've got a relatively small amount of printing happening, then you're getting caught in the output buffer. Flush after each write to be sure that it's gone to disk. Typically you can have up to a page size's worth of unwritten data lying around otherwise.
In the absence of a flush, the only time you can be sure you've got everything on disk is when the program exits. Even a thread terminating won't do it, since output buffers like that aren't per-thread, they're per-process.
Suggestions:
Redirect stderr to a file as well.
Try tail -f your output file(s).
Open a file and fprintf your logging (to help figure out what's going on).
Search for any manual closes/duplication/piping of std* FILE handles or 1-3 file descriptors.
Reduce complexity; cut out big chunks of functionality until printfs work. Then readd them until it breaks again. Continue until you identify the culprit code.
Just for the record, in Perl you would use:
use IO::Handle;
flush STDOUT;
autoflush STDOUT;
I'm writing a utility for running programs, and I need to capture unbuffered stdout and stderr from the programs. I need to:
Capture stdout and stderr to separate files.
Output needs to not be buffered (or be line buffered).
Without modifying the source of the program being run.
The problem is, when piping output to a file, the stdout stream becomes block buffered rather than line buffered. If the program crashes, the output never gets flushed, and is blank. So I need to capture stdout without buffering (or with line buffering).
I think this can be done with pty's but I'm having difficulty finding any examples that do exactly what I want (most ignore stderr). In fact, I'm not sure I've found any pty examples in C at all; most use a higher-level interface like Python's pty and subprocess modules.
Can anyone help (with code snippets or links)? Any help would be appreciated.
EDIT: I think I've solved it. The following two links were pretty helpful.
http://publib.boulder.ibm.com/infocenter/zos/v1r10/index.jsp?topic=/com.ibm.zos.r10.bpxbd00/posixopenpt.htm
http://www.gidforums.com/t-3369.html
My code is available as a repository:
https://bitbucket.org/elliottslaughter/pty
see man 7 pty
In particular:
Unix 98 pseudo-terminals
An unused Unix 98 pseudo-terminal master is opened by calling
posix_openpt(3). (This function opens the master clone device,
/dev/ptmx; see pts(4).) After performing any program-specific initializations,
changing the ownership and permissions of the slave device
using grantpt(3), and unlocking the slave using unlockpt(3)), the corresponding
slave device can be opened by passing the name returned by
ptsname(3) in a call to open(2).
And now that you know the names of the library functions such a code will need to call, you can do two useful things:
Look up their man pages
Google for example code. Since you know what keywords to use with the search engine I suspect you will have much more luck hunting down examples.