So I'm building a Unix minishell in C, and am implementing input, output, and err redirection, and have come across a problem with files. I open my files in a loop where I find redirection operators, and use open(), which returns an fd. I then assign the child's fd accordingly, and call an execute function.
When my shell is just going out and finding programs, and executing them with execvp(), I don't have much of a problem. The only problem is knowing whether or not I need to call close() on the file descriptors before prompting for the next command line. I'm worried about having an fd leak, but don't exactly understand how it works.
My real problem arises when using builtin commands. I have a builtin command called "read", that takes one argument, an environmental variable name(could be one that doesn't yet exist). Read then prompts for a value, and assigns that value to the variable. Here's an example:
% read TESTVAR
test value test value test value
% echo ${TESTVAR}
test value test value test value
Well lets say that I try something like this:
% echo here's another test value > f1
% read TESTVAR < f1
% echo ${TESTVAR}
here's another test value
This works great, keep in mind that read executes inside the parent process, I don't call read with execvp since it's builtin. Read uses gets, which requires a stream variable, not an fd. So after poking around on the irc forums a bit I was told to use fdopen, to get the stream from the file descriptor. So before calling gets, I call:
rdStream = fdopen(inFD, "r");
then call
if(fgets(buffer, envValLen, rdStream) != buffer)
{
if(inFD) fclose(rdStream);
return -1;
}
if(inFD) fclose(rdStream);
As you can see, at the moment I'm closing the stream with fclose(), unless it is equal to stdin(which is 0). Is this necessary? Do I need to close the stream? Or just the file descriptor? Or both? I'm quite confused on which I should close, since they both refer to the same file, in a different manner. At the moment I'm not closing the fd, however I think that I definitely should. I would just like somebody to help make sure my shell isn't leaking any files, as I want it to be able to execute several thousand commands in a single session without leaking memory.
Thanks, if you guys want me to post anymore code just ask.
The standard says:
The fclose() function shall perform the equivalent of a close() on the
file descriptor that is associated with the stream pointed to by
stream.
So calling fclose is enough; it will also close the descriptor.
FILE is a buffering object from standard C library. When you do fclose (standard C function) it will eventually call close (Unix system function) but only after making sure C library buffers are flushed. So, I would say, if you use fopen andfwrite then you should use fclose, and not just close, otherwise you risk loosing some data.
Related
I want to reopen the stdin and stdout (and perhaps stderr while I'm at it) filehandles, so that future calls to printf() or putchar() or puts() will go to a file, and future calls to getc() and such will come from a file.
1) I don't want to permanently lose standard input/output/error. I may want to reuse them later in the program.
2) I don't want to open new filehandles because these filehandles would have to be either passed around a lot or global (shudder).
3) I don't want to use any open() or fork() or other system-dependent functions if I can't help it.
So basically, does it work to do this:
stdin = fopen("newin", "r");
And, if it does, how can I get the original value of stdin back? Do I have to store it in a FILE * and just get it back later?
Why use freopen()? The C89 specification has the answer in one of the endnotes for the section on <stdio.h>:
116. The primary use of the freopen function is to change the file associated with a standard
text stream (stderr,
stdin, or stdout), as those identifiers need not be
modifiable lvalues to which the value
returned by the fopen function
may be assigned.
freopen is commonly misused, e.g. stdin = freopen("newin", "r", stdin);. This is no more portable than fclose(stdin); stdin = fopen("newin", "r");. Both expressions attempt to assign to stdin, which is not guaranteed to be assignable.
The right way to use freopen is to omit the assignment: freopen("newin", "r", stdin);
I think you're looking for something like freopen()
This is a modified version of Tim Post's method; I used /dev/tty instead of /dev/stdout. I don't know why it doesn't work with stdout
(which is a link to /proc/self/fd/1):
freopen("log.txt","w",stdout);
...
...
freopen("/dev/tty","w",stdout);
By using /dev/tty the output is redirected to the terminal from where the app was launched.
Hope this info is useful.
freopen("/my/newstdin", "r", stdin);
freopen("/my/newstdout", "w", stdout);
freopen("/my/newstderr", "w", stderr);
... do your stuff
freopen("/dev/stdin", "r", stdin);
...
...
This peaks the needle on my round-peg-square-hole-o-meter, what are you trying to accomplish?
Edit:
Remember that stdin, stdout and stderr are file descriptors 0, 1 and 2 for every newly created process. freopen() should keep the same fd's, just assign new streams to them.
So, a good way to ensure that this is actually doing what you want it to do would be:
printf("Stdout is descriptor %d\n", fileno(stdout));
freopen("/tmp/newstdout", "w", stdout);
printf("Stdout is now /tmp/newstdout and hopefully still fd %d\n",
fileno(stdout));
freopen("/dev/stdout", "w", stdout);
printf("Now we put it back, hopefully its still fd %d\n",
fileno(stdout));
I believe this is the expected behavior of freopen(), as you can see, you're still only using three file descriptors (and associated streams).
This would override any shell redirection, as there would be nothing for the shell to redirect. However, its probably going to break pipes. You might want to be sure to set up a handler for SIGPIPE, in case your program finds itself on the blocking end of a pipe (not FIFO, pipe).
So, ./your_program --stdout /tmp/stdout.txt --stderr /tmp/stderr.txt should be easily accomplished with freopen() and keeping the same actual file descriptors. What I don't understand is why you'd need to put them back once changing them? Surely, if someone passed either option, they would want it to persist until the program terminated?
The os function dup2() should provide what you need (if not references to exactly what you need).
More specifically, you can dup2() the stdin file descriptor to another file descriptor, do other stuff with stdin, and then copy it back when you want.
The dup() function duplicates an open file descriptor. Specifically, it provides an alternate interface to the service provided by the fcntl() function using the F_DUPFD constant command value, with 0 for its third argument. The duplicated file descriptor shares any locks with the original.
On success, dup() returns a new file descriptor that has the following in common with the original:
Same open file (or pipe)
Same file pointer (both file descriptors share one file pointer)
Same access mode (read, write, or read/write)
freopen solves the easy part. Keeping old stdin around is not hard if you haven't read anything and if you're willing to use POSIX system calls like dup or dup2. If you're started to read from it, all bets are off.
Maybe you can tell us the context in which this problem occurs?
I'd encourage you to stick to situations where you're willing to abandon old stdin and stdout and can therefore use freopen.
And in the meantime, there's a C source code library that will do all this for you, redirecting stdout or stderr. But the cool part is that it lets you assign as many callback functions as you want to the intercepted streams, allowing you then to very easily send a single message to multiple destinations, a DB, a text file, etc.
On top of that, it makes it trivial to create new streams that look and behave the same as stdout and stderr, where you can redirect these new streams to multiple locations as well.
look for U-Streams C library on *oogle.
This is the most readily available, handy and useful way to do
freopen("dir","r",stdin);
I know already how to implement methods regarding usual freopen(), popen() or similar stdout/stdin/stderr -based redirecting mechanisms, but I wondered how should I apply the said mechanism to static (own) libraries in C? Say, I want to use a library to capture any program with printf() commands or so into a file (for instance) without letting it appear on the console - are there some things I need to acknowledge before applying simple fd dups and just calling the library in the main program? Even piping seems to be complex seeing as execing here is risky...
thanks in advance.
There's an old-timers' trick to force the entire process, regardless of what library the code comes from, to have one of the standard IO ports connected to a different filehandle. You simply close the filehandle in question, then open a new one. If you close(1), then open('some_file', 'w'), then ALL calls that would result in a write to stdout will go to some_file from that point forward.
This works because open() always uses the first file descriptor that isn't currently in use. Presuming that you haven't closed stdin (fd=0), the call to open will get a file descriptor of 1.
There are some caveats. FILE outputs that haven't flushed their buffers will have undefined behavior, but you probably won't be doing this in the middle of execution. Set it up as your process starts and you'll be golden.
Some months ago I write a CGI application for Linux that uses popen() to read the output of a command, and then I close the pipe with fclose().
Now, I read that for close pipes is needs use pclose().
The manual says:
The return value from popen() is a normal standard I/O stream in all
respects save that it must be closed with pclose() rather than
fclose(3).
My code is like this:
if ((NULL != (f = popen(command.value, "r")))) {
//do something
fclose(f);
}
My question is:
My mistake have a security concern? It program is currently in production. In tests it not do anything problem. Is really needed, patch it using pclose() instead fclose() ? Note: I only open the PIPE one time in the program.
Today, in my local home I do some test and fclose() and pclose() not return EOF indicating failure.
According to this thread, using fclose instead of pclose means that the process at the other end of the pipe doesn't get reaped, so it stays zombied.
If you use fclose on the pipe, you will have file descriptor leaks, since fclose will not free the file pointer in the kernel (which is created when you create the pipe since its a file).
While your testing so far hasn't shown any problems, run your program 3000 times (or how ever many file descriptors are allowed, upwards of an int I think) and watch when you will n o longer be able to create pipes.
I just found out (after 10 years) that I was mistakingly using fclose for some popen calls, running on windows 2008 server. It worked (i.e. did not crash), and I didn't care about the return code on those calls anyway.
But I needed the return code of the last popen stream, and close was done properly with pclose.
It has the strange effect of returning a 0 error code (maybe collecting the return code of previously not pclosed process), even if the command failed, creating a very strange bug in the code, which could have led to catastrophic errors because the caller thinks that the command worked.
So it's not only a matter of leaking descriptors, it can introduce functional bugs in your code, (even if the application runs for a few seconds and you don't care about leaking descriptors)
I want to implement behavior in my C program so that if a SIGINT happens, I close all open file descriptors. Is there a simple way to get a list of them?
I'd use brute force: for (i = 0; i < fd_max; ++i) close (i);. Quick and pretty portable.
Keep track of all of your open file descriptors and close them individually.
In the general case, a library you're using might have an open file, and closing it will cause that library to misbehave.
In fact, the same problem could exist in your own code, because if you close file descriptors indiscriminately but another part of your program still remembers the file descriptor and tries to use it, it will get an unexpected error or (if other files have been opened since) operate on the wrong file. It is much better for the component responsible for opening a file to also be responsible for closing it.
You could read out the content of /proc/<pid>/fd., if available.
But be aware of the potiential race, that might occur if your application closes some or opens new ones in between your read out /proc/<pid>/fd and you are going to close what you read.
So conculding I want to recommend Kevin Reid's approach to this.
My solution for POSIX systems:
All opened fd's are the lowest value possible.
Make a wrapper function upon open(2).
Your new function open (and return) the requested fd and pass his value to a function called define_if_is_the_higtest_fd_and_store_it().
You should have a int hightest_fd_saved accessible only for a singleton function (there is only 1 'descriptor table') named save_fd() (initial value is 3 (cuz stderr is 2)).
Configure SIGINT to your signal function. Inside, you do a loop from [3, return_fd()].
I think that's it...
In C, how should I execute external program and get its results as if it was ran in the console?
if there is an executable called dummy, and it displays 4 digit number in command prompt when executed, I want to know how to run that executable and get the 4 digit number that it had generated. In C.
popen() handles this quite nicely. For instance if you want to call something and read the results line by line:
char buffer[140];
FILE *in;
extern FILE *popen();
if(! (in = popen(somecommand, "r"""))){
exit(1);
}
while(fgets(buff, sizeof(buff), in) != NULL){
//buff is now the output of your command, line by line, do with it what you will
}
pclose(in);
This has worked for me before, hopefully it's helpful. Make sure to include stdio in order to use this.
You can use popen() on UNIX.
This is not actually something ISO C can do on its own (by that I mean the standard itself doesn't provide this capability) - possibly the most portable solution is to simply run the program, redirecting its standard output to a file, like:
system ("myprog >myprog.out");
then use the standard ISO C fopen/fread/fclose to read that output into a variable.
This is not necessarily the best solution since that may depend on the underlying environment (and even the ability to redirect output is platform-specific) but I thought I'd add it for completeness.
There is popen() on unix as mentioned before, which gives you a FILE* to read from.
Alternatively on unix, you can use a combination of pipe(), fork(), exec(), select(), and read(), and wait() to accomplish the task in a more generalized/flexible way.
The popen library call invokes fork and pipe under the hood to do its work. Using it, you're limited to simply reading whatever the process dumps to stdout (which you could use the underlying shell to redirect). Using the lower-level functions you can do pretty much whatever you want, including reading stderr and writing stdin.
On windows, see calls like CreatePipe() and CreateProcess(), with the IO members of STARTUPINFO set to your pipes. You can get a file descriptor to do read()'s using _open_ofshandle() with the process handle. Depending on the app, you may need to read multi-threaded, or it may be okay to block.