Some months ago I write a CGI application for Linux that uses popen() to read the output of a command, and then I close the pipe with fclose().
Now, I read that for close pipes is needs use pclose().
The manual says:
The return value from popen() is a normal standard I/O stream in all
respects save that it must be closed with pclose() rather than
fclose(3).
My code is like this:
if ((NULL != (f = popen(command.value, "r")))) {
//do something
fclose(f);
}
My question is:
My mistake have a security concern? It program is currently in production. In tests it not do anything problem. Is really needed, patch it using pclose() instead fclose() ? Note: I only open the PIPE one time in the program.
Today, in my local home I do some test and fclose() and pclose() not return EOF indicating failure.
According to this thread, using fclose instead of pclose means that the process at the other end of the pipe doesn't get reaped, so it stays zombied.
If you use fclose on the pipe, you will have file descriptor leaks, since fclose will not free the file pointer in the kernel (which is created when you create the pipe since its a file).
While your testing so far hasn't shown any problems, run your program 3000 times (or how ever many file descriptors are allowed, upwards of an int I think) and watch when you will n o longer be able to create pipes.
I just found out (after 10 years) that I was mistakingly using fclose for some popen calls, running on windows 2008 server. It worked (i.e. did not crash), and I didn't care about the return code on those calls anyway.
But I needed the return code of the last popen stream, and close was done properly with pclose.
It has the strange effect of returning a 0 error code (maybe collecting the return code of previously not pclosed process), even if the command failed, creating a very strange bug in the code, which could have led to catastrophic errors because the caller thinks that the command worked.
So it's not only a matter of leaking descriptors, it can introduce functional bugs in your code, (even if the application runs for a few seconds and you don't care about leaking descriptors)
Related
This question already has answers here:
What happens if I don't call fclose() in a C program?
(4 answers)
Closed 8 years ago.
Suppose that we have opened a file using fopen() in C and we unintentionally forget to close it using fclose() then what could be the consequences of it? Also what are the solutions to it if we are not provided with the source code but only executable?
The consequences are that a file descriptor is "leaked". The operating system uses some descriptor, and has some resources associated with that open file. If you fopen and don't close, then that descriptor won't be cleaned up, and will persist until the program closes.
This problem is compounded if the file can potentially be opened multiple times. As the program runs more and more descriptors will be leaked, until eventually the operating system either refuses or is unable to create another descriptor, in which case the call to fopen fails.
If you are only provided with the executable, not the source code, your options are very limited. At that point you'd have to either try decompiling or rewriting the assembly by hand, neither of which are attractive options.
The correct thing to do is file a bug report, and then get an updated/fixed version.
If there are a lot of files open but not closed properly, the program will eventually run out of file handles and/or memory space and crash.
Suggest you engage your developer to update their code.
The consequences is implementation dependent based on the fclose / fopen and associated functions -- they are buffered input/output functions. So things write are written to a "file" is in fact first written to an internal buffer -- the buffer is only flushed to output when the code "feels like it" -- that could be every line, every write of every full block depending on the smartness of the implementation.
The fopen will most likely use open to get an actual file descriptor to the operating system -- on most systems (Linux, Windows etc) the os file descriptor will be closed by the OS when the process terminates -- however if the program does not terminates, the os file descriptor will leak and you will eventually run out of file descriptors and die.
Some standard may mandate a specific behavior when the program terminates either cleanly or through a crash, but the fact is that you cannot reply in this as not all implementations may follow this.
So your risk is that you will loose some of the data which you program believed that it had written -- that would be the data which was sitting in the internal buffer but never flushed -- or you may run out of file descriptors and die.
So, fix the code.
I am writing a C program for an embedded Linux (debian-arm) device. In some cases, e.g. if a fatal error occurs on the system/program, I want the program to reboot the system by system("reboot");after logging the error(s) via syslog(). My program includes multithreads, UDP sockets, severalfwrite()/fopen(), malloc() calls, ..
I would like to ask a few question what (how) the program should perform processes just before rebooting the system apart from the syslog. I would appreciate to know how these things are done by the experienced programmers.
Is it necessary to close the open sockets (UDP) and threads just before rebooting? If it is the case, is there a function/system call that closes the all open sockets and threads? If the threads needs to be closed and there is no such global function/call to end them, how I suppose to execute pthread_exit(NULL); for each specific threads? Do I need go use something like goto to end the each threads?
How should the program closes files that fopen and fwrite uses? Is there a global call to close the files in use or do I need to find out the files in use manually then use fclose for the each file? I see see some examples on the forums fflush(), flush(), sync(),.. are used, which one(s) would you recommend to use? In a generic case, would it cause any problem if all of these functions are used (although these could be used unnecessary)?
It is not necessary to free the variables that malloc allocated space, is it?
Do you suggest any other tasks to be performed?
The system automatically issues SIGTERM signals to all processes as one of the steps in rebooting. As long as you correctly handle SIGTERM, you need not do anything special after invoking the reboot command. The normal idiom for "correctly handling SIGTERM" is:
Create a pipe to yourself.
The signal handler for SIGTERM writes one byte (any value will do) to that pipe.
Your main select loop includes the read end of that pipe in the set of file descriptors of interest. If that pipe ever becomes readable, it's time to exit.
Furthermore, when a process exits, the kernel automatically closes all its open file descriptors, terminates all of its threads, and deallocates all of its memory. And if you exit cleanly, i.e. by returning from main or calling exit, all stdio FILEs that are still open are automatically flushed and closed. Therefore, you probably don't have to do very much cleanup on the way out -- the most important thing is to make sure you finish generating any output files and remove any temporary files.
You may find the concept of crash-only software useful in figuring out what does and does not need cleaning up.
The only cleanup you need to do is anything your program needs to start up in a consistent state. For example, if you collect some data internally then write it to a file, you will need to ensure this is done before exiting. Other than that, you do not need to close sockets, close files, or free all memory. The operating system is designed to release these resources on process exit.
After a call like freopen(file_name, open_mode, stderr), do I need to call fclose(stderr) before the process ends or it is done automatically ?
thanks in advance and sorry for my English
It is not necessary to close an open stream as all open streams are closed at program termination but it is good practice to have a fclose for every fopen call. In the case of stderr here, this stream is already open at startup (you didn't have to call fopen) and so I see no reason to explicitly close it even if some freopen calls were issued.
When a process ends, all its handles are closed automatically. However, it is good style to close every handle you acquire - for example, somebody may want to convert your program to a library, or you may be looking for leaks. Whether you call freopen does not matter for that.
However, in the case of stderr, the situation is a little bit different. Since you didn't specifically open that stream, you shouldn't close it. It's also very likely to be used by other components out of your control, for example atexit functions or stack smashing detection.
Yes, that is good practice, but not strictly necessary if your process will end immediately since the OS will clean everything up for you when the process dies.
Of course. And simply fclose(stderr); will do.
So I'm building a Unix minishell in C, and am implementing input, output, and err redirection, and have come across a problem with files. I open my files in a loop where I find redirection operators, and use open(), which returns an fd. I then assign the child's fd accordingly, and call an execute function.
When my shell is just going out and finding programs, and executing them with execvp(), I don't have much of a problem. The only problem is knowing whether or not I need to call close() on the file descriptors before prompting for the next command line. I'm worried about having an fd leak, but don't exactly understand how it works.
My real problem arises when using builtin commands. I have a builtin command called "read", that takes one argument, an environmental variable name(could be one that doesn't yet exist). Read then prompts for a value, and assigns that value to the variable. Here's an example:
% read TESTVAR
test value test value test value
% echo ${TESTVAR}
test value test value test value
Well lets say that I try something like this:
% echo here's another test value > f1
% read TESTVAR < f1
% echo ${TESTVAR}
here's another test value
This works great, keep in mind that read executes inside the parent process, I don't call read with execvp since it's builtin. Read uses gets, which requires a stream variable, not an fd. So after poking around on the irc forums a bit I was told to use fdopen, to get the stream from the file descriptor. So before calling gets, I call:
rdStream = fdopen(inFD, "r");
then call
if(fgets(buffer, envValLen, rdStream) != buffer)
{
if(inFD) fclose(rdStream);
return -1;
}
if(inFD) fclose(rdStream);
As you can see, at the moment I'm closing the stream with fclose(), unless it is equal to stdin(which is 0). Is this necessary? Do I need to close the stream? Or just the file descriptor? Or both? I'm quite confused on which I should close, since they both refer to the same file, in a different manner. At the moment I'm not closing the fd, however I think that I definitely should. I would just like somebody to help make sure my shell isn't leaking any files, as I want it to be able to execute several thousand commands in a single session without leaking memory.
Thanks, if you guys want me to post anymore code just ask.
The standard says:
The fclose() function shall perform the equivalent of a close() on the
file descriptor that is associated with the stream pointed to by
stream.
So calling fclose is enough; it will also close the descriptor.
FILE is a buffering object from standard C library. When you do fclose (standard C function) it will eventually call close (Unix system function) but only after making sure C library buffers are flushed. So, I would say, if you use fopen andfwrite then you should use fclose, and not just close, otherwise you risk loosing some data.
Firstly, I'm aware that opening a file with fopen() and not closing it is horribly irresponsible, and bad form. This is just sheer curiosity, so please humour me :)
I know that if a C program opens a bunch of files and never closes any of them, eventually fopen() will start failing. Are there any other side effects that could cause problems outside the code itself? For instance, if I have a program that opens one file, and then exits without closing it, could that cause a problem for the person running the program? Would such a program leak anything (memory, file handles)? Could there be problems accessing that file again once the program had finished? What would happen if the program was run many times in succession?
As long as your program is running, if you keep opening files without closing them, the most likely result is that you will run out of file descriptors/handles available for your process, and attempting to open more files will fail eventually. On Windows, this can also prevent other processes from opening or deleting the files you have open, since by default, files are opened in an exclusive sharing mode that prevents other processes from opening them.
Once your program exits, the operating system will clean up after you. It will close any files you left open when it terminates your process, and perform any other cleanup that is necessary (e.g. if a file was marked delete-on-close, it will delete the file then; note that that sort of thing is platform-specific).
However, another issue to be careful of is buffered data. Most file streams buffer data in memory before writing it out to disk. If you're using FILE* streams from the stdio library, then there are two possibilities:
Your program exited normally, either by calling the exit(3) function, or by returning from main (which implicitly calls exit(3)).
Your program exited abnormally; this can be via calling abort(3) or _Exit(3), dying from a signal/exception, etc.
If your program exited normally, the C runtime will take care of flushing any buffered streams that were open. So, if you had buffered data written to a FILE* that wasn't flushed, it will be flushed on normal exit.
Conversely, if your program exited abnormally, any buffered data will not be flushed. The OS just says "oh dear me, you left a file descriptor open, I better close that for you" when the process terminates; it has no idea there's some random data lying somewhere in memory that the program intended to write to disk but did not. So be careful about that.
The C standard says that calling exit (or, equivalently, returning from main) causes all open FILE objects to be closed as-if by fclose. So this is perfectly fine, except that you forfeit the opportunity to detect write errors.
EDIT: There is no such guarantee for abnormal termination (abort, a failed assert, receipt of a signal whose default behavior is to abnormally terminate the program -- note that there aren't necessarily any such signals -- and other implementation-defined means). As others have said, modern operating systems will clean up all externally visible resources, such as open OS-level file handles, regardless; however, FILEs are likely not to be flushed in that case.
There certainly have been OSes that did not clean up externally visible resources on abnormal termination; it tends to go along with not enforcing hard privilege boundaries between "kernel" and "user" code and/or between distinct user space "processes", simply because if you don't have those boundaries it may not be possible to do so safely in all cases. (Consider, for instance, what happens if you write garbage over the open-file table in MS-DOS, as you are perfectly able to do.)
Assuming you exit under control, using the exit() system call or returning from main(), then the open file streams are closed after flushing. The C Standard (and POSIX) mandate this.
If you exit out of control (core dump, SIGKILL) etc, or if you use _exit() or _Exit(), then the open file streams are not flushed (but the file descriptors end up closed, assuming a POSIX-like system with file descriptors - Standard C does not mandate file descriptors). Note that _Exit() is mandated by the C99 standard, but _exit() is mandated by POSIX (but they behave the same on POSIX systems). Note that file descriptors are separate from file streams. See the discussion of 'Consequences of Program Termination' on the POSIX page for _exit() to see what happens when a program terminates under Unix.
When the process dies, most modern operating systems (the kernel specifically) will free all of your handles and allocated memory.