If I have a buffer which contains the data of a file, how can I get a file descriptor from it?
This is a question derived from how to untar file in memory
I wrote a simple example how to make filedescriptor to a memory area:
#include <unistd.h>
#include <stdio.h>
#include <string.h>
char buff[]="qwer\nasdf\n";
int main(){
int p[2]; pipe(p);
if( !fork() ){
for( int buffsize=strlen(buff), len=0; buffsize>len; )
len+=write( p[1], buff+len, buffsize-len );
return 0;
}
close(p[1]);
FILE *f = fdopen( p[0], "r" );
char buff[100];
while( fgets(buff,100,f) ){
printf("from child: '%s'\n", buff );
}
puts("");
}
Not possible in plain C. In plain C all file access happens via FILE * handles and these can only be created with fopen() and freopen() and in both cases must refer to a file path. As C tries to be as portable as possible, it limits I/O to the absolute bare minimum that probably all systems can support in some way.
If you have POSIX API available (e.g. Linux, macOS, iOS, FreeBSD, most other UNIX systems), you can use fmemopen():
char dataInMemory[] = "This is some data in memory";
FILE * fileDescriptor = fmemopen(dataInMemory, sizeof(dataInMemory), "r");
This is a true file handle that can be used with all C file API. It should also allow seeking, something not possible if you work with pipes as pipes support no seeking (you can emulate forward seeking but there is no way to ever seek backwards).
You can't. Unlike C++, the C model of file I/O isn't open to extension.
Related
If I want to use a physical file along with other types of streams such as a socket, I can simply convert a file handle into a file descriptor:
#include <stdlib.h>
#include <stdio.h>
int main(void) {
FILE *f = fopen("uniquefilename.ext", "w");
int fd = fileno(f);
printf("%d\n", fd);
fclose(f);
return 0;
}
Does the GNU Standard Library provide a way to obtain a physical file's descriptor directly? Something to the effect of:
int fd = some_call("file_name.ext", "mode");
It seems I need to note I am completely aware of how a descriptor is not implicitly bound to any specific file. I was misleading when I wrote "obtain a physical file's descriptor"; what I should have wrote is something like "create a descriptor enabling access to a specific physical file".
It does not.
However, you can use the open function directly! This is part of Linux itself, not the C standard library (technically the C standard library provides a small wrapper to allow you to call it as a C function).
Example usage:
int fd = open("file_name.ext", O_RDWR); // not fopen
// do stuff with fd
close(fd); // not fclose
Note: The man page recommends including <sys/types.h>, <sys/stat.h>, and <fcntl.h>, and for close you need <unistd.h>. That's quite a few headers, and I don't know if they're all necessary.
This question already has answers here:
Closed 12 years ago.
Possible Duplicate:
Getting Filename from file descriptor in C
Is there a simple and (reasonably) portable way of getting the filename from a FILE*?
I open a file using f = fopen(filename, ...) and then pass down f to various other functions, some of which may report an error. I'd like to report the filename in the error message but avoid having to pass around the extra parameter.
I could create a custom wrapper struct { FILE *f, const char *name }, but is there perhaps a simpler way? (If the FILE* wasn't opened using fopen I don't care about the result.)
On some platforms (such as Linux), you may be able to fetch it by reading the link of /proc/self/fd/<number>, as so:
#include <stdio.h>
#include <unistd.h>
#include <string.h>
int main(void)
{
char path[1024];
char result[1024];
/* Open a file, get the file descriptor. */
FILE *f = fopen("/etc/passwd", "r");
int fd = fileno(f);
/* Read out the link to our file descriptor. */
sprintf(path, "/proc/self/fd/%d", fd);
memset(result, 0, sizeof(result));
readlink(path, result, sizeof(result)-1);
/* Print the result. */
printf("%s\n", result);
}
This will, on my system, print out /etc/passwd, as desired.
It's a bit difficult, because a FILE* can read/write from a file handle which isn't associated with a named file at all (for example an unnamed pipe or a socket). You can obtain the file handle with fileno() and then there are system specific ways to learn about the file name. Here's a discussion on how to do this under Linux:
Getting Filename from file descriptor in C
and under Windows this isn't much easier either:
http://msdn.microsoft.com/en-us/library/aa366789(VS.85).aspx (as an extra step here, you use _get_osfhandle() to get the Windows file handle from the c-library file descriptor)
This question already has answers here:
Closed 12 years ago.
Possible Duplicate:
Getting Filename from file descriptor in C
Is there a simple and (reasonably) portable way of getting the filename from a FILE*?
I open a file using f = fopen(filename, ...) and then pass down f to various other functions, some of which may report an error. I'd like to report the filename in the error message but avoid having to pass around the extra parameter.
I could create a custom wrapper struct { FILE *f, const char *name }, but is there perhaps a simpler way? (If the FILE* wasn't opened using fopen I don't care about the result.)
On some platforms (such as Linux), you may be able to fetch it by reading the link of /proc/self/fd/<number>, as so:
#include <stdio.h>
#include <unistd.h>
#include <string.h>
int main(void)
{
char path[1024];
char result[1024];
/* Open a file, get the file descriptor. */
FILE *f = fopen("/etc/passwd", "r");
int fd = fileno(f);
/* Read out the link to our file descriptor. */
sprintf(path, "/proc/self/fd/%d", fd);
memset(result, 0, sizeof(result));
readlink(path, result, sizeof(result)-1);
/* Print the result. */
printf("%s\n", result);
}
This will, on my system, print out /etc/passwd, as desired.
It's a bit difficult, because a FILE* can read/write from a file handle which isn't associated with a named file at all (for example an unnamed pipe or a socket). You can obtain the file handle with fileno() and then there are system specific ways to learn about the file name. Here's a discussion on how to do this under Linux:
Getting Filename from file descriptor in C
and under Windows this isn't much easier either:
http://msdn.microsoft.com/en-us/library/aa366789(VS.85).aspx (as an extra step here, you use _get_osfhandle() to get the Windows file handle from the c-library file descriptor)
#include <stdio.h>
int main() {
printf("This goes to screen\n");
freopen("out.txt", "a", stdout);
printf("This goes to out.txt");
freopen("/dev/stdout", "a", stdout);
printf("This should go to screen too, but doesn't\n");
return 0;
}
I call freopen to redirect the stdout to out.txt then I print something on the file, now I want to redirect it back to the screen, but freopen("/dev/stdout", "a", stdout); doesn't work. Is there any way to do that using ANSI C or POSIX system calls?
I can't think of a way to do this in a cross-platform manner, but on GNU/Linux systems (and maybe other POSIX-compliant ones, too) you can freopen ("/dev/tty", "a", stdout). Is this what you were trying to do?
Unfortunately, there doesn't seem to be a good way:
http://c-faq.com/stdio/undofreopen.html
The best recommendation is not to use freopen in this circumstance.
Generally speaking, you can't. You have closed the file, which could've been pipe or whatever. It's not reopenable. You might have saved stdout value, then assign some fopen to it and then close it and copy the old value back. Example:
FILE *o = stdout;
stdout=fopen("/tmp/crap.txt","a");
printf("Oh no!\n");
fclose(stdout);
stdout = o;
Mike Weller suggested below in comments that stdout might not always be writable. In this case something like that might help:
int o = dup(fileno(stdout));
freopen("/tmp/crap.txt","a",stdout);
printf("Oh no!\n");
dup2(o,fileno(stdout));
close(o);
Another edit: if you're using it to redirect output from the child process like your comment elsewhere suggest, you can redirect it after the fork.
Use fdopen() and dup() as well as freopen().
int old_stdout = dup(1); // Preserve original file descriptor for stdout.
FILE *fp1 = freopen("out.txt", "w", stdout); // Open new stdout
...write to stdout... // Use new stdout
FILE *fp2 = fdopen(old_stdout, "w"); // Open old stdout as a stream
...Now, how to get stdout to refer to fp2?
...Under glibc, I believe you can use:
fclose(stdout); // Equivalent to fclose(fp1);
stdout = fp2; // Assign fp2 to stdout
// *stdout = *fp2; // Works on Solaris and MacOS X, might work elsewhere.
close(old_stdout); // Close the file descriptor so pipes work sanely
I'm not sure whether you can do the assignment reliably elsewhere.
Dubious code that does actually work
The code below worked on Solaris 10 and MacOS X 10.6.2 - but I'm not confident that it is reliable. The structure assignment may or may not work with Linux glibc.
#include <stdio.h>
#include <unistd.h>
int main(void)
{
printf("This goes to screen\n");
int old_stdout = dup(1); // Consider dup(STDOUT_FILENO) or dup(fileno(stdout))
FILE *fp1 = freopen("out.txt", "a", stdout);
printf("This goes to out.txt\n");
fclose(stdout);
FILE *fp2 = fdopen(old_stdout, "w");
*stdout = *fp2; // Unreliable!
printf("This should go to screen too, but doesn't\n");
return 0;
}
You can't say you weren't warned — this is playing with fire!
If you're on a system with the /dev/fd file system, you could create the name of the file implied by the file descriptor returned from dup() with sprintf(buffer, "/dev/fd/%d", old_stdout) and then use freopen() with that name. This would be a lot more reliable than the assignment used in this code.
The better solutions either make the code use 'fprintf(fp, ...)' everywhere, or use a cover function that allows you set your own default file pointer:
mprintf.c
#include "mprintf.h"
#include <stdarg.h>
static FILE *default_fp = 0;
void set_default_stream(FILE *fp)
{
default_fp = fp;
}
int mprintf(const char *fmt, ...)
{
va_list args;
va_start(args, fmt);
if (default_fp == 0)
default_fp = stdout;
int rv = vfprintf(default_fp, fmt, args);
va_end(args);
return(rv);
}
mprintf.h
#ifndef MPRINTF_H_INCLUDED
#define MPRINTF_H_INCLUDED
#include <stdio.h>
extern void set_default_stream(FILE *fp);
extern int mprintf(const char *fmt, ...);
#endif
Clearly, you can create an mvprintf() and other functions as needed.
Example use of mprintf()
Then, in place of the original code, you can use:
#include "mprintf.h"
int main()
{
mprintf("This goes to screen\n");
FILE *fp1 = fopen("out.txt", "w");
set_default_stream(fp1);
mprintf("This goes to out.txt\n");
fclose(fp1);
set_default_stream(stdout);
mprintf("This should go to screen too, but doesn't\n");
return 0;
}
(Warning: untested code - confidence level too high. Also, all code written assuming you use a C99 compiler, primarily because I declare variables when I first need them, not at the beginning of the function.)
Caution:
Note that if the original program is invoked as ./original_program > file or ./original_program | grep something (with redirected output) or is run from a cron job, then opening /dev/tty is not usually appropriate as a way to reopen standard output because the original standard output was not the terminal.
Also, note that if the redirection of standard output is used prior to forking and execing a child program and the original standard output is reinstated in the parent, then the sequence of operations is wrong. You should fork and then adjust the I/O of the child (only), without modifying the parent's I/O at all.
On Windows, you can open "CONOUT$".
freopen("test.txt", "w", stdout);
printf("this goes to test.txt");
freopen("CONOUT$", "w", stdout);
printf("this goes to the console\n");
This probably doesn't work if stdout is redirected to start with.
The following code (SwapIOB) is used in Testbenches that want to store
the stdout stream for comparison to an expected results file.
Background: File streams are managed using an _IOB structure that is stored in an array of 20 _IOB entries. This includes stdout stream. The IOBs are stored in an array. When a file is created the application code gets a ptr to an element in that array. The application code then passes that ptr to the OS for processing I/O calls. Thus, the OS does NOT itself contain or rely on its own pointers to the application's IOB.
Requirement: When running a testbench the stdout messages issued by an application should be re-directed to a file. However, after the module under test has completed then stdout messages should be re-redirected to the console.
This routine was tested and is currently used on Windows XP/Pro system.
void SwapIOB(FILE *A, FILE *B) {
FILE temp;
// make a copy of IOB A (usually this is "stdout")
memcpy(&temp, A, sizeof(struct _iobuf));
// copy IOB B to A's location, now any output
// sent to A is redirected thru B's IOB.
memcpy(A, B, sizeof(struct _iobuf));
// copy A into B, the swap is complete
memcpy(B, &temp, sizeof(struct _iobuf));
} // end SwapIOB;
Application code uses SwapIOB() similar to:
FILE *fp;
fp = fopen("X", "w");
SwapIOB(stdout, fp);
printf("text to file X");
SwapIOB(stdout, fp);
fclose(fp);
printf("text to console works, again!");
here's another question about splice(). I'm hoping to use it to copy files, and am trying to use two splice calls joined by a pipe like the example on splice's Wikipedia page. I wrote a simple test case which only tries to read the first 32K bytes from one file and write them to another:
#define _GNU_SOURCE
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
int main(int argc, char **argv) {
int pipefd[2];
int result;
FILE *in_file;
FILE *out_file;
result = pipe(pipefd);
in_file = fopen(argv[1], "rb");
out_file = fopen(argv[2], "wb");
result = splice(fileno(in_file), 0, pipefd[1], NULL, 32768, SPLICE_F_MORE | SPLICE_F_MOVE);
printf("%d\n", result);
result = splice(pipefd[0], NULL, fileno(out_file), 0, 32768, SPLICE_F_MORE | SPLICE_F_MOVE);
printf("%d\n", result);
if (result == -1)
printf("%d - %s\n", errno, strerror(errno));
close(pipefd[0]);
close(pipefd[1]);
fclose(in_file);
fclose(out_file);
return 0;
}
When I run this, the input file seems to be read properly, but the second splice call fails with EINVAL. Anybody know what I'm doing wrong here?
Thanks!
From the splice manpage:
EINVAL Target file system doesn't support splicing; target file is
opened in append mode; neither of the descriptors refers to a
pipe; or offset given for non-seekable device.
We know one of the descriptors is a pipe, and the file's not open in append mode. We also know no offset is given (0 is equivalent to NULL - did you mean to pass in a pointer to a zero offset?), so that's not the problem. Therefore, the filesystem you're using doesn't support splicing to files.
What kind of file system(s) are you copying to/from?
Your example runs on my system when both files are on ext3 but fails when I use an external drive (I forget offhand if it is DOS or NTFS). My guess is that one or both of your files are on a file system that splice does not support.
The splice(2) system call is for copying between files and pipes and not between files, so it can not be used to copy between files, as has been pointed out by the other answers.
As of Linux 4.5 however a new copy_file_range(2) system call is available that can copy between files. In the case of NFS it can even cause server side copying.
The linked man page contains a full example program.