Is there a Windows equivalent to fdopen for HANDLEs? - c

In Unix, if you have a file descriptor (e.g. from a socket, pipe, or inherited from your parent process), you can open a buffered I/O FILE* stream on it with fdopen(3).
Is there an equivalent on Windows for HANDLEs? If you have a HANDLE that was inherited from your parent process (different from stdin, stdout, or stderr) or a pipe from CreatePipe, is it possible to get a buffered FILE* stream from it? MSDN does document _fdopen, but that works with integer file descriptors returned by _open, not generic HANDLEs.

Unfortunately, HANDLEs are completely different beasts from FILE*s and file descriptors. The CRT ultimately handles files in terms of HANDLEs and associates those HANDLEs to a file descriptor. Those file descriptors in turn backs the structure pointer by FILE*.
Fortunately, there is a section on this MSDN page that describes functions that "provide a way to change the representation of the file between a FILE structure, a file descriptor, and a Win32 file handle":
_fdopen, _wfdopen: Associates a stream with a file that was
previously opened for low-level I/O and returns a pointer to the open
stream.
_fileno: Gets the file descriptor associated with a stream.
_get_osfhandle: Return operating-system file handle associated
with existing C run-time file descriptor
_open_osfhandle: Associates C run-time file descriptor with an
existing operating-system file handle.
Looks like what you need is _open_osfhandle followed by _fdopen to obtain a FILE* from a HANDLE.
Here's an example involving HANDLEs obtained from CreateFile(). When I tested it, it shows the first 255 characters of the file "test.txt" and appends " --- Hello World! --- " at the end of the file:
#include <windows.h>
#include <io.h>
#include <fcntl.h>
#include <cstdio>
int main()
{
HANDLE h = CreateFile("test.txt", GENERIC_READ | GENERIC_WRITE, 0, 0,
OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, 0);
if(h != INVALID_HANDLE_VALUE)
{
int fd = _open_osfhandle((intptr_t)h, _O_APPEND | _O_RDONLY);
if(fd != -1)
{
FILE* f = _fdopen(fd, "a+");
if(f != 0)
{
char rbuffer[256];
memset(rbuffer, 0, 256);
fread(rbuffer, 1, 255, f);
printf("read: %s\n", rbuffer);
fseek(f, 0, SEEK_CUR); // Switch from read to write
const char* wbuffer = " --- Hello World! --- \n";
fwrite(wbuffer, 1, strlen(wbuffer), f);
fclose(f); // Also calls _close()
}
else
{
_close(fd); // Also calls CloseHandle()
}
}
else
{
CloseHandle(h);
}
}
}
This should work for pipes as well.

Here is a more elegant way of doing this instead of CreateFile: specify "N" in fopen(). It's a Microsoft-specific extension to fopen, but since this code is platform-specific anyway, it's ok. When called with "N", fopen adds _O_NOINHERIT flag when calling _open internally.
Based on this:
Windows C Run-Time _close(fd) not closing file

Related

C File descriptor duplication without sharing offset or flags

I need to concurrently read from a file in different offsets using C.
dup unforunately creates a file descriptor that shares offset and flags with the original.
Is there a function like dup that does not share the offset and flags?
EDIT I only have access to the file pointer FILE* fp; I do not have the file path
EDIT This program is compiled for windows in addition to mac and many flavors of linux
SOLUTION
We can use pread on posix systems, and I wrote a pread function for windows which solves this problem
https://github.com/Storj/libstorj/blob/master/src/utils.c#L227
On Linux, you can recover the filename from /proc/self/fd/N, where N is the integral value of the file descriptor:
sprintf( linkname, "/proc/self/fd/%d", fd );
Then use readlink() on the resulting link name.
If the file has been renamed or deleted, you may be out of luck.
But why do you need another file descriptor? You can use pread() and/or pwrite() on the original file descriptor to read/write from/to the file without affecting the current offset. (caveat: on Linux, pwrite() to a file opened in append mode is buggy - POSIX states that pwrite() to a file opened in append mode will write to the offset specified in the pwrite() call, but the Linux pwrite() implementation is broken and will ignore the offset and append the data to the end of the file - see the BUGS section of the Linux man page)
No, neither C nor POSIX (since you mention dup()) has a function for opening a new, independent file handle based on an existing file handle. As you observed, you can dup() a file descriptor, but the result refers to the same underlying open file description.
To get an independent handle, you need to open() or fopen() the same path (which is possible only if the FILE refers to an object accessible through the file system). If you don't know what path that is, or if there isn't any in the first place, then you'll need a different approach.
Some alternatives to consider:
buffer some or all of the file contents in memory, and read as needed from the buffer to serve your needs for independent file offsets;
build an internal equivalent of the tee command; this will probably require a second thread, and you'll probably not be able to read one file too far ahead of the other, or to seek in either one;
copy the file contents to a temp file with a known name, and open that as many times as you want;
if the FILE corresponds to a regular file, map it into memory and access its contents there. The POSIX function fmemopen() could be useful in this case to adapt the memory mapping to your existing stream-based usage.
On windows (assuming VisualStudio), you can get access to the OS file handle from the stdio FILE handle.
From there, reopen it and convert back to a new FILE handle.
This is windows only, but I think Andrews answer will work for Linux and probably the Mac as well - unfortunately there is no portable way to have it work on all systems.
#include <Windows.h>
#include <fcntl.h>
#include <io.h>
#include <stdio.h>
FILE *jreopen(FILE* f)
{
int n = _fileno(f);
HANDLE h = (HANDLE)_get_osfhandle(n);
HANDLE h2 = ReOpenFile(h, GENERIC_READ, FILE_SHARE_READ, 0);
int n2 = _open_osfhandle((intptr_t)h2, _O_RDONLY);
FILE* g = _fdopen(n2, "r");
return g;
}
I was able to use pread and pwrite on POSIX systems, and I wrapped ReadFile/WriteFile on Windows Systems into pread and pwrite functions
#ifdef _WIN32
ssize_t pread(int fd, void *buf, size_t count, uint64_t offset)
{
long unsigned int read_bytes = 0;
OVERLAPPED overlapped;
memset(&overlapped, 0, sizeof(OVERLAPPED));
overlapped.OffsetHigh = (uint32_t)((offset & 0xFFFFFFFF00000000LL) >> 32);
overlapped.Offset = (uint32_t)(offset & 0xFFFFFFFFLL);
HANDLE file = (HANDLE)_get_osfhandle(fd);
SetLastError(0);
bool RF = ReadFile(file, buf, count, &read_bytes, &overlapped);
// For some reason it errors when it hits end of file so we don't want to check that
if ((RF == 0) && GetLastError() != ERROR_HANDLE_EOF) {
errno = GetLastError();
// printf ("Error reading file : %d\n", GetLastError());
return -1;
}
return read_bytes;
}
ssize_t pwrite(int fd, const void *buf, size_t count, uint64_t offset)
{
long unsigned int written_bytes = 0;
OVERLAPPED overlapped;
memset(&overlapped, 0, sizeof(OVERLAPPED));
overlapped.OffsetHigh = (uint32_t)((offset & 0xFFFFFFFF00000000LL) >> 32);
overlapped.Offset = (uint32_t)(offset & 0xFFFFFFFFLL);
HANDLE file = (HANDLE)_get_osfhandle(fd);
SetLastError(0);
bool RF = WriteFile(file, buf, count, &written_bytes, &overlapped);
if ((RF == 0)) {
errno = GetLastError();
// printf ("Error reading file :%d\n", GetLastError());
return -1;
}
return written_bytes;
}
#endif

Create GNU C File Descriptor Without File Handle

If I want to use a physical file along with other types of streams such as a socket, I can simply convert a file handle into a file descriptor:
#include <stdlib.h>
#include <stdio.h>
int main(void) {
FILE *f = fopen("uniquefilename.ext", "w");
int fd = fileno(f);
printf("%d\n", fd);
fclose(f);
return 0;
}
Does the GNU Standard Library provide a way to obtain a physical file's descriptor directly? Something to the effect of:
int fd = some_call("file_name.ext", "mode");
It seems I need to note I am completely aware of how a descriptor is not implicitly bound to any specific file. I was misleading when I wrote "obtain a physical file's descriptor"; what I should have wrote is something like "create a descriptor enabling access to a specific physical file".
It does not.
However, you can use the open function directly! This is part of Linux itself, not the C standard library (technically the C standard library provides a small wrapper to allow you to call it as a C function).
Example usage:
int fd = open("file_name.ext", O_RDWR); // not fopen
// do stuff with fd
close(fd); // not fclose
Note: The man page recommends including <sys/types.h>, <sys/stat.h>, and <fcntl.h>, and for close you need <unistd.h>. That's quite a few headers, and I don't know if they're all necessary.

Why does the calling of dup2 go wrong?

As you see, the program has two file pointer sport and fruit point to the file fruit.txt. The problem is that after run the program, sport.txt is empty and fruit.txt contains Chinese characters. I expected that the sport.txt should contains the word "basketball" because it is written to the file before redirecting happens. So, what is wrong here?
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <string.h>
#include "../cus_header/cus_header.h"
int main(){
FILE *fruit = fopen("fruit.txt", "w");
if(!fruit)
error("cannot open fruit.txt");
FILE *sport = fopen("sport.txt", "w");
if(!sport)
error("cannot open sport.txt");
int de_sport = fileno(sport);
int de_fruit = fileno(fruit);
printf("file number of sport.txt: %i and of fruit.txt: %i\n", de_sport, de_fruit);
fwrite("basketball", sizeof(char), 10, sport);
fwrite("apple", sizeof(char), 6, fruit);
if(dup2(de_fruit, de_sport) == -1)
error("cannot redirect");
fwrite("basketball", sizeof(char), 10, sport); //???
fwrite("apple", sizeof(char), 6, fruit); // ???
fclose(sport);
fclose(fruit);
return 0;
}
As the comments already mention, you shouldn't mix file manipulation with streams (using FILE*, fopen, fwrite, fclose) with raw file manipulation (using file descriptors, open, write, close, dup2). And especially don't mix them on the same file pointer/descriptor like you are doing in this piece of code.
Let's go through the code to see why it behaves the way it does:
FILE *fruit = fopen("fruit.txt", "w");
...
FILE *sport = fopen("sport.txt", "w");
You shouldn't care about how the FILE structure looks like, let's just suppose it keeps the underlying file descriptor somewhere.
int de_sport = fileno(sport);
int de_fruit = fileno(fruit);
You create local variables holding the same file descriptors as the two FILE* refer.
fwrite("basketball", sizeof(char), 10, sport);
fwrite("apple", sizeof(char), 6, fruit);
You write something in each of the two files. Because C file streams are buffered by default, the actual writing in the file on disk might not happen right away (and in your case it doesn't).
dup2(de_fruit, de_sport)
This closes the file descriptor de_sport and makes it refer to the same file as de_fruit. The actual numerical values remain the same, only the actual files that they refer to are changed. This means that the two FILE handles will write to the same file after the dup2 call.
fwrite("basketball", sizeof(char), 10, sport); //???
fwrite("apple", sizeof(char), 6, fruit); // ???
This will write to the same underlying file because the two descriptors now refer to the same file. But again, because streams are buffered, this might actually just append to the buffers of those two FILE*s.
fclose(sport);
fclose(fruit);
This flushes the buffers, so the actual writing to disk happens here. Because the descriptors have been changed, if no flushing happened until now, both streams will actually flush to the same file on disk.
This is probably why you're seeing that behavior, but keep in mind that what you're doing is not safe and that the behavior or file contents might differ.

dup() and cache flush

I am a C beginner, trying to use dup(), I wrote a program to test this function, the result is a little different from what I expected.
Code:
// unistd.h, dup() test
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
extern void dup_test();
int main() {
dup_test();
}
// dup()test
void dup_test() {
// open a file
FILE *f = fopen("/tmp/a.txt", "w+");
int fd = fileno(f);
printf("original file descriptor:\t%d\n",fd);
// duplicate file descriptor of an opened file,
int fd_dup = dup(fd);
printf("duplicated file descriptor:\t%d\n",fd_dup);
FILE *f_dup = fdopen(fd_dup, "w+");
// write to file, use the duplicated file descriptor,
fputs("hello\n", f_dup);
fflush(f_dup);
// close duplicated file descriptor,
fclose(f_dup);
close(fd_dup);
// allocate memory
int maxSize = 1024; // 1 kb
char *buf = malloc(maxSize);
// move to beginning of file,
rewind(f);
// read from file, use the original file descriptor,
fgets(buf, maxSize, f);
printf("%s", buf);
// close original file descriptor,
fclose(f);
// free memory
free(buf);
}
The program try write via the duplicated fd, then close the duplicated fd, then try to read via the original fd.
I expected that when I close the duplicated fd, the io cache will be flushed automatically, but it's not, if I remove the fflush() function in the code, the original fd won't be able to read the content written by the duplicated fd which is already closed.
My question is:
Does this means when close the duplicated fd, it won't do flush automatically?
#Edit:
I am sorry, my mistake, I found the reason, in my initial program it has:
close(fd_dup);
but don't have:
fclose(f_dup);
after use fclose(f_dup); to replace close(f_dup); it works.
So, the duplicated fd do automatically flush if close in a proper way, write() & close() is a pair, fwrite() & fclose() is a pair, should not mix them.
Actually, in the code I could have use the duplicated fd_dup directly with write() & close(), and there is no need to create a new FILE at all.
So, the code could simply be:
// unistd.h, dup() test
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#define BUF_SIZE 1024 // 1 kb
extern void dup_test();
int main() {
dup_test();
}
// dup()test
void dup_test() {
// open a file
FILE *f = fopen("/tmp/a.txt", "w+");
int fd = fileno(f);
printf("original file descriptor:\t%d\n",fd);
// duplicate file descriptor of an opened file,
int fd_dup = dup(fd);
printf("duplicated file descriptor:\t%d\n",fd_dup);
// write to file, use the duplicated file descriptor,
write(fd_dup, "hello\n", BUF_SIZE);
// close duplicated file descriptor,
close(fd_dup);
// allocate memory
char *buf = malloc(BUF_SIZE);
// move to beginning of file,
rewind(f);
// read from file, use the original file descriptor,
fgets(buf, BUF_SIZE, f);
printf("%s", buf);
// close original file descriptor,
fclose(f);
// free memory
free(buf);
}
From dup man pages:
After a successful return from one of these system calls, the old and new file descriptors maybe used interchangeably. They refer to the same open file description (see open(2))and thus share file offset and file status flags; for example, if the file offset is modified by using lseek(2) on one of the descriptors, the offset is also changed for the other.
It means the seek pointer is changed when you write to the duplicated file descriptor, so, reading from the first file descriptor after writing to the duplication shouldn't read any data.
You are using fdopen to create separated seek_ptr and end_ptr of the duplicated stream, in that way, the fd_dup stops being a duplication. That's why you can read data after flushing and closing the stream.
I couldn't find any strong facts about why you can't read if you don't flush the second file descriptor. I can add that it may be related to sync system call.
After all, if you need a IO buffer, you might be using the wrong mechanism, check named pipes and other buffering OS mechanism.
I cannot really understand your problem. I tested it under Microsoft VC2008 (had to replace unistd.h with io.h) and gcc 4.2.1.
I commented out fflush(f_dup) because it is no use before a close and close(fd_dup); because the file descriptor was already closed, so the piece of code now looks like :
// write to file, use the duplicated file descriptor,
fputs("hello\n", f_dup);
// fflush(f_dup);
// close duplicated file descriptor,
fclose(f_dup);
// close(fd_dup);
And it works correctly. I get on both systems :
original file descriptor: 3
duplicated file descriptor: 4
hello

How to detect if a C filestream points to a file or a serial device?

I'm working on a C application that evaluates data from a USB laser scanner, which acts as a serial device. For testing purpose, I'm also allowing test data to be read from a file, because it is not convenient to always have the scanner connected.
I open the file/device like this:
FILE *fp = fopen(argv[1], "a+b");
And depending on whether I want to read from a file or the device, I pass a file path or something like /dev/cu.usbmodemfd121 (I'm on a Mac).
This works fine as long as I've previously initialized the laser scanner, but I'd rather have my application do that. In order to do that, though, I must first figure out if I'm reading from a file or the device. How can I do that, given the FILE * returned by fopen?
I've tried to use fseek(fp, 1, SEEK_END) which I expected to fail for the scanner, since it's stream doesn't have an "end", but for some reasons fseek does not fail..
You could get the file descriptor using fileno and then do a fstat on it. The struct stat it populates contains thinks like st_mode which shows the type of fd. I am guessing for your non-file device S_ISCHR will be true or at least S_ISREG will be false.
If you have control over it, don't do fopen at all. Use open directly to get the file descriptor and then use fdopen if you really want C streams.
#cnicutar's solution worked just fine. Here's what I ended up with, in case it helps somebody (error checking removed for clarity):
#include <fcntl.h> /* open syscall */
#include <sys/stat.h>
int fd = -1;
int status;
FILE *fp = NULL;
struct stat fd_stat;
bool serial_device = false;
fd = open("/foo/bar/baz", O_RDONLY);
fp = fdopen(fd, "rb");
status = fstat(fd, &fd_stat);
printf("S_ISCHR: %d\n", S_ISCHR(fd_stat.st_mode));
printf("S_ISREG %d\n", S_ISREG(fd_stat.st_mode));
if(!S_ISREG(fd_stat.st_mode)) {
serial_device = true;
}

Resources