dup() and cache flush - c

I am a C beginner, trying to use dup(), I wrote a program to test this function, the result is a little different from what I expected.
Code:
// unistd.h, dup() test
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
extern void dup_test();
int main() {
dup_test();
}
// dup()test
void dup_test() {
// open a file
FILE *f = fopen("/tmp/a.txt", "w+");
int fd = fileno(f);
printf("original file descriptor:\t%d\n",fd);
// duplicate file descriptor of an opened file,
int fd_dup = dup(fd);
printf("duplicated file descriptor:\t%d\n",fd_dup);
FILE *f_dup = fdopen(fd_dup, "w+");
// write to file, use the duplicated file descriptor,
fputs("hello\n", f_dup);
fflush(f_dup);
// close duplicated file descriptor,
fclose(f_dup);
close(fd_dup);
// allocate memory
int maxSize = 1024; // 1 kb
char *buf = malloc(maxSize);
// move to beginning of file,
rewind(f);
// read from file, use the original file descriptor,
fgets(buf, maxSize, f);
printf("%s", buf);
// close original file descriptor,
fclose(f);
// free memory
free(buf);
}
The program try write via the duplicated fd, then close the duplicated fd, then try to read via the original fd.
I expected that when I close the duplicated fd, the io cache will be flushed automatically, but it's not, if I remove the fflush() function in the code, the original fd won't be able to read the content written by the duplicated fd which is already closed.
My question is:
Does this means when close the duplicated fd, it won't do flush automatically?
#Edit:
I am sorry, my mistake, I found the reason, in my initial program it has:
close(fd_dup);
but don't have:
fclose(f_dup);
after use fclose(f_dup); to replace close(f_dup); it works.
So, the duplicated fd do automatically flush if close in a proper way, write() & close() is a pair, fwrite() & fclose() is a pair, should not mix them.
Actually, in the code I could have use the duplicated fd_dup directly with write() & close(), and there is no need to create a new FILE at all.
So, the code could simply be:
// unistd.h, dup() test
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#define BUF_SIZE 1024 // 1 kb
extern void dup_test();
int main() {
dup_test();
}
// dup()test
void dup_test() {
// open a file
FILE *f = fopen("/tmp/a.txt", "w+");
int fd = fileno(f);
printf("original file descriptor:\t%d\n",fd);
// duplicate file descriptor of an opened file,
int fd_dup = dup(fd);
printf("duplicated file descriptor:\t%d\n",fd_dup);
// write to file, use the duplicated file descriptor,
write(fd_dup, "hello\n", BUF_SIZE);
// close duplicated file descriptor,
close(fd_dup);
// allocate memory
char *buf = malloc(BUF_SIZE);
// move to beginning of file,
rewind(f);
// read from file, use the original file descriptor,
fgets(buf, BUF_SIZE, f);
printf("%s", buf);
// close original file descriptor,
fclose(f);
// free memory
free(buf);
}

From dup man pages:
After a successful return from one of these system calls, the old and new file descriptors maybe used interchangeably. They refer to the same open file description (see open(2))and thus share file offset and file status flags; for example, if the file offset is modified by using lseek(2) on one of the descriptors, the offset is also changed for the other.
It means the seek pointer is changed when you write to the duplicated file descriptor, so, reading from the first file descriptor after writing to the duplication shouldn't read any data.
You are using fdopen to create separated seek_ptr and end_ptr of the duplicated stream, in that way, the fd_dup stops being a duplication. That's why you can read data after flushing and closing the stream.
I couldn't find any strong facts about why you can't read if you don't flush the second file descriptor. I can add that it may be related to sync system call.
After all, if you need a IO buffer, you might be using the wrong mechanism, check named pipes and other buffering OS mechanism.

I cannot really understand your problem. I tested it under Microsoft VC2008 (had to replace unistd.h with io.h) and gcc 4.2.1.
I commented out fflush(f_dup) because it is no use before a close and close(fd_dup); because the file descriptor was already closed, so the piece of code now looks like :
// write to file, use the duplicated file descriptor,
fputs("hello\n", f_dup);
// fflush(f_dup);
// close duplicated file descriptor,
fclose(f_dup);
// close(fd_dup);
And it works correctly. I get on both systems :
original file descriptor: 3
duplicated file descriptor: 4
hello

Related

How read remember the last offset of file?

How read function know the next position to read from a file.
or How can I manage to made a function that can remember last offset of file even after open another file a changing it's file descriptor.
Is there is a way to know that a file descriptor is already opened and pointed to a file?
like this:
int main()
{
int fd;
char *file;
file = (char *)malloc(sizeof(char) * 32);
fd = open("file.txt", O_RDONLY);
read_file(fd, *file); /* reading the first line from file.txt */
fd = open("file1.txt", O_RDONLY);
read_file(fd, *file); /* reading the first line from file1.txt */
fd = open("file.txt", O_RDONLY);
read_file(fd, *file); /* Now it should read the second line from file file.txt, how can I manage to do that*/
close(fd);
return (0);
}
The current location in the file is maintained by the kernel I think, the file descriptor serves as the key to all the information associated with the open file.
If you need to open and read from two files at the same time, they should of course not share the file descriptor. Just use two, one per file.
const int fd1 = open("file.txt", O_RDONLY);
const int fd2 = open("file1.txt", O_RDONLY);
The treatment of char *file in your code makes no sense, but at this point you can mix accesses to fd1 and fd2.
Remember to close the files when you're done:
close(fd2);
close(fd1);
In real code you would also check that the open-calls succeeded, before trying to do I/O from the file(s), of course.
Is there a way to know that a file descriptor is already opened and
pointed to a file?
If you can lseek(fd, 0, SEEK_CUR) successfully, that means that fd is opened and seekable (so probably a file, but remember that "file" includes directories and device files as well as regular files).
If it returns (off_t)-1 and errno==EBADF then the descriptor is not open; if returns (off_t)-1 and errno==ESPIPE, then it's a pipe, socket, or FIFO.

Why does the calling of dup2 go wrong?

As you see, the program has two file pointer sport and fruit point to the file fruit.txt. The problem is that after run the program, sport.txt is empty and fruit.txt contains Chinese characters. I expected that the sport.txt should contains the word "basketball" because it is written to the file before redirecting happens. So, what is wrong here?
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <string.h>
#include "../cus_header/cus_header.h"
int main(){
FILE *fruit = fopen("fruit.txt", "w");
if(!fruit)
error("cannot open fruit.txt");
FILE *sport = fopen("sport.txt", "w");
if(!sport)
error("cannot open sport.txt");
int de_sport = fileno(sport);
int de_fruit = fileno(fruit);
printf("file number of sport.txt: %i and of fruit.txt: %i\n", de_sport, de_fruit);
fwrite("basketball", sizeof(char), 10, sport);
fwrite("apple", sizeof(char), 6, fruit);
if(dup2(de_fruit, de_sport) == -1)
error("cannot redirect");
fwrite("basketball", sizeof(char), 10, sport); //???
fwrite("apple", sizeof(char), 6, fruit); // ???
fclose(sport);
fclose(fruit);
return 0;
}
As the comments already mention, you shouldn't mix file manipulation with streams (using FILE*, fopen, fwrite, fclose) with raw file manipulation (using file descriptors, open, write, close, dup2). And especially don't mix them on the same file pointer/descriptor like you are doing in this piece of code.
Let's go through the code to see why it behaves the way it does:
FILE *fruit = fopen("fruit.txt", "w");
...
FILE *sport = fopen("sport.txt", "w");
You shouldn't care about how the FILE structure looks like, let's just suppose it keeps the underlying file descriptor somewhere.
int de_sport = fileno(sport);
int de_fruit = fileno(fruit);
You create local variables holding the same file descriptors as the two FILE* refer.
fwrite("basketball", sizeof(char), 10, sport);
fwrite("apple", sizeof(char), 6, fruit);
You write something in each of the two files. Because C file streams are buffered by default, the actual writing in the file on disk might not happen right away (and in your case it doesn't).
dup2(de_fruit, de_sport)
This closes the file descriptor de_sport and makes it refer to the same file as de_fruit. The actual numerical values remain the same, only the actual files that they refer to are changed. This means that the two FILE handles will write to the same file after the dup2 call.
fwrite("basketball", sizeof(char), 10, sport); //???
fwrite("apple", sizeof(char), 6, fruit); // ???
This will write to the same underlying file because the two descriptors now refer to the same file. But again, because streams are buffered, this might actually just append to the buffers of those two FILE*s.
fclose(sport);
fclose(fruit);
This flushes the buffers, so the actual writing to disk happens here. Because the descriptors have been changed, if no flushing happened until now, both streams will actually flush to the same file on disk.
This is probably why you're seeing that behavior, but keep in mind that what you're doing is not safe and that the behavior or file contents might differ.

Percent code for file name in C [duplicate]

This question already has answers here:
Closed 12 years ago.
Possible Duplicate:
Getting Filename from file descriptor in C
Is there a simple and (reasonably) portable way of getting the filename from a FILE*?
I open a file using f = fopen(filename, ...) and then pass down f to various other functions, some of which may report an error. I'd like to report the filename in the error message but avoid having to pass around the extra parameter.
I could create a custom wrapper struct { FILE *f, const char *name }, but is there perhaps a simpler way? (If the FILE* wasn't opened using fopen I don't care about the result.)
On some platforms (such as Linux), you may be able to fetch it by reading the link of /proc/self/fd/<number>, as so:
#include <stdio.h>
#include <unistd.h>
#include <string.h>
int main(void)
{
char path[1024];
char result[1024];
/* Open a file, get the file descriptor. */
FILE *f = fopen("/etc/passwd", "r");
int fd = fileno(f);
/* Read out the link to our file descriptor. */
sprintf(path, "/proc/self/fd/%d", fd);
memset(result, 0, sizeof(result));
readlink(path, result, sizeof(result)-1);
/* Print the result. */
printf("%s\n", result);
}
This will, on my system, print out /etc/passwd, as desired.
It's a bit difficult, because a FILE* can read/write from a file handle which isn't associated with a named file at all (for example an unnamed pipe or a socket). You can obtain the file handle with fileno() and then there are system specific ways to learn about the file name. Here's a discussion on how to do this under Linux:
Getting Filename from file descriptor in C
and under Windows this isn't much easier either:
http://msdn.microsoft.com/en-us/library/aa366789(VS.85).aspx (as an extra step here, you use _get_osfhandle() to get the Windows file handle from the c-library file descriptor)

Is there a Windows equivalent to fdopen for HANDLEs?

In Unix, if you have a file descriptor (e.g. from a socket, pipe, or inherited from your parent process), you can open a buffered I/O FILE* stream on it with fdopen(3).
Is there an equivalent on Windows for HANDLEs? If you have a HANDLE that was inherited from your parent process (different from stdin, stdout, or stderr) or a pipe from CreatePipe, is it possible to get a buffered FILE* stream from it? MSDN does document _fdopen, but that works with integer file descriptors returned by _open, not generic HANDLEs.
Unfortunately, HANDLEs are completely different beasts from FILE*s and file descriptors. The CRT ultimately handles files in terms of HANDLEs and associates those HANDLEs to a file descriptor. Those file descriptors in turn backs the structure pointer by FILE*.
Fortunately, there is a section on this MSDN page that describes functions that "provide a way to change the representation of the file between a FILE structure, a file descriptor, and a Win32 file handle":
_fdopen, _wfdopen: Associates a stream with a file that was
previously opened for low-level I/O and returns a pointer to the open
stream.
_fileno: Gets the file descriptor associated with a stream.
_get_osfhandle: Return operating-system file handle associated
with existing C run-time file descriptor
_open_osfhandle: Associates C run-time file descriptor with an
existing operating-system file handle.
Looks like what you need is _open_osfhandle followed by _fdopen to obtain a FILE* from a HANDLE.
Here's an example involving HANDLEs obtained from CreateFile(). When I tested it, it shows the first 255 characters of the file "test.txt" and appends " --- Hello World! --- " at the end of the file:
#include <windows.h>
#include <io.h>
#include <fcntl.h>
#include <cstdio>
int main()
{
HANDLE h = CreateFile("test.txt", GENERIC_READ | GENERIC_WRITE, 0, 0,
OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, 0);
if(h != INVALID_HANDLE_VALUE)
{
int fd = _open_osfhandle((intptr_t)h, _O_APPEND | _O_RDONLY);
if(fd != -1)
{
FILE* f = _fdopen(fd, "a+");
if(f != 0)
{
char rbuffer[256];
memset(rbuffer, 0, 256);
fread(rbuffer, 1, 255, f);
printf("read: %s\n", rbuffer);
fseek(f, 0, SEEK_CUR); // Switch from read to write
const char* wbuffer = " --- Hello World! --- \n";
fwrite(wbuffer, 1, strlen(wbuffer), f);
fclose(f); // Also calls _close()
}
else
{
_close(fd); // Also calls CloseHandle()
}
}
else
{
CloseHandle(h);
}
}
}
This should work for pipes as well.
Here is a more elegant way of doing this instead of CreateFile: specify "N" in fopen(). It's a Microsoft-specific extension to fopen, but since this code is platform-specific anyway, it's ok. When called with "N", fopen adds _O_NOINHERIT flag when calling _open internally.
Based on this:
Windows C Run-Time _close(fd) not closing file

How can I use Linux's splice() function to copy a file to another file?

here's another question about splice(). I'm hoping to use it to copy files, and am trying to use two splice calls joined by a pipe like the example on splice's Wikipedia page. I wrote a simple test case which only tries to read the first 32K bytes from one file and write them to another:
#define _GNU_SOURCE
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
int main(int argc, char **argv) {
int pipefd[2];
int result;
FILE *in_file;
FILE *out_file;
result = pipe(pipefd);
in_file = fopen(argv[1], "rb");
out_file = fopen(argv[2], "wb");
result = splice(fileno(in_file), 0, pipefd[1], NULL, 32768, SPLICE_F_MORE | SPLICE_F_MOVE);
printf("%d\n", result);
result = splice(pipefd[0], NULL, fileno(out_file), 0, 32768, SPLICE_F_MORE | SPLICE_F_MOVE);
printf("%d\n", result);
if (result == -1)
printf("%d - %s\n", errno, strerror(errno));
close(pipefd[0]);
close(pipefd[1]);
fclose(in_file);
fclose(out_file);
return 0;
}
When I run this, the input file seems to be read properly, but the second splice call fails with EINVAL. Anybody know what I'm doing wrong here?
Thanks!
From the splice manpage:
EINVAL Target file system doesn't support splicing; target file is
opened in append mode; neither of the descriptors refers to a
pipe; or offset given for non-seekable device.
We know one of the descriptors is a pipe, and the file's not open in append mode. We also know no offset is given (0 is equivalent to NULL - did you mean to pass in a pointer to a zero offset?), so that's not the problem. Therefore, the filesystem you're using doesn't support splicing to files.
What kind of file system(s) are you copying to/from?
Your example runs on my system when both files are on ext3 but fails when I use an external drive (I forget offhand if it is DOS or NTFS). My guess is that one or both of your files are on a file system that splice does not support.
The splice(2) system call is for copying between files and pipes and not between files, so it can not be used to copy between files, as has been pointed out by the other answers.
As of Linux 4.5 however a new copy_file_range(2) system call is available that can copy between files. In the case of NFS it can even cause server side copying.
The linked man page contains a full example program.

Resources