As you see, the program has two file pointer sport and fruit point to the file fruit.txt. The problem is that after run the program, sport.txt is empty and fruit.txt contains Chinese characters. I expected that the sport.txt should contains the word "basketball" because it is written to the file before redirecting happens. So, what is wrong here?
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <string.h>
#include "../cus_header/cus_header.h"
int main(){
FILE *fruit = fopen("fruit.txt", "w");
if(!fruit)
error("cannot open fruit.txt");
FILE *sport = fopen("sport.txt", "w");
if(!sport)
error("cannot open sport.txt");
int de_sport = fileno(sport);
int de_fruit = fileno(fruit);
printf("file number of sport.txt: %i and of fruit.txt: %i\n", de_sport, de_fruit);
fwrite("basketball", sizeof(char), 10, sport);
fwrite("apple", sizeof(char), 6, fruit);
if(dup2(de_fruit, de_sport) == -1)
error("cannot redirect");
fwrite("basketball", sizeof(char), 10, sport); //???
fwrite("apple", sizeof(char), 6, fruit); // ???
fclose(sport);
fclose(fruit);
return 0;
}
As the comments already mention, you shouldn't mix file manipulation with streams (using FILE*, fopen, fwrite, fclose) with raw file manipulation (using file descriptors, open, write, close, dup2). And especially don't mix them on the same file pointer/descriptor like you are doing in this piece of code.
Let's go through the code to see why it behaves the way it does:
FILE *fruit = fopen("fruit.txt", "w");
...
FILE *sport = fopen("sport.txt", "w");
You shouldn't care about how the FILE structure looks like, let's just suppose it keeps the underlying file descriptor somewhere.
int de_sport = fileno(sport);
int de_fruit = fileno(fruit);
You create local variables holding the same file descriptors as the two FILE* refer.
fwrite("basketball", sizeof(char), 10, sport);
fwrite("apple", sizeof(char), 6, fruit);
You write something in each of the two files. Because C file streams are buffered by default, the actual writing in the file on disk might not happen right away (and in your case it doesn't).
dup2(de_fruit, de_sport)
This closes the file descriptor de_sport and makes it refer to the same file as de_fruit. The actual numerical values remain the same, only the actual files that they refer to are changed. This means that the two FILE handles will write to the same file after the dup2 call.
fwrite("basketball", sizeof(char), 10, sport); //???
fwrite("apple", sizeof(char), 6, fruit); // ???
This will write to the same underlying file because the two descriptors now refer to the same file. But again, because streams are buffered, this might actually just append to the buffers of those two FILE*s.
fclose(sport);
fclose(fruit);
This flushes the buffers, so the actual writing to disk happens here. Because the descriptors have been changed, if no flushing happened until now, both streams will actually flush to the same file on disk.
This is probably why you're seeing that behavior, but keep in mind that what you're doing is not safe and that the behavior or file contents might differ.
Related
I'm currently developing a program that mimics UNIX file system. I've prepared my disk as file (1 MB) got all data blocks inside it. Now what I'm doing is implementing some simple commands like mkdir, ls etc. In order to work with those commands, I need to read specific offset(no problem with that) and write the modified blocks to specific location.
Simply my goal is:
SIIIDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD (Current Disk)
I wan't to change three blocks with AAA after 16.byte so it will be like:
SIIIDDDDDDDDDDDDAAADDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD (Modified Disk)
I'm not going to provide all of my implementation here I just want to have some ideas about it how can I implement it without buffering all the 1 MB data in my program. In short I know locations of my data blocks so I just want to replace that part of my file not whole file. Can't I simply do this with file stream functions ?
Another example:
fseek(from_disk,superblock.i_node_bit_map_starting_addr , SEEK_SET); //seek to known offset.
read_bit_map(&from_disk); // I can read at specific location without problem
... manipulate bit map ...
fseek(to_disk,superblock.i_node_bit_map_starting_addr , SEEK_SET); //seek to known offset.
write_bit_map(&to_disk); //Write back the data.
//This will destroy the current data of file. (Tried with w+, a modes.)
Note: Not provided in example but I have two file pointers both writing and reading and I'm aware I need to close one before opening another.
I think you are looking for the r+ (potentially rb+ mode). Here is a complete example, afterwards you can run grep -n hello data.txt to verify for yourself the result. You can run it with make prog && ./prog.
#include <stdio.h>
#include <unistd.h>
#include <string.h>
int main(int argc, char const *argv[])
{
FILE *file;
file = fopen("data.txt", "w+");
char dummy_data[] = "This is stackoverflow.com\n";
int dummy_data_length = strlen(dummy_data);
for (int i = 0; i < 1000; ++i)
fwrite(dummy_data, dummy_data_length, 1, file);
fclose(file);
file = fopen("data.txt", "r+");
fseek(file, 500, SEEK_CUR);
fwrite("hello", 5, 1, file);
fclose(file);
return 0;
}
I'm basically trying to create a simple program that simulates a screen whose value is generated randomly and are outputted to a bmp file. However, there are some trouble in the file operation.. when the fwrite is called, the file stays at zero bytes, meaning nothing was written to that file.. I've tried changing the fopen modes (e.g "w+b", "wb+", "wb") but still no luck.. here is the code anyway
char resultFileName[BUFSIZ];
char currentDirectory[BUFSIZ];
//char generatedFileFolder[] = "\\generatedFile";
FILE *resultFile;
getcwd(currentDirectory, sizeof(currentDirectory));
snprintf(resultFileName, sizeof(resultFileName), "%s%s", currentDirectory, "\\result.bmp");
resultFile = fopen(resultFileName, "w+b");
BITMAPFILEHEADER bmfh;
BITMAPINFOHEADER bmih;
fwrite(&bmfh, sizeof(bmfh), 1, resultFile);
fwrite(&bmih, sizeof(bmih), 1, resultFile);
what am I doing wrong here?
You need to call fclose.
The I/O functions starting with f is buffered, which means they only write periodically (when reaching the maximum buffer size), or when you close it.
Since C is not a garbage-collected language (or language with the concept of destructor), your buffers are not flushed unless you flush it or close the file.
I am a C beginner, trying to use dup(), I wrote a program to test this function, the result is a little different from what I expected.
Code:
// unistd.h, dup() test
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
extern void dup_test();
int main() {
dup_test();
}
// dup()test
void dup_test() {
// open a file
FILE *f = fopen("/tmp/a.txt", "w+");
int fd = fileno(f);
printf("original file descriptor:\t%d\n",fd);
// duplicate file descriptor of an opened file,
int fd_dup = dup(fd);
printf("duplicated file descriptor:\t%d\n",fd_dup);
FILE *f_dup = fdopen(fd_dup, "w+");
// write to file, use the duplicated file descriptor,
fputs("hello\n", f_dup);
fflush(f_dup);
// close duplicated file descriptor,
fclose(f_dup);
close(fd_dup);
// allocate memory
int maxSize = 1024; // 1 kb
char *buf = malloc(maxSize);
// move to beginning of file,
rewind(f);
// read from file, use the original file descriptor,
fgets(buf, maxSize, f);
printf("%s", buf);
// close original file descriptor,
fclose(f);
// free memory
free(buf);
}
The program try write via the duplicated fd, then close the duplicated fd, then try to read via the original fd.
I expected that when I close the duplicated fd, the io cache will be flushed automatically, but it's not, if I remove the fflush() function in the code, the original fd won't be able to read the content written by the duplicated fd which is already closed.
My question is:
Does this means when close the duplicated fd, it won't do flush automatically?
#Edit:
I am sorry, my mistake, I found the reason, in my initial program it has:
close(fd_dup);
but don't have:
fclose(f_dup);
after use fclose(f_dup); to replace close(f_dup); it works.
So, the duplicated fd do automatically flush if close in a proper way, write() & close() is a pair, fwrite() & fclose() is a pair, should not mix them.
Actually, in the code I could have use the duplicated fd_dup directly with write() & close(), and there is no need to create a new FILE at all.
So, the code could simply be:
// unistd.h, dup() test
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#define BUF_SIZE 1024 // 1 kb
extern void dup_test();
int main() {
dup_test();
}
// dup()test
void dup_test() {
// open a file
FILE *f = fopen("/tmp/a.txt", "w+");
int fd = fileno(f);
printf("original file descriptor:\t%d\n",fd);
// duplicate file descriptor of an opened file,
int fd_dup = dup(fd);
printf("duplicated file descriptor:\t%d\n",fd_dup);
// write to file, use the duplicated file descriptor,
write(fd_dup, "hello\n", BUF_SIZE);
// close duplicated file descriptor,
close(fd_dup);
// allocate memory
char *buf = malloc(BUF_SIZE);
// move to beginning of file,
rewind(f);
// read from file, use the original file descriptor,
fgets(buf, BUF_SIZE, f);
printf("%s", buf);
// close original file descriptor,
fclose(f);
// free memory
free(buf);
}
From dup man pages:
After a successful return from one of these system calls, the old and new file descriptors maybe used interchangeably. They refer to the same open file description (see open(2))and thus share file offset and file status flags; for example, if the file offset is modified by using lseek(2) on one of the descriptors, the offset is also changed for the other.
It means the seek pointer is changed when you write to the duplicated file descriptor, so, reading from the first file descriptor after writing to the duplication shouldn't read any data.
You are using fdopen to create separated seek_ptr and end_ptr of the duplicated stream, in that way, the fd_dup stops being a duplication. That's why you can read data after flushing and closing the stream.
I couldn't find any strong facts about why you can't read if you don't flush the second file descriptor. I can add that it may be related to sync system call.
After all, if you need a IO buffer, you might be using the wrong mechanism, check named pipes and other buffering OS mechanism.
I cannot really understand your problem. I tested it under Microsoft VC2008 (had to replace unistd.h with io.h) and gcc 4.2.1.
I commented out fflush(f_dup) because it is no use before a close and close(fd_dup); because the file descriptor was already closed, so the piece of code now looks like :
// write to file, use the duplicated file descriptor,
fputs("hello\n", f_dup);
// fflush(f_dup);
// close duplicated file descriptor,
fclose(f_dup);
// close(fd_dup);
And it works correctly. I get on both systems :
original file descriptor: 3
duplicated file descriptor: 4
hello
#include <stdio.h>
int main() {
printf("This goes to screen\n");
freopen("out.txt", "a", stdout);
printf("This goes to out.txt");
freopen("/dev/stdout", "a", stdout);
printf("This should go to screen too, but doesn't\n");
return 0;
}
I call freopen to redirect the stdout to out.txt then I print something on the file, now I want to redirect it back to the screen, but freopen("/dev/stdout", "a", stdout); doesn't work. Is there any way to do that using ANSI C or POSIX system calls?
I can't think of a way to do this in a cross-platform manner, but on GNU/Linux systems (and maybe other POSIX-compliant ones, too) you can freopen ("/dev/tty", "a", stdout). Is this what you were trying to do?
Unfortunately, there doesn't seem to be a good way:
http://c-faq.com/stdio/undofreopen.html
The best recommendation is not to use freopen in this circumstance.
Generally speaking, you can't. You have closed the file, which could've been pipe or whatever. It's not reopenable. You might have saved stdout value, then assign some fopen to it and then close it and copy the old value back. Example:
FILE *o = stdout;
stdout=fopen("/tmp/crap.txt","a");
printf("Oh no!\n");
fclose(stdout);
stdout = o;
Mike Weller suggested below in comments that stdout might not always be writable. In this case something like that might help:
int o = dup(fileno(stdout));
freopen("/tmp/crap.txt","a",stdout);
printf("Oh no!\n");
dup2(o,fileno(stdout));
close(o);
Another edit: if you're using it to redirect output from the child process like your comment elsewhere suggest, you can redirect it after the fork.
Use fdopen() and dup() as well as freopen().
int old_stdout = dup(1); // Preserve original file descriptor for stdout.
FILE *fp1 = freopen("out.txt", "w", stdout); // Open new stdout
...write to stdout... // Use new stdout
FILE *fp2 = fdopen(old_stdout, "w"); // Open old stdout as a stream
...Now, how to get stdout to refer to fp2?
...Under glibc, I believe you can use:
fclose(stdout); // Equivalent to fclose(fp1);
stdout = fp2; // Assign fp2 to stdout
// *stdout = *fp2; // Works on Solaris and MacOS X, might work elsewhere.
close(old_stdout); // Close the file descriptor so pipes work sanely
I'm not sure whether you can do the assignment reliably elsewhere.
Dubious code that does actually work
The code below worked on Solaris 10 and MacOS X 10.6.2 - but I'm not confident that it is reliable. The structure assignment may or may not work with Linux glibc.
#include <stdio.h>
#include <unistd.h>
int main(void)
{
printf("This goes to screen\n");
int old_stdout = dup(1); // Consider dup(STDOUT_FILENO) or dup(fileno(stdout))
FILE *fp1 = freopen("out.txt", "a", stdout);
printf("This goes to out.txt\n");
fclose(stdout);
FILE *fp2 = fdopen(old_stdout, "w");
*stdout = *fp2; // Unreliable!
printf("This should go to screen too, but doesn't\n");
return 0;
}
You can't say you weren't warned — this is playing with fire!
If you're on a system with the /dev/fd file system, you could create the name of the file implied by the file descriptor returned from dup() with sprintf(buffer, "/dev/fd/%d", old_stdout) and then use freopen() with that name. This would be a lot more reliable than the assignment used in this code.
The better solutions either make the code use 'fprintf(fp, ...)' everywhere, or use a cover function that allows you set your own default file pointer:
mprintf.c
#include "mprintf.h"
#include <stdarg.h>
static FILE *default_fp = 0;
void set_default_stream(FILE *fp)
{
default_fp = fp;
}
int mprintf(const char *fmt, ...)
{
va_list args;
va_start(args, fmt);
if (default_fp == 0)
default_fp = stdout;
int rv = vfprintf(default_fp, fmt, args);
va_end(args);
return(rv);
}
mprintf.h
#ifndef MPRINTF_H_INCLUDED
#define MPRINTF_H_INCLUDED
#include <stdio.h>
extern void set_default_stream(FILE *fp);
extern int mprintf(const char *fmt, ...);
#endif
Clearly, you can create an mvprintf() and other functions as needed.
Example use of mprintf()
Then, in place of the original code, you can use:
#include "mprintf.h"
int main()
{
mprintf("This goes to screen\n");
FILE *fp1 = fopen("out.txt", "w");
set_default_stream(fp1);
mprintf("This goes to out.txt\n");
fclose(fp1);
set_default_stream(stdout);
mprintf("This should go to screen too, but doesn't\n");
return 0;
}
(Warning: untested code - confidence level too high. Also, all code written assuming you use a C99 compiler, primarily because I declare variables when I first need them, not at the beginning of the function.)
Caution:
Note that if the original program is invoked as ./original_program > file or ./original_program | grep something (with redirected output) or is run from a cron job, then opening /dev/tty is not usually appropriate as a way to reopen standard output because the original standard output was not the terminal.
Also, note that if the redirection of standard output is used prior to forking and execing a child program and the original standard output is reinstated in the parent, then the sequence of operations is wrong. You should fork and then adjust the I/O of the child (only), without modifying the parent's I/O at all.
On Windows, you can open "CONOUT$".
freopen("test.txt", "w", stdout);
printf("this goes to test.txt");
freopen("CONOUT$", "w", stdout);
printf("this goes to the console\n");
This probably doesn't work if stdout is redirected to start with.
The following code (SwapIOB) is used in Testbenches that want to store
the stdout stream for comparison to an expected results file.
Background: File streams are managed using an _IOB structure that is stored in an array of 20 _IOB entries. This includes stdout stream. The IOBs are stored in an array. When a file is created the application code gets a ptr to an element in that array. The application code then passes that ptr to the OS for processing I/O calls. Thus, the OS does NOT itself contain or rely on its own pointers to the application's IOB.
Requirement: When running a testbench the stdout messages issued by an application should be re-directed to a file. However, after the module under test has completed then stdout messages should be re-redirected to the console.
This routine was tested and is currently used on Windows XP/Pro system.
void SwapIOB(FILE *A, FILE *B) {
FILE temp;
// make a copy of IOB A (usually this is "stdout")
memcpy(&temp, A, sizeof(struct _iobuf));
// copy IOB B to A's location, now any output
// sent to A is redirected thru B's IOB.
memcpy(A, B, sizeof(struct _iobuf));
// copy A into B, the swap is complete
memcpy(B, &temp, sizeof(struct _iobuf));
} // end SwapIOB;
Application code uses SwapIOB() similar to:
FILE *fp;
fp = fopen("X", "w");
SwapIOB(stdout, fp);
printf("text to file X");
SwapIOB(stdout, fp);
fclose(fp);
printf("text to console works, again!");
here's another question about splice(). I'm hoping to use it to copy files, and am trying to use two splice calls joined by a pipe like the example on splice's Wikipedia page. I wrote a simple test case which only tries to read the first 32K bytes from one file and write them to another:
#define _GNU_SOURCE
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
int main(int argc, char **argv) {
int pipefd[2];
int result;
FILE *in_file;
FILE *out_file;
result = pipe(pipefd);
in_file = fopen(argv[1], "rb");
out_file = fopen(argv[2], "wb");
result = splice(fileno(in_file), 0, pipefd[1], NULL, 32768, SPLICE_F_MORE | SPLICE_F_MOVE);
printf("%d\n", result);
result = splice(pipefd[0], NULL, fileno(out_file), 0, 32768, SPLICE_F_MORE | SPLICE_F_MOVE);
printf("%d\n", result);
if (result == -1)
printf("%d - %s\n", errno, strerror(errno));
close(pipefd[0]);
close(pipefd[1]);
fclose(in_file);
fclose(out_file);
return 0;
}
When I run this, the input file seems to be read properly, but the second splice call fails with EINVAL. Anybody know what I'm doing wrong here?
Thanks!
From the splice manpage:
EINVAL Target file system doesn't support splicing; target file is
opened in append mode; neither of the descriptors refers to a
pipe; or offset given for non-seekable device.
We know one of the descriptors is a pipe, and the file's not open in append mode. We also know no offset is given (0 is equivalent to NULL - did you mean to pass in a pointer to a zero offset?), so that's not the problem. Therefore, the filesystem you're using doesn't support splicing to files.
What kind of file system(s) are you copying to/from?
Your example runs on my system when both files are on ext3 but fails when I use an external drive (I forget offhand if it is DOS or NTFS). My guess is that one or both of your files are on a file system that splice does not support.
The splice(2) system call is for copying between files and pipes and not between files, so it can not be used to copy between files, as has been pointed out by the other answers.
As of Linux 4.5 however a new copy_file_range(2) system call is available that can copy between files. In the case of NFS it can even cause server side copying.
The linked man page contains a full example program.