I'm writing a file using a c code on a unix system . I open it , write a few lines and close it. Then i call a shell script, say code B where this file is to be used and then return back to main program. However, when code B tries to read the file, the file is empty.
I checked the file on the file system, its size is shown as 0 and no data is present in file. However after killing the running c code process, file has data present in it.
Here is the piece of code -
void writefile(){
FILE *fp;
fp = fopen("ABC.txt","w");
fputs("Some lines...\n",fp);
fclose(fp);
system("code_B ABC.txt");
}
Please advise how can I read the file in the shell script without stopping the c code process.
If there's some time between the fputs and fclose, add
fflush(fp);
This will cause the contents of the disk file to be written.
You should do fsync() after the fclose(), to guarantee the writing of the file to the disk.
Take a look at this question:
Does Linux guarantee the contents of a file is flushed to disc after close()?
The kernel ensures that data which is written to a file can be read back afterwards from a different process, even if it is not physically written to the disc yet. So, in usual scenarios, there is no need to call fsync() - still, even with fsync(), the filesystem could decide to further delay physical writes.
One common problem is that the C library has not flushed its buffers yet, in which case you would need to call fflush() - however, you are calling fclose() before launching your sub process, and fclose() internally calls fflush().
Actually, since system() is using a shell to launch the command passed as parameter, you can use the following simple SSCCE to verify that it works:
#include <stdio.h>
void writefile(){
FILE *fp;
fp = fopen("ABC.txt","w");
fputs("Some lines...\n",fp);
fclose(fp);
system("cat ABC.txt");
}
int main() {
writefile();
return 0;
}
Here, system() simply calls the cat command to print the file contents. The output is:
$ ./writefile
Some lines...
Related
When we use system call open() and then perform I/O operations (especially read() and lseek()), Do the kernel buffer gets updated if we change the file when program is still running? if not, then how to forcefully synchronize live updating file to kernel buffer.
Here is an example:
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
int main()
{
int fd=0;
char ch='\0';
fd=open("test.dat",O_RDONLY);
while(1)
{
while(read(fd,&ch,1)!=0)
{
printf("%c",ch);
}
printf("\n");
lseek(fd,0,SEEK_SET);
sleep(5);
}
close(fd);
return 0;
}
Now, I have some data in "test.dat" (say: '3 3 34'). I opened a this file and read it to end and then again seek it to start. Meanwhile, in some editor I opened this "test.dat" file and updated its contents and saved them. Since read() and lseek() are system calls, they shall correspond to changes in updated file if kernel/OS buffer regularly syncs with file in Hard disk. But that is not the case. The changes in file are not reflected from read(), instead it continues to print the initial content. For writing, if have solutions like sync(), fsync() etc. But for reading do we have some such functions?
(Note: one trivial solution to this problem is close() and again open() file descriptor which works perfectly but I want to know and understand some alternative without closing file descriptor)
Some editors make a copy of the file they edit, and finally they move it into the original filename (probably renaming the original to something else), so the file you are reading can be different actually from the one the editor has edited. This depends on which editor you use to modify the file and is a complex task as you can have several links to the file, so it is expected this operation is not done by the editor. Ensure you are modifying the original file. Something useful is to have a different contents file and do (instead of editing the file)
$ your_program your_file.txt &
$ cat modified_version.txt > your_file.txt
and see what happens.
You have not exposed what editor you use to edit the file.
I'm writing a curses-based program. In order to make it simpler for me to find errors in this program, I would like to produce debug output. Due to the program already displaying a user interface on the terminal, I cannot put debugging output there.
Instead, I plan to write debugging output to file descriptor 3 unconditionally. You can invoke the program as program 3>/dev/ttyX with /dev/ttyX being a different teletype to see the debugging output. When file descriptor 3 is not opened, write calls fail with EBADF, which I ignore like all errors when writing debugging output.
A problem occurs when I open another file and no debugging output has been requested (i.e. file descriptor 3 has not been opened). In this case, the newly opened file might receive file descriptor 3, causing debugging output to randomly corrupt a file I just opened. This is a bad thing. How can I avoid this? Is there a portable way to mark a file descriptor as “reserved” or such?
Here are a couple of ideas I had and their problems:
I could open /dev/null or a temporary file to file descriptor 3 (e.g. by means of dup2()) before opening any other file. This works but I'm not sure if I can assume this to always succeed as opening /dev/null may not succeed.
I could test if file descriptor 3 is open and not write debugging output if it isn't. This is problematic when I'm attempting to restart the program by calling exec as a different file descriptor might have been opened (and not closed) prior to the exec call. I could intentionally close file descriptor 3 before calling exec when it has not been opened for debugging, but this feels really uggly.
Why use fd 3? Why not use fd 2 (stderr)? It already has a well-defined "I am logging of some sorts" meaning, is always (not true, but sufficiently true...) and you can redirect it before starting your binary, to get the logs where you want.
Another option would be to log messages to syslog, using the LOG_DEBUG level. This entails calling syslog() instead of a normal write function, but that's simply making the logging more explicit.
A simple way of checking if stderr has been redirected or is still pointing at the terminal is by using the isatty function (example code below):
#include <stdio.h>
#include <unistd.h>
int main(void) {
if (isatty(2)) {
printf("stderr is not redirected.\n");
} else {
printf("stderr seems to be redirected.\n");
}
}
In the very beginning of your program, open /dev/null and then assign it to file descriptor 3:
int fd = open ("/dev/null", O_WRONLY);
dup2(fd, 3);
This way, file descriptor 3 won't be taken.
Then, if needed, reuse dup2() to assign file descriptor 3 to your debugging output.
You claim you can't guarantee you can open /dev/null successfully, which is a little strange, but let's run with it. You should be able to use socketpair() to get a pair of FDs. You can then set the write end of the pair non-blocking, and dup2 it. You claim you are already ignoring errors on writes to this FD, so the data going in the bit-bucket won't bother you. You can of course close the other end of the socketpair.
Don't focus on a specific file descriptor value - you can't control it in a portable manner anyway. If you can control it at all. But you can use an environment variable to control debug output to a file:
int debugFD = getDebugFD();
...
int getDebugFD()
{
const char *debugFile = getenv( "DEBUG_FILE" );
if ( NULL == debugFile )
{
return( -1 );
}
int fd = open( debugFile, O_CREAT | O_APPEND | O_WRONLY, 0644 );
// error checking can be here
return( fd );
}
Now you can write your debug output to debugFD. I assume you know enough to make sure debugFD is visible where you need it, and also how to make sure it's initialized before trying to use it.
If you don't pass a DEBUG_FILE envval, you get an invalid file descriptor and your debug calls fail - presumably silently.
I am wondering which option is basically quicker.
What interests me the most is the mechanism of redirection. I suspect the file is opened at the start of the program ./program > file and is closed at the end. Hence every time a program outputs something it should be just written to a file, as simple as it sounds. Is it so? Then I guess both options should be comparable when it comes to speed.
Or maybe it is more complicated process since the operating system has to perform more operations?
There is no much difference between that options (except making file as a strict option reduces flexibility of your program).
To compare both approaches, let's check, what stays behind a magical entity FILE*:
So in both cases we have a FILE* object, a file descriptor fd - a gateway to an OS kernel and in-kernel infrastructure that provides access to files or user terminals, which should (unless libc has some special initializer for stdout or kernel specially handles files with fd = 1).
How does bash redirection work in compare with fopen()?
When bash redirects file:
fork() // new process is created
fd = open("file", ...) // open new file
close(1) // get rid of fd=1 pointing to /dev/pts device
dup2(fd, 1) // make fd=1 point to opened file
close(fd) // get rid of redundant fd
execve("a") // now "a" will have file as its stdout
// in a
stdout = fdopen(1, ...)
When you open file on your own:
fork() // new process is created
execve("a") // now "a" will have file as its stdout
stdout = fdopen(1, ...)
my_file = fopen("file", ...)
fd = open("file", ...)
my_file = fdopen(fd, ...)
So as you can see, the main bash difference is twiddling with file descriptors.
Yes, you are right. The speed will be identical. The only difference in the two cases is which program opens and closes the file. When you redirect it using shell, it is the shell that opens the file and makes the handle available as stdout to the program. When the program opens the file, well, the program opens the file. After that, the handle is a file handle in both the cases, so there should be absolutely no difference in speed.
As a side remark, the program which writes to stdout can be used in more general ways. You can for example say
./program | ssh remotehost bash -c "cat > file"
which will cause the output of the program to be written to file on remotehost. Of course in this case there is no comparison like one you are making in the question.
stdout is a FILE handle, fprintf writes to a file handle, so the speed will be very similar in both cases. In fact printf("Some string") is equivalent to fprintf(stdout, "Some string"). I will say no more :)
How may I write data into a file in real time?
I mean: A program gets data and puts them into a file (we can say out.txt) with a fprintf(...) command, then another program read data from the file (out.txt) and elaborates them.
I have a flow like this:
fp=fopen("out.txt","w+");
while(...) {
...
fprintf (fp,"\n %s", data);
}
fclose(fp);
With that flow I get data into the file (out.txt) after I have closed the file.
Is there a way to write data in real time into the file?
I suspect you mean simultaneously when you say in real time. Simultaneous access
to a file requires to open a shared file. This may be a pipe or some other interprocess
communication. It may also be a simple file on disk, as asked for. Windows allows to open
a shared file by means of _fsopen.
#include <share.h> // required for manifest constants for shflag parameter.
fp = _fsopen("out.txt","w+",_SH_DENYWR); // e.g. _SH_DENYWR denies write access
.
while(...) {
.
.
Another process can read the file while it is open; no need to close the file beforehand. The other process may even write (with shflag = _SH_DENYNO). However, writing and reading simultaneously from different processes requires some more coding effort.
.
fprintf (fp,"%s\n", data); // you may want to have the /n "new line" after writing the data
.
Not all the written content is always immedeately written to the file (physically). Therefore it is required to force the content to be writen to the file after the fprintf.
.
fflush(fp); // force writing to device
.
}
I am trying to redirect the output of a c program to file, even when it generates some errors because of problems with the input data. I can send the output but the error messages to a file.
Does somebody know how to do it?
From within C source code, you can redirect outputs using freopen():
General outputs:
freopen("myfile.txt", "w", stdout);
Errors:
freopen("myfile_err.txt", "w", stderr);
(This answer applies to bash shell, and similar flavors. You didn't specify your environment and this sort of question needs that detail.)
I assume you know about basic redirection with ">". To also capture STDERR in addition to STDOUT, use the following syntax:
command > file-name 2>&1
For some more background on standard streams and numbers:
http://en.wikipedia.org/wiki/Standard_streams#Standard_input_.28stdin.29
This depends on what you mean and what platform you are using. Very often you can accomplish this from the command line, which has been covered in another answer. If you use this method to accomplish this you should be aware that FILE * stderr is typically written immediately (unbuffered) while FILE * stdout may be buffered (usually line buffered) so you could end up with some of your error messages appearing to have been printed earlier than some other messages, but actually the other messages are just being printed late.
From within a C program you can also do something similar within the stdio system using freopen, which will effect the FILE *, so you could make fprintf(stderr, "fungus"); print to something besides what stderr normally would print to.
But if you want to know how to make a program redirect the actual file descriptors under a unix like system you need to learn about the dup and dup2 system calls. They allow you to duplicate a file descriptor.
int fd = open("some_file", O_WRONLY);
dup2(2,fd);
close(fd);
This code will make "some_file" the new stderr at the OS level. The dup2 call will close and replace file descriptor 2 (stderr, which is usually used by FILE * stderr but not necessarily if you call freopen(x,y,stderr) since that may make FILE *stderr use a different file descriptor).
This is how shell programs redirect input and output of programs. The open all of the files that the new program will need, fork, then the child uses dup2 to set up the files descriptors for the new program, then it closes any files that the new program won't need (usually just leaving 0, 1, and 2 open), and then uses one of the exec functions to become the program that the shell was told to run. (some of this isn't entirely accurate because some shells may rely on close on exe flags)
Using a simple linux command you can save the output into the file. here is a simple linux terminal command.
ls > file.txt
The output of this command will be stored into the file.
same as you can store the output of the program like this suppose, object file name is a, run the following command to save output in a file:
./a > file.txt