I'm trying to parse data from stdin in binary mode under Win32.
The first thing my code does is to check for a 4byte header at the beginning:
int riff_header;
fread(&riff_header, sizeof(riff_header), 1, ifp);
// 'RIFF' = little-endian
if (riff_header != 0x46464952) {
fprintf(stderr, "wav2msu: Incorrect header: Invalid format or endianness\n");
fprintf(stderr, " Value was: 0x%x\n", riff_header);
return -1;
}
stdin has been switched to binary mode before reading from it:
if (*argv[argc-1] == '-') {
fprintf(stderr, "Reading from stdin.\n");
infile = stdin;
// We need to switch stdin to binary mode, or else we run
// into problems under Windows
freopen(NULL, "rb", stdin);
}
This code works fine under Linux, however on Win32 (specifically Windows XP), the fread only seems to read a single byte and thus cause the evaluation to fail.
Example:
> ffmeg.exe -i ..\test.mp3 -f wav pipe:1 2> nul |..\foo.exe -o test.bin -
Reading from stdin.
foo: Incorrect header: Invalid format or endianness
Value was: 0x4
What am I doing wrong?
According to the MSDN documentation, it's not permitted to pass NULL for the path parameter of freopen, so the call to freopen is almost certainly failing; have you checked the return value and the value of errno? C89 does not specify the behavior of freopen when path is NULL; C99 does, but the Microsoft C runtime is not (and does not claim to be) C99-compliant.
If you really need to read binary info from stdin, you might have to use platform-specific code and read the raw binary data directly with ReadFile on the file GetStdHandle(STD_INPUT_HANDLE).
At http://pubs.opengroup.org/onlinepubs/009695399/functions/freopen.html I have found the following:
If filename is a null pointer, the freopen() function shall attempt to
change the mode of the stream to that specified by mode, as if the
name of the file currently associated with the stream had been used.
In this case, the file descriptor associated with the stream need not
be closed if the call to freopen() succeeds. It is
implementation-defined which changes of mode are permitted (if any),
and under what circumstances.
Maybe you should check if the change of mode (from text to binary) is allowed by the compiler and libraries you are using. Which compiler are you using?
Update / summary
Using MinGW you can call setmode() to switch the mode of the stdin stream.
You should set the mode to _O_BINARY, which is defined in fcntl.h.
For more information see e.g. http://gnuwin32.sourceforge.net/compile.html
Related
I would like to write a simple API which
allows the user to open a file.
let the user write data to the file
track the write calls and sanity check the written data after each write call.
prevents the data from beeing written to disk if it is not valid -> discard(file)
As a starting point i wrote the test program below, which opens a file in fully buffered "rb+" mode using fopen and setvbuf.
The stream is opened in fully buffered mode for the following reason:
http://www.cplusplus.com/reference/cstdio/setvbuf/
mode
Specifies a mode for file buffering.
Three special macro constants [...]:
_IOFBF Full buffering: On output, data is written once the buffer is full (or flushed). On Input, the buffer is filled when an input
operation is requested and the buffer is empty.
My testprogram contains comments where a validity check could be placed and where the buffer contents should be discarded.
My question is how do i accomplish the discard(file) operation which means the step of getting rid of invalid buffer contents ?
The idea behind this is to assemble some data in the buffer, do a regular validity check after each or several write operations and write the data to disk only, if the data is valid.
Therefore i would need to discard the buffer, if the validity check fails.
When the validity check passes, the whole buffer contents should be written to the file.
My code draft looks like in the following. This is a simplified example:
#include <stdlib.h>
#include <stdio.h>
#include <stdint.h>
int main(void)
{
static uint8_t buffer[10000];
/* The following would be part of mylib_init */
FILE *file = fopen("test", "wb+");
if (file == NULL){
print ("open error!");
exit(-1);
}
if ( 0 != setvbuf(file , buffer, _IOFBF , sizeof(buffer) ) ){
print("Could not set buffer!");
fclose(file);
exit (-2);
}
/* The following would be part of mylib_write_data.
Each write and check resembles one func call */
// Pretend the user writes some data into the file
// ...
// fwrite(x)
if (data_in_buffer_not_valid(buffer)){
discard(file);
}
// ...
// fwrite(y)
//
if (data_in_buffer_not_valid(buffer)){
discard(file);
}
// ...
// fwrite(z)
// ...
// The following would be part of mylib_exit
// Cleanup stuff
fclose(file)
return 0;
}
If you want to have some like "scratch" temporary file that you want to write your data into and then retrieve them later, then the portable interface would be tmpfile() - it's an interface created just for that. Write to that file, rewind if you want, and when you're ready, rewind it and read from it block by block to another file.
On linux you may use fmemopen and fopencookie to write to a buffer via FILE* - these functions are not available on windows.
I would also strongly consider just creating your own interface that would store the result in memory. Writing an interface like struct mystream; mystream_init(struct mystream *); mystream_printf(struct mystream *, const char *fmt, ...); etc. is some of the tasks you sometimes do in C when fopencookie is not available. And consider writing the interface for storing data, so that instead of calling fwrite you would actually call the function that would check the data and write them and process them along the way.
As for setvbuf, note the standard. From C11 7.21.3p3:
When a stream is unbuffered, characters are intended to appear from the source or at the destination as soon as possible. Otherwise characters may be accumulated and transmitted to or from the host environment as a block. When a stream is fully buffered, [...]. When a stream is line buffered, [...] Support for these characteristics is implementation-defined, and may be affected via the setbuf and setvbuf functions.
And these buffering modes may just be not supported at all. And from C11 7.21.5.6:
The setvbuf function may be used only after the stream pointed to by stream has been associated with an open file and before any other operation (other than an unsuccessful call to setvbuf) is performed on the stream. [...] The contents of the array at any time are indeterminate.
You can't count on anything what will be the content of the buffer. Do not expecting any data there.
I have written a C program in which I am logging the results to a file. There is an infinite while loop - this is a requirement. To debug the code, I need to look at the log file, but as the program is running, I don't see anything written there. Closing the program forcibly using ctrl+C does not help either. I see nothing written on the file.
I am using simple fopen and fprintf functions to read the file in write mode and write to it.
FILE *fp = fopen("filename.txt", "w");
fprintf(fp, "this wants itself to be written the moment this statement is executed\n");
PS: There is no bug in the code. If I put a terminating condition in while loop and program exits gracefully, I do see things written in the log file.
A difference between printing to a console and printing to a file is that streams are line buffered by default when attached to the console, but block buffered when attached to a file. Change your code to:
FILE *fp = fopen("filename.txt", "w");
setvbuf(fp,0,_IOLBF,0);
fprintf(fp, "this wants itself to be written the moment this statement is executed\n");
and your output will be line buffered even though the stream is attached to a file. You can also do unbuffered streams.
[EDIT: ]
Ref C11 7.21.5.6:
Synopsis
#include <stdio.h>
int setvbuf(FILE * restrict stream,
char * restrict buf,
int mode, size_t size);
Description
The setvbuf function may be used only after the stream pointed to by
stream has been associated with an open file and before any other
operation (other than an unsuccessful call to setvbuf) is performed on
the stream. The argument mode determines how stream will be buffered,
as follows: _IOFBF causes input/output to be fully buffered; _IOLBF
causes input/output to be line buffered; _IONBF causes input/output to
be unbuffered. If buf is not a null pointer, the array it points to
may be used instead of a buffer allocated by the setvbuf function
and the argument size specifies the size of the array; otherwise, size
may determine the size of a buffer allocated by the setvbuf function.
The contents of the array at any time are indeterminate.
Returns
The setvbuf function returns zero on success, or nonzero if an invalid
value is given for mode or if the request cannot be honored.
You should to see the the function fopen(),if you fopen a file with "w" mode,it means if this file exist,clear this file and then write.I think you should use "a+" mode to append data in the end.
I am practicing some practice questions in FILE IO in C. Below is one of the programs.
#include<stdio.h>
#include<stdlib.h>
int main()
{
char fname[]="poem.txt";
FILE *fp;
char ch;
fp = fopen ( fname, "tr");
if (fp == NULL)
{
printf("Unable to open file...\n");
exit(1);
}
while((ch =fgetc(fp)) != EOF)
{
printf("%c",ch);
}
printf("\n");
return 0;
}
As you can see in the statement
fp = fopen ( fname, "tr");
The mode "tr" is not a valid mode (as I understand). I was expecting gcc to give an error (or a warning) while compiling the above program. However, gcc does not give any error (or warning) while compiling it.
However, as expected, when i run the program it exits printing "Unable to open file..." which means fopen() returned NULL , because there was error while opening file.
-bash-4.1$ ./a.out
Unable to open file...
-bash-4.1$
(The file poem.txt exists so this is because of the invalid mode given to fopen(). I checked by changing the mode to "r" and it works fine displaying the content of "poem.txt")
-bash-4.1$ ./a.out
THis is a poem.
-bash-4.1$
I was expecting gcc to give an error (or warning) message for the invalid mode.
Why gcc does not give any error (or warning) for this ?
the compiler doesn't check what you do, it only checks the syntax.
However, at run time, if the code is written like so:
#include<stdio.h>
#include<stdlib.h>
int main()
{
char fname[]="poem.txt";
FILE *fp;
char ch;
fp = fopen ( fname, "tr");
if (fp == NULL)
{
perror( "fopen for poem.txt failed");
exit( EXIT_FAILURE );
}
while((ch =fgetc(fp)) != EOF)
{
printf("%c",ch);
}
printf("\n");
return 0;
}
then a proper error message is output:
...$ ./untitled
fopen for poem.txt failed: Invalid argument
This is Undefined Behavior:
Per Annex J.2 "Undefined Behavior", it is UDB if:
—The string pointed to by the mode argument in a call to the fopen function does not exactly match one of the specified character sequences (7.19.5.3).
Although Annex J is informative, looking at §7.19.5.3:
/3 The argument mode points to a string. If the string is one of the following, the file is open in the indicated mode. Otherwise, the behavior is undefined.
Basically, the compiler can blow you off here - a standard library function name (and behavior) can be used outside of the inclusion of a standard header (for example, non-standard extensions, completely user-defined behavior, etc.). The Standard specifies what a conforming library implementation shall include and how it shall behave, but does not require you to use that standard library (or define behavior for a specific implementation explicitly specified as UDB territory: at this point, if your parameter types match it's a legal function call).
A really good lint utility might help you here.
How is the compiler supposed to know what the valid arguments for a function are?
To do it you'd be building too much knowledge in the compiler - it would have to recognize functions and their parameters by name. What if you want to override the function? What if different modes are valid on different platforms?
In Windows programming, "tr" is a valid mode is not a valid mode, although "rt" is. The t means text and the r means read. (If you are using gcc and linking to MS's C runtime then you will be able to use this).
However you still don't see t very often because it is the default and therefore redundant; the other option for this setting is b meaning binary. But MS do seem to explicitly use t in their examples to make it clear that translation is intended.
The behaviour of text mode and binary mode for a stream is implementation-defined, although the intent is that binary mode reads the characters exactly as they appear on disk, and text mode may perform translations relevant to text processing; most famously, converting \r\n in MS text files to \n in your program.
I was toying around with some code which was opening, reading, and modifying a text file. A quick (simplified) example would be:
#include <stdio.h>
int main()
{
FILE * fp = fopen("test.txt", "r+");
char line[100] = {'\0'};
int count = 0;
int ret_code = 0;
while(!feof(fp)){
fgets(line, 100, fp);
// do some processing on line...
count++;
if(count == 4) {
ret_code = fprintf(fp, "replaced this line\n");
printf("ret code was %d\n", ret_code);
perror("Error was: ");
}
}
fclose(fp);
return 0;
}
Now on Linux, compiled with gcc (4.6.2) this code runs, and modifies the file's 5th line. The same code, running on Windows7 compiled with Visual C++2010 runs and claims to have succeeded (reports a return code of 19 characters and perror says "No error") but fails to replace the line.
On Linux my file has full permissions:
-rw-rw-rw- 1 mike users 191 Feb 14 10:11 test.txt
And as far as I can tell it's the same on Windows:
test.txt (right click) -> properties -> Security
"Allow" is checked for Read & Write for user, System, and Admin.
I get the same results using MinGW's gcc on Windows so I know it's not a Visual C++ "feature".
Am I missing something obvious, or is the fact that I get no errors, but also no output just an undocumented "feature" of using r+ with fopen() on Windows?
EDIT: Seems even at Microsoft's site they say "r+" should open for reading and writting. They also made this note:
When the "r+", "w+", or "a+" access type is specified, both reading and writing are allowed (the file is said to be open for "update"). However, when you switch between reading and writing, there must be an intervening fflush, fsetpos, fseek, or rewind operation. The current position can be specified for the fsetpos or fseek operation, if desired.
So I tried:
...
if(count == 4) {
fflush(fp);
ret_code = fprintf(fp, "replaced this line\n");
fflush(fp);
printf("ret code was %d\n", ret_code);
...
to no avail.
According to the Linux man page for fopen():
Reads and writes may be intermixed on read/write streams in any order.
Note that ANSI C requires that a file positioning function intervene
between output and input, unless an input operation encounters
end-of-file. (If this condition is not met, then a read is allowed to
return the result of writes other than the most recent.) Therefore it
is good practice (and indeed sometimes necessary under Linux) to put
an fseek(3) or fgetpos(3) operation between write and read operations
on such a stream. This operation may be an apparent no-op (as in
fseek(..., 0L, SEEK_CUR) called for its synchronizing side effect.
So, you should always call fseek() (as, eg. fseek(..., 0, SEEK_CUR)) when switching between reading and writing from a file.
Before performing output after input, an fflush() isn't any good - you need to perform a seek operation. Something like:
fseek(fp, ftell(fp), SEEK_SET); // not fflush(fp);
from the C99 standard (7.19.5.3/6 "The fopen functoin):
When a file is opened with update mode ('+' as the second or third
character in the above list of mode argument values), both input and
output may be performed on the associated stream. However, output
shall not be directly followed by input without an intervening call to
the fflush function or to a file positioning function (fseek,
fsetpos, or rewind), and input shall not be directly followed by output
without an intervening call to a file positioning function, unless the
input operation encounters end-of-file.
I have implemented fputc and fgetc in retarget.c to successfully use printf via UART0 on a Cortex-M3.
However, I want a second uart channel for additional debug information. How can I integrate this as nicely as I can UART0 using printf?
For example, using fprintf to a custom target and checking in fputc which target to send the character to..
E.g. for normal output fprintf(UART0,".."); and for debug output fprintf(UART1,"..");
But I cannot see if fopen is called for stdout so I am struggling to see how to manually implement this. (If I just call fprintf(RANDOM_VALUE,..), I don't know how this will behave.
I guess that once I have it directed to a different 'FILE', then it is simply a matter of checking which is being pointed to within fputc but it is the initial setting of the FILE pointer that I am struggling with.
Perhaps some way to differentiate between stdout and stderr, although then I still have the same problem for getting input from the two separate channels.
Also is fprintf in the microlib? If not, is there a better way to implement this?
Thanks!
fputc() takes a stream pointer argument, there are two standard output streams stdin, stdout and stderr. At the lower level of the retargeting these are associated with the file descriptors 0, 1, and 2 respectively, you can use this information to associate stderr with the alternate UART at the device driver level.
You can then output debug data using stderr thus:
fprintf (stderr, "Error reading file" ) ;
for example.
A minimal retargeting (specific to Keil ARM-MDK/RealView) might look like this:
struct __FILE
{
int handle;
};
enum
{
STDIN_HANDLE,
STDOUT_HANDLE,
STDERR_HANDLE
} ;
FILE __stdin = {STDIN_HANDLE} ;
FILE __stdout = {STDOUT_HANDLE} ;
FILE __stderr = {STDERR_HANDLE} ;
int fputc(int ch, FILE *f)
{
int ret = EOF ;
switch( f->handle )
{
case STDOUT_HANDLE :
// Write character to UART0
...
ret = ch ;
break ;
case STDERR_HANDLE :
// Write character to UART1
...
ret = ch ;
break ;
default :
break ;
return ret ;
}
Obviously this is also where you might hook in a filesystem if you needed, in which case your __FILE struct would no doubt have additional members.
If you don't want to use stderr for this purpose, you will have to retarget fopen() to translate a device name ("dbg:" for example) into a file descriptor for the desired port and then use stdio to output to the associated stream.
Also is fprintf in the microlib? If not, is there a better way to implement this?
The documentation will tell you, but yes. Microlib stdio support is controlled by the #pragma import(__use_full_stdio) directive, the documentation is not clear about what is excluded if this is not used. Try it without and use it if anything is missing. That said I would imagine that printf() is implemented as an fprintf() to the stdout stream, so if you have printf() you have fprintf().