So here's a fun one...
we have a few C libraries which should be platform independent, even though they were developed on linux, because they only rely on the c standard library as defined in ISO/IEC 9899:1999. When we compiled those libraries with MinGW everything seemed to work fine at first, but today we found out that the snprintf() implementation of msvcrt is braindea... sorry, i meant "incompatible" with the definition in the C99 standard.
I would have expected a warning from MinGW, telling me that -std=c99 actually isn't fully supported. Because otherwise, how am i supposed to know?
Is there any alternative c standard library for windows, and most importantly: can MinGW somehow be told to link against it instead of msvcrt?
If not, we would at the very least need a list or something where we can check for other potential portability problems concerning msvcrt and c99.
PS:
I know about Cygwin and MSYS2, but i'd rather have native windows binaries (in part because we also use our libraries in Matlab).
Edit:
Sorry, i should have explained more clearly what exactly my problem with msvcrt's snprintf() is. According to the standard, snprintf() is required to output a '\0' as the last character, if the output doesn't fit. However, msvcrt just doesn't do that, so the resulting string is no longer properly terminated. I have no idea why anyone would choose to implement snprintf() that way, because to me just omitting the '\0' doesn't make any sense at all.
We have also tried the suggested __USE_MINGW_ANSI_STDIO, but i guess that just fixes the missing format specifiers? At least it didn't seem to make a difference for our specific problem.
The standard enforces snprintf to behave like this:
Otherwise, output characters beyond the n-1st are
discarded rather than being written to the array, and a null character is written at the end of the characters actually written into the array.
snprintf in msvcrt is indeed not the standard one, but a Microsoft version as explained here:
Is snprintf() ALWAYS null terminating?
The following code gives non-compliant results:
#include <stdio.h>
int main (void)
{
char dst[3];
snprintf(dst, 3, "%c%c%c", 'A', 'B', 'C');
for(size_t i=0; i<3; i++)
{
printf("%.2X ", dst[i]);
}
}
I get output 41 42 43 which is not standard compliant. To get standard C, you have to add this before the stdio.h include:
#define __USE_MINGW_ANSI_STDIO 1
And now you get 41 42 00 which is compliant.
The root of all these program security problems is Microsoft, who have been using non-compliant C libs in their products for the past 20 years.
If you need C99 stdio from MinGW-w64, you can define __USE_MINGW_ANSI_STDIO so that you bypass the msvcrt implementation. It's best to define this through a compiler argument
-D__USE_MINGW_ANSI_STDIO
Alternatively, you could try to use a MinGW-w64 build that is set up to link with the new ucrt, but I don't know of any pre-existing easy-to-use stable builds that are set up that way.
Related
I'm trying to work with some legacy C89 code, and am having trouble getting it to build. My usual environment is Visual Studio, but that only seems to support C99, and some C99 features (such as stdio etc. not necessarily being constant) break the code - a lot. Before I start tampering with the code I want to write some tests, so I don't break the old behaviour, but I can't test the tests, so to speak, before I can get the code to build.
So is there still any way to compile C89 code on Windows?
Edit: Steve Summit has identified that stdio and so on has never been guaranteed; it's just a feature of some compilers that my legacy code happens to depend on, in a rather deeply embedded way. So my question shifts to whether there is any Windows C compiler available (preferably free!) for Windows that supports that assumption. Alternatively, I have an Ubuntu installation in a virtual machine, although I have little experience using it - is there such a compiler available in Ubuntu?
MSVC is a C++ compiler and has just gained C99 support recently. Previously it supports only C89 with some MS extensions. To compile in strict C89 mode use the /Za option. Make sure to also enable /Tc to use C mode
/Za, /Ze (Disable Language Extensions)
The /Za compiler option disables and emits errors for Microsoft extensions to C that aren't compatible with ANSI C89/ISO C90. The deprecated /Ze compiler option enables Microsoft extensions. Microsoft extensions are enabled by default.
See Enforce ANSI C Standard in Visual Studio 2015
Most other compilers use other options like -ansi, -std=c90 or -std=iso9899:1990
However if this is just about stdin/stdout not being constant while using in a static initializer list then it's completely irrelevant to C89 and is actually an XY problem. The following snippet compiles without problem in VS2019 C++ mode, so if you don't have any possible conflict just compile the code in C++ mode
#include <stdio.h>
FILE* ifp = stdout;
int main()
{
fprintf(ifp, "test\n");
return 0;
}
Otherwise it's easy to fix that to compile in C mode by moving the initialization into main()
FILE* ifp = NULL;
int main()
{
ifp = stdout;
fprintf(ifp, "test\n");
return 0;
}
[This isn't really an answer, but it's too elaborate for a comment.]
If you've got code that does things like
#include <stdio.h>
FILE *ifp = stdin;
int main() { ... }
and if the problem you're having is errors stating that stdin is not a compile-time constant suitable for a static initializer, I think you're going to have to rewrite that aspect of your code. I could be wrong, but if I remember correctly, the idea that stdin et al. were compile-time constants was never a guarantee, just a useful property of the earliest Unix implementations. It wasn't necessarily true of all old implementations, so the "change" to the Standard that explicitly said they weren't necessarily constant wasn't a change per se, but rather, more or less a codification of the divergence of existing practice.
(In other words, if you've got a compiler that's rejecting the code, and even if it has a backwards-compatibility mode, I'd be surprised if the backwards-compatibility mode turned stdin into a compile-time constant.)
All supported (and even older) versions of Visual Studio are perfectly capable of compiling C89 code. Also C99 is backward compatible with previous revisions of the language, so a C99 compiler should be able to compile just fine C89 code.
Although you might get some warnings, the code should compile and work just fine if the code is portable of course.
I'm trying the below code to see if the optional string functions in C are supported (I've got Mac OS X El Capitan and XCode installed)...
#include <stdio.h>
int main(void)
{
#if defined __STDC_LIB_EXT1__
printf("Optional functions are defined.\n");
#else
printf("Optional functions are not defined.\n");
#endif
return 0;
}
...but it suggests they aren't.
I've tried all the different compilers I have from XCode (cc, gcc, llvm-gcc, clang).
I've also tried brew install gcc assuming that the GNU C compiler would give me these extra functions, but it doesn't.
Is there a way to simply install a C11 compatible compiler on Mac OS that'll give me these additional (i.e. safe) string functions.
Summary: You won't get it to work. There are better ways to make sure your code is correct. For now, use the address sanitizer instead.
Also known as "Annex K" of the C11 standard or TR 24731, these functions are not widely implemented. The only commonly available implementation is part of Microsoft Visual Studio, other common C implementations have rejected (explicitly, even) the functionality in annex K. So, while annex K is technically part of the standard, for practical purposes it should be treated as a Microsoft-specific extension.
See Field Experience With Annex K — Bounds Checking Interfaces (document N1967) for more information. According to this report, there are only four implementations of annex K, two are for Windows, one is considered "very incomplete" and the remaining one is "unsuitable for production use without considerable changes."
However, the argument that these string functions are "safe" is a bit misleading. These functions merely add bounds checking, which only works if the functions are called correctly—but then again, the "non-safe" functions only work if they are called correctly too. From the report cited above,
Despite more than a decade since the original proposal and nearly ten years since the ratification of ISO/IEC TR 24731-1:2007, and almost five years since the introduction of the Bounds checking interfaces into the C standard, no viable conforming implementations has emerged. The APIs continue to be controversial and requests for implementation continue to be rejected by implementers.
The design of the Bounds checking interfaces, though well-intentioned, suffers from far too many problems to correct. Using the APIs has been seen to lead to worse quality, less secure software than relying on established approaches or modern technologies. More effective and less intrusive approaches have become commonplace and are often preferred by users and security experts alike.
Therefore, we propose that Annex K be either removed from the next revision of the C standard, or deprecated and then removed.
I suggest using the address sanitizer as an alternative.
Do not use strncpy, strncat or the like as "safe" functions, they're not designed to do that and they are not drop-in replacements for strcpy, strcat, etc., unlike strcpy_s, strcat_s, which are drop-in replacements.
If you are not using Windows or Embarcadero you need to use the external safeclib: https://github.com/rurban/safeclib/releases
No other libc's comes with the safe C11 Annex K extensions.
For an overview of the various libc quirks regarding this see https://rurban.github.io/safeclib/doc/safec-3.3/d1/dae/md_doc_libc-overview.html
I recently started porting a TON of my C programs to a Windows environment, from my previous Linux development PC. I noticed something a bit off about mingw's Windows GCC implementation.
In Windows, I found a lovely function called getch. It's easy, it's immediate... and it's also non-standard.
I'd like to focus of the "non-standard" part of it. Specifically, I want to know why mingw-gcc allows me to use it, without using anything but the standard libraries.
Assume we have a program that prints "Hello, World!", a NL and CR, and then waits for a key and a return:
#include <stdio.h>
int main(void)
{
char str[14] = "Hello, World!"; //13 characters and a terminator
printf("%s\n\r", str);
scanf("%c");
return 0;
}
Now, let's change a bit of that program to use getch:
#include <stdio.h>
int main(void)
{
char str[14] = "Hello, World!"; //Again, 13 characters and a terminator
printf("%s\n\r", str);
getch(); //See? now it uses getch.
return 0;
}
The interesting part is, Isn't getch a call made by the conio.h library for old DOS/Win32 environments? The compiler doesn't even give a warning. Why does this work?
Here's something I find even a bit more unsettling:
int main(void) //literally NOTHING included
{
getch();
return 0;
}
What on earth? I know for a fact that getch does not exist on Linux environments (natively, anyways). So, where is the compiler getting this call from?
My best guess (please correct me if I am wrong) is that the decision to link whatever has getch is made at link time, not compile time.
In any case, this seems a little odd to me. Why does an implementation of GCC automatically include clearly non-standard capability on Windows?
The compilation step works (though it should produce a warning) because, traditionally, C allows you to call functions that haven't been declared, provided that they return an int which getch() does.
The linking step works because the C runtime library that MinGW uses is a single library, i.e., it provides all the Visual C runtime library functions, including the non-standard ones. MinGW presumably links with it by default because (apart from very rare edge cases) it is always needed, even if all you want is a main() function that does nothing. :-)
It should also be mentioned that the library in question does not officially support third-party use, except by applications built in Visual Studio 6. The more modern runtimes have deprecated getch in favour of the equivalent but standards compliant _getch, but VS6 predates that change.
The compile step will probably 'work' because compilers (at least older versions of compilers — we know that Visual Studio is not up to date with the latest C standards) will assume the return type and the parameters are all int.
The link step will need the appropriate library linked in.
I got this book "Beginning C" by Ivor Horton and I'm half way through it and I like it; so far so good. I use Code::Blocks on Windows as my IDE, and now I've run into the problem I cannot solve for about 3 days now.
The author mentions some "optional" functions in <string.h>, like strnlen_s(), and also says that these are available in the new standard — C11 (the book is from 2013; I don't know how new C11 actually is), and he also gives a piece of code that will determine "whether the standard library that comes with your C compiler supports these optional functions".
This is the code:
#include <stdio.h>
int main(void)
{
#if defined __STDC_LIB_EXT1__
printf("Optional functions are defined.\n");
#else
printf("Optional functions are not defined.\n");
#endif
return 0;
}
So I run the code to check if GCC in Code::Blocks does and determine that it doesn't. The book didn't recommend the compiler nor the IDE; I picked up Code::Blocks with GCC on my own, since that's what I do my exams in at college, so I figured I should get familiar with the environment.
The thing is, I have no idea how to "fix" this, since strnlen() doesn't work, strnlen_s() doesn't work, and bunch of others, and I can't really continue through a book. Not that I need them, or that I can't do it any other way (strlen() works just fine) but it would be nice to know how to use non-standard functions.
Up to date versions of GCC certainly do support C11, you need to enable it with the compiler flag -std=c11.
I presume you're using some flavour of MinGW with Code::Blocks - I recommend using MinGW-W64 as it is actively maintained and very up to date.
Also, bundled toolchains of MinGW-W64's gcc are available at TDM-GCC.
The Code::Blocks IDE itself doesn't care which version of C you're using, that doesn't affect what libraries you have available.
You are speaking of the optional Annex K Microsoft pushed through.
K.2 Scope
1 This annex specifies a series of optional extensions that can be useful in the mitigation of
security vulnerabilities in programs, and comprise new functions, macros, and types
declared or defined in existing standard headers.
2 An implementation that defines __STDC_LIB_EXT1__ shall conform to the
specifications in this annex.380)
3 Subclause K.3 should be read as if it were merged into the parallel structure of named
subclauses of clause 7.
It is generally seen as deeply flawed, and Microsoft trying to force it's use as a severe nuisance.
That's especially the case as they are the only major player implementing them, and their versions are non-conformant.
glibc with gcc for example provide most supposed advantages of that annex without introducing new functions, discouraging use of half the standard-library and forcing such a cumbersome API on programmers.
You might want to read the C tag-wiki, and especially grab a draft of the C11 standard (which is from 2011, as the name should imply).
The optional Annex K from the C11 Standard is not widely adopted yet (see Deduplicator's comment below). For instance as of February 2015 it hasn't been merged into glibc.
The good news is that you might try an alternative compiler. For instance Pelles C for Windows is a modified LCC with enhanced support for newest C11 features (like atomics and C11 threads model, that I believe are also mentioned in your book). Here is some basic program, that compiles and runs in it:
#include <stdio.h>
#include <string.h>
int main(void)
{
#if defined __STDC_LIB_EXT1__
printf("Optional functions are defined.\n");
#else
printf("Optional functions are not defined.\n");
#endif
char *str = "Hello Annex K";
printf("%zu\n", strnlen_s(str, 5));
return 0;
}
Output is:
Optional functions are defined.
5
Press any key to continue...
Okay, I've run into a strange issue compiling a C file with MinGW (GCC 4.6.2) on Windows 7. The file in question contains the following C code:
#include <stdio.h>
int main(int argc, char *argv[]) {
printf("%2hhX\n", 250);
char c[80];
snprintf(c, sizeof(c), "%2hhX", 250);
printf("%s\n", c);
return 0;
}
The compilation turns out like this:
$ gcc.exe -std=c99 -pedantic -Wall test.c
test.c: In function 'main':
test.c:6:2: warning: unknown conversion type character 'h' in format [-Wformat]
test.c:6:2: warning: too many arguments for format [-Wformat-extra-args]
Now, what's strange to me is that it complains about the snprintf call on line 6, but not the printf call on line 4. Am I missing something or is the warning just incorrect? Also, is there perhaps a better equivalent for the format string "%2hhX"? (I'm trying to print char variables as hexadecimal values.)
Historically, MinGW has been in a bit of an odd situation, especially as far as C99 support goes. MinGW relies mostly on the msvcrt.dll runtime that's distributed with Windows, and that runtime doesn't support C99.
So with older versions of MinGW, you can run into problems in C99 mode when using C99-specific format specifiers. Also historically, GCC didn't make any special accommodations for msvcrt.dll's lack of support for C99 specifiers. So you'd get into situations where -Wformat wouldn't warn about a format that wouldn't work.
Things are improving on both sides - GCC has specific support for -Wformat when used with the MS runtime, such as:
-Wpedantic-ms-format so that GCC won't complain about "I32" and "I64" (even though it's documented, I still get a complaint about it being unrecognized even in 4.7.0 - maybe it's brand new)
the ms_printf option to __attribute__((__format__))
On the other side, MinGW has provided its own snprintf() for a while, since MSVC's variant, _snprintf(), behaves quite differently. However, MinGW relied for a long while on the printf() in msvcrt.dll, so C99 format specifiers for printf() didn't work. At some point MinGW started providing it's own version of printf() and friends so that you could get proper C99 (and GNU?) support. However, it seems that to be on the conservative side, these didn't replace the msvcrt.dll versions initially. They have names like __mingw_printf().
It looks like at some point between 4.6.1 and 4.7.0, the MinGW headers started using the MinGW supplied versions as replacements for the msvcrt.dll function (at least if you've specifed C99).
However, it seems that with the newer versions, GCC and MinGW are still a little out of sync. Where as before GCC would not warn about specifiers that wouldn't actually work on MinGW, not it complains about spcifiers that will.
You may want to try the following snipet of code to see how well your version of MinGW support "hhX":
printf("%hhX\n", 0x11223344);
__mingw_printf("%hhX\n", 0x11223344);
I'm not sure what to suggest to fix the problem you're running into - I think that you may be able to patch the MinGW stdio.h header so that it has a __attribute__((__format__ (gnu_printf, ...))) attribute on the printf functions (they're not there in the newer stdio.h, so GCC will use it's default idea of what the format support is).
In addition to the other answer, here is some more info on printf format checks in GCC:
When you say __attribute__((__format__ (FORMAT, ...))), the value of FORMAT can be (as far as printf is concerned) one of the following: printf, gnu_printf, ms_printf.
ms_printf makes GCC assume that function takes a format string intended for Microsoft Visual Studio CRT printf family functions. It means that GCC will complain about z, hh and ll, but will pass I64 without warning.
gnu_printf makes GCC assume GNU libc printf implementation underneath (or maybe just a POSIX/C99-compliant printf implementation, i'm not sure). Therefore GCC will complain about I64 and other Microsoft extensions, but will accept z, hh and ll.
printf is an alias for ms_printf when compiling for Windows, and an alias for gnu_printf otherwise.
Note that this check is completely orthogonal to the actual printf implementation being used. This is easy to see if you write your own printf-like function and put __attribute__((__format__ (FORMAT, ...))) on it - GCC will complain about different things depending on FORMAT, but you can do whatever you want inside the function.
Available printf implementations that i know of:
MinGW ANSI STDIO (compile with -D__USE_MINGW_ANSI_STDIO=1) in MinGW.org and MinGW-w64 toolchains. Complies with ms_printf (fully?) and gnu_printf format (partially - does not support positional arguments).
MSVCRT (compile without -D__USE_MINGW_ANSI_STDIO=1). Complies with ms_printf (duh...), compliance with gnu_printf is very low and depends on runtime version (old versions did not support ll, new ones do; z and hh are not supported in any version so far; GCC is blissfully unaware of these developments though, and assumes the worst case, msvcrt from VC 6.0 era, it seems).
gnulib. Complies with ms_printf and gnu_printf completely (or near-completely).
The stdio.h header in MinGW.org does not use attribute format.
The stdio.h header in MinGW-w64 uses attribute format gnu_printf for MinGW ANSI STDIO implementation, but does not use anything for MSVCRT implementation. FIXED: In newer versions of MinGW-w64 headers stdio.h will use attribute format ms_printf for MSVCRT implementation.
gnulib is fully aware of the difference between printf and gnu_printf, and will pick one or the other depending on some complicated macros (presumably, accompanying it with a proper implementation that supports what the format says it does).
Pieces of software that are known (at the moment) to have problems with GCC format checks:
glib - uses printf format, but implementation is from gnulib; there's an outstanding bug for changing it to gnu_printf
CPython - the code is full of z formats, but official binaries are built against MSVCRT; it also uses printf format in its extension headers, even though extensions often use z as well