Given the following:
len = strlen(str);
/* code that does not read from len */
len = new_len;
Can I depend on the compiler to remove the first line?
I'm writing a code generating script. The first line is generated in one place, while everything that follows is generated elsewhere (by a virtual function in a descendant class). Most of the time, len isn't needed. Sometimes though, it is. I wonder if I can just set it in all cases and let the compiler get rid of the line if it isn't required.
No, of course you cannot "depend" on the optimization choices done by the compiler.
They can change at the user's whim (compiler command line options), or with different compiler versions.
Since the behavior you're describing is not required by the language standard, you cannot depend on compilers to implement it.
On the other hand, you can surely test it, and try to "coax" it into being optimized out, by (again) asking your compiler to do as much optimization as possible, for instance.
Compilers are pretty clever. But to rely on the compiler removing "unused" function calls is probably not a good idea. For one thing, the compiler will NEED to understand the function you are calling (so strlen is a good example here, because most compilers understand what strlen does and how it affects other things) - if the function is not one that the compiler "understands", then it can't optimise it out.
What if you did:
x = printf("Hello, World!\n");
x = printf("World, Hello!\n");
Would you think they compiler had done the right thing by removing the first printf? Probably not... So, with any function the compiler can't determine "is side-effect free", the compiler MUST call the function, even if the result is not used. Side-effect free means under normal circumstances - e.g. there is a side-effect of calling strlen() with an invalid pointer - your code will probably crash - but that's not "normal circumstances" - you'd be pretty daft to use strlen() just to check if your pointer is a valid one or not, right?
So, in other words, you probably want to make sure your call to strlen() is really needed before you call strlen - or live with the fact that the compiler may wall generate an unnecessary strlen call.
Modern compilers do seem to do an excellent job of eliminating redundant assignments. I ran the following snippet through VS 2008 and gcc 4.6 and the call to strlen (the inlined code, rather) is in fact removed. Both compilers managed to see that the both something() and something_else() produce no side effects. Calls to these are removed as well.
This occurs at /O1 in VC and /O1 in gcc.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int something(int len) {
if(len != len) {
printf("%d\n", len);
}
return 0;
}
int something_else(int len) {
return len * len;
}
int main(void) {
char *s = malloc(1000);
size_t len;
strcpy(s, "Hello world");
len = strlen(s);
something(len);
printf("%s\n", s);
len += 8;
something_else(len);
len = 5;
printf("%d\n", len);
return 0;
}
It depends on compiler & the optimization level you specify.
Here it gets assigned initially from a function call. What if that function has a side effect?
So you cannot just assume the compiler to optimize it for you.
If the initial assignment statement is free from any side effects & you specify optimization level good enough (e.g. -O3 in case of gcc) it may optimize it for you.
Related
I'm learning C in school and am doing some input and string comparisons and am running into what looks like a casting issue.
Here is my code:
size_t unit_match_index(char *userInput) {
char* unit = malloc(strlen(userInput) + 1);
strcpy(unit, userInput);
//convert to lowercase
for (size_t i = 0; i < strlen(unit); ++i) {
unit[i] = tolower(unit[i]);
/*C6385: invalid data from 'unit': the readable size is 'strlen(userInput)+1' bytes, but 2bytes may be read
C6386: buffer overrun while writing to 'unit': the writable size is [same as above]
*/
}
//...
}
After doing a little bit of research, it looks like tolower() looks for an int and returns an int (2 bytes), and thinks strlen(userInput)+1 may equate to 1, making the total unit array size only 1 byte.
Is there something I should be doing to avoid this, or is this just the analyzer being a computer (computers are dumb)? I'm concerned because I will lose marks on my assignment if there are errors.
As suggested in an answer to this related question, these two warnings are caused by a "bug" in the MSVC Code Analyser.
I even tried the 'fix' I suggested in my answer to that question (that is, using char* unit = malloc(max(strlen(userInput), 0) + 1);) – but it didn't work in your code (not sure why).
However, what did work (and I have no idea why) is to use the strdup function in place of your calls to malloc and strcpy – it does the same thing but in one fell swoop.
Adding the casts (correctly)1 suggested in the comments, here's a version of your code that doesn't generate the spurious C6385 and C6386 warnings:
#include <stdlib.h>
#include <string.h>
#include <ctype.h>
size_t unit_match_index(char* userInput)
{
char* unit = strdup(userInput);
//convert to lowercase
for (size_t i = 0; i < strlen(unit); ++i) {
unit[i] = (char)tolower((unsigned char)unit[i]);
}
//...
return 0;
}
However, MSVC will now generate a different (but equally spurious) warning:
warning C4996: 'strdup': The POSIX name for this item is deprecated.
Instead, use the ISO C and C++ conformant name: _strdup. See online
help for details.
As it happens, the strdup function (without the leading underscore) is adopted as part of the ISO Standard since C23 (6/2019).
1 On the reasons for the casts when using the tolower function, see: Do I need to cast to unsigned char before calling toupper(), tolower(), et al.?. However, simply adding those casts does not silence the two code analysis warnings.
Consider this code:
int __attribute__((warn_unused_result)) foo(void)
{
return 42;
}
int main(void)
{
foo();
}
When compiled, it emits a warning:
$ gcc main.c
main.c: In function ‘main’:
main.c:8:5: warning: ignoring return value of ‘foo’, declared with attribute warn_unused_result [-Wunused-result]
8 | foo();
| ^~~~~
This is as expected. What I wonder is if there is any rationale why some many standard library functions is not declared with this attribute. I'm talking about functions like scanf where checking the return value is crucial in most cases, and also functions like malloc which is completely pointless if you do not use the return value. However, realloc seems to have this.
Is there any reason for not declaring scanf, malloc etc functions with __attribute__((warn_unused_result))? I think this could have prevented many bugs.
It may be enlightening to think about why these specific functions get warn_unused_result—this may help us figure out why other functions don’t get the attribute. On my Glibc system, there are only two such functions:
extern void *realloc (void *__ptr, size_t __size)
__THROW __attribute_warn_unused_result__ __attribute_alloc_size__ ((2));
extern void *reallocarray (void *__ptr, size_t __nmemb, size_t __size)
__THROW __attribute_warn_unused_result__
__attribute_alloc_size__ ((2, 3));
On my macOS system, there are three:
void *malloc(size_t __size) __result_use_check __alloc_size(1);
void *calloc(size_t __count, size_t __size) __result_use_check __alloc_size(1,2);
void *realloc(void *__ptr, size_t __size) __result_use_check __alloc_size(2);
So, why does this super-useful attribute only get used on realloc() and another near-identical function in Glibc?
The answer is because this attribute is designed to prevent a very specific bug. Let's say we have a 100-element array and want to resize it to add a 101st element (at index 100) (and let's ignore error handling):
// Create 100-element array.
int *arr = malloc(sizeof(int) * 100);
arr[99] = 1234;
// Resize and add a 101st element.
realloc(arr, sizeof(int) * 101); // bug
arr[100] = 1234;
Spot the bug? This is a serious memory error but it is also a memory error that may or may not get noticed very quickly. The original malloc will often round up to a larger size in the first place, and if realloc succeeds, you may end up with the same address anyway. The actual bug will only get noticed once you start writing into another object, or once another object gets allocated at the old address for arr.
This is something that might take you a while to debug.
So, the warning is designed to catch this serious bug that is difficult to debug. Compare with the following:
malloc(100 * sizeof(int));
This bug isn’t very serious at all—it’s just a memory leak. It’s not even incorrect code!
As a fun exercise, try putting the above code into Godbolt and looking at the assembly output—GCC will actually remove the call to malloc entirely.
As for printf, scanf, and others—
int r = printf(...);
if (r == -1) {
abort();
}
If GCC gave warnings for not checking the return of printf, people would get frustrated because they would see too many warnings in their code—and they would respond by turning the warnings off.
So sometimes it is better to only trigger warnings for the most serious cases, and let some other cases slide.
As a minor technical note, this is somewhat more an issue for the standard library maintainers, like the Glibc team, rather than GCC or Clang. Even though the standard library and compiler are tightly integrated, they are separate projects—and you can even swap out different standard libraries. For example, you might use musl, glibc, or Apple’s libSystem.
It is not portable.
Standard does not require it.
You can always write the wrapper if you want be warned.
static inline void * __attribute__((warn_unused_result, always_inline)) mymalloc(size_t size)
{
return malloc(size);
}
int main(void)
{
mymalloc(300);
}
I understood, that __builtin_object_size should give information about the size of an object, if it can be determined at compile time or the relevant call to malloc uses the attribute alloc_size. I think I compiled the code ok with gcc 9.3 on ubuntu 20.04 with
gcc -g -std=gnu11 -Wall -O2 compstrcpyos.c -o compstrcpyos.exe -lbsd
The code should at least find out in doit the size, but it doesn't. I get -1 as result.
void doit(char *buf1, char *buf2, const char *src) {
printf("bos1=%zd, bos2=%zd\n",
__builtin_object_size(buf1, 0),
__builtin_object_size(buf2, 0));
strlcpy(buf1, src, __builtin_object_size(buf1, 0));
strlcpy(buf2, src, __builtin_object_size(buf2, 0));
for (size_t i=0; i < strlen(buf1); ++i) {
buf1[i] = tolower(src[i]);
}
for (size_t i=0; i < strlen(buf2); ++i) {
buf2[i] = tolower(src[i]);
}
}
void *malloc(size_t s) __attribute__((alloc_size (1)));
int main(int argc, char *argv[]) {
if (argc < 2) { return -1; }
char buf1[20];
char *buf2 = malloc(20);
printf("bos1=%zd, bos2=%zd\n",
__builtin_object_size(buf1, 0),
__builtin_object_size(buf2, 0));
doit(buf1, buf2, argv[1]);
printf("%s\n%s\n", buf1, buf2);
}
Running the code gives.
$ ./compstrcpyos.exe ABCDefghijklmnopq
bos1=20, bos2=20
bos1=-1, bos2=-1
abcdefghijklmnopq
abcdefghijklmnopq
I tried to add the option -fsanitize=object-size, I tried with the constants 1, 2 and 3 instead 0 in the call to __builtin_object_size, but never got 20 in doit. What do I need to do get 20, 20 on the second line (that is in doit) as well.
Any help appreciated.
I think you are over-estimating the reach of the compiler:
I understood, that __builtin_object_size should give information about the size of an object, if it can be determined at compile time
Up to here, fine.
or the relevant call to malloc uses the attribute alloc_size.
But this is not a disjunction ("or"). The compiler can only know what is known at compile time. So there is no alternative to being able to determine the size at compile-time. Now, if an object is allocated dynamically, the compiler may still be able to determine its size, provided that:
It is allocated by a function which the compiler recognises as a dynamic allocation function, and
The compiler can deduce from the argument(s) to the dynamic allocation function what the size of the allocated object is, and
The compiler can determine the values of those argument(s).
For the first two points, the compiler relies on the alloc_size attribute. If the allocation function is not marked with this attribute, the compiler cannot determine the size of the object. So it's a conjunction ("and"). But even that is not sufficient; the compiler also needs to be able to determine the value of the arguments used when the object was allocated.
For that to work through a function call, the function call basically must be inlined. Static functions will normally be inlined if they are only called in one place. (They may be inlined elsewhere, but a large function is unlikely to be inlined if it is called -- or potentially called -- from more than one call site.)
In short, you would have been more accurate had you said:
__builtin_object_size should give information about the size of an object, if it can be determined at compile time, which requires the relevant call to malloc to have the attribute alloc_size and that its arguments can be determined at compile time.
Indirecting through the function call erases the ability of the compiler to feel confident in determining the sizes. In theory, that function could be called by other callers, including ones outside the translation unit (file). If you add a static qualifier to the doit function declaration that would signal that there are never external callers, and it's then possible (in this particular example) to make a definitive compile-time determination.
For what it's worth, __builtin_object_size is not the idiomatic way of saying "how big is this thing". It appears intended to be used in ways like the example here in super defensive coding inside one scope.
The idiomatic way of passing a buffer with an explicit size to a function is to pass the buffer and also the size of it as separate arguments (or as part of a context struct or whatnot)
Since I found this particular documentation on https://www.tutorialspoint.com/c_standard_library/c_function_rand.htm,I have been thinking about this particular line of code srand((unsigned)time(&t));.Whenever I had to generate some stuff,I used srand(time(NULL)) in order not to generate the same stuff everytime I run the program,but when I came across this,I have been wondering :Is there any difference between srand((unsigned)time(&t)) and srand(time(NULL))?Because to me they seem like they do the same thing.Why is a time_t variable used?And why is the adress operator used in srand()?
#include <stdio.h>
#include<stdlib.h>
int main(){
int i,n;
time_t t;
n = 5;
srand((unsigned)time(&t));
for (i = 0; i < n; i++) {
printf("%d\n", rand() % 50);
}
return(0);
}
Yes, it will yield the same result. But the example is badly written.
I would be careful reading Tutorialspoint. It's a site known for bad C code, and many bad habits you see in questions here at SO can be traced to that site. Ok, it's anecdotal evidence, but I did ask a user here why they cast the result of malloc, and they responded that they had learned that on Tutorialspoint. You can actually see (at least) four examples in this short snippet.
They cast the result from the call to time() which is completely unnecessary and just clutters the code.
For some reason they use the variable t, which is completely useless in this example. If you read the documentation for time() you'll see that just passing NULL is perfectly adequate in this example.
Why use the variable n? For this short example it's perfectly ok with a hardcoded value. And when you use variables to avoid hardcoded values, you should declare them const and give them a much more descriptive name than n. (Ok, I realize I was a bit on the edge when writing this. Omitting const isn't that big of a deal, even if it's preferable. And "n" is a common name meaning "number of iterations". And using a variable instead of a hard coded value is in general a good thing. )
Omitted #include<time.h> which would be ok if they also omitted the rest of the includes.
Using int main() instead of int main(void).
For 5, I'd say that in most cases, this does not matter for the main function, but declaring other functions as for example int foo() with empty parenthesis instead of int foo(void) could cause problems, because they mean different things. From the C standard:
The use of function declarators with empty parentheses (not prototype-format parameter type declarators) is an obsolescent feature.
Here is a question related to that: What are the semantics of function pointers with empty parentheses in each C standard?
One could also argue about a few other things, but some people would disagree about these.
Why declare i outside the for loop? Declaring it inside have been legal since C99, which is 20 years old.
Why end the function with return 0? Omitting this is also ok since C99. You only need to have a return in main if you want to return something else than 0. Personally, in general I find "it's good practice" as a complete nonsense statement unless there are some good arguments to why it should be good practice.
These are good to remember if your goal is to maintain very old C code in environments where you don't have compilers that supports C99. But how common is that?
So if I got to rewrite the example at tutorialspoint, i'd write it like this:
#include<stdio.h>
#include<stdlib.h>
#include<time.h>
int main(void){
srand(time(NULL));
for (int i = 0; i < 5; i++) {
printf("%d\n", rand() % 50);
}
}
Another horrible example can be found here: https://www.tutorialspoint.com/c_standard_library/c_function_gets.htm
The function gets is removed from standard C, because it's very dangerous. Yet, the site does not even mention that.
Also, they teach you to cast the result of malloc https://www.tutorialspoint.com/c_standard_library/c_function_malloc.htm which is completely unnecessary. Read why here: Do I cast the result of malloc?
And although they mention that malloc returns NULL on failure, they don't show in the examples how to properly error check it. Same goes for functions like scanf.
I used the following piece of code to read data from files as part of a larger program.
double data_read(FILE *stream,int code) {
char data[8];
switch(code) {
case 0x08:
return (unsigned char)fgetc(stream);
case 0x09:
return (signed char)fgetc(stream);
case 0x0b:
data[1] = fgetc(stream);
data[0] = fgetc(stream);
return *(short*)data;
case 0x0c:
for(int i=3;i>=0;i--)
data[i] = fgetc(stream);
return *(int*)data;
case 0x0d:
for(int i=3;i>=0;i--)
data[i] = fgetc(stream);
return *(float*)data;
case 0x0e:
for(int i=7;i>=0;i--)
data[i] = fgetc(stream);
return *(double*)data;
}
die("data read failed");
return 1;
}
Now I am told to use -O2 and I get following gcc warning:
warning: dereferencing type-punned pointer will break strict-aliasing rules
Googleing I found two orthogonal answers:
Concluding: there's no need to worry; gcc tries to be more law obedient than
the actual law.
vs
So basically if you have an int* and a float* they are not allowed to point to the same memory location. If your code does not respect this, then the compiler's optimizer will most likely break your code.
In the end I don't want to ignore the warnings. What would you recommend?
[update] I substituted the toy example with the real function.
The problem occurs because you access a char-array through a double*:
char data[8];
...
return *(double*)data;
But gcc assumes that your program will never access variables though pointers of different type. This assumption is called strict-aliasing and allows the compiler to make some optimizations:
If the compiler knows that your *(double*) can in no way overlap with data[], it's allowed to all sorts of things like reordering your code into:
return *(double*)data;
for(int i=7;i>=0;i--)
data[i] = fgetc(stream);
The loop is most likely optimized away and you end up with just:
return *(double*)data;
Which leaves your data[] uninitialized. In this particular case the compiler might be able to see that your pointers overlap, but if you had declared it char* data, it could have given bugs.
But, the strict-aliasing rule says that a char* and void* can point at any type. So you can rewrite it into:
double data;
...
*(((char*)&data) + i) = fgetc(stream);
...
return data;
Strict aliasing warnings are really important to understand or fix. They cause the kinds of bugs that are impossible to reproduce in-house because they occur only on one particular compiler on one particular operating system on one particular machine and only on full-moon and once a year, etc.
It looks a lot as if you really want to use fread:
int data;
fread(&data, sizeof(data), 1, stream);
That said, if you do want to go the route of reading chars, then reinterpreting them as an int, the safe way to do it in C (but not in C++) is to use a union:
union
{
char theChars[4];
int theInt;
} myunion;
for(int i=0; i<4; i++)
myunion.theChars[i] = fgetc(stream);
return myunion.theInt;
I'm not sure why the length of data in your original code is 3. I assume you wanted 4 bytes; at least I don't know of any systems where an int is 3 bytes.
Note that both your code and mine are highly non-portable.
Edit: If you want to read ints of various lengths from a file, portably, try something like this:
unsigned result=0;
for(int i=0; i<4; i++)
result = (result << 8) | fgetc(stream);
(Note: In a real program, you would additionally want to test the return value of fgetc() against EOF.)
This reads a 4-byte unsigned from the file in little-endian format, regardless of what the endianness of the system is. It should work on just about any system where an unsigned is at least 4 bytes.
If you want to be endian-neutral, don't use pointers or unions; use bit-shifts instead.
Using a union is not the correct thing to do here. Reading from an unwritten member of the union is undefined - i.e. the compiler is free to perform optimisations that will break your code (like optimising away the write).
This doc summarizes the situation: http://dbp-consulting.com/tutorials/StrictAliasing.html
There are several different solutions there, but the most portable/safe one is to use memcpy(). (The function calls may be optimized out, so it's not as inefficient as it appears.) For example, replace this:
return *(short*)data;
With this:
short temp;
memcpy(&temp, data, sizeof(temp));
return temp;
Basically you can read gcc's message as guy you are looking for trouble, don't say I didn't warn ya.
Casting a three byte character array to an int is one of the worst things I have seen, ever. Normally your int has at least 4 bytes. So for the fourth (and maybe more if int is wider) you get random data. And then you cast all of this to a double.
Just do none of that. The aliasing problem that gcc warns about is innocent compared to what you are doing.
The authors of the C Standard wanted to let compiler writers generate efficient code in circumstances where it would be theoretically possible but unlikely that a global variable might have its value accessed using a seemingly-unrelated pointer. The idea wasn't to forbid type punning by casting and dereferencing a pointer in a single expression, but rather to say that given something like:
int x;
int foo(double *d)
{
x++;
*d=1234;
return x;
}
a compiler would be entitled to assume that the write to *d won't affect x. The authors of the Standard wanted to list situations where a function like the above that received a pointer from an unknown source would have to assume that it might alias a seemingly-unrelated global, without requiring that types perfectly match. Unfortunately, while the rationale strongly suggests that authors of the Standard intended to describe a standard for minimum conformance in cases where a compiler would otherwise have no reason to believe that things might alias, the rule fails to require that compilers recognize aliasing in cases where it is obvious and the authors of gcc have decided that they'd rather generate the smallest program it can while conforming to the poorly-written language of the Standard, than generate code which is actually useful, and instead of recognizing aliasing in cases where it's obvious (while still being able to assume that things that don't look like they'll alias, won't) they'd rather require that programmers use memcpy, thus requiring a compiler to allow for the possibility that pointers of unknown origin might alias just about anything, thus impeding optimization.
Apparently the standard allows sizeof(char*) to be different from sizeof(int*) so gcc complains when you try a direct cast. void* is a little special in that everything can be converted back and forth to and from void*.
In practice I don't know many architecture/compiler where a pointer is not always the same for all types but gcc is right to emit a warning even if it is annoying.
I think the safe way would be
int i, *p = &i;
char *q = (char*)&p[0];
or
char *q = (char*)(void*)p;
You can also try this and see what you get:
char *q = reinterpret_cast<char*>(p);