So there's this gcc warning that bothers me:
warning: assuming signed overflow does not occur when simplifying multiplication
The code it points at looks like this:
/* Move the memory block of entries after the removed one - if any. */
if (database->entries + database->entries_size - 1 != database_entry) {
memmove(
database_entry,
database_entry + 1,
sizeof(spm_database_entry_t)
* (
(database->entries + database->entries_size)
- database_entry - 1
)
);
}
As you can easily guess it moves part of the container's memory after element removal to allow its further reallocation (shrinking).
database_entry is a pointer of type spm_database_entry_t* to the removed element
database->entries is a pointer to array of spm_database_entry_t
database->entries_size is a size_t representing number database->entries elements before the removal
How to get rid of the warning? Can I prevent the multiplication simplifying or maybe there's better way to calculate how much memory needs moving?
edit
Are you sure that database_entry < database->entries + database->entries_size?
Positive.
What are the compiler flags you're using?
-Wall -Wextra -Wshadow -Wpointer-arith -Wcast-qual -Wstrict-prototypes
-Wmissing-prototypes -Wdeclaration-after-statement -Wwrite-strings
-Winit-self -Wcast-align -Wstrict-aliasing=2 -Wformat=2
-Wmissing-declarations -Wmissing-include-dirs -Wno-unused-parameter
-Wuninitialized -Wold-style-definition -Wno-missing-braces
-Wno-missing-field-initializers -Wswitch-default -Wswitch-enum
-Wbad-function-cast -Wstrict-overflow=5 -Winline -Wundef -Wnested-externs
-Wunreachable-code -Wfloat-equal -Wredundant-decls
-pedantic -ansi
-fno-omit-frame-pointer -ffloat-store -fno-common -fstrict-aliasing
edit2
Casting to unsigned int before the multiplication seem to do the trick, but casting to size_t doesn't. I don't get it - standard says size_t is always unsigned...
edit3
If context can be of any use: https://github.com/msiedlarek/libspm/blob/master/libspm/database.c#L116
edit4
Solution based on steveha's answer:
/* Calculate how meny entries need moving after the removal. */
size_t entries_to_move = (
(database->entries + database->entries_size)
- database_entry - 1
);
/* Move the memory block of entries after the removed one - if any. */
memmove(
database_entry,
database_entry + 1,
sizeof(spm_database_entry_t) * entries_to_move
);
Personally, I favor additional intermediate temporary variables. The compiler will see that they are used for only the one calculation, and will optimize the variables away; but in a debug build, you can single-step, examine the variables, and make sure it really is doing what you expect.
/* Move the memory block of entries after the removed one - if any. */
assert(database_entry >= database->entries &&
database_entry < database->entries + database->entries_size);
size_t i_entry = database_entry - database->entries;
size_t count_to_move = (database->entries_size - 1) - i_entry;
size_t bytes_to_move = count_to_move * sizeof(spm_database_entry_t);
memmove(database_entry, database_entry + 1, bytes_to_move);
Most of the time, bytes_to_move will not be 0, but if it is 0 then memmove() will simply move 0 bytes and no harm done. So we can remove that if statement, unless you had something else inside it that needs doing only when the move happens.
Also, if you do it this way, and you are still getting the warning, you will get a line number that will point you right at what the compiler is worried about.
I suspect the issue relates to the fact that size_t, as returned by sizeof(spm_database_entry_t), is always an unsigned type (usually just a type synonym for unsigned int or unsigned long int, if I remember correctly). There's a theoretical possibility, however, that if the value of database_entry exceeds database->entries + database->entries_size that you'll end up multiplying a signed quantity by an unsigned type, raising the possibility of bugs or integer overflows. Normally, when signed and unsigned types get mixed like this, the smaller type is cast/coerced into the larger type, or, if they're equally ranked, the signed type is coerced to an unsigned. I don't know what the rest of your code looks like, so it's difficult to suggest an improvement.
Related
Given this snippet
#include <inttypes.h>
uint8_t test(uint32_t foo)
{
if (foo > 0xFFFFFFFF)
{
return 0;
}
if (foo < 0)
{
return 0;
}
return 1;
}
compiling with -Wall -Wextra using gcc 12.2 gives only the following warning:
warning: comparison of unsigned expression in '< 0' is always false [-Wtype-limits]
9 | if (foo < 0)
| ^
I don't understand why the first line if (foo > 0xFFFFFFFF) does not trigger the same warning.
Trying if (foo > UINT32_MAX) does not trigger any compiler warning.
is it a bug or a feature?
These warnings are not mandated by the language rules, it's up to the compiler vendors to add them, or not. The fact the gcc warns you about the second if can be considered a courtesy. For this reason I believe we can't consider this a bug, unless you can spot this particular feature on the gcc documentation, that could change my view.
I checked clang for these warnings and sure enough it doesn't warn you at all, at least not with those flags:
https://godbolt.org/z/c7hbM3K5d
It's only when you try -Weverything it warns you of both cases:
<source>:5:13: warning: result of comparison 'uint32_t' (aka 'unsigned int') > 4294967295 is always false [-Wtautological-type-limit-compare]
if (foo > 0xFFFFFFFF)
~~~ ^ ~~~~~~~~~~
<source>:9:13: warning: result of comparison of unsigned expression < 0 is always false [-Wtautological-unsigned-zero-compare]
if (foo < 0)
~~~ ^ ~
https://godbolt.org/z/9b9q13rTe
+1 for clang in my book.
If you want to see these warnings in your build be sure to use the specific flag (-Wtautological-unsigned-zero-compare and -Wtautological-type-limit-compare), using -Weverything would be too verbose and flag many situations you don't need as mentioned in the comments.
Back to gcc, if you use a wider type for the comparison like for example if (foo > 0xFFFFFFFF0) then you get your warning:
<source>:5:13: warning: comparison is always false due to limited range of data type [-Wtype-limits]
5 | if (foo > 0xFFFFFFFF0)
| ^
https://godbolt.org/z/Edvz7aq3b
We can assume they forgot to include the max value for uint32_t, it can be an off by one mistake, or it can be that the warning implementation focuses on types rather than values, one can only speculate. You can file a report if you're adamant on having this particular warning working for UINT32_MAX, or at least get to the end of it, and find out what is the reason for this behavior.
Problem
In using automated CI tests I found some code, which breaks if optimization of gcc is set to -O2. The code should increment a counter if a double value crosses a threshold in either direction.
Workaround
Going down to -O1 or using -ffloat-store option works around the problem.
Example
Here is a small example which shows the same problem.
The update() function should return true whenever a sequence of *pNextState * 1e-6 crosses a threshold of 0.03.
I used call by reference because the values are part of a large struct in the full code.
The idea behind using < and >= is that if a sequence hits the value exactly, the function should return 1 this time and return 0 the next cycle.
test.h:
extern int update(double * pState, double * pNextState);
test.c:
#include "test.h"
int update(double * pState, double * pNextState_scaled) {
static double threshold = 0.03;
double oldState = *pState;
*pState = *pNextState_scaled * 1e-6;
return oldState < threshold && *pState >= threshold;
}
main.c:
#include <stdio.h>
#include <stdlib.h>
#include "test.h"
int main(void) {
double state = 0.01;
double nextState1 = 20000.0;
double nextState2 = 30000.0;
double nextState3 = 40000.0;
printf("%d\n", update(&state, &nextState1));
printf("%d\n", update(&state, &nextState2));
printf("%d\n", update(&state, &nextState3));
return EXIT_SUCCESS;
}
Using gcc with at least -O2 the output is:
0
0
0
Using gcc with -O1, -O0 or -ffloat-store produces the desired output
0
1
0
As i understand the problem from debugging a problem arises if the compiler optimizes out the local variable oldstate on stack and instead compares against an intermediate result in an floting point register with higher precision (80bit) and the value *pState is a tiny bit smaller than the threshold.
If the value for comparison is stored in 64bit precision, the logic can't miss crossing the threshold. Because of the multiplication by 1e-6 the result is probably stored in an floating point register.
Would you consider this a gcc bug?
clang does not show the problem.
I am using gcc version 9.2.0 on an Intel Core i5, Windows and msys2.
Update
It is clear to me that floating point comparison is not exact and i would consider the following result as valid:
0
0
1
The idea was that, if (*pState >= threshold) == false in one cycle then comparing the same value (oldstate = *pState) against the same threshold in a subsequent call (*pState < threshold) has to be true.
[Disclaimer: This is a generic, shoot-from-the-hip answer. Floating-point issues can be subtle, and I have not analyzed this one carefully. Once in a while, suspicious-looking code like this can be made to work portably and reliably after all, and per the accepted answer, that appears to be the case here. Nevertheless, the general-purpose answer stands in the, well, general case.]
I would consider this a bug in the test case, not in gcc. This sounds like a classic example of code that's unnecessarily brittle with respect to exact floating-point equality.
I would recommend either:
rewriting the test case, or perhaps
removing the test case.
I would not recommend:
working around it by switching compilers
working around it by using different optimization levels or other compiler options
submitting compiler bug reports [although in this case, it appears there was a compiler bug, although it needn't be submitted as it has been submitted already]
I've analysed your code and my take is that it is sound according the standard but you are hit by gcc bug 323 about which you may find more accessible information in gcc FAQ.
A way to modify your function and render it robust in presence of gcc bug is to store the fact that the previous state was below the threshold instead (or in addition) of storing that state. Something like this:
int update(int* pWasBelow, double* pNextState_scaled) {
static double const threshold = 0.03;
double const nextState = *pNextState_scaled * 1e-6;
int const wasBelow = *pWasBelow;
*pWasBelow = nextState < threshold;
return wasBelow && !*pWasBelow;
}
Note that this does not guarantee reproductibility. You may get 0 1 0 in one set-up and 0 0 1 in another, but you'll detect the transition sooner or later.
I am putting this as an answer because I don't think I can do real code in a comment, but #SteveSummit should get the credit - I would not have likely found this without their comment above.
The general advice is: don't do exact comparisons with floating point values, and it seems like that's what this is doing. If a computed value is almost exactly 0.03 but due to internal representations or optimizations, it's ever so slightly off and not exactly, then it's going to look like a threshold crossing.
So one can resolve this by adding an epsilon for how close one can be to the threshold without having considered to cross it yet.
int update(double * pState, double * pNextState_scaled) {
static const double threshold = 0.03;
static const double close_enough = 0.0000001f; // or whatever
double oldState = *pState;
*pState = *pNextState_scaled * 1e-6;
// if either value is too close to the threshold, it's not a crossing
if (fabs(oldState - threshold) < close_enough) return 0;
if (fabs(*pState - threshold) < close_enough) return 0;
return oldState < threshold && *pState >= threshold;
}
I imagine you'd have to know your application in order to know how to tune this value appropriately, but a coupla orders of magnitude smaller than the value you're comparing with seems in the neighborhood.
GCC versions released before May 2019 (and maybe later) fail to optimize this piece of code:
// Replace the k'th byte within an int
int replace_byte(int v1 ,char v2, size_t k)
{
memcpy( (void*) (((char*)&v1)+k) , &v2 , sizeof(v2) );
return v1;
}
as can be seen here (GodBolt): clang optimizes this code properly GCC and MSVC do not. This is GCC bug 90271, which will at some point be fixed. But - it won't be fixed for GCC versions that are out today, and I want to write this code today...
So: Is there a workaround which will make GCC generate the same code as clang for this function, or at least - code of comparable performance, keeping data in registers and not resorting to pointers and the stack?
Notes:
I marked this as C, since the code snippet is in C. I assume a workaround, if one exists, can also be implemented in C.
I'm interested in optimizing both the non-inlined function and the inlined version.
This question is related to this one, but only regards GCC and the specific approach in the piece of code here; and is in C rather than C++.
This makes the non-inlined version a little longer, but the inlined version is optimized for all three compilers:
int replace_bytes(int v1 ,char v2, size_t k)
{
return (v1 & ~(0xFF << k * 8)) | ((unsigned char)v2 << k * 8);
}
The cast of v2 to an unsigned char before the shift is necessary if char is a signed type. When that's the case, without the case v2 will be sign extended to an integer, which will cause unwanted bits to be set to 1 in the result.
I was looking for the source for glibc's rand() function, which an answer here links to.
Following the link, I'm puzzled about the code for the __random_r() TYPE_0 branch:
int32_t val = state[0];
val = ((state[0] * 1103515245) + 12345) & 0x7fffffff;
state[0] = val;
*result = val;
What is the point of the val variable, getting assigned and then immediately overwritten? The random_data struct that holds state is nothing unusual.
As one would expect, compiling with -O2 on godbolt gives the same code if you just eliminate val. Is there a known reason for this pattern?
UPDATE: This seems it was an aberration in the version linked to from that answer, I've updated the links there to the 2.28 version. It might have been something that was done temporarily to aid in debugging by making the contents of state[0] easier to see in a local watchlist?
Wow, that is indeed some unbelievable garbage code.
There is no justification for this.
And not only is initialization of val not needed, the fact is that state[0] is an int32_t, and multiplication by 1103515245 will trigger undefined behaviour in GCC (integer overflow) on any platform with 32-bit ints (= basically every one). And GCC is the compiler most often used to compile Glibc.
As noted by HostileFork, the code in more recent 2.28 reads:
int32_t val = ((state[0] * 1103515245U) + 12345U) & 0x7fffffff;
state[0] = val;
*result = val;
With this, not only is the useless initialization removed, but the U suffix makes the multiplication happen with unsigned integers, avoiding undefined behaviour. The & 0x7fffffff ensures that the resulting value fits into an int32_t and is positive.
I have the following program
#include <stdio.h>
#include <stdlib.h>
#include <inttypes.h>
int main(void) {
uint16_t o = 100;
uint32_t i1 = 30;
uint32_t i2 = 20;
o = (uint16_t) (o - (i1 - i2)); /*Case A*/
o -= (uint16_t) (i1 - i2); /*Case B*/
(void)o;
return 0;
}
Case A compiles with no errors.
Case B causes the following error
[error: conversion to ‘uint16_t’ from ‘int’ may alter its value [-Werror=conversion]]
The warning options I'm using are:
-Werror -Werror=strict-prototypes -pedantic-errors -Wconversion -pedantic -Wall -Wextra -Wno-unused-function
I'm using GCC 4.9.2 on Ubuntu 15.04 64-bits.
Why do I get this error in Case B but not in Case A?
PS:
I ran the same example with clang compiler and both cases are compiled fine.
Integer Promotion is a strange thing. Basically, all integer values, of any smaller size, are promoted to int so they can be operated on efficiently, and then converted back to the smaller size when stored. This is mandated by the C standard.
So, Case A really looks like this:
o = (uint16_t) ((int)o - ((uint32_t)i1 - (uint32_t)i2));
(Note that uint32_t does not fit in int, so needs no promotion.)
And, Case B really looks like this:
o = (int)o - (int)(uint16_t) ((uint32_t)i1 - (uint32_t)i2);
The main difference is that Case A has an explicit cast, whereas Case B has an implicit conversion.
From the GCC manual:
-Wconversion
Warn for implicit conversions that may alter a value. ....
So, only Case B gets a warning.
Your case B is equivalent to:
o = o - (uint16_t) (i1 - i2); /*Case B*/
The result is an int which may not fit in uint16_t, so, per your extreme warning options, it produces a warning (and thus an error since you're treating warnings as errors).