I have read the related questions to this Lint warning about the suspicious truncation, but here it is a purely C case.
The following line is where the Warning #647 pops up:
pCont->sig -= (signed int64_t)((sub2 << 8)/pCont->freq + 1);
where pCont->sig is also 64 bit signed (type signed int64_t), and both sub2 and freq are 32 bit unsigned. All this is compiled with armcc.
Already tried, without success, to cast the 1 to unsigned 32 bit, but the problem persists.
Any idea on what I might try, or what is going wrong here?
From this reference about the warning
For example:
(long) (n << 8)
might elicit this message if n is unsigned int, whereas
(long) n << 8
would not. In the first case, the shift is done at int precision
and the high order 8 bits are lost even though there is a
subsequent conversion to a type that might hold all the bits. In
the second case, the shifted bits are retained.
This seems to fit your case exactly and also show you how to fix it.
Related
In case of unsigned short I shifted 383 by 11 positions towards left and again in the same instruction shifted it by 15 positions right, I expected the value to be 1 but it was 27. But when I used both the shift operations in different instructions one after another(first left shift and then right), output was 1.
here is a sample code :-
unsigned short seed = 383;
printf("size of short: %d\n",sizeof(short));
unsigned short seedout,seed1,seed2,seed3,seedout1;
seed1 = (seed<<11);
seed2 = (seed1>>15);
seed3 = ((seed<<11)>>15);
printf("seed1 :%d\t seed2: %d\t seed3: %d\n",seed1,seed2,seed3);
and its output was :
size of short: 2
seed1 :63488 seed2: 1 seed3: 23
seedout1: 8 seedout :382
Process returned 0 (0x0) execution time : 0.154 s
For clarity, you compare
unsigned short seed1 = (seed<<11);
unsigned short seed2 = (seed1>>15);
on one hand and
unsigned short seed3 = ((seed<<11)>>15);
on the other hand.
The first one takes the result of the shift operation, stores it in an unsigned short variable (which apparently is 16 bit on your platform) and shifts this result right again.
The second one shifts the result immediately.
The reason why this is different resp. the bits which are shifted out to the left are retained is the following:
Although seed is unsigned short, seed<<11 is signed int. Thus, these bits are not cut off as it is the case when storing the result, but they are kept in the intermediate signed int. Only the assignment to seed1 makes the value unsigned short, which leads to a clipping of the bits.
In other words: your second example is merely equivalent to
int seed1 = (seed<<11); // instead of short
unsigned short seed2 = (seed1>>15);
Regarding left shifting, type signedness & implicit promotion:
Whenever something is left shifted into the sign bit of a signed integer type, we invoke undefined behavior. Similarly, we also invoke undefined behavior when left-shifting a negative value.
Therefore we must always ensure that the left operand of << is unsigned. And here's the problem, unsigned short is a small integer type, so it is subject to implicit type promotion whenever used in an expression. Shift operators always integer promote the left operand:
C17 6.5.7:
The integer promotions are performed on each of the operands. The type of the result is that of the promoted left operand.
(This makes shifts in particular a special case, since they don't care about the type of the right operand but only look at the left one.)
So in case of a 16 bit system, you'll run into the case where unsigned short gets promoted to unsigned int, because a 16 bit int cannot hold all values of a 16 bit unsigned short. And that's fine, it's not a dangerous conversion.
On a 32 bit system however, the unsigned short gets promoted to int which is signed. Should you left shift a value like 0x8000 (MSB) set 15 bits or more, you end up shifting data into the sign bit of the promoted int, which is a subtle and possibly severe bug. For example, this prints "oops" on my Windows computer:
#include <stdio.h>
int main (void)
{
unsigned short x=0x8000;
if((x<<16) < 0) ) // undefined behavior
puts("oops");
}
But the compiler could as well have assumed that a left shift of x can never result in a value < x and removed the whole machine code upon optimization.
We need to be sure that we never end up with a signed type by accident! Meaning we must know how implicit type promotion works in C.
As for left-shifting unsigned int or larger unsigned types, that's perfectly well-defined as long as we don't shift further than the width of the (promoted) type itself (more than 31 bits on a 32 bit system). Any bits shifted out well be discarded, and if you right shift, it will always be a logical shift where zeroes are shifted in from the right.
To answer the actual question:
Your unsigned short is integer promoted to an int on a 32 bit system. This allows to shift beyond the 16 bits of an unsigned short, but if you discard those extra bits by saving the result in an unsigned short, you end up with this:
383 = 0x17F
0x17f << 11 = 0xBF800
0xBF800 truncated to 16 bits = 0xF800 = 63488
0xF800 >> 15 = 0x1
However, if skipping the middle step truncation to 15 bits, you have this instead:
0xBF800 >> 15 = 0x17 = 23
But again, this is only by luck since this time we didn't end up shifting data into the sign bit.
Another example, when executing this code, you might expect to get either the value 0 or the value 32768:
unsigned short x=32768;
printf("%d", x<<16>>16);
But it prints -32768 on my 2's complement PC. The x<<16 invokes undefined behavior, and the >>16 then apparently sign extended the result.
These kind of subtle shift bugs are common, particularly in embedded systems. A frightening amount of all C programs out there are written by people who didn't know about implicit promotions.
I shifted 383 by 11 positions towards left and again in the same instruction shifted it by 15 positions right, I expected the value to be 1 but it was 27
Simple math, you've shifted it 4 bits to the right, which is equivalent to dividing by 16.
Divide 383 by 16, and you get 27 (integer-division, of course).
Note that the "simply shifted it 4 bits" part holds because:
You've used an unsigned operand, which means that you did not "drag 1s" when shifting right
The shift-left operation likely returns an unsigned integer (32 bits) on your platform, so no data was loss during that part.
BTW, with regards to the 2nd bullet above - when you do this in parts and store the intermediate result into an unsigned short, you do indeed lose data and get a different result.
In other words, when doing seed<<11, the compiler uses 32-bit operations, and when storing it into seed1, only the LSB part of the previous result is preserved.
EDIT:
27 above should be 23. I copied that from your description without checking, though I see that you did mention 23 further down your question, so I'm assuming that 27 was a simple typo...
I have a function that takes an int data_length and does the following:
unsigned char *message = (unsigned char*)malloc(65535 * sizeof(char));
message[2] = (unsigned char)((data_length >> 56) & 255);
I'm getting the following:
warning: right shift count >= width of type [-Wshift-count-overflow]
message[2] = (unsigned char)((data_length >> 56) & 255);
The program works as expected, but how can I remove the compiler warning (without disabling it)?
Similar questions didn't seem to use a variable as the data to be inserted so it seemed the solution was to cast them to int or such.
Shifting by an amount greater than the bit width of the type in question is not allowed by the standard, and doing so invokes undefined behavior.
This is detailed in section 6.5.7p3 of the C standard regarding bitwise shift operators.
The integer promotions are performed on each of the operands. The
type of the result is that of the promoted left operand. If
the value of the right operand is negative or is greater than
or equal to the width of the promoted left operand, the behavior is
undefined.
If the program appears to be working, it is by luck. You could make a unrelated change to your program or simply build it on a different machine and suddenly things will stop working.
If the size of data_length is 32 bits or less, then shifting right by 56 is too big. You can only shift by 0 - 31.
The problem is simple. You're using data_length as int when it should be unsigned as negative lengths hardly make sense. Also to be able to shift 56 bits the value must be at least 56 57 bits wide. Otherwise the behaviour is undefined.
In practice processors are known to do wildly different things. In one, shifting a 32-bit value right by 32 bits will clear the variable. In another, the value is shifted by 0 bits (32 % 32!). And then in some, perhaps the processor considers it invalid opcode and the OS kills the process.
Simple solution: declare uint64_t data_length.
If you really have limited yourself to 32-bit datatypes, then you can just assign 0 to these bytes that signify the most significant bytes. Or just cast to uint64_t or unsigned long long before the shift.
Previously, I had the following C code, through which I intended to do sign extension of variable 'sample' after a cast to 'signed short' of variable 'sample_unsigned'.
unsigned short sample_unsigned;
signed short sample;
sample = ((signed short) sample_unsigned << 4) >> 4;
In binary representation, I would expect 'sample' to have its most significant bit repeated 4 times. For instance, if:
sample_unsigned = 0x0800 (corresponding to "100000000000" in binary)
I understand 'sample' should result being:
sample = 0xF800 (corresponding to "1111100000000000" in binary)
However, 'sample' always ended being the same as 'sample_unsigned', and I had to split the assignment statement as below, which worked. Why this?
sample = ((signed short) sample_unsigned << 4);
sample >>= 4;
Your approach will not work. There is no gaurantee right shifting will preserve the sign. Even if, it would only work for 16 bit int. For >=32 bit int you have to replicate the sign manually into the upper bits, otherwise it just shifts the same data back and forth. In general, bitshifts of signed values are critical - see the [standard](http://port70.net/~nsz/c/c11/n1570.html#6.5.7 for details. Some constellations invoke undefined behaviour. It is better to avoid them and just work with unsigned integers.
For most platforms, the following works, however. It is not necessarily slower (on platforms with 16 bit int, it is likely even faster):
uint16_t usample;
int16_t ssample;
ssample = (int16_t)usample;
if ( ssample & 0x800 )
ssample |= ~0xFFF;
The cast to int16_t is implementation defined; your compiler shall specify how it is performed. For (almost?) all recent implementations no extra operation is performed. Just verify in the generated code or your compiler documentation. The logical-or relies on intX_t using 2s complement which is guaranteed by the standard - as opposed to the standard types.
On 32 bit platforms, there might be an intrinsic instruction to sign-extend (e.g. ARM Cortex-M3/4 SBFX). Or the compiler provides a builtin function. Depending on your use-case and speed requirements, it might be suitable to use them.
Update:
An alternative approach would be using a bitfield structure:
struct {
int16_t val : 12; // assuming 12 bit signed value like above
} extender;
extender.val = usample;
ssample = extender.val;
This might result in using the same assembler instructions I proposed above.
It is because (signed short) sample_unsigned is automatically converted to int as operand due to interger promotion.
sample = (signed short)((signed short) sample_unsigned << 4) >> 4;
will work as well.
The number 1, right shifted by anything greater than 0, should be 0, correct? Yet I can type in this very simple program which prints 1.
#include <stdio.h>
int main()
{
int b = 0x80000000;
int a = 1 >> b;
printf("%d\n", a);
}
Tested with gcc on linux.
6.5.7 Bitwise shift operators:
If the value of the right operand is negative or is greater than or equal to the width of the promoted left operand, the behavior is undefined.
The compiler is at license to do anything, obviously, but the most common behaviors are to optimize the expression (and anything that depends on it) away entirely, or simply let the underlying hardware do whatever it does for out-of-range shifts. Many hardware platforms (including x86 and ARM) mask some number of low-order bits to use as a shift-amount. The actual hardware instruction will give the result you are observing on either of those platforms, because the shift amount is masked to zero. So in your case the compiler might have optimized away the shift, or it might be simply letting the hardware do whatever it does. Inspect the assembly if you want to know which.
according to the standard, shifting for more than the bits actually existing can result in undefined behavior. So we cannot blame the compiler for that.
The motivation probably resides in the "border meaning" of 0x80000000 that sits on the boundary of the maximum positive and negative together (and that is "negative" having the highmost bit set) and on certain check that should be done and that the compiled program doesn't to to avoid to waste time verifying "impossible" things (do you really want the processor to shift bits 3 billion times?).
It's very probably not attempting to shift by some large number of bits.
INT_MAX on your system is probably 2**31-1, or 0x7fffffff (I'm using ** to denote exponentiation). If that's the case, then In the declaration:
int b = 0x80000000;
(which was missing a semicolon in the question; please copy-and-paste your exact code) the constant 0x80000000 is of type unsigned int, not int. The value is implicitly converted to int. Since the result is outside the bounds of int, the result is implementation-defined (or, in C99, may raise an implementation-defined signal, but I don't know of any implementation that does that).
The most common way this is done is to reinterpret the bits of the unsigned value as a 2's-complement signed value. The result in this case is -2**31, or -2147483648.
So the behavior isn't undefined because you're shifting by value that equals or exceeds the width of type int, it's undefined because you're shifting by a (very large) negative value.
Not that it matters, of course; undefined is undefined.
NOTE: The above assumes that int is 32 bits on your system. If int is wider than 32 bits, then most of it doesn't apply (but the behavior is still undefined).
If you really wanted to attempt to shift by 0x80000000 bits, you could do it like this:
unsigned long b = 0x80000000;
unsigned long a = 1 >> b; // *still* undefined
unsigned long is guaranteed to be big enough to hold the value 0x80000000, so you avoid part of the problem.
Of course, the behavior of the shift is just as undefined as it was in your original code, since 0x80000000 is greater than or equal to the width of unsigned long. (Unless your compiler has a really big unsigned long type, but no real-world compiler does that.)
The only way to avoid undefined behavior is not to do what you're trying to do.
It's possible, but vanishingly unlikely, that your original code's behavior is not undefined. That can only happen if the implementation-defined conversion of 0x80000000 from unsigned int to int yields a value in the range 0 .. 31. IF int is smaller than 32 bits, the conversion is likely to yield 0.
well read that maybe can help you
expression1 >> expression2
The >> operator masks expression2 to avoid shifting expression1 by too much.
That because if the shift amount exceeded the number of bits in the data type of expression1, all the original bits would be shifted away to give a trivial result.
Now for ensure that each shift leaves at least one of the original bits,
the shift operators use the following formula to calculate the actual shift amount:
mask expression2 (using the bitwise AND operator) with one less than the number of bits in expression1.
Example
var x : byte = 15;
// A byte stores 8 bits.
// The bits stored in x are 00001111
var y : byte = x >> 10;
// Actual shift is 10 & (8-1) = 2
// The bits stored in y are 00000011
// The value of y is 3
print(y); // Prints 3
That "8-1" is because x is 8 bytes so the operacion will be with 7 bits. that void remove last bit of original chain bits
while ((1U << i) < nSize) {
i++;
}
Any particular reason to use 1U instead of 1?
On most compliers, both will give a result with the same representation. However, according to the C specification, the result of a bit shift operation on a signed argument gives implementation-defined results, so in theory 1U << i is more portable than 1 << i. In practice all C compilers you'll ever encounter treat signed left shifts the same as unsigned left shifts.
The other reason is that if nSize is unsigned, then comparing it against a signed 1 << i will generate a compiler warning. Changing the 1 to 1U gets rid of the warning message, and you don't have to worry about what happens if i is 31 or 63.
The compiler warning is most likely the reason why 1U appears in the code. I suggest compiling C with most warnings turned on, and eliminating the warning messages by changing your code.
1U is unsigned. It can carry values twice as big, but without negative values.
Depending on the environment, when using U, i can be a maximum of either 31 or 15, without causing an overflow. Without using U, i can be a maximum of 30 or 14.
31, 30 are for 32 bit int
15, 14 are for 16 bit int
If nSize is an int, it can be maximum of 2147483647 (2^31-1). If you use 1 instead of 1U then 1 << 30 will get you 1073741824 and 1 << 31 will be -2147483648, and so the while loop will never end if nSize is larger than 1073741824.
With 1U << i, 1U << 31 will evaluate to 2147483648, and so you can safely use it for nSize up to 2147483647. If nSize is an unsigned int, it is also possible that the loop never ends, as in that case nSize can be larger than 1U << 31.
Edit: So I disagree with the answers telling you nSize should be unsigned, but if it is signed then it should not be negative...
1U is unsigned.
The reason why they used an unsigned value in that is expression is (I guess) because nSize is unsigned too, and compilers (when invoked with certain parameters) give warnings when comparing a signed and an unsigned values.
Another reason (less likely, in my opinion, but we cannot know without knowing wath value nSize is supposed to assume) is that unsigned values can be twice as big as signed ones, so nSize could be up to ~4*10^9 instead of ~2*10^9.