Related
This post is meant to be used as a FAQ regarding implicit integer promotion in C, particularly implicit promotion caused by the usual arithmetic conversions and/or the integer promotions.
Example 1)
Why does this give a strange, large integer number and not 255?
unsigned char x = 0;
unsigned char y = 1;
printf("%u\n", x - y);
Example 2)
Why does this give "-1 is larger than 0"?
unsigned int a = 1;
signed int b = -2;
if(a + b > 0)
puts("-1 is larger than 0");
Example 3)
Why does changing the type in the above example to short fix the problem?
unsigned short a = 1;
signed short b = -2;
if(a + b > 0)
puts("-1 is larger than 0"); // will not print
(These examples were intended for a 32 or 64 bit computer with 16 bit short.)
C was designed to implicitly and silently change the integer types of the operands used in expressions. There exist several cases where the language forces the compiler to either change the operands to a larger type, or to change their signedness.
The rationale behind this is to prevent accidental overflows during arithmetic, but also to allow operands with different signedness to co-exist in the same expression.
Unfortunately, the rules for implicit type promotion cause much more harm than good, to the point where they might be one of the biggest flaws in the C language. These rules are often not even known by the average C programmer and therefore cause all manner of very subtle bugs.
Typically you see scenarios where the programmer says "just cast to type x and it works" - but they don't know why. Or such bugs manifest themselves as rare, intermittent phenomena striking from within seemingly simple and straight-forward code. Implicit promotion is particularly troublesome in code doing bit manipulations, since most bit-wise operators in C come with poorly-defined behavior when given a signed operand.
Integer types and conversion rank
The integer types in C are char, short, int, long, long long and enum.
_Bool/bool is also treated as an integer type when it comes to type promotions.
All integers have a specified conversion rank. C11 6.3.1.1, emphasis mine on the most important parts:
Every integer type has an integer conversion rank defined as follows:
— No two signed integer types shall have the same rank, even if they have the same representation.
— The rank of a signed integer type shall be greater than the rank of any signed integer type with less precision.
— The rank of long long int shall be greater than the rank of long int, which shall be greater than the rank of int, which shall be greater than the rank of short int, which shall be greater than the rank of signed char.
— The rank of any unsigned integer type shall equal the rank of the corresponding signed integer type, if any.
— The rank of any standard integer type shall be greater than the rank of any extended integer type with the same width.
— The rank of char shall equal the rank of signed char and unsigned char.
— The rank of _Bool shall be less than the rank of all other standard integer types.
— The rank of any enumerated type shall equal the rank of the compatible integer type (see 6.7.2.2).
The types from stdint.h sort in here too, with the same rank as whatever type they happen to correspond to on the given system. For example, int32_t has the same rank as int on a 32 bit system.
Further, C11 6.3.1.1 specifies which types are regarded as the small integer types (not a formal term):
The following may be used in an expression wherever an int or unsigned int may
be used:
— An object or expression with an integer type (other than int or unsigned int) whose integer conversion rank is less than or equal to the rank of int and unsigned int.
What this somewhat cryptic text means in practice, is that _Bool, char and short (and also int8_t, uint8_t etc) are the "small integer types". These are treated in special ways and subject to implicit promotion, as explained below.
The integer promotions
Whenever a small integer type is used in an expression, it is implicitly converted to int which is always signed. This is known as the integer promotions or the integer promotion rule.
Formally, the rule says (C11 6.3.1.1):
If an int can represent all values of the original type (as restricted by the width, for a bit-field), the value is converted to an int; otherwise, it is converted to an unsigned int. These are called the integer promotions.
This means that all small integer types, no matter signedness, get implicitly converted to (signed) int when used in most expressions.
This text is often misunderstood as: "all small signed integer types are converted to signed int and all small, unsigned integer types are converted to unsigned int". This is incorrect. The unsigned part here only means that if we have for example an unsigned short operand, and int happens to have the same size as short on the given system, then the unsigned short operand is converted to unsigned int. As in, nothing of note really happens. But in case short is a smaller type than int, it is always converted to (signed) int, regardless of it the short was signed or unsigned!
The harsh reality caused by the integer promotions means that almost no operation in C can be carried out on small types like char or short. Operations are always carried out on int or larger types.
This might sound like nonsense, but luckily the compiler is allowed to optimize the code. For example, an expression containing two unsigned char operands would get the operands promoted to int and the operation carried out as int. But the compiler is allowed to optimize the expression to actually get carried out as an 8-bit operation, as would be expected. However, here comes the problem: the compiler is not allowed to optimize out the implicit change of signedness caused by the integer promotion because there is no way for the compiler to tell if the programmer is purposely relying on implicit promotion to happen, or if it is unintentional.
This is why example 1 in the question fails. Both unsigned char operands are promoted to type int, the operation is carried out on type int, and the result of x - y is of type int. Meaning that we get -1 instead of 255 which might have been expected. The compiler may generate machine code that executes the code with 8 bit instructions instead of int, but it may not optimize out the change of signedness. Meaning that we end up with a negative result, that in turn results in a weird number when printf("%u is invoked. Example 1 could be fixed by casting the result of the operation back to type unsigned char.
With the exception of a few special cases like ++ and sizeof operators, the integer promotions apply to almost all operations in C, no matter if unary, binary (or ternary) operators are used.
The usual arithmetic conversions
Whenever a binary operation (an operation with 2 operands) is done in C, both operands of the operator have to be of the same type. Therefore, in case the operands are of different types, C enforces an implicit conversion of one operand to the type of the other operand. The rules for how this is done are named the usual artihmetic conversions (sometimes informally referred to as "balancing"). These are specified in C11 6.3.18:
(Think of this rule as a long, nested if-else if statement and it might be easier to read :) )
6.3.1.8 Usual arithmetic conversions
Many operators that expect operands of arithmetic type cause conversions and yield result
types in a similar way. The purpose is to determine a common real type for the operands
and result. For the specified operands, each operand is converted, without change of type
domain, to a type whose corresponding real type is the common real type. Unless
explicitly stated otherwise, the common real type is also the corresponding real type of
the result, whose type domain is the type domain of the operands if they are the same,
and complex otherwise. This pattern is called the usual arithmetic conversions:
First, if the corresponding real type of either operand is long double, the other operand is converted, without change of type domain, to a type whose corresponding real type is long double.
Otherwise, if the corresponding real type of either operand is double, the other operand is converted, without change of type domain, to a type whose corresponding real type is double.
Otherwise, if the corresponding real type of either operand is float, the other operand is converted, without change of type domain, to a type whose corresponding real type is float.
Otherwise, the integer promotions are performed on both operands. Then the
following rules are applied to the promoted operands:
If both operands have the same type, then no further conversion is needed.
Otherwise, if both operands have signed integer types or both have unsigned
integer types, the operand with the type of lesser integer conversion rank is
converted to the type of the operand with greater rank.
Otherwise, if the operand that has unsigned integer type has rank greater or
equal to the rank of the type of the other operand, then the operand with
signed integer type is converted to the type of the operand with unsigned
integer type.
Otherwise, if the type of the operand with signed integer type can represent
all of the values of the type of the operand with unsigned integer type, then
the operand with unsigned integer type is converted to the type of the
operand with signed integer type.
Otherwise, both operands are converted to the unsigned integer type
corresponding to the type of the operand with signed integer type.
Notable here is that the usual arithmetic conversions apply to both floating point and integer variables. In the case of integers, we can also note that the integer promotions are invoked from within the usual arithmetic conversions. And after that, when both operands have at least the rank of int, the operators are balanced to the same type, with the same signedness.
This is the reason why a + b in example 2 gives a strange result. Both operands are integers and they are at least of rank int, so the integer promotions do not apply. The operands are not of the same type - a is unsigned int and b is signed int. Therefore the operator b is temporarily converted to type unsigned int. During this conversion, it loses the sign information and ends up as a large value.
The reason why changing type to short in example 3 fixes the problem, is because short is a small integer type. Meaning that both operands are integer promoted to type int which is signed. After integer promotion, both operands have the same type (int), no further conversion is needed. And then the operation can be carried out on a signed type as expected.
According to the previous post, I want to give more information about each example.
Example 1)
int main(){
unsigned char x = 0;
unsigned char y = 1;
printf("%u\n", x - y);
printf("%d\n", x - y);
}
Since unsigned char is smaller than int, we apply the integer promotion on them, then we have (int)x-(int)y = (int)(-1) and unsigned int (-1) = 4294967295.
The output from the above code:(same as what we expected)
4294967295
-1
How to fix it?
I tried what the previous post recommended, but it doesn't really work.
Here is the code based on the previous post:
change one of them to unsigned int
int main(){
unsigned int x = 0;
unsigned char y = 1;
printf("%u\n", x - y);
printf("%d\n", x - y);
}
Since x is already an unsigned integer, we only apply the integer promotion to y. Then we get (unsigned int)x-(int)y. Since they still don't have the same type, we apply the usual arithmetic converions, we get (unsigned int)x-(unsigned int)y = 4294967295.
The output from the above code:(same as what we expected):
4294967295
-1
Similarly, the following code gets the same result:
int main(){
unsigned char x = 0;
unsigned int y = 1;
printf("%u\n", x - y);
printf("%d\n", x - y);
}
change both of them to unsigned int
int main(){
unsigned int x = 0;
unsigned int y = 1;
printf("%u\n", x - y);
printf("%d\n", x - y);
}
Since both of them are unsigned int, no integer promotion is needed. By the usual arithmetic converison(have the same type), (unsigned int)x-(unsigned int)y = 4294967295.
The output from the above code:(same as what we expected):
4294967295
-1
One of possible ways to fix the code:(add a type cast in the end)
int main(){
unsigned char x = 0;
unsigned char y = 1;
printf("%u\n", x - y);
printf("%d\n", x - y);
unsigned char z = x-y;
printf("%u\n", z);
}
The output from the above code:
4294967295
-1
255
Example 2)
int main(){
unsigned int a = 1;
signed int b = -2;
if(a + b > 0)
puts("-1 is larger than 0");
printf("%u\n", a+b);
}
Since both of them are integers, no integer promotion is needed. By the usual arithmetic conversion, we get (unsigned int)a+(unsigned int)b = 1+4294967294 = 4294967295.
The output from the above code:(same as what we expected)
-1 is larger than 0
4294967295
How to fix it?
int main(){
unsigned int a = 1;
signed int b = -2;
signed int c = a+b;
if(c < 0)
puts("-1 is smaller than 0");
printf("%d\n", c);
}
The output from the above code:
-1 is smaller than 0
-1
Example 3)
int main(){
unsigned short a = 1;
signed short b = -2;
if(a + b < 0)
puts("-1 is smaller than 0");
printf("%d\n", a+b);
}
The last example fixed the problem since a and b both converted to int due to the integer promotion.
The output from the above code:
-1 is smaller than 0
-1
If I got some concepts mixed up, please let me know. Thanks~
Integer and floating point rank and promotion rules in C and C++
I'd like to take a stab at this to summarize the rules so I can quickly reference them. I've fully studied the question and both of the other two answers here, including the main one by #Lundin. If you want more examples beyond the ones below, go study that answer in detail as well, while referencing my "rules" and "promotion flow" summaries below.
I've also written my own example and demo code here: integer_promotion_overflow_underflow_undefined_behavior.c.
Despite normally being incredibly verbose myself, I'm going to try to keep this a short summary, since the other two answers plus my test code already have sufficient detail via their necessary verbosity.
Integer and variable promotion quick reference guide and summary
3 simple rules
For any operation where multiple operands (input variables) are involved (ex: mathematical operations, comparisons, or ternary), the variables are promoted as required to the required variable type before the operation is performed.
Therefore, you must manually, explicitly cast the output to any desired type you desire if you do not want it to be implicitly chosen for you. See the example below.
All types smaller than int (int32_t on my 64-bit Linux system) are "small types". They cannot be used in ANY operation. So, if all input variables are "small types", they are ALL first promoted to int (int32_t on my 64-bit Linux system) before performing the operation.
Otherwise, if at least one of the input types is int or larger, the other, smaller input type or types are promoted to this largest-input-type's type.
Example
Example: with this code:
uint8_t x = 0;
uint8_t y = 1;
...if you do x - y, they first get implicitly promoted to int (which is int32_t on my 64-bit
system), and you end up with this: (int)x - (int)y, which results in an int type with value
-1, rather than a uint8_t type of value 255. To get the desired 255 result, manually
cast the result back to uint8_t, by doing this: (uint8_t)(x - y).
Promotion flow
The promotion rules are as follows. Promotion from smallest to largest types is as follows.
Read "-->" as "gets promoted to".
The types in square brackets (ex: [int8_t]) are the typical "fixed-width integer types" for the given standard type on a typical 64-bit Unix (Linux or Mac) architecture. See, for example:
https://www.cs.yale.edu/homes/aspnes/pinewiki/C(2f)IntegerTypes.html
https://www.ibm.com/docs/en/ibm-mq/7.5?topic=platforms-standard-data-types
And even better, test it for yourself on your machine by running my code here!: stdint_sizes.c from my eRCaGuy_hello_world repo.
1. For integer types
Note: "small types" = bool (_Bool), char [int8_t], unsigned char [uint8_t], short [int16_t], unsigned short [uint16_t].
SMALL TYPES: bool (_Bool), char [int8_t], unsigned char [uint8_t], short [int16_t], unsigned short [uint16_t]
--> int [int32_t]
--> unsigned int [uint32_t]
--> long int [int64_t]
--> unsigned long int [uint64_t]
--> long long int [int64_t]
--> unsigned long long int [uint64_t]
Pointers (ex: void*) and size_t are both 64-bits, so I imagine they fit into the uint64_t category above.
2. For floating point types
float [32-bits] --> double [64-bits] --> long double [128-bits]
I would like to add two clarifications to #Lundin's otherwise excellent answer, regarding example 1, where there are two operands of identical integer type, but are "small types" that require integer promotion.
I'm using the N1256 draft since I don't have access to a paid copy of the C standard.
First: (normative)
6.3.1.1's definition of integer promotion isn't the triggering clause of actually doing integer promotion. In reality it is 6.3.1.8 Usual arithmetic conversions.
Most of the time, the "usual arithmetic conversions" apply when the operands are of different types, in which case at least one operand must be promoted. But the catch is that for integer types, integer promotion is required in all cases.
[clauses of floating-point types come first]
Otherwise, the integer promotions are performed on both operands. Then the
following rules are applied to the promoted operands:
If both operands have the same type, then no further conversion is needed.
Otherwise, if both operands have signed integer types or both have unsigned
integer types, the operand with the type of lesser integer conversion rank is
converted to the type of the operand with greater rank.
Otherwise, if the operand that has unsigned integer type has rank greater or
equal to the rank of the type of the other operand, then the operand with
signed integer type is converted to the type of the operand with unsigned
integer type.
Otherwise, if the type of the operand with signed integer type can represent
all of the values of the type of the operand with unsigned integer type, then
the operand with unsigned integer type is converted to the type of the
operand with signed integer type.
Otherwise, both operands are converted to the unsigned integer type
corresponding to the type of the operand with signed integer type.
Second: (non-normative)
There is an explicit example cited by the standard to demonstrate this:
EXAMPLE 2 In executing the fragment
char c1, c2;
/* ... */
c1 = c1 + c2;
the "integer promotions" require that the abstract machine promote the value of each variable to int size
and then add the two ints and truncate the sum. Provided the addition of two chars can be done without
overflow, or with overflow wrapping silently to produce the correct result, the actual execution need only
produce the same result, possibly omitting the promotions.
I'm doing a few integer for myself, where I'm trying to fully understand integer overflow.
I kept reading about how it can be dangerous to mix integer types of different sizes. For that reason i wanted to have an example where a short would overflow much faster than a int.
Here is the snippet:
unsigned int longt;
longt = 65530;
unsigned short shortt;
shortt = 65530;
if (longt > (shortt+10)){
printf("it is bigger");
}
But the if-statement here is not being run, which must mean that the short is not overflowing. Thus I conclude that in the expression shortt+10 a conversion happens from short to integer.
This is a bit strange to me, when the if statement evaluates expressions, does it then have the freedom to assign a new integer type as it pleases?
I then thought that if I was adding two short's then that would surely evaluate to a short:
unsigned int longt;
longt = 65530;
unsigned short shortt;
shortt = 65530;
shortt = shortt;
short tmp = 10;
if (longt > (shortt+tmp)){
printf("Ez bigger");
}
But alas, the proporsition still evaluates to false.
I then try do do something where I am completely explicit, where I actually do the addition into a short type, this time forcing it to overflow:
unsigned int longt;
longt = 65530;
unsigned short shortt;
shortt = 65530;
shortt = shortt;
short tmp = shortt + 10;
if (longt > tmp){
printf("Ez bigger");
}
Finally this worked, which also would be really annoying if it did'nt.
This flusters me a little bit though, and it reminds me of a ctf exercise that I did a while back, where I had to exploit this code snippet:
#include <stdio.h>
int main() {
int impossible_number;
FILE *flag;
char c;
if (scanf("%d", &impossible_number)) {
if (impossible_number > 0 && impossible_number > (impossible_number + 1)) {
flag = fopen("flag.txt","r");
while((c = getc(flag)) != EOF) {
printf("%c",c);
}
}
}
return 0;
}
Here, youre supposed to trigger a overflow of the "impossible_number" variable which was actually possible on the server that it was deployed upon, but would make issues when run locally.
int impossible_number;
FILE *flag;
char c;
if (scanf("%d", &impossible_number)) {
if (impossible_number > 0 && impossible_number > (impossible_number + 1)) {
flag = fopen("flag.txt","r");
while((c = getc(flag)) != EOF) {
printf("%c",c);
}
}
}
return 0;
You should be able to give "2147483647" as input, and then overflow and hit the if statement. However this does not happen when run locally, or running at an online compiler.
I don't get it, how do you get an expression to actually overflow the way that is is actually supossed to do, like in this example from 247ctf?
I hope someone has a answer for this
How you avoid implicit conversion from short to integer during addition?
You don't.
C has no arithmetic operations on integer types narrower than int and unsigned int. There is no + operator for type short.
Whenever an expression of type short is used as the operand of an arithmetic operator, it is implicitly converted to int.
For example:
short s = 1;
s = s + s;
In s + s, s is promoted from short to int and the addition is done in type int. The assignment then implicitly converts the result of the addition from int to short.
Some compilers might have an option to enable a warning for the narrowing conversion from int to short, but there's no way to avoid it.
What you're seeing is a result of integer promotions. What this basically means it that anytime an integer type smaller than int is used in an expression it is converted to int.
This is detailed in section 6.3.1.1p2 of the C standard:
The following may be used in an expression wherever an int or unsigned int may be used:
An object or expression with an integer type (other than int or unsigned int) whose integer conversion rank is less than or equal to
the rank of int and unsigned int.
A bit-field of type _Bool, int, signed int, or unsigned int.
If an int can represent all values of the original type (as restricted
by the width, for a bit-field), the value is converted to an int;
otherwise, it is converted to an unsigned int. These are called the
integer promotions. All other types are unchanged by the integer
promotions
That is what's happening here. So let's look at the first expression:
if (longt > (shortt+10)){
Here we have a unsigned short with value 65530 being added to the constant 10 which has type int. The unsigned short value is converted to an int value, so now we have the int value 65530 being added to the int value 10 which results in the int value 65540. We now have 65530 > 65540 which is false.
The same happens in the second case where both operands of the + operator are first promoted from unsigned short to int.
In the third case, the difference happens here:
short tmp = shortt + 10;
On the right side of the assignment, we still have the int value 65540 as before, but now this value needs to be assigned back to a short. This undergoes an implementation defined conversion to short, which is detailed in section 6.3.1.3:
1 When a value with integer type is converted to another integer type other than _Bool, if the value can be represented by the new
type, it is unchanged.
2 Otherwise, if the new type is unsigned, the value is converted by repeatedly adding or subtracting one more than the maximum value that
can be represented in the new type until the value is in the range of
the new type.
3 Otherwise, the new type is signed and the value cannot be represented in it; either the result is implementation-defined or an
implementation-defined signal is raised.
Paragraph 3 takes effect in this particular case. In most implementations you're likely to come across, this will typically mean "wraparound" of the value.
So how do you work with this? The closest thing you can do is either what you did, i.e. assign the intermediate result to a variable of the desired type, or cast the intermediate result:
if (longt > (short)(shortt+10)) {
As for the "impossible" input in the CTF example, that actually causes signed integer overflow as a result of the the addition, and that triggers undefined behavior. For example, when I ran it on my machine, I got into the if block if I compiled with -O0 or -O1 but not with -O2.
How you avoid implicit conversion from short to integer during addition?
Not really avoidable.
On 16-bit and wider machines, the conversion short to int and unsigned short to unsigned does not affect the value. But addition overflow and the implicit conversion from int to unsigned renders a different result in 16-but vs. 32-bit for OP's values. For in 16-bit land, unsigned short to int does not implicitly occur. Instead, code does unsigned short to unsigned.
int/unsigned as 16-bit
If int/unsigned were 16-bit -common on many embedded processors, then shortt would not convert to an int, but to unsigned.
// Given 16-bit int/unsigned
unsigned int longt;
longt = 65530; // 32-bit long constant assigned to 16-bit unsigned - no value change as value in range.
unsigned short shortt;
shortt = 65530; // 32-bit long constant assigned to 16-bit unsigned short - no value change as value in range.
// (shortt+10)
// shortt+10 is a unsigned short + int
// unsigned short promotes to unsigned - no value change.
// Then since unsigned + int, the int 10 converts to unsigned 10 - no value change.
// unsigned 65530 + unsigned 10 exceeds unsigned range so 65536 subtracted.
// Sum is 4.
// Statment is true.
if (longt > (shortt+10)){
printf("it is bigger");
}
It is called an implicit conversion.
From C standard:
Several operators convert operand values from one type to another
automatically. This subclause specifies the result required from such
an implicit conversion, as well as those that result from a cast
operation (an explicit conversion ). The list in 6.3.1.8 summarizes
the conversions performed by most ordinary operators; it is
supplemented as required by the discussion of each operator in 6.5
Every integer type has an integer conversion rank defined as follows:
No two signed integer types shall have the same rank, even if they
have the same representation.
The rank of a signed integer type
shall be greater than the rank of any signed integer type with less
precision.
The rank of long long int shall be greater than the rank
of long int, which shall be greater than the rank of int, which shall
be greater than the rank of short int, which shall be greater than the
rank of signed char.
The rank of any unsigned integer type shall
equal the rank of the corresponding signed integer type, if any.
The
rank of any standard integer type shall be greater than the rank of
any extended integer type with the same width.
The rank of char
shall equal the rank of signed char and unsigned char.
The rank of
_Bool shall be less than the rank of all other standard integer types.
The rank of any enumerated type shall equal the rank of the
compatible integer type (see 6.7.2.2).
The rank of any extended
signed integer type relative to another extended signed integer type
with the same precision is implementation-defined, but still subject
to the other rules for determining the integer conversion rank.
For
all integer types T1, T2, and T3, if T1 has greater rank than T2 and
T2 has greater rank than T3, then T1 has greater rank than T3.
The
following may be used in an expression wherever an int or unsigned int
may be used:
— An object or expression with an integer type (other than int or unsigned
int) whose integer conversion rank is less than or equal to the rank
of int and unsigned int.
A bit-field of type _Bool, int, signed int,
or unsigned int. If an int can represent all v alues of the original
type (as restricted by the width, for a bit-field), the value is
converted to an int; otherwise, it is converted to an unsigned int.
These are called the integer promotions.58) All other types are
unchanged by the integer promotions.
The integer promotions preserve
value including sign. As discussed earlier, whether a ‘‘plain’’ char
is treated as signed is implementation-defined.
You cant avoid implicit conversion but you can cast the result of the operation to the required type
if (longt > (short)(shortt+tmp))
{
printf("Ez bigger");
}
https://godbolt.org/z/39Exa8E7K
But this conversion invokes Undefined Behaviour as your short integer overflows. You have to be very careful doing it as it can be a source of very hard to find and debug errors.
This post is meant to be used as a FAQ regarding implicit integer promotion in C, particularly implicit promotion caused by the usual arithmetic conversions and/or the integer promotions.
Example 1)
Why does this give a strange, large integer number and not 255?
unsigned char x = 0;
unsigned char y = 1;
printf("%u\n", x - y);
Example 2)
Why does this give "-1 is larger than 0"?
unsigned int a = 1;
signed int b = -2;
if(a + b > 0)
puts("-1 is larger than 0");
Example 3)
Why does changing the type in the above example to short fix the problem?
unsigned short a = 1;
signed short b = -2;
if(a + b > 0)
puts("-1 is larger than 0"); // will not print
(These examples were intended for a 32 or 64 bit computer with 16 bit short.)
C was designed to implicitly and silently change the integer types of the operands used in expressions. There exist several cases where the language forces the compiler to either change the operands to a larger type, or to change their signedness.
The rationale behind this is to prevent accidental overflows during arithmetic, but also to allow operands with different signedness to co-exist in the same expression.
Unfortunately, the rules for implicit type promotion cause much more harm than good, to the point where they might be one of the biggest flaws in the C language. These rules are often not even known by the average C programmer and therefore cause all manner of very subtle bugs.
Typically you see scenarios where the programmer says "just cast to type x and it works" - but they don't know why. Or such bugs manifest themselves as rare, intermittent phenomena striking from within seemingly simple and straight-forward code. Implicit promotion is particularly troublesome in code doing bit manipulations, since most bit-wise operators in C come with poorly-defined behavior when given a signed operand.
Integer types and conversion rank
The integer types in C are char, short, int, long, long long and enum.
_Bool/bool is also treated as an integer type when it comes to type promotions.
All integers have a specified conversion rank. C11 6.3.1.1, emphasis mine on the most important parts:
Every integer type has an integer conversion rank defined as follows:
— No two signed integer types shall have the same rank, even if they have the same representation.
— The rank of a signed integer type shall be greater than the rank of any signed integer type with less precision.
— The rank of long long int shall be greater than the rank of long int, which shall be greater than the rank of int, which shall be greater than the rank of short int, which shall be greater than the rank of signed char.
— The rank of any unsigned integer type shall equal the rank of the corresponding signed integer type, if any.
— The rank of any standard integer type shall be greater than the rank of any extended integer type with the same width.
— The rank of char shall equal the rank of signed char and unsigned char.
— The rank of _Bool shall be less than the rank of all other standard integer types.
— The rank of any enumerated type shall equal the rank of the compatible integer type (see 6.7.2.2).
The types from stdint.h sort in here too, with the same rank as whatever type they happen to correspond to on the given system. For example, int32_t has the same rank as int on a 32 bit system.
Further, C11 6.3.1.1 specifies which types are regarded as the small integer types (not a formal term):
The following may be used in an expression wherever an int or unsigned int may
be used:
— An object or expression with an integer type (other than int or unsigned int) whose integer conversion rank is less than or equal to the rank of int and unsigned int.
What this somewhat cryptic text means in practice, is that _Bool, char and short (and also int8_t, uint8_t etc) are the "small integer types". These are treated in special ways and subject to implicit promotion, as explained below.
The integer promotions
Whenever a small integer type is used in an expression, it is implicitly converted to int which is always signed. This is known as the integer promotions or the integer promotion rule.
Formally, the rule says (C11 6.3.1.1):
If an int can represent all values of the original type (as restricted by the width, for a bit-field), the value is converted to an int; otherwise, it is converted to an unsigned int. These are called the integer promotions.
This means that all small integer types, no matter signedness, get implicitly converted to (signed) int when used in most expressions.
This text is often misunderstood as: "all small signed integer types are converted to signed int and all small, unsigned integer types are converted to unsigned int". This is incorrect. The unsigned part here only means that if we have for example an unsigned short operand, and int happens to have the same size as short on the given system, then the unsigned short operand is converted to unsigned int. As in, nothing of note really happens. But in case short is a smaller type than int, it is always converted to (signed) int, regardless of it the short was signed or unsigned!
The harsh reality caused by the integer promotions means that almost no operation in C can be carried out on small types like char or short. Operations are always carried out on int or larger types.
This might sound like nonsense, but luckily the compiler is allowed to optimize the code. For example, an expression containing two unsigned char operands would get the operands promoted to int and the operation carried out as int. But the compiler is allowed to optimize the expression to actually get carried out as an 8-bit operation, as would be expected. However, here comes the problem: the compiler is not allowed to optimize out the implicit change of signedness caused by the integer promotion because there is no way for the compiler to tell if the programmer is purposely relying on implicit promotion to happen, or if it is unintentional.
This is why example 1 in the question fails. Both unsigned char operands are promoted to type int, the operation is carried out on type int, and the result of x - y is of type int. Meaning that we get -1 instead of 255 which might have been expected. The compiler may generate machine code that executes the code with 8 bit instructions instead of int, but it may not optimize out the change of signedness. Meaning that we end up with a negative result, that in turn results in a weird number when printf("%u is invoked. Example 1 could be fixed by casting the result of the operation back to type unsigned char.
With the exception of a few special cases like ++ and sizeof operators, the integer promotions apply to almost all operations in C, no matter if unary, binary (or ternary) operators are used.
The usual arithmetic conversions
Whenever a binary operation (an operation with 2 operands) is done in C, both operands of the operator have to be of the same type. Therefore, in case the operands are of different types, C enforces an implicit conversion of one operand to the type of the other operand. The rules for how this is done are named the usual artihmetic conversions (sometimes informally referred to as "balancing"). These are specified in C11 6.3.18:
(Think of this rule as a long, nested if-else if statement and it might be easier to read :) )
6.3.1.8 Usual arithmetic conversions
Many operators that expect operands of arithmetic type cause conversions and yield result
types in a similar way. The purpose is to determine a common real type for the operands
and result. For the specified operands, each operand is converted, without change of type
domain, to a type whose corresponding real type is the common real type. Unless
explicitly stated otherwise, the common real type is also the corresponding real type of
the result, whose type domain is the type domain of the operands if they are the same,
and complex otherwise. This pattern is called the usual arithmetic conversions:
First, if the corresponding real type of either operand is long double, the other operand is converted, without change of type domain, to a type whose corresponding real type is long double.
Otherwise, if the corresponding real type of either operand is double, the other operand is converted, without change of type domain, to a type whose corresponding real type is double.
Otherwise, if the corresponding real type of either operand is float, the other operand is converted, without change of type domain, to a type whose corresponding real type is float.
Otherwise, the integer promotions are performed on both operands. Then the
following rules are applied to the promoted operands:
If both operands have the same type, then no further conversion is needed.
Otherwise, if both operands have signed integer types or both have unsigned
integer types, the operand with the type of lesser integer conversion rank is
converted to the type of the operand with greater rank.
Otherwise, if the operand that has unsigned integer type has rank greater or
equal to the rank of the type of the other operand, then the operand with
signed integer type is converted to the type of the operand with unsigned
integer type.
Otherwise, if the type of the operand with signed integer type can represent
all of the values of the type of the operand with unsigned integer type, then
the operand with unsigned integer type is converted to the type of the
operand with signed integer type.
Otherwise, both operands are converted to the unsigned integer type
corresponding to the type of the operand with signed integer type.
Notable here is that the usual arithmetic conversions apply to both floating point and integer variables. In the case of integers, we can also note that the integer promotions are invoked from within the usual arithmetic conversions. And after that, when both operands have at least the rank of int, the operators are balanced to the same type, with the same signedness.
This is the reason why a + b in example 2 gives a strange result. Both operands are integers and they are at least of rank int, so the integer promotions do not apply. The operands are not of the same type - a is unsigned int and b is signed int. Therefore the operator b is temporarily converted to type unsigned int. During this conversion, it loses the sign information and ends up as a large value.
The reason why changing type to short in example 3 fixes the problem, is because short is a small integer type. Meaning that both operands are integer promoted to type int which is signed. After integer promotion, both operands have the same type (int), no further conversion is needed. And then the operation can be carried out on a signed type as expected.
According to the previous post, I want to give more information about each example.
Example 1)
int main(){
unsigned char x = 0;
unsigned char y = 1;
printf("%u\n", x - y);
printf("%d\n", x - y);
}
Since unsigned char is smaller than int, we apply the integer promotion on them, then we have (int)x-(int)y = (int)(-1) and unsigned int (-1) = 4294967295.
The output from the above code:(same as what we expected)
4294967295
-1
How to fix it?
I tried what the previous post recommended, but it doesn't really work.
Here is the code based on the previous post:
change one of them to unsigned int
int main(){
unsigned int x = 0;
unsigned char y = 1;
printf("%u\n", x - y);
printf("%d\n", x - y);
}
Since x is already an unsigned integer, we only apply the integer promotion to y. Then we get (unsigned int)x-(int)y. Since they still don't have the same type, we apply the usual arithmetic converions, we get (unsigned int)x-(unsigned int)y = 4294967295.
The output from the above code:(same as what we expected):
4294967295
-1
Similarly, the following code gets the same result:
int main(){
unsigned char x = 0;
unsigned int y = 1;
printf("%u\n", x - y);
printf("%d\n", x - y);
}
change both of them to unsigned int
int main(){
unsigned int x = 0;
unsigned int y = 1;
printf("%u\n", x - y);
printf("%d\n", x - y);
}
Since both of them are unsigned int, no integer promotion is needed. By the usual arithmetic converison(have the same type), (unsigned int)x-(unsigned int)y = 4294967295.
The output from the above code:(same as what we expected):
4294967295
-1
One of possible ways to fix the code:(add a type cast in the end)
int main(){
unsigned char x = 0;
unsigned char y = 1;
printf("%u\n", x - y);
printf("%d\n", x - y);
unsigned char z = x-y;
printf("%u\n", z);
}
The output from the above code:
4294967295
-1
255
Example 2)
int main(){
unsigned int a = 1;
signed int b = -2;
if(a + b > 0)
puts("-1 is larger than 0");
printf("%u\n", a+b);
}
Since both of them are integers, no integer promotion is needed. By the usual arithmetic conversion, we get (unsigned int)a+(unsigned int)b = 1+4294967294 = 4294967295.
The output from the above code:(same as what we expected)
-1 is larger than 0
4294967295
How to fix it?
int main(){
unsigned int a = 1;
signed int b = -2;
signed int c = a+b;
if(c < 0)
puts("-1 is smaller than 0");
printf("%d\n", c);
}
The output from the above code:
-1 is smaller than 0
-1
Example 3)
int main(){
unsigned short a = 1;
signed short b = -2;
if(a + b < 0)
puts("-1 is smaller than 0");
printf("%d\n", a+b);
}
The last example fixed the problem since a and b both converted to int due to the integer promotion.
The output from the above code:
-1 is smaller than 0
-1
If I got some concepts mixed up, please let me know. Thanks~
Integer and floating point rank and promotion rules in C and C++
I'd like to take a stab at this to summarize the rules so I can quickly reference them. I've fully studied the question and both of the other two answers here, including the main one by #Lundin. If you want more examples beyond the ones below, go study that answer in detail as well, while referencing my "rules" and "promotion flow" summaries below.
I've also written my own example and demo code here: integer_promotion_overflow_underflow_undefined_behavior.c.
Despite normally being incredibly verbose myself, I'm going to try to keep this a short summary, since the other two answers plus my test code already have sufficient detail via their necessary verbosity.
Integer and variable promotion quick reference guide and summary
3 simple rules
For any operation where multiple operands (input variables) are involved (ex: mathematical operations, comparisons, or ternary), the variables are promoted as required to the required variable type before the operation is performed.
Therefore, you must manually, explicitly cast the output to any desired type you desire if you do not want it to be implicitly chosen for you. See the example below.
All types smaller than int (int32_t on my 64-bit Linux system) are "small types". They cannot be used in ANY operation. So, if all input variables are "small types", they are ALL first promoted to int (int32_t on my 64-bit Linux system) before performing the operation.
Otherwise, if at least one of the input types is int or larger, the other, smaller input type or types are promoted to this largest-input-type's type.
Example
Example: with this code:
uint8_t x = 0;
uint8_t y = 1;
...if you do x - y, they first get implicitly promoted to int (which is int32_t on my 64-bit
system), and you end up with this: (int)x - (int)y, which results in an int type with value
-1, rather than a uint8_t type of value 255. To get the desired 255 result, manually
cast the result back to uint8_t, by doing this: (uint8_t)(x - y).
Promotion flow
The promotion rules are as follows. Promotion from smallest to largest types is as follows.
Read "-->" as "gets promoted to".
The types in square brackets (ex: [int8_t]) are the typical "fixed-width integer types" for the given standard type on a typical 64-bit Unix (Linux or Mac) architecture. See, for example:
https://www.cs.yale.edu/homes/aspnes/pinewiki/C(2f)IntegerTypes.html
https://www.ibm.com/docs/en/ibm-mq/7.5?topic=platforms-standard-data-types
And even better, test it for yourself on your machine by running my code here!: stdint_sizes.c from my eRCaGuy_hello_world repo.
1. For integer types
Note: "small types" = bool (_Bool), char [int8_t], unsigned char [uint8_t], short [int16_t], unsigned short [uint16_t].
SMALL TYPES: bool (_Bool), char [int8_t], unsigned char [uint8_t], short [int16_t], unsigned short [uint16_t]
--> int [int32_t]
--> unsigned int [uint32_t]
--> long int [int64_t]
--> unsigned long int [uint64_t]
--> long long int [int64_t]
--> unsigned long long int [uint64_t]
Pointers (ex: void*) and size_t are both 64-bits, so I imagine they fit into the uint64_t category above.
2. For floating point types
float [32-bits] --> double [64-bits] --> long double [128-bits]
I would like to add two clarifications to #Lundin's otherwise excellent answer, regarding example 1, where there are two operands of identical integer type, but are "small types" that require integer promotion.
I'm using the N1256 draft since I don't have access to a paid copy of the C standard.
First: (normative)
6.3.1.1's definition of integer promotion isn't the triggering clause of actually doing integer promotion. In reality it is 6.3.1.8 Usual arithmetic conversions.
Most of the time, the "usual arithmetic conversions" apply when the operands are of different types, in which case at least one operand must be promoted. But the catch is that for integer types, integer promotion is required in all cases.
[clauses of floating-point types come first]
Otherwise, the integer promotions are performed on both operands. Then the
following rules are applied to the promoted operands:
If both operands have the same type, then no further conversion is needed.
Otherwise, if both operands have signed integer types or both have unsigned
integer types, the operand with the type of lesser integer conversion rank is
converted to the type of the operand with greater rank.
Otherwise, if the operand that has unsigned integer type has rank greater or
equal to the rank of the type of the other operand, then the operand with
signed integer type is converted to the type of the operand with unsigned
integer type.
Otherwise, if the type of the operand with signed integer type can represent
all of the values of the type of the operand with unsigned integer type, then
the operand with unsigned integer type is converted to the type of the
operand with signed integer type.
Otherwise, both operands are converted to the unsigned integer type
corresponding to the type of the operand with signed integer type.
Second: (non-normative)
There is an explicit example cited by the standard to demonstrate this:
EXAMPLE 2 In executing the fragment
char c1, c2;
/* ... */
c1 = c1 + c2;
the "integer promotions" require that the abstract machine promote the value of each variable to int size
and then add the two ints and truncate the sum. Provided the addition of two chars can be done without
overflow, or with overflow wrapping silently to produce the correct result, the actual execution need only
produce the same result, possibly omitting the promotions.
while writing the code I observe one thing in my code its related to the comparison of bit-field value with negative integers.
I have one structure member of unsigned of size one bit and one unsigned int. When I compare the negative value with unsigned int variable I am getting expected result as 1 but when I compare the structure member with the negative value I am getting the opposite result as 0.
#include <stdio.h>
struct S0
{
unsigned int bit : 1;
};
struct S0 s;
int main (void)
{
int negVal = -3;
unsigned int p = 123;
printf ("%d\n", (negVal > p)); /*Result as 1 */
printf ("%d\n", (negVal > s.bit));/*Result as 0 but expected 1 */
return 0;
}
My doubt is if I compare the negative value with unsigned int then balancing will happen (implicit type casting). But if I compare structure member of unsigned int why implicit type casting is not happening. Correct me if I miss any basics of bit fields?
(move my remark as an answer)
gcc promotes s.bit to an int, so (negVal > s.bit) does (-3 > 0) valuing 0
See Should bit-fields less than int in size be the subject of integral promotion? but your question is not a duplicate of it.
(negVal > p) returns 1 because negVal is promoted to unsigned producing a big value, see Signed/unsigned comparisons
For illustration, the following uses a 32-bit int and a 32-bit unsigned int.
In negVal > p:
negVal is an int with value −3.
p is an unsigned int with value 123.
C 2018 6.5.8 3, which is discusses > and the other relational operators, tells us that the usual arithmetic conversions are performed on the operands.
6.3.1.8 1 defines the usual arithmetic conversions. For integer types, the first step of the usual arithmetic conversions is to perform the integer promotions on each operand.
6.3.1.1 2 defines the integer promotions. int, unsigned int, and integer types wider than these are unchanged. For other integer types, it says: ”If an int can represent all values of the original type (as restricted by the width, for a bit-field), the value is converted to an int; otherwise, it is converted to an unsigned int.”
Since negVal is an int, it is unchanged by the integer promotions.
Since p is an unsigned int, it is unchanged by the integer promotions.
The next step in the usual arithmetic conversions is to convert one operand to the type of the other. For int and unsigned int, the int is converted to unsigned int.
Converting the int −3 to unsigned int results in 4,294,967,293. (The conversion is defined to add or subtracting UINT_MAX + 1, which is 4,294,967,296, to the value as many times as necessary to bring it in range. This is equivalent to “wrapping” modulo 4,294,967,296 or to reinterpreting the two’s complement representation of −3 as an unsigned int.)
After the conversions, the expression negVal > p has become 4294967293u > 123u.
This comparison is true, so the result is 1.
In negVal > s.bit:
negVal is an int with value −3.
s.bit is a one-bit bit-field with value 0.
As above, the usual arithmetic conversions are performed on the operands.
As above, the first step of the usual arithmetic conversions is to perform the integer promotions on each operand.
Since negVal is an int, it is unchanged by the integer promotions.
Since s.bit is a bit-field narrower than an int, it will be converted by the integer promotions. This one-bit bit-field can represent either 0 or 1. Both of these can be represented by an int, and therefore the rule “If an int can represent all values of the original type (as restricted by the width, for a bit-field), the value is converted to an int” applies.
Converting 0 to int results in 0.
The next step in the usual arithmetic conversions would be to convert one operand to the type of the other. Since both operands are now int, no conversion is needed.
After the conversions, the expression negVal > s.bit has become -3 > 0.
This comparison is false, so the result is 0.
I am learning the c language using the K&R book. In the second chapter book, the author talks about implicit conversion. There book says this:
Conversion rules are more complicated when unsigned operands are involved. The problem is that
comparisons between signed and unsigned values are machine-dependent, because they depend on the sizes of the various integer types. For example, suppose that int is 16 bits and long is 32 bits. Then -1L < 1U, because 1U, which is an unsigned int, is promoted to a signed long. But -1L >
1UL because -1L is promoted to unsigned long and thus appears to be a large positive number.
I tried the code below in two different scenarios:
compiled on an x86 64bits platform and executed. Where sizeof(-1L) -> 8byte and sizeof(1U) -> 4 bytes
compiled on an x86 32bits platform and executed. Where sizeof(-1L) -> 4byte and sizeof(1U) -> 4 bytes
The code:
int main() {
if(-1L > 1U)
printf("true");
else
printf("false");
return 0;
}
The results:
x86 64bits: false
x86 32bits: true
so I'm getting two different OP in each case.
As author says, for 2 different data sizes one being 16 and the other 32, it holds good in my x86-64 case.
But im not able to understand why in the second case for 32 bits, I'm getting true.
As author says unsigned int is promoted to signed long int, if this is true then both
should be 4 bytes wide, then why is it printing true instead of false? As now both should be signed long.
As the author says it is machine dependent, then both long and int should have same byte size, so how the implicit conversion is happening here?
My understanding is that -1 is stored as two's complement i.e 0xFFFFFFFF > 0x1 so in the second case it should be true.
But this explanation contradicts the 1st case.
Please correct me if what I think is wrong, as I am new to implicit conversion.
Can anyone please explain this behaviour?
lets explain the rank system first
6.3.1 Arithmetic operand(c99 standard)
A) The rank of a signed integer type shall be greater than the rank of any signed integer
type with less precision(more bytes higher precision higher rank)
B) The rank of long long int shall be greater than the rank of long int, which shall be
greater than the rank of int, which shall be greater than the rank of short int, which
shall be greater than the rank of signed char.
C) The rank of any unsigned integer type shall equal the rank of the corresponding signed
integer type, if any.
(in other words if your system unsigned int is 32bits and your int is 32bits then the
ranks of these are the same.)
the above explains the rank.
now coming to arithmetic conversions.
6.3.1.8 Usual arithmetic conversions (c99 standard)
1)If both operands have the same type, then no further conversion is needed.
2)Otherwise, if both operands have signed integer types or both have unsigned integer
types, the operand with the type of lesser integer conversion rank is converted to the
type of the operand with greater rank.(similar to 1)
3)Otherwise, if the operand that has unsigned integer type has rank greater or equal to
the rank of the type of the other operand, then the operand with signed integer type is
converted to the type of the operand with unsigned integer type.
4)Otherwise, if the type of the operand with signed integer type can represent all of the
values of the type of the operand with unsigned integer type, then the operand with
unsigned integer type is converted to the type of the operand with signed integer type
5)Otherwise, both operands are converted to the unsigned integer type corresponding to the
type of the operand with signed integer type.
2) compiled on an x86 32bits platform and executed. Where sizeof(-1L) -> 4byte and sizeof(1U) -> 4 bytes
in your case look at statement 3 & C. the unsigned value(4bytes) has rank equal to the signed value(4btyes) therefore the singed value is converted to an unsigned value, when this happens the, the sign bit makes this look like a extremely large value. -1L > 1U therefore is true
1) compiled on an x86 64bits platform and executed. Where sizeof(-1L) -> 8byte and sizeof(1U) -> 4 bytes
in this case, the unsigned value rank is less than the rank of the singed value. look at 4).
the signed integer(8bytes) can represent any 4byte unsigned value. therefore the unsigned 4byte value is converted to a signed value.(this will preserve the sign bit, sign bit is 0)
therefore -1L > 1U is false
But im not able to understand why in second case in 32 bit its OP-->true. As author says unsigned int is promoted to signed long int if so then both are 4 byte wide, why its printing true instead of false.? since now both are signed long.
The auther says, that if int and long have different size, then unsigned int is promoted to signed long.
If int and long have the same size, then long is too small to hold all values of unsigned int and therefore both are converted to unsigned long.
For binary arithmetic and relational operators:
If either operand has type long double, the other operand is converted to long double. Otherwise, if either operand has type double, the other operand is converted to double. Otherwise, if either operand has type float, the other operand is converted to float. Otherwise the integral promotions are performed on both operands.
(Integral promotion: A char, a short int, or an int bit-field, or their signed or unsigned varieties, or an enumeration type, may be used in an expression wherever an int or unsigned int may be used. If an int can represent all the values of the original type, the value is converted to an int; otherwise it is converted to an unsigned int.)
Then if either operand has type unsigned long int, the other operand is converted to unsigned long int. Otherwise, if one operand has type long int and the other has type unsigned int, if a long int can represent all values of an unsigned int the operand of type unsigned int is converted to long int; if a long int cannot represent all the values of an unsigned int, both operands are converted to unsigned long int. Otherwise, if either operand has type long int, the other operand is converted to long int. Otherwise, if either operand has type unsigned int, the other operand is converted to unsigned int. Otherwise, both operands have type int.
The sentence in bold explains your second case, where long int has the same width as unsigned int thus cannot hold all values of unsigned int.
(The above description lacks the type unsigned long long int and long long it, but the rules are basically the same.)
As all who answered above are correct, Just to add more clarity and my understanding writing here to get more clarity.
-->if one operand has type long int and the other has type unsigned int,
-->if a long int can represent all values of an unsigned int the operand of type unsigned int is converted to long int;
-->if a long int cannot represent all the values of an unsigned int, both operands are converted to unsigned long int.
So from above one operand has type long int i.e -1L and the other has type unsigned int i.e 1U
suppose sizeof -1L is --->8byte and sizeof 1U is 4 byte
0X0000-0XFFFFF values can be represented using in long int whose sizeof is 8 byte
so in this case long int can represent all values of an unsigned int i.e using 8byte ---> it can represent all the values unsigned int 1U.
so----> here operand of type unsigned int is converted to long int ---> -1L > 1U --> is false
coming 2nd case
if a long int cannot represent all the values of an unsigned int
i.e sizeof -1L -->4byte and sizeof 1U -->4byte
here long int cannot represent all the values i.e using 4 bytes--> it cannot represent all the values of unsigned int 1U. so both operands are converted to unsigned long int
-1L appears to large value since its unsigned now when compared to 1U.
i.e---->0xFFFFFFFF > 0x1 ---> its true