Using this answer is good because it is very portable, correct and passes compilers set to be strict, but it is less efficient than I want and than it could be, because it doesn't use the x86 bswap instruction. (Maybe other instruction sets have similar efficient instructions.)
If the system I'm running on supports ntohl() then I would expect ntohl() to use the bswap instruction and gets me close. ntohl() does exactly the right thing, but it only works on uint32_t, not on a float. Casting between a uint32_t and a float is type punning and not allowed by strict compilers. Making a union with a float and a uint32_t runs into undefined compiler behavior (per previous posts here).
My understanding from previous posts is that casting from any pointer type to a char * or vice versa is explicitly allowed. So what is wrong with this solution? I haven't seen it mentioned in any answers yet.
char NetworkOrderFloat[4]; // Assume it contains network-order float bytes
uint32_t HostOrderInt = ntohl(*(uint32_t *)NetworkOrderFloat);
char *Pointer = (char *)&HostOrderInt;
float HostOrderFloat = *(float *)Pointer;
The ideal solution here seems to be more environments supporting ntohf(), but that doesn't seem to have happened yet.
Your proposal breaks “strict aliasing rules”, the first time when it does *(uint32_t *)NetworkOrderFloat. The expression (uint32_t *)NetworkOrderFloat is still the address of an array of chars, and accessing it with an lvalue of type uint32_t is against these rules. Details and more examples can be found in this article.
Using a union to convert a float's representation to uint32_t, on the other hand, is not forbidden by the C standard as far as I know. But you can always use memcpy if you worry that it is.
float NetworkOrderFloat = ...;
uint32_t tmp;
_Static_assert(sizeof(uint32_t)==sizeof(float),"unsupported arch");
memcpy(&tmp, &NetworkOrderFloat, sizeof(float));
tmp = ntohl(tmp);
memcpy(&HostOrderFloat, &tmp, sizeof(float));
A decent modern compiler should compile the memcpy calls to nothing and ntohl to bswap.
Related
I read this on a blog:
Violating Type Rules: It is undefined behavior to cast an int* to a float* and dereference it (accessing the "int" as if it were a "float"). C requires that these sorts of type conversions happen through memcpy: using pointer casts is not correct and undefined behavior results. The rules for this are quite nuanced and I don't want to go into the details here (there is an exception for char*, vectors have special properties, unions change things, etc). This behavior enables an analysis known as "Type-Based Alias Analysis" (TBAA) which is used by a broad range of memory access optimizations in the compiler, and can significantly improve performance of the generated code. For example, this rule allows clang to optimize this function:
How can you use the memcpy function for type coercion? And what about the exception to char*?
I don't understand how to use the memcpy function for type coercion？
Suppose you have the float value 1.25. And suppose you want to confirm that its actual IEEE-754 representation in hexadecimal is 3fa00000. There are at least four different ways you might try to do this:
(1) Take a float pointer and cast it to an integer pointer, and indirect on it:
float f = 1.25;
printf("%08x\n", *(uint32_t *)&f);
(This fragment quietly assumes 32-bit int. For better portability, you could use printf("%08" PRIx32 "\n", *(uint32_t *)&f);.)
(2) Use a union:
union {float f; uint32_t i;} u;
u.f = f;
printf("%08x\n", u.i);
(3) Use a char pointer, and iterate/index:
unsigned char *p = (unsigned char *)&f;
for(int i = 3; i >= 0; i--) printf("%02x", p[i]);
(Note that this code fragment assumes little-endian.)
(4) Use memcpy:
uint32_t x;
memcpy(&x, &f, 4);
printf("%08x\n", x);
Now, the take-home lesson is that not all of these methods work reliably any more, because of the strict aliasing rule.
In particular, method (1) is flatly illegal. It's a textbook example of what the strict aliasing rule disallows.
I think you're still allowed to use a union as in method 2, but you may have to put on a language lawyer hat to convince yourself of it. (See also the comments on this answer below.)
Methods (3) and (4), however, continue to work, because they take advantage of an explicit exception to the strict aliasing rule, namely that you are allowed to access the bits of an object using a punned pointer of the "wrong" type, as long as the "wrong type" is specifically a character pointer.
So I think this is clear, but in answer to your specific questions:
How can you use the memcpy function for type coercion?
As in method (4).
And what about the exception to char *?
That's the explicit exception in the strict aliasing rule that allows method (3) to work.
The rules, by the way, are significantly different here in C than in C++. Strictly speaking, I believe, in C++ not even method (3) is legal, and the only way you're allowed to do this sort of thing any more is with method (4) and an implicit call to memcpy. (However, I'm told that optimizing compilers tend to treat calls to memcpy very specially these days, not only replacing explicit function calls with inline register moves, but sometimes even optimizing out the copy altogether, and doing something like method 1 or 2 internally, if they know they can get away with it.)
For example, can this
unsigned f(float x) {
unsigned u = *(unsigned *)&x;
return u;
}
cause unpredictable results on a platform where,
unsigned and float are both 32-bit
a pointer has a fixed size for all types
unsigned and float can be stored to and loaded from the same part of memory.
I know about strict aliasing rules, but most examples showing problematic cases of violating strict aliasing is like the following.
static int g(int *i, float *f) {
*i = 1;
*f = 0;
return *i;
}
int h() {
int n;
return g(&n, (float *)&n);
}
In my understanding, the compiler is free to assume that i and f are implicitly restrict. The return value of h could be 1 if the compiler thinks *f = 0; is redundant (because i and f can't alias), or it could be 0 if it puts into account that the values of i and f are the same. This is undefined behaviour, so technically, anything else can happen.
However, the first example is a bit different.
unsigned f(float x) {
unsigned u = *(unsigned *)&x;
return u;
}
Sorry for my unclear wording, but everything is done "in-place". I can't think of any other way the compiler might interpret the line unsigned u = *(unsigned *)&x;, other than "copy the bits of x to u".
In practice, all compilers for various architectures I tested in https://godbolt.org/ with full optimization produce the same result for the first example, and varying results (either 0 or 1) for the second example.
I know it's technically possible that unsigned and float have different sizes and alignment requirements, or should be stored in different memory segments. In that case even the first code won't make sense. But on most modern platforms where the following holds, is the first example still undefined behaviour (can it produce unpredictable results)?
unsigned and float are both 32-bit
a pointer has a fixed size for all types
unsigned and float can be stored to and loaded from the same part of memory.
In real code, I do write
unsigned f(float x) {
unsigned u;
memcpy(&u, &x, sizeof(x));
return u;
}
The compiled result is the same as using pointer casting, after optimization. This question is about interpretation of the standard about strict aliasing rules for code such as the first example.
Is it always undefined behaviour to copy the bits of a variable through an incompatible pointer?
Yes.
The rule is https://port70.net/~nsz/c/c11/n1570.html#6.5p7 :
An object shall have its stored value accessed only by an lvalue expression that has one of
the following types:
a type compatible with the effective type of the object,
a qualified version of a type compatible with the effective type of the object,
a type that is the signed or unsigned type corresponding to the effective type of the
object,
a type that is the signed or unsigned type corresponding to a qualified version of the
effective type of the object,
an aggregate or union type that includes one of the aforementioned types among its
members (including, recursively, a member of a subaggregate or contained union), or
a character type.
The effective type of the object x is float - it is defined with that type.
unsigned is not compatible with float,
unsigned is not a qualified version of float,
unsigned is not a signed or unsigned type of float,
unsigned is not a signed or unsigned type corresponding to qualified version of float,
unsigned is not an aggregate or union type
and unsigned is not a character type.
The "shall" is violated, it is undefined behavior (see https://port70.net/~nsz/c/c11/n1570.html#4p2 ). There is no other interpretation.
We also have https://port70.net/~nsz/c/c11/n1570.html#J.2 :
The behavior is undefined in the following circumstances:
An object has its stored value accessed other than by an lvalue of an allowable type (6.5).
As Kamil explains, it's UB. Even int and long (or long and long long) aren't alias-compatible even when they're the same size. (But interestingly, unsigned int is compatible with int)
It's nothing to do with being the same size, or using the same register-set as suggested in a comment, it's mainly a way to let compilers assume that different pointers don't point to overlapping memory when optimizing. They still have to support C99 union type-punning, not just memcpy. So for example a dst[i] = src[i] loop doesn't need to check for possible overlap when unrolling or vectorizing, if dst and src have different types.1
If you're accessing the same integer data, the standard requires that you use the exact same type, modulo only things like signed vs. unsigned and const. Or that you use (unsigned) char*, which is like GNU C __attribute__((may_alias)).
The other part of your question seems to be why it appears to work in practice, despite the UB.
Your godbolt link forgot to link the actual compilers you tried.
https://godbolt.org/z/rvj3d4e4o shows GCC4.1, from before GCC went out of its way to support "obvious" local compile-time-visible cases like this, to sometimes not break people's buggy code using non-portable idioms like this.
It loads garbage from stack memory, unless you use -fno-strict-aliasing to make it movd to that location first. (Store/reload instead of movd %xmm0, %eax is a missed-optimization bug that's been fixed in later GCC versions for most cases.)
f: # GCC4.1 -O3
movl -4(%rsp), %eax
ret
f: # GCC4.1 -O3 -fno-strict-aliasing
movss %xmm0, -4(%rsp)
movl -4(%rsp), %eax
ret
Even that old GCC version warns warning: dereferencing type-punned pointer will break strict-aliasing rules which should make it obvious that GCC notices this and does not consider it well-defined. Later GCC that do choose to support this code still warn.
It's debatable whether it's better to sometimes work in simple cases, but break other times, vs. always failing. But given that GCC -Wall does still warn about it, that's probably a good tradeoff between convenience for people dealing with legacy code or porting from MSVC. Another option would be to always break it unless people use -fno-strict-aliasing, which they should if dealing with codebases that depend on this behaviour.
Being UB doesn't mean required-to-fail
Just the opposite; it would take tons of extra work to actually trap on every signed overflow in the C abstract machine, for example, especially when optimizing stuff like 2 + c - 3 into c - 1. That's what gcc -fsanitize=undefined tries to do, adding x86 jo instructions after additions (except it still does constant-propagation so it's just adding -1, not detecting temporary overflow on INT_MAX. https://godbolt.org/z/WM9jGT3ac). And it seems strict-aliasing is not one of the kinds of UB it tries to detect at run time.
See also the clang blog article: What Every C Programmer Should Know About Undefined Behavior
An implementation is free to define behaviour the ISO C standard leaves undefined
For example, MSVC always defines this aliasing behaviour, like GCC/clang/ICC do with -fno-strict-aliasing. Of course, that doesn't change the fact that pure ISO C leaves it undefined.
It just means that on those specific C implementations, the code is guaranteed to work the way you want, rather than happening to do so by chance or by de-facto compiler behaviour if it's simple enough for modern GCC to recognize and do the more "friendly" thing.
Just like gcc -fwrapv for signed-integer overflows.
Footnote 1: example of strict-aliasing helping code-gen
#define QUALIFIER // restrict
void convert(float *QUALIFIER pf, const int *pi) {
for(int i=0 ; i<10240 ; i++){
pf[i] = pi[i];
}
}
Godbolt shows that with the -O3 defaults for GCC11.2 for x86-64, we get just a SIMD loop with movdqu / cvtdq2ps / movups and loop overhead. With -O3 -fno-strict-aliasing, we get two versions of the loop, and an overlap check to see if we can run the scalar or the SIMD version.
Is there actual cases where strict aliasing helps better code generation, in which the same cannot be achieved with restrict
You might well have a pointer that might point into either of two int arrays, but definitely not at any float variable, so you can't use restrict on it. Strict-aliasing will let the compiler still avoid spill/reload of float objects around stores through the pointer, even if the float objects are global vars or otherwise aren't provably local to the function. (Escape analysis.)
Or a struct node * that definitely isn't the same type as the payload in a tree.
Also, most code doesn't use restrict all over the place. It could get quite cumbersome. Not just in loops, but in every function that deals with pointers to structs. And if you get it wrong and promise something that's not true, your code's broken.
The Standard was never intended to fully, accurately, and unambiguously partition programs that have defined behavior and those that don't(*), but instead relies upon compiler writers to exercise a certain amount of common sense.
(*) If it was intended for that purpose, it fails miserably, as evidenced by the amount of confusion stemming from it.
Consider the following two code snippets:
/* Assume suitable declarations of u are available everywhere */
union test { uint32_t ww[4]; float ff[4]; } u;
/* Snippet #1 */
uint32_t proc1(int i, int j)
{
u.ww[i] = 1;
u.ff[j] = 2.0f;
return u.ww[i];
}
/* Snippet #2, part 1, in one compilation unit */
uint32_t proc2a(uint32_t *p1, float *p2)
{
*p1 = 1;
*p2 = 2.0f;
return *p1;
}
/* Snippet #2, part 2, in another compilation unit */
uint32_t proc2(int i, int j)
{
return proc2a(u.ww+i, u.ff+j);
}
It is clear that the authors of the Standard intended that the first version of the code be processed meaningfully on platforms where that would make sense, but it's also clear that at least some of the authors of C99 and later versions did not intend to require that the second version be processed likewise (some of the authors of C89 may have intended that the "strict aliasing rule" only apply to situations where a directly named object would be accessed via pointer of another type, as shown in the example given in the published Rationale; nothing in the Rationale suggests a desire to apply it more broadly).
On the other hand, the Standard defines the [] operator in such a fashion that proc1 is semantically equivalent to:
uint32_t proc3(int i, int j)
{
*(u.ww+i) = 1;
*(u.ff+j) = 2.0f;
return *(u.ww+i);
}
and there's nothing in the Standard that would imply that proc() shouldn't have the same semantics. What gcc and clang seem to do is special-case the [] operator as having a different meaning from pointer dereferencing, but nothing in the Standard makes such a distinction. The only way to consistently interpret the Standard is to recognize that the form with [] falls in the category of actions which the Standard doesn't require that implementations process meaningfully, but relies upon them to handle anyway.
Constructs such as yours example of using a directly-cast pointer to access storage associated with an object of the original pointer's type fall in a similar category of constructs which at least some authors of the Standard likely expected (and would have demanded, if they didn't expect) that compilers would handle reliably, with or without a mandate, since there was no imaginable reason why a quality compiler would do otherwise. Since then, however, clang and gcc have evolved to defy such expectations. Even if clang and gcc would normally generate useful machine code for a function, they seek to perform aggressive inter-procedural optimizations that make it impossible to predict what constructs will be 100% reliable. Unlike some compilers which refrain from applying potential optimizing transforms unless they can prove that they are sound, clang and gcc seek to perform transforms that can't be proven to affect program behavior.
I am utilizing Microchip sample nvmem.c file function to write data into particular memory address of PIC32 Microcontroller. When I am trying to use it showing following MISRA error: I just posted sample code where I got an error. My whole code is compiled and working fine.
1] explicit cast from 'unsigned int' to 'void ' [MISRA 2012 Rule 11.6, required] at NVMemWriteWord((void)APP_FLASH_MARK_ADDRESS,(UINT)_usermark);
How can I resolve this error?
nvmem.c
uint8_t NVMemWriteWord(void* address, uint32_t data)
{
uint8_t res;
NVMADDR = KVA_TO_PA((uint32_t)address); //destination address to write
NVMDATA = data;
res = NVMemOperation(NVMOP_WORD_PGM);
}
test.c
#define ADDRESS 0x9D007FF0U;
NVMemWriteWord((void*)ADDRESS,(uint32_t)_usermark);
Use
uint8_t NVMemWriteWord(unsigned int address, uint32_t data)
{
uint8_t res;
NVMADDR = KVA_TO_PA(address);
NVMDATA = data;
res = NVMemOperation(NVMOP_WORD_PGM);
}
and
#define ADDRESS 0x9D007FF0U
NVMemWriteWord(ADDRESS,(uint32_t)_usermark);
instead. Functionally it is exactly equivalent to the example, it just avoids the cast from a void pointer to an unsigned integer address.
Suggest:
#define ADDRESS (volatile uint32_t*)0x9D007FF0U
NVMemWriteWord( ADDRESS, _usermark) ;
Never cast to void* - the purpose of void* is that you can assign any other pointer type to it safely and without explicit cast. The cast of _usermark may or may not be necessary, but unnecessary explicit casts should be avoided - they can suppress important compiler warnings. You should approach type conversions in the following order of preference:
Type agreement - exactly same types.
Type compatibility - smaller type to larger type, same signedness.
Type case - last resort (e.g. larger to smaller type, signedness mismatch, integer to/from pointer).
In this instance since NVMemWriteWord simply casts address to an integer, then the use of void* may not be appropriate. If in other contexts you are actually using a pointer, then it may be valid.
The whole of MISRA-C:2012 chapter 12 regarding pointer conversions is quite picky. And rightly so, since this is very dangerous territory.
11.6 is a sound rule that bans conversions from integers to void*. The rationale is to block alignment bugs. There aren't many reasons why you would want to do such conversions anyway.
Notably, there's also two rigid but advisory rules 11.4 which bans conversions from integers to pointers, and 11.5 which pretty much bans the use of void* entirely. It isn't possible to do hardware-related programming and follow 11.4, so that rule has to be ignored. But you have little reason to use void*.
In this specific cast you can get away by using uint32_t and avoiding pointers entirely.
In the general case of register access, you must do a conversion with volatile-qualified pointers: (volatile uint32_t*)ADDRESS, assuming that the MCU uses 32 bit registers.
I have a function foo(void* pBuf). I need to pass it a 64 bit address but I can't seem to get the right typecast when I'm passing by value.
Example: foo(address). Where- uint64_t address=0x00000000DEADBEEF
EDIT: Compiling using an ARM compiler.
uint64_t foo(void *pBuf){
uint64_t retAddr = (uint64_t) pBuf;
retAddr += 0x100000;
return retAddr;
}
I'm on a 32-bit ARM and sizeof(void *) is 4
Clarification: Why I needed a 64-bit address on a 32-bit ARM?
Because my memory map uses 36-bit addressing.
Call it this way:
uint64_t address = 0xDEADBEEF;
foo((void*)address);
That is, you cast the address to a void-pointer to be compatible with the function signature.
Sorry to necro this question, but none of these answers seem reasonable to me. This is a fairly straightforward type conversion problem. It seems as though people were caught up on 64-bit addressing on a 32-bit system, when this could easily be for a peripheral or some other address space besides the system itself.
In the OP's case, a cast directly to uint64_t would cause undefined behavior because of the additional four bytes that do not exist in void *. In the case of the M4 calling convention, p would typically be passed in a single register, likely r0. There are no additional upper bytes for uint64_t to alias, so your compiler is rightly issuing a warning for this.
Under the GCC 7.3 arm-none-eabi port, void * can be safely cast to size_t (aka unsigned int) because they both have size and alignment of 4. Once that is done, you can safely promote unsigned int to uint64_t (aka unsigned long long int) by assignment. The promotion is better defined behavior than a cast.
uint64_t foo(void *p){
uint64_t a = (size_t) p;
a += 0x100000;
return a;
}
You should not use a 64-bits type for an address, as it is undefined behavior for 32-bits (or any non-64 bits) systems.
Rather, prefer using uintptr_t, which is standard C.
See this question for more details or this page for references.
Then a solution could be :
uintptr_t address = 0xDEADBEEF; /* will trigger a warning if the constant is > max possible memory size */
foo((void*)address);
Note : if uintptr_t is not available on your system, size_t is usually a good second choice.
Part 2 :
Looks like, in your rephrased question, you want to convert an address into a 64-bits integer.
In which case, a direct cast from ptr to integer is likely to trigger a compiler warning, due to potential differences in wideness.
Prefer a double cast :
uint64_t value = (uint64_t)(size_t) ptr;
I can think of two ways to get this right. Got a solution to my problem by calling foo the first way
foo((void*)(uint32_t)address)
This works only because my input to foo is always a 32-bit value. The returned value can be 64-bit.
Of course, a proper fix would be to change foo itself, if I could modify it.
I could just pass foo(&address). Inside foo, retAddr = *pBuf.
Thanks for all the suggestions!
This question already has answers here:
Should I use cstdint?
(6 answers)
Closed 8 years ago.
I have 2 questions about C programming:
For int and uint16_t, long and uint32_t, and so on. When should I use the u*_t types instead of int, long, and so on? I found it confusing to choose which one is best for my program.
When do I need to cast type?
I have the following statement in my program:
long * src;
long * dst;
...
memcpy(dst, src, len);
My friend changes this to
memcpy((char *)dst, (char *)src, len).
This is just example I encountered. Generally, I am confused when cast is required?
Use the plain types (int etc) except when you need a precisely-sized type. You might need the precisely sized type if you are working with a wire protocol which defines that the size field shall be a 2-byte unsigned integer (hence uint16_t), but for most work, most of the time, use the plain types. (There are some caveats to this, but most of the time, most people can work with the plain types for simple numeric work. If you are working to a set of interfaces, use the types dictated by the interfaces. If you're using multiple interfaces and the types clash, you'll have to consider using casting some of the time — or change one or both interfaces. Etc.)
The casts added by your friend are pointless. The actual prototype of memcpy() is:
void *memcpy(void * restrict s1, const void * restrict s2, size_t n);
The compiler converts the long * values to void * (nominally via char * because of the cast), all of which is almost always a no-op.
More generally, you use a cast when you need to change the type of something. One place you might need it is in bitwise operations, where you want a 64-bit result but the operands are 32-bit and leaving the conversion until after the bitwise operations gives a different result from the one you wanted. For example, assuming a system where int is 32 bits and long is 64 bits.
unsigned int x = 0x012345678;
unsigned long y = (~x << 22) | 0x1111;
This would calculate ~x as a 32-bit quantity, and the shift would be performed on a 32-bit quantity, losing a number of bits. By contrast:
unsigned long z = (~(unsigned long)x << 22) | 0x1111;
ensures that the calculation is done in 64-bit arithmetic and doesn't lose any bits from the original value.
The size of "classical" types like int and long int can vary between systems. This can cause problems when, for example, accessing files with fixed-width data structures. For example, int long is currently a 64-bit integer on new systems, but only 32 bits on older systems.
The intN_t and uintN_t types were introduced with C99 and are defined in <inttypes.h>. Since they explicitly specify the number of bits, they eliminate any ambiguity. As a rule, you should use these types in preference if you are at all concerned about making your code portable.
Wikipedia has more information
If you do not want to rely on your compiler use predefined types provided by standard library headers. Every C library you'd compile with is guaranteed to assign proper types to have at least size to store values of size their types declare.
In your friend specific case one can assume that he made this type cast just because he wanted to point other readers that two pointers actually hold symbol characters. Or maybe he is kind of old-fashion guy who remembers the times when there was no void type and the "lowest common divisor" was pointer to char. In my developer life, if I want to emphasize some of my actions I'll make an explicit type cast even if it is, in fact, redundant.
For you 1st question, look at : https://stackoverflow.com/questions/11786113/difference-between-different-integer-types
Basically, the _t is the real standard type name and without, it's a define of the same type.
the u is for unsigned which doesn't allow negative number.
As for your second question, you often need to cast when the function called needs arguments of another type that what you're passing. You can look here for casting tips, or here...