I have the following program where I initialize two buffers in a seemingly fast way by casting the 8-bit buffer to 32 and 64-bit values.
#include <stdio.h>
#include <stdint.h>
typedef struct {
uint32_t a[2];
uint16_t b[4];
} ctx_t;
void inita(ctx_t *ctx, uint8_t *aptr)
{
*(uint64_t *) (ctx->a) = *(uint64_t *) (aptr);
}
void initb(ctx_t *ctx, uint8_t *bptr)
{
for (int i = 0; i < 2; i++) {
*((uint32_t *) (ctx->b) + i) = *(uint32_t *) (bptr);
}
}
int main()
{
uint8_t a[8] = {0x01, 0x23, 0x45, 0x67, 0x89, 0xab, 0xcd, 0xef};
uint8_t b[8] = {0x01, 0x23, 0x45, 0x67, 0x89, 0xab, 0xcd, 0xef};
ctx_t ctx;
inita(&ctx, a);
initb(&ctx, b);
printf("a: ");
for (int i = 0; i < 8; i++) {
printf("%02x", a[i]);
}
printf("\nb: ");
for (int i = 0; i < 8; i++) {
printf("%02x", b[i]);
}
printf("\n");
}
When compiling using GCC version 8.2.1, I get the following warning message:
> gcc -std=c99 -Wall -Wextra -Wshadow -fsanitize=address,undefined -O2 main.c
main.c: In function ‘inita’:
main.c:11:3: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
*(uint64_t *) (ctx->a) = *(uint64_t *) (aptr);
^~~~~~~~~~~~~~~~~~~~~
I read about the strict aliasing rules and it makes sense that something could go wrong by breaking this.
However, why am I not getting the same warning in the initb() function? Here I am also casting pointers and buffers to other sizes than what was declared.
The program works and yields the expected output:
a: 0123456789abcdef
b: 0123456789abcdef
If I fix the warning, by doing:
void inita(ctx_t *ctx, uint8_t *aptr)
{
*(uint32_t *) (ctx->a) = *(uint32_t *) (aptr);
*(uint32_t *) (ctx->a + 1) = *(uint32_t *) (aptr + 4);
}
Then, I now get the same result as before but without warnings.
Do I still have aliasing problems in my code (due to initb) or is it safe to do this?
Do I still have aliasing problems in my code (due to initb) or is it safe to do this?
Yes there is aliasing problems in both of your functions, and no it is not safe. There are no guarantees it will work or keep working. Compiler simply got confused when trying to detect it.
You need to explictly disable strict aliasing for not to risk compiler generating incorrect code. You also haven't taken care of potential alignment issues. When you fix those, you have also made your code harder to port.
Use memcpy instead:
void inita(ctx_t *ctx, uint8_t *aptr)
{
memcpy(ctx->a, aptr, sizeof uint64_t);
}
In the languages Dennis Ritchie invented, called C, and described in both editions of The C Programming Language (but extended to include 64-bit types) the one problem with the above code is that there is no guarantee that the automatic char[] objects will be aligned in a fashion suitable for access via 64-bit pointer. This would not be a problem on x86 or x64 platforms, but would be a problem on platforms based on other architectures like the ARM Cortex-M0.
Dialects processed by implementations such as MSVC or icc that uphold the Spirit of C described by the Standards Committee, including the principle "Don't prevent the programmer from doing what needs to be done", and make an effort to support low-level programming, will recognize that a construct of the form *(T*)pointerOfTypeU might access an object of type U. Although the Standard doesn't mandate such support, that's likely because the authors expected that implementations would make at least some effort to to recognize situations where a pointer of one type was formed from an lvalue of another type, and thought such recognition could be left as a quality-of-implementation issue. Note that there is nothing in the Standard that would require that a compiler given:
struct foo {int x[10]; };
void test(struct foo *p, int i)
{
struct foo temp;
temp = *p;
temp.x[i] = 1;
*p = temp;
}
recognize that an object of type struct foo might be affected by the actions of forming an int* with the address foo.x+i, dereferencing it, then writing to the resulting lvalue of type int. Instead, the authors of the Standard rely upon quality implementations to make some effort to recognize obvious cases where a pointer or lvalue of one type is derived from a pointer or lvalue of another.
The icc compiler, given:
int test1(int *p1, float *q1)
{
*p1 = 1;
*q1 = 2.0f;
return *p1;
}
int test2(int *p2, int *q2)
{
*p2 = 1;
*(float*)q2 = 2.0f;
return *p2;
}
int test3(int *p3, float volatile *q3)
{
*p3 = 1;
*q3 = 2.0f;
return *p3;
}
will assume that because p1 and p2 are different types, and have no discernible relationship, they will not alias, but will recognize that because p2 and q2 have the same type, they could identify the same object. It will further recognize that the lvalue *(float*)q2 is quite obviously based on q2. Consequently, it will recognize that an access to *(float*)q2 might be an access to *p2. Additionally, icc will treat volatile as a "don't assume you understand everything that's going on here" indication, and thus allow for the possibility that an access via q3 might affect other objects in weird ways.
Some compilers like clang and gcc, unless forced via -O0 or -fno-strict-aliasing to behave in a manner suitable for low-level programming, interpret the Standard as an excuse to ignore obvious relationships among lvalues of different types except in cases where doing so would break language constructs so badly as to make them pretty much useless. Although they happen to recognize that an access to someUnion.array[i] is an access to someUnion, they do recognize *(someUnion.array+i) likewise, even though the definition of someUnion.array[i] is *(someUnion.array+i). Given that the Standard treats support for almost anything having to do with mixed-type storage as a "quality of implementation" issue, all that can really be said is that compilers that are suitable for different purposes support different combinations of constructs.
Related
Consider this artificial example:
#include <stddef.h>
static inline void nullify(void **ptr) {
*ptr = NULL;
}
int main() {
int i;
int *p = &i;
nullify((void **) &p);
return 0;
}
&p (an int **) is casted to void **, which is then dereferenced. Does this break the strict aliasing rules?
According to the standard:
An object shall have its stored value accessed only by an lvalue
expression that has one of the following types:
a type compatible with the effective type of the object,
So unless void * is considered compatible with int *, this violates the strict aliasing rules.
However, this is not what is suggested by gcc warnings (even if it proves nothing).
While compiling this sample:
#include <stddef.h>
void f(int *p) {
*((float **) &p) = NULL;
}
gcc warns about strict aliasing:
$ gcc -c -Wstrict-aliasing -fstrict-aliasing a.c
a.c: In function ‘f’:
a.c:3:7: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
*((float **) &p) = NULL;
~^~~~~~~~~~~~~
However, with a void **, it does not warn:
#include <stddef.h>
void f(int *p) {
*((void **) &p) = NULL;
}
So is it valid regarding the strict aliasing rules?
If it is not, how to write a function to nullify any pointer (for example) which does not break the strict aliasing rules?
There is no general requirement that implementations use the same representations for different pointer types. On a platform that would use a different representation for e.g. an int* and a char*, there would be no way to support a single pointer type void* that could act upon both int* and char* interchangeably. Although an implementation that can handle pointers interchangeably would facilitate low-level programming on platforms which use compatible representations, such ability would not be supportable on all platforms. Consequently, the authors of the Standard had no reason to mandate support for such a feature rather than treating it as a quality of implementation issue.
From what I can tell, quality compilers like icc which are suitable for low-level programming, and which target platforms where all pointers have the same representation, will have no difficulty with constructs like:
void resizeOrFail(void **p, size_t newsize)
{
void *newAddr = realloc(*p, newsize);
if (!newAddr) fatal_error("Failure to resize");
*p = newAddr;
}
anyType *thing;
... code chunk #1 that uses thing
resizeOrFail((void**)&thing, someDesiredSize);
... code chunk #2 that uses thing
Note that in this example, both the act of taking thing's address, and all use the of resulting pointer, visibly occur between the two chunks of code that use thing. Thus, there is no actual aliasing, and any compiler which is not willfully blind will have no trouble recognizing that the act of passing thing's address to reallocorFail might cause thing to be modified.
On the other hand, if the usage had been something like:
void **myptr;
anyType *thing;
myptr = &thing;
... code chunk #1 that uses thing
*myptr = realloc(*myptr, newSize);
... code chunk #2 that uses thing
then even quality compilers might not realize that thing might be affected between the two chunks of code that use it, since there is no reference to anything of type anyType* between those two chunks. On such compilers, it would be necessary to write the code as something like:
myptr = &thing;
... code chunk #1 that uses thing
*(void *volatile*)myptr = realloc(*myptr, newSize);
... code chunk #2 that uses thing
to let the compiler know that the operation on *mtptr is doing something "weird". Quality compilers intended for low-level programming will regard this as a sign that they should avoid caching the value of thing across such an operation, but even the volatile qualifier won't be enough for implementations like gcc and clang in optimization modes that are only intended to be suitable for purposes that don't involve low-level programming.
If a function like reallocOrFail needs to work with compiler modes that aren't really suitable for low-level programming, it could be written as:
void resizeOrFail(void **p, size_t newsize)
{
void *newAddr;
memcpy(&newAddr, p, sizeof newAddr);
newAddr = realloc(newAddr, newsize);
if (!newAddr) fatal_error("Failure to resize");
memcpy(p, &newAddr, sizeof newAddr);
}
This would, however, require that compilers allow for the possibility that resizeOrFail might alter the value of an arbitrary object of any type--not merely data pointers--and thus needlessly impair what should be useful optimizations. Worse, if the pointer in question happens to be stored on the heap (and isn't of type void*), a conforming compilers that isn't suitable for low-level programming would still be allowed to assume that the second memcpy can't possibly affect it.
A key part of low-level programming is ensuring that one chooses implementations and modes that are suitable for that purpose, and knowing when they might need a volatile qualifier to help them out. Some compiler vendors might claim that any code which requires that compilers be suitable for its purposes is "broken", but attempting to appease such vendors will result in code that is less efficient than could be produced by using a quality compiler suitable for one's purposes.
Some C compilers can sometimes deduce a casted pointer as still a lvalue, but gcc defaults won't compile it. Instead of having to port (by human-error-prone way)legacy code base to:
p=(int *)p+1 ; /* increment by sizeof(int) */
can gcc be made to allow this code below (even if not technically correct)?
void f() {
void *p ; /* type to be cast later for size */
((int *)p)++ ; /* gcc -c f.c => lvalue required error */ }
Edit: even if technically incorrect, I assume the only programmer intent for such code is for p to remain lvalue and "increment" it, generating same code as my long form, right? (perreal flagged as "lvalue cast")
Edit2: We all agree refactoring to std C is best, but if something like Mateo's -fpermissive worked, it might not catch future programming errors, but hoping initial gcc porting effort will be less faulty... Any other similar suggestions?
If you want to clean up your code(remove excessive casts) , you can concentrate the ugly casts inside a macro or, even better, an inlined function.
The below fragment only makes use of implicit casts to/from void*
You are ,of course, still responsible for the proper alignment.
#include <stdio.h>
static void *increment(void * p,size_t offset)
{
char *tmp = p;
return tmp+offset;
}
int main(void)
{
void *ptr = "Hell0 world!\n";
ptr = increment( ptr, sizeof(int) );
printf("%s", (char*) ptr);
return 0;
}
I'm fixing some code not written by me, so I found this:
#define get_u_int16_t(X,O) (*(u_int16_t *)(((u_int8_t *)X) + O))
How can I change it to keep the rule, if it violating it ?
The macro is called in this way:
if(get_u_int16_t(packet->payload, i) == ...) { ... }
where payload is a const unsigned char * and i is an unsigned int .
The situation is:
struct orig {
[...]
struct pkt packet;
}*;
struct pkt {
[...]
const u_int8_t *payload;
}*;
Called in this way:
struct orig * flow;
struct pkt * packet = &flow->packet;
payload is a string
i begins with a value of 0 and it is inside a for that loop for the lenght of payload ( u_int16_t len ):
for(i = 0; i < len; i++) {
if(get_u_int16_t(packet->payload, a) == /*value*/) {
// do stuff
}
The macro itself doesn't violate the strict aliasing rule; it depends how you use it. If you only use it to read already existing objects of type u_int16_t or a compatible type then it's fine; if on the other hand you use it to read e.g. parts of a 64-bit integer or a floating-point object then that would be a strict aliasing violation, as well as (possibly) an alignment violation.
As always, you can make the code safe using memcpy:
inline u_int16_t read_u_int16_t(const void *p) {
u_int16_t val;
memcpy(&val, p, sizeof(val));
return val;
}
#define get_u_int16_t(X,O) (read_u_int16_t(((u_int8_t *)X) + O))
As #2501 points out this may be invalid if u_int8_t is not a character type, so you should just use char * for pointer arithmetic:
#define get_u_int16_t(X,O) (read_u_int16_t(((char *)X) + O))
There are two ways to write code of the type you're interested in:
Use pointers of type "unsigned char*" to access anything, assemble values out of multiple bytes, and tolerate the performance hit.
If you know that pointer alignment won't be an issue, use a dialect of C that doesn't semantically gut it. GCC can implement such a dialect if invoked with -fno-strict-aliasing, but since gcc has no way to prevent obtuse "optimizations" without blocking useful and sensible optimizations, getting good performance from such a dialect may require learning how to use restrict.
Some people would argue that approach #1 is better, and for many PC-side programming purposes I'd tend to agree. For embedded software, however, I would suggest staying away from gcc's default dialect since there's no guarantee that the things gcc regards as defined behavior today will remain so tomorrow. I'm not sure where the authors of gcc got the notion that the authors of the Standard were trying to specify all of the forms of aliasing that a quality microprocessor implementation should support, rather than establish a minimum baseline for platforms where no forms of type punning other than "unsigned char" would be useful.
Folks,
Consider this (abominable) piece of code:
volatile unsigned long a[1];
unsigned long T;
void main(void)
{
a[0] = 0x6675636b; /* first access of a */
T = *a;
*(((char *)a) + 3) = 0x64; /* second access of a */
T = *a;
}
...the question: is ((char *)a) volatile or non-volatile?
This begs a larger question: should there be a dependence between the two accesses of a? That is, human common sense says there is, but the C99 standard says that volatile things don't alias non-volatile things -- so if ((char *)a) is non-volatile, then the two accesses don't alias, and there isn't a dependence.
More correctly, C99 6.7.3 (para 5) reads:
"If an attempt is made to refer to an object defined with a volatile-qualified type through use of an lvalue with non-volatile-qualified type, the behavior is undefined."
So when we typecast a, does the volatile qualifier apply?
When in doubt, run some code :) I whipped up some similar (slightly less abominable) test code (win32 C++ app in msvs 2k10)
int _tmain(int argc, _TCHAR* argv[]) {
int a = 0;
volatile int b = 0;
a = 1; //breakpoint 1
b = 2; //breakpoint 2
*(int *) &b = 0; //breakpoint 3
*(volatile int *) &b = 0; //breakpoint 4
return 0;
}
When compiled for release, I am allowed to breakpoint at 2 and 4, but not 1 and 3.
My conclusion is that the typecast determines the behavior and 1 and 3 were optimized away. Intuition supports this - otherwise compiler would have to keep some type of list of all locations of memory listed as volatile and check on every access (hard, ugly), rather than just associating it with the type of the identifier (easier and more intuitive).
I also suspect it's compiler specific (and possibly flag specific even within a compiler) and would test on any platform before depending on this behavior.
Actually scratch that, I would simply try to not depend on this behavior :)
Also, I know you were asking specifically about arrays, but I doubt that makes a difference. You can easily whip up similar test code for arrays.
like you said, its "undefined". Which means demons can come out of your nose. Please stick to the "defined" behaviours as much as possible. A volatile specifier will ask the compiler to not optimize the value, since its an "important" and critical value that might cause problems if changed due to different optmization mechanisms. But that's all it can do.
In C, what exactly are the performance benefits that come with observing strict aliasing?
There is a page that describes aliasing very thoroughly here.
There are also some SO topics here and here.
To summarize, the compiler cannot assume the value of data when two pointers of different types are accessing the same location (i.e. it must read the value every time and therefore cannot make optimizations).
This only occurs when strict aliasing is not being enforced. Strict aliasing options:
gcc: -fstrict-aliasing [default] and -fno-strict-aliasing
msvc:
Strict aliasing is off by default.
(If somebody knows how to turn it on,
please say so.)
Example
Copy-paste this code into main.c:
void f(unsigned u)
{
unsigned short* const bad = (unsigned short*)&u;
}
int main(void)
{
f(5);
return 0;
}
Then compile the code with these options:
gcc main.c -Wall -O2
And you will get:
main.c:3:
warning: dereferencing type-punned
pointer will break strict-aliasing
rules
Disable aliasing with:
gcc main.c -fno-strict-aliasing
-Wall -O2
And the warning goes away. (Or just take out -Wall but...don't compile without it)
Try as I might I could not get MSVC to give me a warning.
The level of performance improvement that will result from applying type-based aliasing will depend upon:
The extent to which code caches things in automatic-duration objects, or via the restrict qualifier, indicates that compilers may do so without regard for whether they might be affected by certain pointer-based operations.
Whether the aliasing assumptions made by a compiler are consistent with what a programmer needs to do (if they're not, reliable processing would require disabling type-based aliasing, negating any benefits it could otherwise have offered).
Consider the following two code snippets:
struct descriptor { uint32_t size; uint16_t *dat; };
void test(struct descriptor *ptr)
{
for (uint32_t i=0; i < ptr->size; i++)
ptr->dat[i] = 1234;
}
void test2(struct descriptor *ptr)
{
int size = ptr->size;
short *dat = ptr->dat;
for (uint32_t i=0; i < size; i++)
dat[i] = 1234;
}
In the absence of type-based aliasing rules, a compiler given test1() would have to allow for the possibility that ptr->dat might point to an address within ptr->size or ptr->dat. This would in turn require that it either check whether ptr->dat was in range to access those things, or else reload the contents of ptr->size and ptr->dat on every iteration of the loop. In this scenario, type-based aliasing rules might allow for a 10x speedup.
On the other hand, a compiler given test2() could generate code equivalent to the optimized version of test1() without having to care about type-based aliasing rules. In this case, performing the same operation, type-based aliasing rules would not offer any speedup.
Now consider the following functions:
uint32_t *ptr;
void set_bottom_16_bits_and_advance_v1(uint16_t value)
{
((uint16_t)ptr)[IS_BIG_ENDIAN] = value;
ptr++;
}
void set_bottom_16_bits_and_advance_v2(uint16_t value)
{
((unsigned char*)ptr)[3*IS_BIG_ENDIAN] = value & 255;
((unsigned char*)ptr)[(3*IS_BIG_ENDIAN) ^ 1] = value >> 8;
ptr++;
}
void test1(unsigned n)
{
for (unsigned i=0; i<n; i++)
set_bottom_16_bits_v1(i);
}
void test2(unsigned n, int value)
{
for (unsigned i=0; i<n; i++)
set_bottom_16_bits_v2(value);
}
If a compiler given set_bottom_16_bits_and_advance_v1 and test1 were--even with type-based aliasing enabled--accommodate the possibility that it might modify an object of type uint32_t (since its execution makes use of a value of type uint32_t*), it would not need to allow for the possibility that ptr might hold its own address. If a compiler could not handle the possibility of the first function accessing a uint32_t without disabling type-based aliasing entirely, however, it would need to reload ptr on every iteration of the loop. Almost any compiler(*), with or without type-based aliasing analysis, which is given set_bottom_16_bits_and_advance_v1 and test2, however, would be required to reload ptr every time through the loop, reducing to zero any performance benefits type-based aliasing could have offered.
(*) The CompCert C dialect expressly disallows the use of character pointers, or any other pointer-to-integer type, to modify the values of stored pointer object, since making allowance for such accesses would not only degrade performance, but also make it essentially impossible to identify all corner cases that would need to be evaluated to guarantee that the behavior of a compiler's generated machine code will match the specified behavior of the source.