I thought I knew C pretty well, but I'm confused by the following code:
typedef struct {
int type;
} cmd_t;
typedef struct {
int size;
char data[];
} pkt_t;
int func(pkt_t *b)
{
int *typep;
char *ptr;
/* #1: Generates warning */
typep = &((cmd_t*)(&(b->data[0])))->type;
/* #2: Doesn't generate warning */
ptr = &b->data[0];
typep = &((cmd_t*)ptr)->type;
return *typep;
}
When I compile with GCC, I get the "dereferencing type-punned pointer will break strict-aliasing rules" warning.
Why am I getting this warning at all? I'm dereferencing at char array. Casting a char * to anything is legal. Is this one of those cases where an array is not exactly the same as a pointer?
Why aren't both assignments generating the warning? The 2nd assignment is the equivalent of the first, isn't it?
When strict aliasing is turned on, the compiler is allowed to assume that two pointers of different type (char* vs cmt_t* in this instance) will not point to the same memory location. This allows for a greater range of optimizations which you would otherwise not want to be applied if they do indeed point to the same memory location. Various examples/horror-stories can be found in this question.
This is why, under strict-aliasing, you have to be careful how you do type punning. I believe that the standard doesn't allow for any type-puning what-so-ever (don't quote me on that) but most compilers have exemption for unions (my google-fu is failing in turning up the relevant manual pages):
union float_to_int {
double d;
uint64_t i;
};
union float_to_int ftoi;
ftoi.d = 1.0;
... = ftoi.i;
Unfortunately, this doesn't quite work for your situation as you would have to memcpy the content of the array into the union which is less then ideal. A simpler approach would be to simply to turn off strict-aliasing via the -fno-strict-aliasing switch. This will ensure that your code is correct and it's unlikely to have a significant performance impact (do measure if performance matters).
As for why the warning doesn't show up when the line is broken up, I don't know. Chances are that the modifications to the source code manages to confuse the compiler's static analysis pass enough that it doesn't see the type-punning. Note that the static analysis pass responsible for detecting type-punning is unrelated and doesn't talk to the various optimization passes that assume strict-aliasing. You can think of any static analysis done by compilers (unless otherwise specified) as a best-effort type of thing. In other words, the absence of warning doesn't mean that there are no errors which means that simply breaking up the line doesn't magically make your type-punning safe.
Related
Are there limits on what I can do to allocated memory?(standard-wise)
For example
#include <stdio.h>
#include <stdlib.h>
struct str{
long long a;
long b;
};
int main(void)
{
long *x = calloc(4,sizeof(long));
x[0] = 2;
x[3] = 7;
//is anything beyond here legal( if you would exclude possible illegal operations)
long long *y = x;
printf("%lld\n",y[0]);
y[0] = 2;
memset (x,0,16);
struct str *bar = x;
bar->b = 4;
printf("%lld\n",bar->a);
return 0;
}
To summarize:
Can I recast the pointer to other datatypes and structs, as long as the size fits?
Can I read before I write, then?
If not can I read after I wrote?
Can I use it with a struct smaller than the allocated memory?
Reading from y[0] violates the strict aliasing rule. You use an lvalue of type long long to read objects of effective type long.
Assuming you omit that line; the next troublesome part is memset(x,0,16);. This answer argues that memset does not update the effective type. The standard is not clear.
Assuming that memset leaves the effective type unchanged; the next issue is the read of bar->a.
The C Standard is unclear on this too. Some people say that bar->a implies (*bar).a and this is a strict aliasing violation because we did not write a bar object to the location first.
Others (including me) say that it is fine: the only lvalue used for access is bar->a; that is an lvalue of type long long, and it accesses an object of effective type long long (the one written by y[0] = 2;).
There is a C2X working group that is working on improving the specification of strict aliasing to clarify these issues.
Can I recast the pointer to other datatypes, as long as the size fits?
You can recast1 to any data type that is at most as large as the memory you allocated. You must write a value however to change the effective type of the allcoated object according to 6.5p6
Can I read before I write, then?
If not can I read after I wrote?
No. Except when otherwise specified (calloc is the otherwise)2, the value in the memory is indeterminate. It may contain trap values. A cast in order to reinterpret a value as another type is UB, and a violation of strict aliasing (6.5p7)
Can I use it with a struct smaller than the allocated memory?
Yes, but that's a waste.
1 You'll need to cast to void* first. Otherwise you'd get a rightful complaint from the compiler about incompatible pointer types.
2 Even then some types may trap on a completely 0 bit pattern, so it depends.
Most compilers offer a mode where reads and writes of pointers will act upon the underlying storage, in the sequence they are performed, regardless of the data types involved. The Standard does not require compilers to offer such a mode, but as far as I can tell all quality compilers do so.
According to their published rationale, the authors of the Standard added aliasing restrictions to the language with the stated purpose of avoiding compilers to make pessimistic aliasing assumptions when given code like:
float f;
float test(int *p)
{
f=1.0f;
*p = 2;
return f;
}
Note that in the example given in the rationale [very much like the above], even if it were legal to modify the storage used by f via pointer p, a reasonable person looking at the code would have no reason to think it likely that such a thing would ever happen. On the other hand, many compiler writers recognized that if given something like:
float f;
float test(float *p)
{
f=1.0f;
*(int*)p = 2;
return f;
}
one would have to be deliberately obtuse to think that the code would be unlikely to modify the storage used by a float, and there was consequently no reason why a quality compiler should not regard the write to *(int*)p as a potential write to a float.
Unfortunately, in the intervening years, compiler writers have become increasingly aggressive with type-based aliasing "optimizations", sometimes in ways that go clearly and undeniably beyond what the Standard would allow. Unless a program will never need to access any storage as different types at different times, I'd suggest using -fno-strict-aliasing option on compilers that support it. Otherwise one may have code that complies with the Standard and works today, but fails in a future version of the compiler which has become even more aggressive with its "optimizations".
PS--Disabling type-based aliasing may impact the performance of code in some situations, but proper use of restrict-qualified variables and parameters should avoid the costs of pessimistic aliasing assumptions. With a little care, use of those qualifiers will enable the same optimizations as aggressive aliasing could have done, but much more safely.
static int* p= (int*)(&foo);
I just know p points to a memory in the code segment.
But I don't know what exactly happens in this line.
I thought maybe it's a pointer to a function but the format to point a function is:
returnType (*pointerName) (params,...);
pointerName = &someFunc; // or pointerName=someFunc;
You take the address of foo and cast it to pointer to int.
If foo and p are of different types, the compiler might issue a warning about type mismatch. The cast is to supress that warning.
For example, consider the following code, which causes a warning from the compiler (initialization from incompatible pointer type):
float foo = 42;
int *p = &foo;
Here foo is a float, while p points to an int. Clearly - different types.
A typecasting makes the compiler treat one variable as if it was of different type. You typecast by putting new type name in parenthesis. Here we will make pointer to float to be treated like a pointer to int and the warning will be no more:
float foo = 5;
int *p = (int*)(&foo);
You could've omitted one pair of parenthesis as well and it'd mean the same:
float foo = 5;
int *p = (int*)&foo;
The issue is the same if foo is a function. We have a pointer to a function on right side of assignment and a pointer to int on left side. A cast would be added to make a pointer to function to be treated as an address of int.
A pointer of a type which points to an object (i.e. not void* and not a
pointer to a function) cannot be stored to a pointer to any other kind of
object without a cast, except in a few cases where the types are identical
except for qualifiers. Conforming compilers are required to issue a
diagnostic if that rule is broken.
Beyond that, the Standard allows compilers to interpret code that casts
pointers in nonsensical fashion unless code aides by some restrictions
which, as written make such casts essentially useless, for the nominal
purpose of promoting optimization. When the rules were written, most
compilers would probably do about half of the optimizations that would
be allowed under the rules, but would still process pointer casts sensibly
since doing so would cost maybe 5% of the theoretically-possible
optimizations. Today, however, it is more fashionable for compiler writers
to seek out all cases where an optimization would be allowed by the
Standard without regard for whether they make sense.
Compilers like gcc have an option -fno-strict-aliasing that blocks this
kind of optimization, both in cases where it would offer big benefits and
little risk, as well as in the cases where it would almost certainly break
code and be unlikely to offer any real benefit. It would be helpful if they
had an option to block only the latter, but I'm unaware of one. Thus, my
recommendation is that unless one wants to program in a very limited subset
of Dennis Ritchie's language, I'd suggest targeting the -fno-strict-aliasing
dialect.
When designing a C interface, it is common to let into the public interface (.h) only what needs to be known by the user program.
Hence for example, the inner components of structures should remain hidden if the user program does not need to know them. This is indeed good practice, as the content and behavior of the struct could change in the future, without affecting the interface.
A great way to achieve that objective is to use incomplete types.
typedef struct foo opaqueType;
Now an interface using only pointers to opaqueType can be built, without the user program ever needing to know the inner working of struct foo.
But sometimes, it can be required to allocate such structure statically, typically on stack, for performance and memory fragmentation issues. Obviously, with above construction, opaqueType is incomplete, so its size is unknown, so it cannot be statically allocated.
A work around is to allocate a "shell type", such as :
typedef struct { int faketable[8]; } opaqueType;
Above construction enforces a size and an alignment, but doesn't go farther into describing what the structure really contains. So it matches the objective of keeping the type "opaque".
It mostly works. But in one circumstance (GCC 4.4), the compiler complains that it breaks strict-aliasing, and it generates buggy binary.
Now, I've read a ton of things about strict aliasing, so I guess I understand now what it means.
The question is : is there a way to define an opaque type which can nonetheless be allocated on stack, and without breaking strict aliasing rule ?
Note that I've attempted the union method described in this excellent article but it still generates the same warning.
Note also that visual, clang and gcc 4.6 and later don't complain and work fine with this construction.
[Edit] Information complement :
According to tests, the problem only happens in the following circumstances :
Private and public type different. I'm casting the public type to private inside the .c file. It doesn't matter apparently if they are part of the same union. It doesn't matter if the public type contains char.
If all operations on private type are just reads, there's no problem. Only writes cause problems.
I also suspect that only functions which are automatically inlined get into trouble.
Problem only happens on gcc 4.4 at -O3 setting. -O2 is fine.
Finally, my target is C90. Maybe C99 if there really is no choice.
You can force the alignment with max_align_t and you can avoid the strict aliasing issues using an array of char since char is explicitly allowed to alias any other type.
Something along the lines of:
#include <stdint.h>
struct opaque
{
union
{
max_align_t a;
char b[32]; // or whatever size you need.
} u;
};
If you want to support compiler that do not have the max_align_t, or if you know the alignment requirements of the real type, then you can use any other type for the a union member.
UPDATE: If you are targetting C11, then you may also use alignas():
#include <stdint.h>
#include <stdalign.h>
struct opaque
{
alignas(max_align_t) char b[32];
};
Of course, you can replace the max_align_t with whatever type you think appropriate. Or even an integer.
UPDATE #2:
Then, the use of this type in the library would be something along the lines of:
void public_function(struct opaque *po)
{
struct private *pp = (struct private *)po->b;
//use pp->...
}
This way, since you are type-punning a pointer to char you are not breaking the strict aliasing rules.
What you desire is some kind of equivalent of the C++ private access control in C. As you know, no such equivalent exists. The approach you give is approximately what I would do. However, I would make the opaqueType opaque to the inner components implementing the type, so I would be forced to cast it to the real type within the inner components. The forced cast should not generate the warning you are mentioning.
Although cumbersome to use, you can define an interface that provides "stack allocated" memory to an opaque type without exposing a sized structure. The idea is that the implementation code is in charge of the stack allocation, and the user passes in a callback function to get a pointer to the allocated type.
typedef struct opaqueType_raii_callback opqaueType_raii_callback;
struct opaqueType_raii_callback {
void (*func)(opqaueType_raii_callback *, opqaueType *);
};
extern void opaqueType_raii (opaqueType_raii_callback *);
extern void opaqueType_raii_v (opaqueType_raii_callback *, size_t);
void opaqueType_raii (opaqueType_raii_callback *cb) {
opaqueType_raii_v(cb, 1);
}
void opqaueType_raii_v (opaqueType_raii_callback *cb, size_t n) {
opaqueType x[n];
cb->func(cb, x);
}
The definitions above look a bit esoteric, but it is the way I normally implement a callback interface.
struct foo_callback_data {
opaqueType_raii_callback cb;
int my_data;
/* other data ... */
};
void foo_callback_function (opaqueType_raii_callback *cb, opaqueType *x) {
struct foo_callback_data *data = (void *)cb;
/* use x ... */
}
void foo () {
struct foo_callback_data data;
data.cb.func = foo_callback_function;
opaqueType_raii(&data.cb);
}
For me this seems to be something which just shouldn't be done.
The point of having an opaque pointer is to hide the implementation details. The type and alignment of memory where the actual structure is allocated, or whether the library manages additional data beyond what's pointed to are also implementation details.
Of course not that you couldn't document that one or another thing was possible, but the C language uses this approach (strict aliasing), which you can only more or less hack around by Rodrigo's answer (using max_align_t). By the rule you can't know by the interface what kind of constraints the particular compiler would impose on the actual structure within the implementation (for some esoteric microcontrollers, even the type of memory may matter), so I don't think this can be done reliably in a truly cross platform manner.
If I have structure definitions, for example, like these:
struct Base {
int foo;
};
struct Derived {
int foo; // int foo is common for both definitions
char *bar;
};
Can I do something like this?
void foobar(void *ptr) {
((struct Base *)ptr)->foo = 1;
}
struct Derived s;
foobar(&s);
In other words, can I cast the void pointer to Base * to access its foo member when its type is actually Derived *?
You should do
struct Base {
int foo;
};
struct Derived {
struct Base base;
char *bar;
};
to avoid breaking strict aliasing; it is a common misconception that C allows arbitrary casts of pointer types: although it will work as expected in most implementations, it's non-standard.
This also avoids any alignment incompatibilities due to usage of pragma directives.
Many real-world C programs assume the construct you show is safe, and there is an interpretation of the C standard (specifically, of the "common initial sequence" rule, C99 §6.5.2.3 p5) under which it is conforming. Unfortunately, in the five years since I originally answered this question, all the compilers I can easily get at (viz. GCC and Clang) have converged on a different, narrower interpretation of the common initial sequence rule, under which the construct you show provokes undefined behavior. Concretely, experiment with this program:
#include <stdio.h>
#include <string.h>
typedef struct A { int x; int y; } A;
typedef struct B { int x; int y; float z; } B;
typedef struct C { A a; float z; } C;
int testAB(A *a, B *b)
{
b->x = 1;
a->x = 2;
return b->x;
}
int testAC(A *a, C *c)
{
c->a.x = 1;
a->x = 2;
return c->a.x;
}
int main(void)
{
B bee;
C cee;
int r;
memset(&bee, 0, sizeof bee);
memset(&cee, 0, sizeof cee);
r = testAB((A *)&bee, &bee);
printf("testAB: r=%d bee.x=%d\n", r, bee.x);
r = testAC(&cee.a, &cee);
printf("testAC: r=%d cee.x=%d\n", r, cee.a.x);
return 0;
}
When compiling with optimization enabled (and without -fno-strict-aliasing), both GCC and Clang will assume that the two pointer arguments to testAB cannot point to the same object, so I get output like
testAB: r=1 bee.x=2
testAC: r=2 cee.x=2
They do not make that assumption for testAC, but — having previously been under the impression that testAB was required to be compiled as if its two arguments could point to the same object — I am no longer confident enough in my own understanding of the standard to say whether or not that is guaranteed to keep working.
That will work in this particular case. The foo field in the first member of both structures and hit has the same type. However this is not true in the general case of fields within a struct (that are not the first member). Items like alignment and packing can make this break in subtle ways.
As you seem to be aiming at Object Oriented Programming in C I can suggest you to have a look at the following link:
http://www.planetpdf.com/codecuts/pdfs/ooc.pdf
It goes into detail about ways of handling oop principles in ANSI C.
In particular cases this could work, but in general - no, because of the structure alignment.
You could use different #pragmas to make (actually, attempt to) the alignment identical - and then, yes, that would work.
If you're using microsoft visual studio, you might find this article useful.
There is another little thing that might be helpful or related to what you are doing ..
#define SHARED_DATA int id;
typedef union base_t {
SHARED_DATA;
window_t win;
list_t list;
button_t button;
}
typedef struct window_t {
SHARED_DATA;
int something;
void* blah;
}
typedef struct window_t {
SHARED_DATA;
int size;
}
typedef struct button_t {
SHARED_DATA;
int clicked;
}
Now you can put the shared properties into SHARED_DATA and handle the different types via the "superclass" packed into the union.. You could use SHARED_DATA to store just a 'class identifier' or store a pointer.. Either way it turned out handy for generic handling of event types for me at some point. Hope i'm not going too much off-topic with this
I know this is an old question, but in my view there is more that can be said and some of the other answers are incorrect.
Firstly, this cast:
(struct Base *)ptr
... is allowed, but only if the alignment requirements are met. On many compilers your two structures will have the same alignment requirements, and it's easy to verify in any case. If you get past this hurdle, the next is that the result of the cast is mostly unspecified - that is, there's no requirement in the C standard that the pointer once cast still refers to the same object (only after casting it back to the original type will it necessarily do so).
However, in practice, compilers for common systems usually make the result of a pointer cast refer to the same object.
(Pointer casts are covered in section 6.3.2.3 of both the C99 standard and the more recent C11 standard. The rules are essentially the same in both, I believe).
Finally, you've got the so called "strict aliasing" rules to contend with (C99/C11 6.5 paragraph 7); basically, you are not allowed to access an object of one type via a pointer of another type (with certain exceptions, which don't apply in your example). See "What is the strict-aliasing rule?", or for a very in-depth discussion, read my blog post on the subject.
In conclusion, what you attempt in your code is not guaranteed to work. It might be guaranteed to always work with certain compilers (and with certain compiler options), and it might work by chance with many compilers, but it certainly invokes undefined behavior according to the C language standard.
What you could do instead is this:
*((int *)ptr) = 1;
... I.e. since you know that the first member of the structure is an int, you just cast directly to int, which bypasses the aliasing problem since both types of struct do in fact contain an int at this address. You are relying on knowing the struct layout that the compiler will use and you are still relying on the non-standard semantics of pointer casting, but in practice this is significantly less likely you give you problems.
The great/bad thing about C is that you can cast just about anything -- the problem is, it might not work. :) However, in your case, it will*, since you have two structs whose first members are both of the same type; see this program for an example. Now, if struct derived had a different type as its first element -- for example, char *bar -- then no, you'd get weird behavior.
* I should qualitfy that with "almost always", I suppose; there're a lot of different C compilers out there, so some may have different behavior. However, I know it'll work in GCC.
I used the following piece of code to read data from files as part of a larger program.
double data_read(FILE *stream,int code) {
char data[8];
switch(code) {
case 0x08:
return (unsigned char)fgetc(stream);
case 0x09:
return (signed char)fgetc(stream);
case 0x0b:
data[1] = fgetc(stream);
data[0] = fgetc(stream);
return *(short*)data;
case 0x0c:
for(int i=3;i>=0;i--)
data[i] = fgetc(stream);
return *(int*)data;
case 0x0d:
for(int i=3;i>=0;i--)
data[i] = fgetc(stream);
return *(float*)data;
case 0x0e:
for(int i=7;i>=0;i--)
data[i] = fgetc(stream);
return *(double*)data;
}
die("data read failed");
return 1;
}
Now I am told to use -O2 and I get following gcc warning:
warning: dereferencing type-punned pointer will break strict-aliasing rules
Googleing I found two orthogonal answers:
Concluding: there's no need to worry; gcc tries to be more law obedient than
the actual law.
vs
So basically if you have an int* and a float* they are not allowed to point to the same memory location. If your code does not respect this, then the compiler's optimizer will most likely break your code.
In the end I don't want to ignore the warnings. What would you recommend?
[update] I substituted the toy example with the real function.
The problem occurs because you access a char-array through a double*:
char data[8];
...
return *(double*)data;
But gcc assumes that your program will never access variables though pointers of different type. This assumption is called strict-aliasing and allows the compiler to make some optimizations:
If the compiler knows that your *(double*) can in no way overlap with data[], it's allowed to all sorts of things like reordering your code into:
return *(double*)data;
for(int i=7;i>=0;i--)
data[i] = fgetc(stream);
The loop is most likely optimized away and you end up with just:
return *(double*)data;
Which leaves your data[] uninitialized. In this particular case the compiler might be able to see that your pointers overlap, but if you had declared it char* data, it could have given bugs.
But, the strict-aliasing rule says that a char* and void* can point at any type. So you can rewrite it into:
double data;
...
*(((char*)&data) + i) = fgetc(stream);
...
return data;
Strict aliasing warnings are really important to understand or fix. They cause the kinds of bugs that are impossible to reproduce in-house because they occur only on one particular compiler on one particular operating system on one particular machine and only on full-moon and once a year, etc.
It looks a lot as if you really want to use fread:
int data;
fread(&data, sizeof(data), 1, stream);
That said, if you do want to go the route of reading chars, then reinterpreting them as an int, the safe way to do it in C (but not in C++) is to use a union:
union
{
char theChars[4];
int theInt;
} myunion;
for(int i=0; i<4; i++)
myunion.theChars[i] = fgetc(stream);
return myunion.theInt;
I'm not sure why the length of data in your original code is 3. I assume you wanted 4 bytes; at least I don't know of any systems where an int is 3 bytes.
Note that both your code and mine are highly non-portable.
Edit: If you want to read ints of various lengths from a file, portably, try something like this:
unsigned result=0;
for(int i=0; i<4; i++)
result = (result << 8) | fgetc(stream);
(Note: In a real program, you would additionally want to test the return value of fgetc() against EOF.)
This reads a 4-byte unsigned from the file in little-endian format, regardless of what the endianness of the system is. It should work on just about any system where an unsigned is at least 4 bytes.
If you want to be endian-neutral, don't use pointers or unions; use bit-shifts instead.
Using a union is not the correct thing to do here. Reading from an unwritten member of the union is undefined - i.e. the compiler is free to perform optimisations that will break your code (like optimising away the write).
This doc summarizes the situation: http://dbp-consulting.com/tutorials/StrictAliasing.html
There are several different solutions there, but the most portable/safe one is to use memcpy(). (The function calls may be optimized out, so it's not as inefficient as it appears.) For example, replace this:
return *(short*)data;
With this:
short temp;
memcpy(&temp, data, sizeof(temp));
return temp;
Basically you can read gcc's message as guy you are looking for trouble, don't say I didn't warn ya.
Casting a three byte character array to an int is one of the worst things I have seen, ever. Normally your int has at least 4 bytes. So for the fourth (and maybe more if int is wider) you get random data. And then you cast all of this to a double.
Just do none of that. The aliasing problem that gcc warns about is innocent compared to what you are doing.
The authors of the C Standard wanted to let compiler writers generate efficient code in circumstances where it would be theoretically possible but unlikely that a global variable might have its value accessed using a seemingly-unrelated pointer. The idea wasn't to forbid type punning by casting and dereferencing a pointer in a single expression, but rather to say that given something like:
int x;
int foo(double *d)
{
x++;
*d=1234;
return x;
}
a compiler would be entitled to assume that the write to *d won't affect x. The authors of the Standard wanted to list situations where a function like the above that received a pointer from an unknown source would have to assume that it might alias a seemingly-unrelated global, without requiring that types perfectly match. Unfortunately, while the rationale strongly suggests that authors of the Standard intended to describe a standard for minimum conformance in cases where a compiler would otherwise have no reason to believe that things might alias, the rule fails to require that compilers recognize aliasing in cases where it is obvious and the authors of gcc have decided that they'd rather generate the smallest program it can while conforming to the poorly-written language of the Standard, than generate code which is actually useful, and instead of recognizing aliasing in cases where it's obvious (while still being able to assume that things that don't look like they'll alias, won't) they'd rather require that programmers use memcpy, thus requiring a compiler to allow for the possibility that pointers of unknown origin might alias just about anything, thus impeding optimization.
Apparently the standard allows sizeof(char*) to be different from sizeof(int*) so gcc complains when you try a direct cast. void* is a little special in that everything can be converted back and forth to and from void*.
In practice I don't know many architecture/compiler where a pointer is not always the same for all types but gcc is right to emit a warning even if it is annoying.
I think the safe way would be
int i, *p = &i;
char *q = (char*)&p[0];
or
char *q = (char*)(void*)p;
You can also try this and see what you get:
char *q = reinterpret_cast<char*>(p);