Strict aliasing prevents us from accessing the same memory location using an incompatible type.
int* i = malloc( sizeof( int ) ) ; //assuming sizeof( int ) >= sizeof( float )
*i = 123 ;
float* f = ( float* )i ;
*f = 3.14f ;
this would be illegal according to C standard, because the compiler "knows" that int cannot accessed by a float lvalue.
What if I use that pointer to point to correct memory, like this:
int* i = malloc( sizeof( int ) + sizeof( float ) + MAX_PAD ) ;
*i = 456 ;
First I allocate memory for int, float and the last part is memory which will allow float to be stored on aligned address. float requires to be aligned on multiples of 4. MAX_PAD is usually 8 of 16 bytes depending on the system. In any case, MAX_PAD is large enough so float can be aligned properly.
Then I write an int into i, so far so good.
float* f = ( float* )( ( char* )i + sizeof( int ) + PaddingBytesFloat( (char*)i ) ) ;
*f= 2.71f ;
I use the pointer i, increment it with the size of int and align it correctly with the function PaddingBytesFloat(), which returns the number of bytes required to align a float, given an address. Then I write a float into it.
In this case, f points to a different memory location that doesn't overlap; it has a different type.
Here are some parts from the standard (ISO/IEC 9899:201x) 6.5 , I was relying on when writing this example.
Aliasing is when more than one lvalue points to the same memory location. Standard requires that those lvalues have a compatible type with the effective type of the object.
What is effective type, quote from standard:
The effective type of an object for an access to its stored value is the declared type of the
object, if any.87)If a value is stored into an object having no declared type through an
lvalue having a type that is not a character type, then the type of the lvalue becomes the
effective type of the object for that access and for subsequent accesses that do not modify
the stored value. If a value is copied into an object having no declared type using
memcpy or memmove, or is copied as an array of character type, then the effective type
of the modified object for that access and for subsequent accesses that do not modify the
value is the effective type of the object from which the value is copied, if it has one. For all other accesses to an object having no declared type, the effective type of the object is
simply the type of the lvalue used for the access.
87) Allocated objects have no declared type.
I'm trying to connect the pieces and figure out if this is allowed. In my interpretation the effective type of an allocated object can be changed depending on the type of the lvalue used on that memory, because of this part: For
all other accesses to an object having no declared type, the effective type of the object is simply the type of the lvalue used for the access.
Is this legal? If not, what if I used a void pointer as lvalue instead of an int pointer i in my second example? If even that wouldn't work, what if I got the address, which is assigned to the float pointer in the second example, as a memcopied value, and that address was never used as an lvalue before.
I think that yes, it is legal.
To illustrate my point, let's see this code:
struct S
{
int i;
float f;
};
char *p = malloc(sizeof(struct S));
int *i = p + offsetof(struct S, i); //this offset is 0 by definition
*i = 456;
float *f = p + offsetof(struct S, f);
*f= 2.71f;
This code is, IMO, clearly legal, and it is equivalent to yours from a compiler point of view, for appropriate values of PaddingBytesFloat() and MAX_PAD.
Note that my code does not use any l-value of type struct S, it is only used to ease the calculation of the paddings.
As I read the standard, in malloc'ed memory has no declared type until something is written there. Then the declared type is whatever is written. Thus the declared type of such memory can be changed any time, overwriting the memory with a value of different type, much like an union.
TL; DR: My conclusion is that with dynamic memory you are safe, with regard to strict-aliasing as long as you read the memory using the same type (or a compatible one) you use to last write to that memory.
Yes, this is legal. To see why, you don't even need to think about strict aliasing rule, because it doesn't apply in this case.
According to the C99 standard, when you do this:
int* i = malloc( sizeof( int ) + sizeof( float ) + MAX_PAD ) ;
*i = 456 ;
malloc will return a pointer to a memory block large enough to hold an object of size sizeof(int)+sizeof(float)+MAX_PAD. However, notice that you are only using a small piece of this size; in particular, you are only using the first sizeof(int) bytes. Consequently, you are leaving some free space that can be used to store other objects, as long as you store them into a disjoint offset (that is, after the first sizeof(int) bytes). This is tightly related with the definition of what exactly is an object. From C99 section 3.14:
Object: region of data storage in the execution environment, the
contents of which can represent values
The precise meaning of the contents of the object pointed to by i is the value 456; this implies that the integer object itself only takes a small portion of the memory block you allocated. There is nothing in the standard stopping you from storing a new, different object of any type a few bytes ahead.
This code:
float* f = ( float* )( ( char* )i + sizeof( int ) + PaddingBytesFloat( (char*)i ) ) ;
*f= 2.71f ;
Is effectively attaching another object to a sub-block of the allocated memory. As long as the resulting memory location for f doesn't overlap with that of i, and there is enough room left to store a float, you will always be safe. The strict aliasing rule doesn't even apply here, because the pointers point to objects that do not overlap - the memory locations are different.
I think the key point here is to understand that you are effectively manipulating two distinct objects, with two distinct pointers. It just so happens that both pointers point to the same malloc()'d block, but they are far enough from one another, so this is not a problem.
You can have a look at this related question: What alignment issues limit the use of a block of memory created by malloc? and read Eric Postpischil's great answer: https://stackoverflow.com/a/21141161/2793118 - after all, if you can store arrays of different types in the same malloc() block, why wouldn't you store an int and a float? You can even look at your code as the special case in which these arrays are one-element arrays.
As long as you take care of alignment issues, the code is perfectly fine and 100% portable.
UPDATE (follow-up, read comments below):
I believe your reasoning about the standard not enforcing strict aliasing on malloc()'d objects is wrong. It is true that the effective type of a dynamically allocated object can be changed, as conveyed by the standard (it is a matter of using an lvalue expression with a different type to store a new value in there), but note that once you do that, it is your job to ensure that no other lvalue expression with a different type will access the object value. This is enforced by rule 7 on section 6.5, and you quoted it in your question:
An object shall have its stored value accessed only by an lvalue
expression that has one of the following types:
- a type compatible with the effective type of the object;
Thus, by the time you change the effective type of an object, you are implicitly promising to the compiler that you won't access this object using an old pointer with an incompatible type (compared to the new effective type). This should be enough for the purposes of the strict aliasing rule.
I found a nice analogy. You may also find it useful. Quoting from ISO/IEC 9899:TC2 Committee Draft — May 6, 2005 WG14/N1124
6.7.2.1 Structure and union specifiers
[16] As a special case, the last element of a structure with more than one
named member may have an incomplete array type; this is called a
flexible array member. In most situations, the flexible array member
is ignored. In particular, the size of the structure is as if the
flexible array member were omitted except that it may have more
trailing padding than the omission would imply. However, when a . (or
->) operator has a left operand that is (a pointer to) a structure with a flexible array member and the right operand names that member,
it behaves as if that member were replaced with the longest array
(with the same element type) that would not make the structure larger
than the object being accessed; the offset of the array shall remain
that of the flexible array member, even if this would differ from that
of the replacement array. If this array would have no elements, it
behaves as if it had one element but the behavior is undefined if any
attempt is made to access that element or to generate a pointer one
past it.
[17] EXAMPLE After the declaration:
struct s { int n; double d[]; };
the structure struct s has a flexible array member d. A typical
way to use this is:
int m = /* some value */;
struct s *p = malloc(sizeof (struct s) + sizeof (double [m]));
and assuming that the
call to malloc succeeds, the object pointed to by p behaves, for most
purposes, as if p had been declared as:
struct { int n; double d[m]; } > *p;
(there are circumstances in which this equivalence is broken; in particular, the offsets of member d might not be the same).
It would be more fair to use an example like:
struct ss {
double da;
int ia[];
}; // sizeof(double) >= sizeof(int)
In example of above quote, size of struct s is same as int (+ padding) and it is then followed by double. (or some other type, float in your case)
Accessing memory sizeof(int) + PADDING bytes after the struct start as double (using syntactic sugar) looks fine as per this example, so I believe your example is legal C.
The strict aliasing rules are there to allow for more aggressive compiler optimizations, specifically by being able to reorder accesses to different types without having to worry about whether they point to the same location. So for instance, in your first example it is perfectly legal for a compiler to reorder the writes to i and f, and thus your code is an example of undefined behaviour (UB).
There is an exception to this rule and you have the relevant quote from the standards
having a type that is not a character type
Your second bit of code is entirely safe. The memory regions do no overlap so it does not matter if memory accesses are reordered across that boundary. Indeed the behaviour of the two pieces of code is completely different. The first one places an int in a memory region and then a float in to the same memory region, whereas the second one places an int in to a memory region and a float in to a bit of memory next to it. Even if these accesses are reordered then your code will have the same effect. Perfectly, legal.
I get the feeling I have thus missed the real question here.
The safest way to fiddle with low level memory if you really did want the behaviour in your first program is either (a) a union or (b) a char *. Using char * and then casting to the proper type is used in a lot of C code, e.g: in this pcap tutorial (scroll down to "for all those new C programmers who insist that pointers are useless, I smite you."
Related
Is it valid to typecast to a pointer to a variable length array type?
int main() {
int n = 10;
int m = 20;
int (*p)[n][m] = malloc(sizeof(int[n][m]));
void *q = p;
int a = n;
int b = m;
(*(int (*)[a][b])q)[5][5] = 1;
printf("%zd %d\n", sizeof(*p), (*p)[5][5]);
return 0;
}
prints 800 1.
With
int a = 1;
still prints 800 1.
With
int a = 1;
int b = 1;
now prints 800 0.
Up to which point is well-defined behavior?
It's undefined behavior since you access an array out of bounds.
It might be worth pointing out that the C type system is a bit muddy here though. Memory allocated with malloc isn't given a type until accessed. I'll quote C17 6.5/6 and leave comments in between:
The effective type of an object for an access to its stored value is the declared type of the
object, if any.
This "heap chunk" returned from malloc has no declared type.
If a value is stored into an object having no declared type through an
lvalue having a type that is not a character type, then the type of the lvalue becomes the
effective type of the object for that access and for subsequent accesses that do not modify
the stored value.
Meaning that if we store something at a given place in the heap chunk, that address will get an effective type and gets treated as an object (a single int in this case). The compiler doesn't really know if the chunk as whole is to be regarded as an array, a struct or something else though.
For all other accesses to an object having no declared type, the effective type of the object is simply the type of the lvalue used for the access.
Meaning that if we read access this data before writing to it, it will get treated as a variable of the type used for reading. (The heap chunk isn't initialized by malloc though, so reading before writing doesn't make much sense.)
Assuming 32 bit int then malloc(sizeof(int[n][m])); allocates a chunk of 10*20*4 = 800 bytes. This chunk set aside by malloc does not yet have an effective type. The compiler does not yet keep track of what type that is stored inside that data. The fact that you have a certain pointer type point at the data doesn't change that, as long as the data isn't de-referenced, it does not yet have an effective type.
Normally the heap chunk wouldn't get a type until you do the [5][5] de-referencing and write 1 to a location in that chunk. That location can now be said to have effective type int.
However, before you do that, you do *(int (*)[a][b])q. The type of q is irrelevant here - you cast it and say that here is a pointer to array. Then you de-reference that and get an array - this is a lvalue (and it will decay into a pointer to the first element of the array). From that point on, at least in this expression, the compiler can assume that it's dealing with an array - regardless if that array happens to be stored at memory with no effective type. If this array is a VLA and doesn't have dimension large enough for ... [5][5] = 1; access, you end up with plain old array out of bounds access which is undefined behavior.
Specifically, UB as per 6.5.6/8 additive operators:
If both the pointer operand and the result point to elements of the same array object, or one past the last element of the array object, the evaluation shall not produce an overflow; otherwise, the behavior is undefined.
arr[i] being 100% equivalent to *(arr+i), the + operator sets the rules for pointer arithmetic quoted above - you are not allowed to use pointer arithmetic to access an array or object out of bounds - it's undefined behavior. Meaning that the compiler can generate incorrectly behaving code, regardless of whether there's available memory at the supposed address or not.
Are explicit casts from char * to other pointer types fully defined behavior according to ANSI C89 if the pointer is guaranteed to meet the alignment requirements of the type you're casting to? Here's an example of what I mean:
/* process.c */
void *process(size_t elem_size, size_t cap) {
void *arr;
assert(cap > 5);
arr = malloc(elem_size * cap);
/* set id of element 5 to 0xffffff */
*(long *)((char *)arr + elem_size*5) = 0xffffff;
/* rest of the code omitted */
return arr;
}
/* main.c */
struct some_struct { long id; /* other members omitted */ };
struct other_struct { long id; /* other members omitted */ };
int main(int argc, char **argv) {
struct some_struct *s = process(sizeof(struct some_struct), 40);
printf("%lx\n", s[5].id);
return 0;
}
This code compiles without warnings and works as expected on my machine but I'm not fully sure if these kinds of casts are defined behavior.
C89 draft, section 4.10.3 (Memory management functions):
The pointer returned if the allocation succeeds is suitably aligned so that it may be assigned to a pointer to any type of object and then used to access such an object in the space allocated (until the space is explicitly freed or reallocated).
C89 draft, section 3.3.4 (Cast operators):
A pointer to an object or incomplete type may be converted to a pointer to a different object type or a different incomplete type. The resulting pointer might not be valid if it is improperly aligned for the type pointed to. It is guaranteed, however, that a pointer to an object of a given alignment may be converted to a pointer to an object of the same alignment or a less strict alignment and back again; the result shall compare equal to the original pointer.
This clearly specifies what happens if you cast from struct some_struct * to char * and back to struct some_struct * but in my case the code responsible for allocation doesn't have access to the full struct definition so it can't initially specify the pointer type to be struct some_struct * so I'm not sure if the rule still applies.
If the code I posted is technically UB, is there another standards-compliant way to modify array elements without knowing their full type? Are there real-world impementations where you would expect it to do something else than ((struct some_struct *)arr)[5].id = 0xffffff;?
This code compiles without warnings and works as expected on my machine but I'm not fully sure if these kinds of casts are defined behavior.
In general, the casts have defined behavior, but that behavior can be that the result is not a valid pointer. Thus dereferencing the result of the cast may produce UB.
Considering only function process(), then, it is possible that the result of its evaluation of (long *)((char *)arr + elem_size*5) is an invalid pointer. If and only if it is invalid, the attempt to use it to assign a value to the object it hypothetically points to produces UB.
However, main()s particular usage of that function is fine:
In process(), the pointer returned by malloc and stored in arr is suitably aligned for a struct some_struct (and for any other type).
The compiler must choose a size and layout for struct some_struct such that every element and every member of every element of an array of such structures is properly aligned.
Arrays are composed of contiguous objects of the array's element type, without gaps (though the element types themselves may contain padding if they are structures or unions).
Therefore, (char *)arr + n * sizeof(struct some_struct) must be suitably aligned for a struct some_struct, for any integer n such that the result points within or just past the end of the allocated region. This computation is closely related to the computations involved in accessing an array of struct some_struct via the indexing operator.
struct some_struct has a long as its first member, and that member must appear at offset 0 from the beginning of the struct. Therefore, every pointer that is suitably aligned for a struct some_struct must also be suitably aligned for a long.
Determining whether an operation upholds or violates "type aliasing" constraints requires being able to answer two questions:
For purposes of the aliasing rules, when does a region of storage hold an "object" of a particular type.
For purposes of the aliasing rules, is a particular access performed "by" an lvalue expression of a particular type.
For your question, the second issue above is the most the relevant: if a pointer or lvalue of type T1 is used to derive a value of type T2* which is dereferenced to access storage, the access may sometimes need to be regarded for purposes of aliasing as though performed by an lvalue of type T1, and sometimes as though performed by one of type T2, but the Standard fails to offer any guidance as to which interpretation should apply when. Constructs like yours would be processed predictably by implementation that didn't abuse the Standard as an excuse to behave nonsensically, but could be processed nonsensically by conforming but obtuse implementations that do abuse the Standard in such fashion.
The authors of C89 didn't expect anyone to care about the precise boundaries between constructs whose behavior was defined by the Standard, versus those which all implementations were expected to process identically but which the Standard didn't actually define, and thus saw no need to define the terms "object" and "by" with sufficient precision to unambiguously answer the above questions in ways that would yield defined program behavior in all sensible cases.
I've been trying to understand a particular aspect of strict aliasing recently, and I think I have made the smallest possible interesting piece of code. (Interesting for me, that is!)
Update: Based on the answers so far, it's clear I need to clarify the question. The first listing here is "obviously" defined behaviour, from a certain point of view. The real issue is to follow this logic through to custom allocators and custom memory pools. If I malloc a large block of memory at the start, and then write my own my_malloc and my_free that uses that single large block, then is it UB on the grounds that it doesn't use the official free?
I'll stick with C, somewhat arbitrarily. I get the impression it is easier to talk about, that the C standard is a bit clearer.
int main() {
uint32_t *p32 = malloc(4);
*p32 = 0;
free(p32);
uint16_t *p16 = malloc(4);
p16[0] = 7;
p16[1] = 7;
free(p16);
}
It is possible that the second malloc will return the same address as the first malloc (because it was freed in between). That means that it is accessing the same memory with two different types, which violates strict aliasing. So surely the above is undefined behaviour (UB)?
(For simplicity, let's assume the malloc always succeeds. I could add in checks for the return value of malloc, but that would just clutter the question)
If it's not UB, why? Is there an explicit exception in the standard, which says that malloc and free (and calloc/realloc/...) are allowed to "delete" the type associated with a particular address, allowing further accesses to "imprint" a new type on the address?
If malloc/free are special, then does that mean I cannot legally write my own allocator which clones the behaviour of malloc? I'm sure there are plenty of projects out there with custom allocators - are they all UB?
Custom allocators
If we decide, therefore, that such custom allocators must be defined behaviour, then it means the strict aliasing rule is essentially "incorrect". I would update it to say that it is possible to write (not read) through a pointer of a different ('new') type as long as you don't use pointers of the old type any more. This wording could be quietly-ish changed if it was confirmed that all compilers have essentially obeyed this new rule anyway.
I get the impression that gcc and clang essentially respect my (aggressive) reinterpretation. If so, perhaps the standards should be edited accordingly? My 'evidence' regarding gcc and clang is difficult to describe, it uses memmove with an identical source and destination (which is therefore optimized out) in such a way that it blocks any undesirable optimizations because it tells the compiler that future reads through the destination pointer will alias the bit pattern that was previously written through the source pointer. I was able to block the undesirable interpretations accordingly. But I guess this isn't really 'evidence', maybe I was just lucky. UB clearly means that the compiler is also allowed to give me misleading results!
( ... unless, of course, there is another rule that makes memcpy and memmove special in the same way that malloc may be special. That they are allowed to change the type to the type of the destination pointer. That would be consistent with my 'evidence'. )
Anyway, I'm rambling. I guess a very short answer would be: "Yes, malloc (and friends) are special. Custom allocators are not special and are therefore UB, unless they maintain separate memory pools for each type. And, further, see example X for an extreme piece of code where compiler Y does undesirable stuff precisely because compiler Y is very strict in this regard and is contradicting this reinterpretation."
Follow up: what about non-malloced memory? Does the same thing apply. (Local variables, static variables, ...)
Here are the C99 strict aliasing rules in (what I hope is) their entirety:
6.5
(6) The effective type of an object for an access to its stored value is the declared type of the object, if any. If a value is stored into an object having no declared type through an lvalue having a type that is not a character type, then the type of the lvalue becomes the effective type of the object for that access and for subsequent accesses that do not modify the stored value. If a value is copied into an object having no declared type using
memcpy or memmove, or is copied as an array of character type, then the effective type
of the modified object for that access and for subsequent accesses that do not modify the
value is the effective type of the object from which the value is copied, if it has one. For
all other accesses to an object having no declared type, the effective type of the object is
simply the type of the lvalue used for the access.
(7) An object shall have its stored value accessed only by an lvalue expression that has one of
the following types:
— a type compatible with the effective type of the object,
— a qualified version of a type compatible with the effective type of the object,
— a type that is the signed or unsigned type corresponding to the effective type of the object,
— a type that is the signed or unsigned type corresponding to a qualified version of the effective type of the object,
— an aggregate or union type that includes one of the aforementioned types among its members (including, recursively, a member of a subaggregate or contained union), or
— a character type.
These two clauses together prohibit one specific case, storing a value via an lvalue of type X and then subsequently retrieving a value via an lvalue of type Y incompatible with X.
So, as I read the standard, even this usage is perfectly OK (assuming 4 bytes are enough to store either an uint32_t or two uint16_t).
int main() {
uint32_t *p32 = malloc(4);
*p32 = 0;
/* do not do this: free(p32); */
/* do not do this: uint16_t *p16 = malloc(4); */
/* do this instead: */
uint16_t *p16 = (uint16_t *)p32;
p16[0] = 7;
p16[1] = 7;
free(p16);
}
There's no rule that prohibits storing an uint32_t and then subsequently storing an uint16_t at the same address, so we're perfectly OK.
Thus there's nothing that would prohibit writing a fully compliant pool allocator.
Your code is correct C and does not invoke undefined behaviour (except that you do not test malloc return value) because :
you allocate a bloc of memory, use it and free it
you allocate another bloc of memory, use it and free it.
What is undefined is whether p16 will receive same value as p32 had at a different time
What would be undefined behaviour, even if value was the same would be to access p32 after it has been freed. Examples :
int main() {
uint32_t *p32 = malloc(4);
*p32 = 0;
free(p32);
uint16_t *p16 = malloc(4);
p16[0] = 7;
p16[1] = 7;
if (p16 == p32) { // whether p16 and p32 are equal is undefined
uint32_t x = *p32; // accessing *p32 is explicitely UB
}
free(p16);
}
It is UB because you try to access a memory block after it has been freed. And even when it does point to a memory block, that memory block has been initialized as an array of uint16_t, using it as a pointer to another type is formally undefined behaviour.
Custom allocation (assuming a C99 conformant compiler) :
So you have a big chunk of memory and want to write custom free and malloc functions without UB. It is possible. Here I will not go to far into the hard part of management of allocated and free blocs, and just give hints.
you will need to know what it the strictest alignement for the implementation. stdlib malloc knows it because 7.20.3 §1 of C99 language specification (draft n1256) says : The pointer returned if the allocation
succeeds is suitably aligned so that it may be assigned to a pointer to any type of object. It is generally 4 on 32 bits systems and 8 on 64 bits systems, but might be greater or lesser ...
you memory pool must be a char array because 6.3.2.3 §7 says : A pointer to an object or incomplete type may be converted to a pointer to a different
object or incomplete type. If the resulting pointer is not correctly aligned for the pointed-to type, the behavior is undefined. Otherwise, when converted back again, the result shall compare equal to the original pointer. When a pointer to an object is converted to a pointer to a character type, the result points to the lowest addressed byte of
the object. Successive increments of the result, up to the size of the object, yield pointers to the remaining bytes of the object. : that means that provided you can deal with the alignement, a character array of correct size can be converted to a pointer to an arbitrary type (and is the base of malloc implementation)
You must make your memory pool start at an address compatible with the system alignement :
intptr_t orig_addr = chunk;
int delta = orig_addr % alignment;
char *pool = chunk + alignement - delta; /* pool in now aligned */
You now only have to return from your own pool addresses of blocs got as pool + n * alignement and converted to void * : 6.3.2.3 §1 says : A pointer to void may be converted to or from a pointer to any incomplete or object type. A pointer to any incomplete or object type may be converted to a pointer to void and back again; the result shall compare equal to the original pointer.
It would be cleaner with C11, because C11 explicitely added _Alignas and alignof keywords to explictely deal with it and it would be better than the current hack. But it should work nonetheless
Limits :
I must admit that my interpretation of 6.3.2.3 §7 is that a pointer to a correctly aligned char array can be converted to a pointer of another type is not really neat and clear. Some may argue that what is said is just that if it originally pointed to the other type, it can be used as a char pointer. But as I start from a char pointer it is not explicitely allowed. That's true, but it is the best that can be done, it is not explicely marked as undefined behaviour ... and it is what malloc does under the hood.
As alignement is explicitely implementation dependant, you cannot create a general library usable on any implementation.
The actual rules regarding aliasing are laid out in standard section 6.5, paragraph 7. Note the wording:
An object shall have its stored value accessed only by an lvalue expression that has one of the following types:
(emphasis mine)
Aliasing includes the notion of objects, not just general memory. For malloc to have returned the same address on a second use requires the original object to have been deallocated. Even if it has the same address, it is not considered the same object. Any attempts to access the first object through dangling pointers leftover after free are UB for completely different reasons, so there's no aliasing because any continued use of the first pointer p32 is invalid anyway.
I have a structure, defined by as follows:
struct vector
{
(TYPE) *items;
size_t nitems;
};
where type may literally be any type, and I have a type-agnostic structure of similar kind:
struct _vector_generic
{
void *items;
size_t nitems;
};
The second structure is used to pass structures of the first kind of any type to a resizing function, for example like this:
struct vector v;
vector_resize((_vector_generic*)&v, sizeof(*(v->items)), v->nitems + 1);
where vector_resize attempts to realloc memory for the given number of items in the vector.
int
vector_resize (struct _vector_generic *v, size_t item_size, size_t length)
{
void *new = realloc(v->items, item_size * length);
if (!new)
return -1;
v->items = new;
v->nitems = length;
return 0;
}
However, the C standard states that pointers to different types are not required to be of the same size.
6.2.5.27:
A pointer to void shall have the same representation and alignment
requirements as a pointer to a character type.39) Similarly, pointers
to qualified or unqualified versions of compatible types shall have
the same representation and alignment requirements. All pointers to
structure types shall have the same representation and alignment
requirements as each other. All pointers to union types shall have the
same representation and alignment requirements as each other. Pointers
to other types need not have the same representation or alignment
requirements.
Now my question is, should I be worried that this code may break on some architectures?
Can I fix this by reordering my structs such that the pointer type is at the end? for example:
struct vector
{
size_t nitems;
(TYPE) *items;
};
And if not, what can I do?
For reference of what I am trying to achieve, see:
https://github.com/andy-graprof/grapes/blob/master/grapes/vector.h
For example usage, see:
https://github.com/andy-graprof/grapes/blob/master/tests/grapes.tests/vector.exp
You code is undefined.
Accessing an object using an lvalue of an incompatible type results in undefined behavior.
Standard defines this in:
6.5 p7:
An object shall have its stored value accessed only by an lvalue expression that has one of
the following types:
— a type compatible with the effective type of the object,
— a qualified version of a type compatible with the effective type of the object,
— a type that is the signed or unsigned type corresponding to the effective type of the
object,
— a type that is the signed or unsigned type corresponding to a qualified version of the
effective type of the object,
— an aggregate or union type that includes one of the aforementioned types among its
members (including, recursively, a member of a subaggregate or contained union), or
— a character type.
struct vector and struct _vector_generic have incompatible types and do not fit into any of the above categories. Their internal representation is irrelevant in this case.
For example:
struct vector v;
_vector_generic* g = &v;
g->size = 123 ; //undefined!
The same goes for you example where you pass the address of the struct vector to the function and interpret it as a _vector_generic pointer.
The sizes and padding of the structs could also be different causing elements to be positioned at different offsets.
What you can do is use your generic struct, and cast if depending on the type the void pointer holds in the main code.
struct gen
{
void *items;
size_t nitems;
size_t nsize ;
};
struct gen* g = malloc( sizeof(*g) ) ;
g->nitems = 10 ;
g->nsize = sizeof( float ) ;
g->items = malloc( g->nsize * g->nitems ) ;
float* f = g->items ;
f[g->nitems-1] = 1.2345f ;
...
Using the same struct definition you can allocate for a different type:
struct gen* g = malloc( sizeof(*g) ) ;
g->nitems = 10 ;
g->nsize = sizeof( int ) ;
g->items = malloc( g->nsize * g->nitems ) ;
int* i = g->items ;
...
Since you are storing the size of the type and the number of elements, it is obvious how your resize function would look like( try it ).
You will have to be careful to remember what type is used in which variable as the compiler will not warn you because you are using void*.
The code in your question invokes undefined behaviour (UB), because you de-reference a potentially invalid pointer. The cast:
(_vector_generic*)&v
... is covered by 6.3.2.3 paragraph 7:
A pointer to an object type may be converted to a pointer to a different object type. If the resulting pointer is not correctly aligned for the referenced type, the behavior is undefined. Otherwise, when converted back again, the result shall compare equal to the original pointer.
If we assume alignment requirements are met, then the cast does not invoke UB. However, there is no requirement that the converted pointer must "compare equal" with (i.e. point at the same object as) the original pointer, nor even that it points to any object at all - that is to say, the value of the pointer is unspecified - therefore, to dereference this pointer (without first ascertaining that it is equal to the original) invokes undefined behaviour.
(Many people who know C well find this odd. I think this is because they know a pointer cast usually compiles to no operation - the pointer value simply remains as it is - and therefore they see pointer conversion as purely a type conversion. However, the standard does not mandate this).
Even if the pointer after conversion did compare equal with the original pointer, 6.5 paragraph 7 (the so-called "strict aliasing rule") would not allow you to dereference it. Essentially, you cannot access the same object via two pointers with different type, with some limited exceptions.
Example:
struct a { int n; };
struct b { int member; };
struct a a_object;
struct b * bp = (struct b *) &a_object; // bp takes an unspecified value
// Following would invoke UB, because bp may be an invalid pointer:
// int m = b->member;
// But what if we can ascertain that bp points at the original object?:
if (bp == &a_object) {
// The comparison in the line above actually violates constraints
// in 6.5.9p2, but it is accepted by many compilers.
int m = b->member; // UB if executed, due to 6.5p7.
}
Lets for the sake of discussion ignore that the C standard formally says this is undefined behavior. Because undefined behavior simply means that something is beyond the scope of the language standard: anything can happen and the C standard makes no guarantees. There may however be "external" guarantees on the particular system you are using, made by those who made the system.
And in the real world where there is hardware, there are indeed such guarantees. There are just two things that can go wrong here in practice:
TYPE* having a different representation or size than void*.
Different struct padding in each struct type because of alignment requirements.
Both of these seem unlikely and can be dodged with a static asserts:
static void ct_assert (void) // dummy function never linked or called by anyone
{
struct vector v1;
struct _vector_generic v2;
static_assert(sizeof(v1.items) == sizeof(v2.items),
"Err: unexpected pointer format.");
static_assert(sizeof(v1) == sizeof(v2),
"Err: unexpected padding.");
}
Now the only thing left that could go wrong is if a "pointer to x" has same size but different representation compared to "pointer to y" on your specific system. I have never heard of such a system anywhere in the real world. But of course, there are no guarantees: such obscure, unorthodox systems may exist. In that case, it is up to you whether you want to support them, or if it will suffice to just have portability to 99.99% of all existing computers in the world.
In practice, the only time you have more than one pointer format on a system is when you are addressing memory beyond the CPU's standard address width, which is typically handled by non-standard extensions such as far pointers. In all such cases, the pointers will have different sizes and you will detect such cases with static assert above.
The C99 standard states that:
When two pointers are subtracted, both shall point to elements of the same array object, or one past the last element of the array object
Consider the following code:
struct test {
int x[5];
char something;
short y[5];
};
...
struct test s = { ... };
char *p = (char *) s.x;
char *q = (char *) s.y;
printf("%td\n", q - p);
This obviously breaks the above rule, since the p and q pointers are pointing to different "array objects", and, according to the rule, the q - p difference is undefined.
But in practice, why should such a thing ever result in undefined behaviour? After all, the struct members are laid out sequentially (just as array elements are), with any potential padding between the members. True, the amount of padding will vary across implementations and that would affect the outcome of the calculations, but why should that result be "undefined"?
My question is, can we suppose that the standard is just "ignorant" of this issue, or is there a good reason for not broadening this rule? Couldn't the above rule be rephrased to "both shall point to elements of the same array object or members of the same struct"?
My only suspicion are segmented memory architectures where the members might end up in different segments. Is that the case?
I also suspect that this is the reason why GCC defines its own __builtin_offsetof, in order to have a "standards compliant" definition of the offsetof macro.
EDIT:
As already pointed out, arithmetic on void pointers is not allowed by the standard. It is a GNU extension that fires a warning only when GCC is passed -std=c99 -pedantic. I'm replacing the void * pointers with char * pointers.
Subtraction and relational operators (on type char*) between addresses of member of the same struct are well defined.
Any object can be treated as an array of unsigned char.
Quoting N1570 6.2.6.1 paragraph 4:
Values stored in non-bit-field objects of any other object type
consist of n × CHAR_BIT bits, where n is the size of an object of that
type, in bytes. The value may be copied into an object of type
unsigned char [ n ] (e.g., by memcpy); the resulting set of bytes is
called the object representation of the value.
...
My only suspicion are segmented memory architectures where the members
might end up in different segments. Is that the case?
No. For a system with a segmented memory architecture, normally the compiler will impose a restriction that each object must fit into a single segment. Or it can permit objects that occupy multiple segments, but it still has to ensure that pointer arithmetic and comparisons work correctly.
Pointer arithmetic requires that the two pointers being added or subtracted to be part of the same object because it doesn't make sense otherwise.
The quoted section of standard specifically refers to two unrelated objects such as int a[b]; and int b[5]. The pointer arithmetic requires to know the type of the object that the pointers pointing to (I am sure you are aware of this already).
i.e.
int a[5];
int *p = &a[1]+1;
Here p is calculated by knowing the that the &a[1] refers to an int object and hence incremented to 4 bytes (assuming sizeof(int) is 4).
Coming to the struct example, I don't think it can possibly be defined in a way to make pointer arithmetic between struct members legal.
Let's take the example,
struct test {
int x[5];
char something;
short y[5];
};
Pointer arithmatic is not allowed with void pointers by C standard (Compiling with gcc -Wall -pedantic test.c would catch that). I think you are using gcc which assumes void* is similar to char* and allows it.
So,
printf("%zu\n", q - p);
is equivalent to
printf("%zu", (char*)q - (char*)p);
as pointer arithmetic is well defined if the pointers point to within the same object and are character pointers (char* or unsigned char*).
Using correct types, it would be:
struct test s = { ... };
int *p = s.x;
short *q = s.y;
printf("%td\n", q - p);
Now, how can q-p be performed? based on sizeof(int) or sizeof(short) ? How can the size of char something; that's in the middle of these two arrays be calculated?
That should explain it's not possible to perform pointer arithmetic on objects of different types.
Even if all members are of same type (thus no type issue as stated above), then it's better to use the standard macro offsetof (from <stddef.h>) to get the difference between struct members which has the similar effect as pointer arithmetic between members:
printf("%zu\n", offsetof(struct test, y) - offsetof(struct test, x));
So I see no necessity to define pointer arithmetic between struct members by the C standard.
Yes, you are allowed to perform pointer arithmetric on structure bytes:
N1570 - 6.3.2.3 Pointers p7:
... When a pointer to an object is converted to a pointer to a character type,
the result points to the lowest addressed byte of the object. Successive increments of the
result, up to the size of the object, yield pointers to the remaining bytes of the object.
This means that for the programmer, bytes of the stucture shall be seen as a continuous area, regardless how it may have been implemented in the hardware.
Not with void* pointers though, that is non-standard compiler extension. As mentioned on paragraph from the standard, it applies only to character type pointers.
Edit:
As mafso pointed out in comments, above is only true as long as type of substraction result ptrdiff_t, has enough range for the result. Since range of size_t can be larger than ptrdiff_t, and if structure is big enough, it's possible that addresses are too far apart.
Because of this it's preferable to use offsetof macro on structure members and calculate result from those.
I believe the answer to this question is simpler than it appears, the OP asks:
but why should that result be "undefined"?
Well, let's see that the definition of undefined behavior is in the draft C99 standard section 3.4.3:
behavior, upon use of a nonportable or erroneous program construct or
of erroneous data, for which this International Standard imposes no
requirements
it is simply behavior for which the standard does not impose a requirement, which perfectly fits this situation, the results are going to vary depending on the architecture and attempting to specify the results would have probably been difficult if not impossible in a portable manner. This leaves the question, why would they choose undefined behavior as opposed to let's say implementation of unspecified behavior?
Most likely it was made undefined behavior to limit the number of ways an invalid pointer could be created, this is consistent with the fact that we are provided with offsetof to remove the one potential need for pointer subtraction of unrelated objects.
Although the standard does not really define the term invalid pointer, we get a good description in Rationale for International Standard—Programming Languages—C which in section 6.3.2.3 Pointers says (emphasis mine):
Implicit in the Standard is the notion of invalid pointers. In
discussing pointers, the Standard typically refers to “a pointer to an
object” or “a pointer to a function” or “a null pointer.” A special
case in address arithmetic allows for a pointer to just past the end
of an array. Any other pointer is invalid.
The C99 rationale further adds:
Regardless how an invalid pointer is created, any use of it yields
undefined behavior. Even assignment, comparison with a null pointer
constant, or comparison with itself, might on some systems result in
an exception.
This strongly suggests to us that a pointer to padding would be an invalid pointer, although it is difficult to prove that padding is not an object, the definition of object says:
region of data storage in the execution environment, the contents of
which can represent values
and notes:
When referenced, an object may be interpreted as having a particular
type; see 6.3.2.1.
I don't see how we can reason about the type or the value of padding between elements of a struct and therefore they are not objects or at least is strongly indicates padding is not meant to be considered an object.
I should point out the following:
from the C99 standard, section 6.7.2.1:
Within a structure object, the non-bit-field members and the units in which bit-fields
reside have addresses that increase in the order in which they are declared. A pointer to a
structure object, suitably converted, points to its initial member (or if that member is a
bit-field, then to the unit in which it resides), and vice versa. There may be unnamed
padding within a structure object, but not at its beginning.
It isn't so much that the result of pointer subtraction between members is undefined so much as it is unreliable (i.e. not guaranteed to be the same between different instances of the same struct type when the same arithmetic is applied).