I've used structs extensively and I've seen some interesting things, especially *value instead of value->first_value where value is a pointer to struct, first_value is the very first member, is *value safe?
Also note that sizes aren't guaranteed because of alignment, whats the alginment value based on, the architecture/register size?
We align data/code for faster execution can we tell compiler not to do this? so maybe we can guarantee certain things about structs, like their size?
When doing pointer arithmetic on struct members in order to locate member offset, I take it you do - if little endian + for big endian, or does it just depend on the compiler?
what does malloc(0) really allocate?
The following code is for educational/discovery purposes, its not meant to be of production quality.
#include <stdlib.h>
#include <stdio.h>
int main()
{
printf("sizeof(struct {}) == %lu;\n", sizeof(struct {}));
printf("sizeof(struct {int a}) == %lu;\n", sizeof(struct {int a;}));
printf("sizeof(struct {int a; double b;}) == %lu;\n", sizeof(struct {int a; double b;}));
printf("sizeof(struct {char c; double a; double b;}) == %lu;\n", sizeof(struct {char c; double a; double b;}));
printf("malloc(0)) returns %p\n", malloc(0));
printf("malloc(sizeof(struct {})) returns %p\n", malloc(sizeof(struct {})));
struct {int a; double b;} *test = malloc(sizeof(struct {int a; double b;}));
test->a = 10;
test->b = 12.2;
printf("test->a == %i, *test == %i \n", test->a, *(int *)test);
printf("test->b == %f, offset of b is %i, *(test - offset_of_b) == %f\n",
test->b, (int)((void *)test - (void *)&test->b),
*(double *)((void *)test - ((void *)test - (void *)&test->b))); // find the offset of b, add it to the base,$
free(test);
return 0;
}
calling gcc test.c followed by ./a.out
I get this:
sizeof(struct {}) == 0;
sizeof(struct {int a}) == 4;
sizeof(struct {int a; double b;}) == 16;
sizeof(struct {char c; double a; double b;}) == 24;
malloc(0)) returns 0x100100080
malloc(sizeof(struct {})) returns 0x100100090
test->a == 10, *test == 10
test->b == 12.200000, offset of b is -8, *(test - offset_of_b) == 12.200000
Update
this is my machine:
gcc --version
i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5666) (dot 3)
Copyright (C) 2007 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
uname -a
Darwin MacBookPro 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun 7 16:33:36 PDT 2011; root:xnu-1504.15.3~1/RELEASE_I386 i386
From 6.2.5/20:
A structure type describes a sequentially allocated nonempty set of member objects
(and, in certain circumstances, an incomplete array), each of which has an optionally
specified name and possibly distinct type.
To answer:
especially *value instead of value->first_value where value is a pointer to struct, first_value is the very first member, is *value safe?
see 6.7.2.1/15:
15 Within a structure object, the non-bit-field members and the units in which bit-fields
reside have addresses that increase in the order in which they are declared. A pointer to a
structure object, suitably converted, points to its initial member (or if that member is a
bit-field, then to the unit in which it resides), and vice versa. There may be unnamed
padding within a structure object, but not at its beginning.1
There may however be padding bytes at the end of the structure as also in-between members.
In C, malloc( 0 ) is implementation defined. (As a side note, this is one of those little things where C and C++ differ.)
[1] Emphasis mine.
I've used structs extensively and I've seen some interesting things, especially *value instead of value->first_value where value is a pointer to struct, first_value is the very first member, is *value safe?
Yes, *value is safe; it yields a copy of the structure that value points at. But it is almost guaranteed to have a different type from *value->first_value, so the result of *value will almost always be different from *value->first_value.
Counter-example:
struct something { struct something *first_value; ... };
struct something data = { ... };
struct something *value = &data;
value->first_value = value;
Under this rather limited set of circumstances, you would get the same result from *value and *value->first_value. Under that scheme, the types would be the same (even if the values are not). In the general case, the type of *value and *value->first_value are of different types.
Also note that sizes aren't guaranteed because of alignment, but is alignment always on register size?
Since 'register size' is not a defined C concept, it isn't clear what you're asking. In the absence of pragmas (#pragma pack or similar), the elements of a structure will be aligned for optimal performance when the value is read (or written).
We align data/code for faster execution; can we tell compiler not to do this? So maybe we can guarantee certain things about structs, like their size?
The compiler is in charge of the size and layout of struct types. You can influence by careful design and perhaps by #pragma pack or similar directives.
These questions normally arise when people are concerned about serializing data (or, rather, trying to avoid having to serialize data by processing structure elements one at a time). Generally, I think you're better off writing a function to do the serialization, building it up from component pieces.
When doing pointer arithmetic on struct members in order to locate member offset, I take it you do subtraction if little endian, addition for big endian, or does it just depend on the compiler?
You're probably best off not doing pointer arithmetic on struct members. If you must, use the offsetof() macro from <stddef.h> to handle the offsets correctly (and that means you're not doing pointer arithmetic directly). The first structure element is always at the lowest address, regardless of big-endianness or little-endianness. Indeed, endianness has no bearing on the layout of different members within a structure; it only has an affect on the byte order of values within a (basic data type) member of a structure.
The C standard requires that the elements of a structure are laid out in the order that they are defined; the first element is at the lowest address, and the next at a higher address, and so on for each element. The compiler is not allowed to change the order. There can be no padding before the first element of the structure. There can be padding after any element of the structure as the compiler sees fit to ensure what it considers appropriate alignment. The size of a structure is such that you can allocate (N × size) bytes that are appropriately aligned (e.g. via malloc()) and treat the result as an array of the structure.
Calling malloc(0) will return a pointer that may be safely passed to free() at least once. If the same value is returned by multiple malloc(0) calls, it may be freed once for each such call. Obviously, if it returns NULL, that could be passed to free() an unlimited number of times without effect. Every call to malloc(0) which returns non-null should be balanced by a call to free() with the returned value.
If you have an inner structure it is guaranteed to start on the same address as the enclosing one if that is the first declaration of the enclosing structure.
So *value and value->first is accessing memory at the same address (but using different types) in the following
struct St {
long first;
} *value;
Also, the ordering between memebers of the structure is guaranteed to be the same as the declaration order
To adjust alignment, you can use compiler specific directives or use bitfields.
The alignment of structure memebers are usually based on what's best to access the individual members on the target platform
Also, for malloc, it is possible it keeps some bookkeeping near the returned address, so even for zero-size memory it can return a valid address (just don't try to access anything via the returned address)
It is important to learn about the way that size of struct works. for example:
struct foo{
int i;
char c;
}
struct bar{
int i;
int j;
}
struct baz{
int i;
char c;
int j;
}
sizeof(foo) = 8 bytes (32 bit arch)
sizeof(bar) = 8 bytes
sizeof(baz) = 12 bytes
What this means is that struct sizes and offsets have to follow two rules:
1- The struct must be a multiple of it's first element (Why foo is 8 not 5 bytes)
2- A struct element must start on a multiple of itself. (In baz, int j could not start on 6, so bytes 6, 7, and 8 are wasted padding
Related
I'm trying to implement a polymorphic data structure e.g. an intrusive linked list (I already know the kernel has one - this is more of learning experience).
The trouble is casting a nested struct to the containing struct leads to gcc issuing a memory-alignment warning.
The specifics are provided below:
// t.c
#include <stdio.h>
enum OL_TYPE { A, B, C };
struct base {
enum OL_TYPE type;
};
struct overlay {
struct base base;
int i;
};
struct overlay2 {
struct base base;
float f;
int i;
// double d; // --> adding this causes a memory alignment warning
};
void testf(struct base *base) {
if (base->type == A) {
struct overlay *olptr = (struct overlay *)base;
printf("overlay->i = %d\n", olptr->i);
} else
if (base->type == B) {
struct overlay2 *olptr = (struct overlay2 *)base;
printf("overlay->i = %d\n", olptr->i);
}
}
int main(int argc, char *argv[]) {
struct overlay ol;
ol.base.type = A;
ol.i = 3;
testf(&ol.base);
}
Compiled with gcc t.c -std=c99 -pedantic -fstrict-aliasing -Wcast-align=strict -O3 -o q leads to this:
t.c: In function ‘testf’:
t.c:28:34: warning: cast increases required alignment of target type [-Wcast-align]
28 | struct overlay2 *olptr = (struct overlay2 *)base;
| ^
It's interesting to notice that if I comment out the double from overlay2 such that it's not part of the structure anymore, the warning disappears.
In light of this, I have a few questions:
Is there any danger to this?
If yes, what is the danger? If not, does that mean the warning is a false positive and how can it be dealt with?
Why does adding the double suddenly lead to the alignment warning?
Is misalignment / an alignment problem even possible if I cast from base to a struct overlay2 * IF the actual type IS in fact a struct overlay2 with a nested struct base ? Please explain!
Appreciate any answers that can provide some insight
is there any danger to this?
Only if what base points to was not allocated as a struct overlay2 in the first place. If you're just smuggling in a pointer to struct overlay2 as a struct base*, and casting back internally, that should be fine (the address would actually be aligned correctly in the first place).
If yes, what is the danger? If not, does that mean the warning is a false positive and how can it be dealt with?
The danger occurs when you allocate something as something other than struct overlay2 then try to use the unaligned double inside it. For example, a union of struct base and char data[sizeof(struct overlay2)] would be the right size to contain a struct overlay2, but it might be four byte aligned but not eight byte aligned, causing the double (size 8, alignment 8) to be misaligned. On x86 systems, a misaligned field typically just slows your code, but on non-x86 misaligned field access can crash your program.
As for dealing with it, you can silence the warning (with some compiler-specific directives), or you can just force all your structures to be aligned identically, by adding alignas directives (available since C11/C++11):
#include <stdalign.h> // Add at top of file to get alignas macro
struct base {
alignas(double) enum OL_TYPE type;
};
why does adding the double suddenly lead to the alignment warning?
Because it's the only data type in any of the structures that requires eight byte alignment; without it, the structures can all be legally aligned to four bytes; with it, overlay2 needs eight byte alignment while the others only need four byte alignment.
is an alignment even possible if I cast from base to a struct overlay2 * IF the actual type IS in fact a struct overlay2 with a nested struct base ?
As noted above, if it was really pointing to something allocated as a struct overlay2 all along, you're safe.
It's not really a false positive, because the compiler doesn't know that your passed base* argument will be the address of an actual overlay2 structure (even though you do). However, if that condition is always going to be true, then no issue should arise due to inappropriate alignment.
However, to be completely sure (and to silence the warning), you could always make the base structure align to the requirements of a double, if that doesn't cause any other issues:
struct base {
_Alignas(double) enum OL_TYPE type;
};
Why does adding the double suddenly lead to the alignment warning
Because, in general, the alignment requirement of any structure must be at least that of the largest alignment requirement of all its members. Without such an assurance, an array of those structures would – by definition – result in misalignment of such members in any one one of two adjacent elements in the array.
Some processors require that pointers to an item of size S must be aligned to the next power-of-two greater than or equal to S, otherwise you get memory alignment faults.
If you take pointer to a variable (including aggregate) that's lesser alignment (lower power-of-two) and cast it to the type of a variable that has higher alignment requirements, you potentially set yourself up for those alignment faults.
If this is likely to be an issue, one option is to put all the item types that will be cast between into a union, always allocate that union, and then reference the type within the union.
That has the potential to get expensive on memory, so another possibility is to create a copy function that initialises a greater-alignment object from the contents of the lesser-alignment one, ensuring any additional values the larger one has will be set to sensible default values.
Finally, and since we're talking about linked lists anyway, the better option is to have a single struct type that comprises the linked items. It will also contain a void * or union thing * pointer to the actual data and an indication of what type of data the pointer refers to. This way you're pretty economical with memory, you don't have to worry about alignment issues, and you have the cleanest and most adaptable representation.
I believe I've found a way to achieve something like the well-known "struct hack" in portable C89. I'm curious if this really strictly conforms to C89.
The main idea is: I allocate memory large enough to hold an initial struct and the elements of the array. The exact size is (K + N) * sizeof(array_base_type), where K is chosen so that K * sizeof(array_base_type) >= sizeof(the_struct) and N is the number of array elements.
First, I dereference the pointer that malloc() returned to store the_struct, then I use pointer arithmetic to obtain a pointer to the beginning of the array following the struct.
One line of code is worth more than a thousand words, so here is a minimal implementation:
typedef struct Header {
size_t length;
/* other members follow */
} Header;
typedef struct Value {
int type;
union {
int intval;
double fltval;
} v;
} Value;
/* round up to nearest multiple of sizeof(Value) so that a Header struct fits in */
size_t n_hdr = (sizeof(Header) + sizeof(Value) - 1) / sizeof(Value);
size_t n_arr = 42; /* arbitrary array size here */
void *frame = malloc((n_hdr + n_arr) * sizeof(Value));
if (!frame)
return NULL;
Header *hdr = frame;
Value *stack_bottom = (Value *)frame + n_hdr;
My main concern is that the last two assignments (using frame as both a pointer to Header and a pointer to Value) may violate the strict aliasing rule. I do not, however, dereference hdr as a pointer to Value - it's only pointer arithmetic that is performed on frame in order to access the first element of the value array, so I don't effectively access the same object using pointers of different types.
So, is this approach any better than the classic struct hack (which has been officially deemed UB), or is it UB too?
The "obvious" (well... not exactly obvious, but it's what comes to my mind anyway :-) ) way to cause this to break is to use a vectorizing compiler that somehow decides it's OK to load, say, 64 Headers into a vector register from the 42-rounded-up-to-64+ area at hdr which comes from malloc which always allocates enough to vectorize. Storing the vector register back to memory might overwrite one of the Values.
I think this vectorizing compiler could point to the standard (well, if a compiler has fingers...) and claim conformance.
In practice, though, I'd expect this code to work. If you come across a vectorizing compiler, add even more space (do the rounding up with a machine-dependent macro that can insert a minimum) and charge on. :-)
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Why isn’t sizeof for a struct equal to the sum of sizeof of each member?
I can not understand why is it like this:
#include <stdio.h>
#include <stdlib.h>
typedef struct
{
char b;
int a;
} A;
typedef struct
{
char b;
} B;
int main() {
A object;
printf("sizeof char is: %d\n",sizeof(char));
printf("sizeof int is: %d\n",sizeof(int));
printf("==> the sizeof both are: %d\n",sizeof(int)+sizeof(char));
printf("and yet the sizeof struct A is: %d\n",sizeof(object));
printf("why?\n");
B secondObject;
printf("pay attention that the sizeof struct B is: %d which is equal to the "
"sizeof char\n",sizeof(secondObject));
return 0;
}
I think I explained my question in the code and there is no more need to explain. besides I have another question:
I know there is allocation on the: heap/static heap/stack, but what is that means that the allocation location is unknown, How could it be ?
I am talking about this example:
typedef struct
{
char *_name;
int _id;
} Entry;
int main()
{
Entry ** vec = (Entry**) malloc(sizeof(Entry*)*2);
vec[0] = (Entry *) malloc(sizeof (Entry));
vec[0]->_name = (char*)malloc(6);
strcpy (vec[0]->_name, "name");
vec[0]->_id = 0;
return 0;
}
I know that:
vec is on the stack.
*vec is on the heap.
*vec[0] is on the heap.
vec[0]->id is on the heap.
but :
vec[0]->_name is unknown
why ?
There is an unspecified amount of padding between the members of a structure and at the end of a structure. In C the size of a structure object is greater than or equal to the sum of the size of its members.
Take a look at this question as well as this one and many others if you search for CPU and memory alignment. In short, CPUs are happier if they access the memory aligned to the size of the data they are reading. For example, if you are reading a uint16_t, then it would be more efficient (on most CPUs) if you read at an address that is a multiple of 2. The details of why CPUs are designed in such a way is whole other story.
This is why compilers come to the rescue and pad the fields of the structures in such a way that would be most comfortable for the CPU to access them, at the cost of extra storage space. In your case, you are probably given 3 byte of padding between your char and int, assuming int is 4 bytes.
If you look at the C standard (which I don't have nearby right now), or the man page of malloc, you will see such a phrase:
The malloc() and calloc() functions return a pointer to the allocated memory
that is suitably aligned for any kind of variable.
This behavior is exactly due to the same reason I mentioned above. So in short, memory alignment is something to care about, and that's what compilers do for you in struct layout and other places, such as layout of local variables etc.
You're running into structure padding here. The compiler is inserting likely inserting three bytes' worth of padding after the b field in struct A, so that the a field is 4-byte aligned. You can control this padding to some degree using compiler-specific bits; for example, on MSVC, the pack pragma, or the aligned attribute on GCC, but I would not recommend this. Structure padding is there to specify member alignment restrictions, and some architectures will fault on unaligned accesses. (Others might fixup the alignment manually, but typically do this rather slowly.)
See also: http://en.wikipedia.org/wiki/Data_structure_alignment#Data_structure_padding
As to your second question, I'm unsure what you mean by the name is "unknown". Care to elaborate?
The compiler is free to add padding in structures to ensure that datatypes are aligned properly. For example, an int will be aligned to sizeof(int) bytes. So I expect the output for the size of your A struct is 8. The compiler does this, because fetching an int from an unaligned address is at best inefficient, and at worst doesn't work at all - that depends on the processor that the computer uses. x86 will fetch happily from unaligned addresses for most data types, but will take about twice as long for the fetch operation.
In your second code-snippet, you haven't declared i.
So vec[0]->_name is not unknown - it is on the heap, just like anything else you get from "malloc" (and malloc's siblings).
In C, the compiler will lay out members of a struct in the order in which they're declared, with possible padding bytes inserted between members, or after the last member, to ensure that each member is aligned properly.
gcc provides a language extension, __attribute__((packed)), which tells the compiler not to insert padding, allowing struct members to be misaligned. For example, if the system normally requires all int objects to have 4-byte alignment, __attribute__((packed)) can cause int struct members to be allocated at odd offsets.
Quoting the gcc documentation:
The `packed' attribute specifies that a variable or structure field
should have the smallest possible alignment--one byte for a variable,
and one bit for a field, unless you specify a larger value with the
`aligned' attribute.
Obviously the use of this extension can result in smaller data requirements but slower code, as the compiler must (on some platforms) generate code to access a misaligned member a byte at a time.
But are there any cases where this is unsafe? Does the compiler always generate correct (though slower) code to access misaligned members of packed structs? Is it even possible for it to do so in all cases?
Yes, __attribute__((packed)) is potentially unsafe on some systems. The symptom probably won't show up on an x86, which just makes the problem more insidious; testing on x86 systems won't reveal the problem. (On the x86, misaligned accesses are handled in hardware; if you dereference an int* pointer that points to an odd address, it will be a little slower than if it were properly aligned, but you'll get the correct result.)
On some other systems, such as SPARC, attempting to access a misaligned int object causes a bus error, crashing the program.
There have also been systems where a misaligned access quietly ignores the low-order bits of the address, causing it to access the wrong chunk of memory.
Consider the following program:
#include <stdio.h>
#include <stddef.h>
int main(void)
{
struct foo {
char c;
int x;
} __attribute__((packed));
struct foo arr[2] = { { 'a', 10 }, {'b', 20 } };
int *p0 = &arr[0].x;
int *p1 = &arr[1].x;
printf("sizeof(struct foo) = %d\n", (int)sizeof(struct foo));
printf("offsetof(struct foo, c) = %d\n", (int)offsetof(struct foo, c));
printf("offsetof(struct foo, x) = %d\n", (int)offsetof(struct foo, x));
printf("arr[0].x = %d\n", arr[0].x);
printf("arr[1].x = %d\n", arr[1].x);
printf("p0 = %p\n", (void*)p0);
printf("p1 = %p\n", (void*)p1);
printf("*p0 = %d\n", *p0);
printf("*p1 = %d\n", *p1);
return 0;
}
On x86 Ubuntu with gcc 4.5.2, it produces the following output:
sizeof(struct foo) = 5
offsetof(struct foo, c) = 0
offsetof(struct foo, x) = 1
arr[0].x = 10
arr[1].x = 20
p0 = 0xbffc104f
p1 = 0xbffc1054
*p0 = 10
*p1 = 20
On SPARC Solaris 9 with gcc 4.5.1, it produces the following:
sizeof(struct foo) = 5
offsetof(struct foo, c) = 0
offsetof(struct foo, x) = 1
arr[0].x = 10
arr[1].x = 20
p0 = ffbff317
p1 = ffbff31c
Bus error
In both cases, the program is compiled with no extra options, just gcc packed.c -o packed.
(A program that uses a single struct rather than array doesn't reliably exhibit the problem, since the compiler can allocate the struct on an odd address so the x member is properly aligned. With an array of two struct foo objects, at least one or the other will have a misaligned x member.)
(In this case, p0 points to a misaligned address, because it points to a packed int member following a char member. p1 happens to be correctly aligned, since it points to the same member in the second element of the array, so there are two char objects preceding it -- and on SPARC Solaris the array arr appears to be allocated at an address that is even, but not a multiple of 4.)
When referring to the member x of a struct foo by name, the compiler knows that x is potentially misaligned, and will generate additional code to access it correctly.
Once the address of arr[0].x or arr[1].x has been stored in a pointer object, neither the compiler nor the running program knows that it points to a misaligned int object. It just assumes that it's properly aligned, resulting (on some systems) in a bus error or similar other failure.
Fixing this in gcc would, I believe, be impractical. A general solution would require, for each attempt to dereference a pointer to any type with non-trivial alignment requirements either (a) proving at compile time that the pointer doesn't point to a misaligned member of a packed struct, or (b) generating bulkier and slower code that can handle either aligned or misaligned objects.
I've submitted a gcc bug report. As I said, I don't believe it's practical to fix it, but the documentation should mention it (it currently doesn't).
UPDATE: As of 2018-12-20, this bug is marked as FIXED. The patch will appear in gcc 9 with the addition of a new -Waddress-of-packed-member option, enabled by default.
When address of packed member of struct or union is taken, it may
result in an unaligned pointer value. This patch adds
-Waddress-of-packed-member to check alignment at pointer assignment and warn unaligned address as well as unaligned pointer
I've just built that version of gcc from source. For the above program, it produces these diagnostics:
c.c: In function ‘main’:
c.c:10:15: warning: taking address of packed member of ‘struct foo’ may result in an unaligned pointer value [-Waddress-of-packed-member]
10 | int *p0 = &arr[0].x;
| ^~~~~~~~~
c.c:11:15: warning: taking address of packed member of ‘struct foo’ may result in an unaligned pointer value [-Waddress-of-packed-member]
11 | int *p1 = &arr[1].x;
| ^~~~~~~~~
As ams said above, don't take a pointer to a member of a struct that's packed. This is simply playing with fire. When you say __attribute__((__packed__)) or #pragma pack(1), what you're really saying is "Hey gcc, I really know what I'm doing." When it turns out that you do not, you can't rightly blame the compiler.
Perhaps we can blame the compiler for it's complacency though. While gcc does have a -Wcast-align option, it isn't enabled by default nor with -Wall or -Wextra. This is apparently due to gcc developers considering this type of code to be a brain-dead "abomination" unworthy of addressing -- understandable disdain, but it doesn't help when an inexperienced programmer bumbles into it.
Consider the following:
struct __attribute__((__packed__)) my_struct {
char c;
int i;
};
struct my_struct a = {'a', 123};
struct my_struct *b = &a;
int c = a.i;
int d = b->i;
int *e __attribute__((aligned(1))) = &a.i;
int *f = &a.i;
Here, the type of a is a packed struct (as defined above). Similarly, b is a pointer to a packed struct. The type of of the expression a.i is (basically) an int l-value with 1 byte alignment. c and d are both normal ints. When reading a.i, the compiler generates code for unaligned access. When you read b->i, b's type still knows it's packed, so no problem their either. e is a pointer to a one-byte-aligned int, so the compiler knows how to dereference that correctly as well. But when you make the assignment f = &a.i, you are storing the value of an unaligned int pointer in an aligned int pointer variable -- that's where you went wrong. And I agree, gcc should have this warning enabled by default (not even in -Wall or -Wextra).
It's perfectly safe as long as you always access the values through the struct via the . (dot) or -> notation.
What's not safe is taking the pointer of unaligned data and then accessing it without taking that into account.
Also, even though each item in the struct is known to be unaligned, it's known to be unaligned in a particular way, so the struct as a whole must be aligned as the compiler expects or there'll be trouble (on some platforms, or in future if a new way is invented to optimise unaligned accesses).
Using this attribute is definitely unsafe.
One particular thing it breaks is the ability of a union which contains two or more structs to write one member and read another if the structs have a common initial sequence of members. Section 6.5.2.3 of the C11 standard states:
6 One special guarantee is made in order to simplify the use of unions:
if a union contains several structures that share a common
initial sequence (see below), and if the union object
currently contains one of these structures, it is permitted
to inspect the common initial part of any of them anywhere that a
declaration of the completed type of the union is visible. Tw o
structures share a common initial sequence if corresponding
members have compatible types (and, for bit-fields, the same widths)
for a sequence of one or more initial members.
...
9 EXAMPLE 3 The following is a valid fragment:
union {
struct {
int alltypes;
}n;
struct {
int type;
int intnode;
} ni;
struct {
int type;
double doublenode;
} nf;
}u;
u.nf.type = 1;
u.nf.doublenode = 3.14;
/*
...
*/
if (u.n.alltypes == 1)
if (sin(u.nf.doublenode) == 0.0)
/*
...
*/
When __attribute__((packed)) is introduced it breaks this. The following example was run on Ubuntu 16.04 x64 using gcc 5.4.0 with optimizations disabled:
#include <stdio.h>
#include <stdlib.h>
struct s1
{
short a;
int b;
} __attribute__((packed));
struct s2
{
short a;
int b;
};
union su {
struct s1 x;
struct s2 y;
};
int main()
{
union su s;
s.x.a = 0x1234;
s.x.b = 0x56789abc;
printf("sizeof s1 = %zu, sizeof s2 = %zu\n", sizeof(struct s1), sizeof(struct s2));
printf("s.y.a=%hx, s.y.b=%x\n", s.y.a, s.y.b);
return 0;
}
Output:
sizeof s1 = 6, sizeof s2 = 8
s.y.a=1234, s.y.b=5678
Even though struct s1 and struct s2 have a "common initial sequence", the packing applied to the former means that the corresponding members don't live at the same byte offset. The result is the value written to member x.b is not the same as the value read from member y.b, even though the standard says they should be the same.
(The following is a very artificial example cooked up to illustrate.) One major use of packed structs is where you have a stream of data (say 256 bytes) to which you wish to supply meaning. If I take a smaller example, suppose I have a program running on my Arduino which sends via serial a packet of 16 bytes which have the following meaning:
0: message type (1 byte)
1: target address, MSB
2: target address, LSB
3: data (chars)
...
F: checksum (1 byte)
Then I can declare something like
typedef struct {
uint8_t msgType;
uint16_t targetAddr; // may have to bswap
uint8_t data[12];
uint8_t checksum;
} __attribute__((packed)) myStruct;
and then I can refer to the targetAddr bytes via aStruct.targetAddr rather than fiddling with pointer arithmetic.
Now with alignment stuff happening, taking a void* pointer in memory to the received data and casting it to a myStruct* will not work unless the compiler treats the struct as packed (that is, it stores data in the order specified and uses exactly 16 bytes for this example). There are performance penalties for unaligned reads, so using packed structs for data your program is actively working with is not necessarily a good idea. But when your program is supplied with a list of bytes, packed structs make it easier to write programs which access the contents.
Otherwise you end up using C++ and writing a class with accessor methods and stuff that does pointer arithmetic behind the scenes. In short, packed structs are for dealing efficiently with packed data, and packed data may be what your program is given to work with. For the most part, you code should read values out of the structure, work with them, and write them back when done. All else should be done outside the packed structure. Part of the problem is the low level stuff that C tries to hide from the programmer, and the hoop jumping that is needed if such things really do matter to the programmer. (You almost need a different 'data layout' construct in the language so that you can say 'this thing is 48 bytes long, foo refers to the data 13 bytes in, and should be interpreted thus'; and a separate structured data construct, where you say 'I want a structure containing two ints, called alice and bob, and a float called carol, and I don't care how you implement it' -- in C both these use cases are shoehorned into the struct construct.)
I just learned that it's possible to increase the size of the memory you'll allocate to a struct when using the malloc function. For example, you can have a struct like this:
struct test{
char a;
int v[1];
char b;
};
Which clearly has space for only 2 chars and 1 int (pointer to an int in reality, but anyway). But you could call malloc in such a way to make the struct holds 2 chars and as many ints as you wanted (let's say 10):
int main(){
struct test *ptr;
ptr = malloc (sizeof(struct test)+sizeof(int)*9);
ptr->v[9]=50;
printf("%d\n",ptr->v[9]);
return 0;
}
The output here would be "50" printed on the screen, meaning that the array inside the struct was holding up to 10 ints.
My questions for the experienced C programmers out there:
What is happening behind the scenes here? Does the computer allocate 2+4 (2 chars + pointer to int) bytes for the standard "struct test", and then 4*9 more bytes of memory and let the pointer "ptr" put whatever kind of data it wants on those extra bytes?
Does this trick only works when there is an array inside the struct?
If the array is not the last member of the struct, how does the computer manage the memory block allocated?
...Which clearly has space for only 2 chars and 1 int (pointer to an
int in reality, but anyway)...
Already incorrect. Arrays are not pointers. Your struct holds space for 2 chars and 1 int. There's no pointer of any kind there. What you have declared is essentially equivalent to
struct test {
char a;
int v;
char b;
};
There's not much difference between an array of 1 element and an ordinary variable (there's conceptual difference only, i.e. syntactic sugar).
...But you could call malloc in such a way to make it hold 1 char and as
many ints as you wanted (let's say 10)...
Er... If you want it to hold 1 char, why did you declare your struct with 2 chars???
Anyway, in order to implement an array of flexible size as a member of a struct you have to place your array at the very end of the struct.
struct test {
char a;
char b;
int v[1];
};
Then you can allocate memory for your struct with some "extra" memory for the array at the end
struct test *ptr = malloc(offsetof(struct test, v) + sizeof(int) * 10);
(Note how offsetof is used to calculate the proper size).
That way it will work, giving you an array of size 10 and 2 chars in the struct (as declared). It is called "struct hack" and it depends critically on the array being the very last member of the struct.
C99 version of C language introduced dedicated support for "struct hack". In C99 it can be done as
struct test {
char a;
char b;
int v[];
};
...
struct test *ptr = malloc(sizeof(struct test) + sizeof(int) * 10);
What is happening behind the scenes here? Does the computer allocate
2+4 (2 chars + pointer to int) bytes for the standard "struct test",
and then 4*9 more bytes of memory and let the pointer "ptr" put
whatever kind of data it wants on those extra bytes?
malloc allocates as much memory as you ask it to allocate. It is just a single flat block of raw memory. Nothing else happens "behind the scenes". There's no "pointer to int" of any kind in your struct, so any questions that involve "pointer to int" make no sense at all.
Does this trick only works when there is an array inside the struct?
Well, that's the whole point: to access the extra memory as if it belongs to an array declared as the last member of the struct.
If the array is not the last member of the struct, how does the computer manage the memory block allocated?
It doesn't manage anything. If the array is not the last member of the struct, then trying to work with the extra elements of the array will trash the members of the struct that declared after the array. This is pretty useless, which is why the "flexible" array has to be the last member.
No, that does not work. You can't change the immutable size of a struct (which is a compile-time allocation, after all) by using malloc ( ) at run time. But you can allocate a memory block, or change its size, such that it holds more than one struct:
int main(){
struct test *ptr;
ptr = malloc (sizeof(struct test) * 9);
}
That's just about all you can do with malloc ( ) in this context.
In addition to what others have told you (summary: arrays are not pointers, pointers are not arrays, read section 6 of the comp.lang.c FAQ), attempting to access array elements past the last element invokes undefined behavior.
Let's look at an example that doesn't involve dynamic allocation:
struct foo {
int arr1[1];
int arr2[1000];
};
struct foo obj;
The language guarantees that obj.arr1 will be allocated starting at offset 0, and that the offset of obj.arr2 will be sizeof (int) or more (the compiler may insert padding between struct members and after the last member, but not before the first one). So we know that there's enough room in obj for multiple int objects immediately following obj.arr1. That means that if you write obj.arr1[5] = 42, and then later access obj.arr[5], you'll probably get back the value 42 that you stored there (and you'll probably have clobbered obj.arr2[4]).
The C language doesn't require array bounds checking, but it makes the behavior of accessing an array outside its declared bounds undefined. Anything could happen -- including having the code quietly behave just the way you want it to. In fact, C permits array bounds checking; it just doesn't provide a way to handle errors, and most compilers don't implement it.
For an example like this, you're most likely to run into visible problems in the presence of optimization. A compiler (particularly an optimizing compiler) is permitted to assume that your program's behavior is well-defined, and to rearrange the generated code to take advantage of that assumption. If you write
int index = 5;
obj.arr1[index] = 42;
the compiler is permitted to assume that the index operation doesn't go outside the declared bounds of the array. As Henry Spencer wrote, "If you lie to the compiler, it will get its revenge".
Strictly speaking, the struct hack probably involves undefined behavior (which is why C99 added a well-defined version of it), but it's been so widely used that most or all compilers will support it. This is covered in question 2.6 of the comp.lang.c FAQ.