Is gcc's __attribute__((packed)) / #pragma pack unsafe? - c

In C, the compiler will lay out members of a struct in the order in which they're declared, with possible padding bytes inserted between members, or after the last member, to ensure that each member is aligned properly.
gcc provides a language extension, __attribute__((packed)), which tells the compiler not to insert padding, allowing struct members to be misaligned. For example, if the system normally requires all int objects to have 4-byte alignment, __attribute__((packed)) can cause int struct members to be allocated at odd offsets.
Quoting the gcc documentation:
The `packed' attribute specifies that a variable or structure field
should have the smallest possible alignment--one byte for a variable,
and one bit for a field, unless you specify a larger value with the
`aligned' attribute.
Obviously the use of this extension can result in smaller data requirements but slower code, as the compiler must (on some platforms) generate code to access a misaligned member a byte at a time.
But are there any cases where this is unsafe? Does the compiler always generate correct (though slower) code to access misaligned members of packed structs? Is it even possible for it to do so in all cases?

Yes, __attribute__((packed)) is potentially unsafe on some systems. The symptom probably won't show up on an x86, which just makes the problem more insidious; testing on x86 systems won't reveal the problem. (On the x86, misaligned accesses are handled in hardware; if you dereference an int* pointer that points to an odd address, it will be a little slower than if it were properly aligned, but you'll get the correct result.)
On some other systems, such as SPARC, attempting to access a misaligned int object causes a bus error, crashing the program.
There have also been systems where a misaligned access quietly ignores the low-order bits of the address, causing it to access the wrong chunk of memory.
Consider the following program:
#include <stdio.h>
#include <stddef.h>
int main(void)
{
struct foo {
char c;
int x;
} __attribute__((packed));
struct foo arr[2] = { { 'a', 10 }, {'b', 20 } };
int *p0 = &arr[0].x;
int *p1 = &arr[1].x;
printf("sizeof(struct foo) = %d\n", (int)sizeof(struct foo));
printf("offsetof(struct foo, c) = %d\n", (int)offsetof(struct foo, c));
printf("offsetof(struct foo, x) = %d\n", (int)offsetof(struct foo, x));
printf("arr[0].x = %d\n", arr[0].x);
printf("arr[1].x = %d\n", arr[1].x);
printf("p0 = %p\n", (void*)p0);
printf("p1 = %p\n", (void*)p1);
printf("*p0 = %d\n", *p0);
printf("*p1 = %d\n", *p1);
return 0;
}
On x86 Ubuntu with gcc 4.5.2, it produces the following output:
sizeof(struct foo) = 5
offsetof(struct foo, c) = 0
offsetof(struct foo, x) = 1
arr[0].x = 10
arr[1].x = 20
p0 = 0xbffc104f
p1 = 0xbffc1054
*p0 = 10
*p1 = 20
On SPARC Solaris 9 with gcc 4.5.1, it produces the following:
sizeof(struct foo) = 5
offsetof(struct foo, c) = 0
offsetof(struct foo, x) = 1
arr[0].x = 10
arr[1].x = 20
p0 = ffbff317
p1 = ffbff31c
Bus error
In both cases, the program is compiled with no extra options, just gcc packed.c -o packed.
(A program that uses a single struct rather than array doesn't reliably exhibit the problem, since the compiler can allocate the struct on an odd address so the x member is properly aligned. With an array of two struct foo objects, at least one or the other will have a misaligned x member.)
(In this case, p0 points to a misaligned address, because it points to a packed int member following a char member. p1 happens to be correctly aligned, since it points to the same member in the second element of the array, so there are two char objects preceding it -- and on SPARC Solaris the array arr appears to be allocated at an address that is even, but not a multiple of 4.)
When referring to the member x of a struct foo by name, the compiler knows that x is potentially misaligned, and will generate additional code to access it correctly.
Once the address of arr[0].x or arr[1].x has been stored in a pointer object, neither the compiler nor the running program knows that it points to a misaligned int object. It just assumes that it's properly aligned, resulting (on some systems) in a bus error or similar other failure.
Fixing this in gcc would, I believe, be impractical. A general solution would require, for each attempt to dereference a pointer to any type with non-trivial alignment requirements either (a) proving at compile time that the pointer doesn't point to a misaligned member of a packed struct, or (b) generating bulkier and slower code that can handle either aligned or misaligned objects.
I've submitted a gcc bug report. As I said, I don't believe it's practical to fix it, but the documentation should mention it (it currently doesn't).
UPDATE: As of 2018-12-20, this bug is marked as FIXED. The patch will appear in gcc 9 with the addition of a new -Waddress-of-packed-member option, enabled by default.
When address of packed member of struct or union is taken, it may
result in an unaligned pointer value. This patch adds
-Waddress-of-packed-member to check alignment at pointer assignment and warn unaligned address as well as unaligned pointer
I've just built that version of gcc from source. For the above program, it produces these diagnostics:
c.c: In function ‘main’:
c.c:10:15: warning: taking address of packed member of ‘struct foo’ may result in an unaligned pointer value [-Waddress-of-packed-member]
10 | int *p0 = &arr[0].x;
| ^~~~~~~~~
c.c:11:15: warning: taking address of packed member of ‘struct foo’ may result in an unaligned pointer value [-Waddress-of-packed-member]
11 | int *p1 = &arr[1].x;
| ^~~~~~~~~

As ams said above, don't take a pointer to a member of a struct that's packed. This is simply playing with fire. When you say __attribute__((__packed__)) or #pragma pack(1), what you're really saying is "Hey gcc, I really know what I'm doing." When it turns out that you do not, you can't rightly blame the compiler.
Perhaps we can blame the compiler for it's complacency though. While gcc does have a -Wcast-align option, it isn't enabled by default nor with -Wall or -Wextra. This is apparently due to gcc developers considering this type of code to be a brain-dead "abomination" unworthy of addressing -- understandable disdain, but it doesn't help when an inexperienced programmer bumbles into it.
Consider the following:
struct __attribute__((__packed__)) my_struct {
char c;
int i;
};
struct my_struct a = {'a', 123};
struct my_struct *b = &a;
int c = a.i;
int d = b->i;
int *e __attribute__((aligned(1))) = &a.i;
int *f = &a.i;
Here, the type of a is a packed struct (as defined above). Similarly, b is a pointer to a packed struct. The type of of the expression a.i is (basically) an int l-value with 1 byte alignment. c and d are both normal ints. When reading a.i, the compiler generates code for unaligned access. When you read b->i, b's type still knows it's packed, so no problem their either. e is a pointer to a one-byte-aligned int, so the compiler knows how to dereference that correctly as well. But when you make the assignment f = &a.i, you are storing the value of an unaligned int pointer in an aligned int pointer variable -- that's where you went wrong. And I agree, gcc should have this warning enabled by default (not even in -Wall or -Wextra).

It's perfectly safe as long as you always access the values through the struct via the . (dot) or -> notation.
What's not safe is taking the pointer of unaligned data and then accessing it without taking that into account.
Also, even though each item in the struct is known to be unaligned, it's known to be unaligned in a particular way, so the struct as a whole must be aligned as the compiler expects or there'll be trouble (on some platforms, or in future if a new way is invented to optimise unaligned accesses).

Using this attribute is definitely unsafe.
One particular thing it breaks is the ability of a union which contains two or more structs to write one member and read another if the structs have a common initial sequence of members. Section 6.5.2.3 of the C11 standard states:
6 One special guarantee is made in order to simplify the use of unions:
if a union contains several structures that share a common
initial sequence (see below), and if the union object
currently contains one of these structures, it is permitted
to inspect the common initial part of any of them anywhere that a
declaration of the completed type of the union is visible. Tw o
structures share a common initial sequence if corresponding
members have compatible types (and, for bit-fields, the same widths)
for a sequence of one or more initial members.
...
9 EXAMPLE 3 The following is a valid fragment:
union {
struct {
int alltypes;
}n;
struct {
int type;
int intnode;
} ni;
struct {
int type;
double doublenode;
} nf;
}u;
u.nf.type = 1;
u.nf.doublenode = 3.14;
/*
...
*/
if (u.n.alltypes == 1)
if (sin(u.nf.doublenode) == 0.0)
/*
...
*/
When __attribute__((packed)) is introduced it breaks this. The following example was run on Ubuntu 16.04 x64 using gcc 5.4.0 with optimizations disabled:
#include <stdio.h>
#include <stdlib.h>
struct s1
{
short a;
int b;
} __attribute__((packed));
struct s2
{
short a;
int b;
};
union su {
struct s1 x;
struct s2 y;
};
int main()
{
union su s;
s.x.a = 0x1234;
s.x.b = 0x56789abc;
printf("sizeof s1 = %zu, sizeof s2 = %zu\n", sizeof(struct s1), sizeof(struct s2));
printf("s.y.a=%hx, s.y.b=%x\n", s.y.a, s.y.b);
return 0;
}
Output:
sizeof s1 = 6, sizeof s2 = 8
s.y.a=1234, s.y.b=5678
Even though struct s1 and struct s2 have a "common initial sequence", the packing applied to the former means that the corresponding members don't live at the same byte offset. The result is the value written to member x.b is not the same as the value read from member y.b, even though the standard says they should be the same.

(The following is a very artificial example cooked up to illustrate.) One major use of packed structs is where you have a stream of data (say 256 bytes) to which you wish to supply meaning. If I take a smaller example, suppose I have a program running on my Arduino which sends via serial a packet of 16 bytes which have the following meaning:
0: message type (1 byte)
1: target address, MSB
2: target address, LSB
3: data (chars)
...
F: checksum (1 byte)
Then I can declare something like
typedef struct {
uint8_t msgType;
uint16_t targetAddr; // may have to bswap
uint8_t data[12];
uint8_t checksum;
} __attribute__((packed)) myStruct;
and then I can refer to the targetAddr bytes via aStruct.targetAddr rather than fiddling with pointer arithmetic.
Now with alignment stuff happening, taking a void* pointer in memory to the received data and casting it to a myStruct* will not work unless the compiler treats the struct as packed (that is, it stores data in the order specified and uses exactly 16 bytes for this example). There are performance penalties for unaligned reads, so using packed structs for data your program is actively working with is not necessarily a good idea. But when your program is supplied with a list of bytes, packed structs make it easier to write programs which access the contents.
Otherwise you end up using C++ and writing a class with accessor methods and stuff that does pointer arithmetic behind the scenes. In short, packed structs are for dealing efficiently with packed data, and packed data may be what your program is given to work with. For the most part, you code should read values out of the structure, work with them, and write them back when done. All else should be done outside the packed structure. Part of the problem is the low level stuff that C tries to hide from the programmer, and the hoop jumping that is needed if such things really do matter to the programmer. (You almost need a different 'data layout' construct in the language so that you can say 'this thing is 48 bytes long, foo refers to the data 13 bytes in, and should be interpreted thus'; and a separate structured data construct, where you say 'I want a structure containing two ints, called alice and bob, and a float called carol, and I don't care how you implement it' -- in C both these use cases are shoehorned into the struct construct.)

Related

Memory Alignment warning with gcc

I'm trying to implement a polymorphic data structure e.g. an intrusive linked list (I already know the kernel has one - this is more of learning experience).
The trouble is casting a nested struct to the containing struct leads to gcc issuing a memory-alignment warning.
The specifics are provided below:
// t.c
#include <stdio.h>
enum OL_TYPE { A, B, C };
struct base {
enum OL_TYPE type;
};
struct overlay {
struct base base;
int i;
};
struct overlay2 {
struct base base;
float f;
int i;
// double d; // --> adding this causes a memory alignment warning
};
void testf(struct base *base) {
if (base->type == A) {
struct overlay *olptr = (struct overlay *)base;
printf("overlay->i = %d\n", olptr->i);
} else
if (base->type == B) {
struct overlay2 *olptr = (struct overlay2 *)base;
printf("overlay->i = %d\n", olptr->i);
}
}
int main(int argc, char *argv[]) {
struct overlay ol;
ol.base.type = A;
ol.i = 3;
testf(&ol.base);
}
Compiled with gcc t.c -std=c99 -pedantic -fstrict-aliasing -Wcast-align=strict -O3 -o q leads to this:
t.c: In function ‘testf’:
t.c:28:34: warning: cast increases required alignment of target type [-Wcast-align]
28 | struct overlay2 *olptr = (struct overlay2 *)base;
| ^
It's interesting to notice that if I comment out the double from overlay2 such that it's not part of the structure anymore, the warning disappears.
In light of this, I have a few questions:
Is there any danger to this?
If yes, what is the danger? If not, does that mean the warning is a false positive and how can it be dealt with?
Why does adding the double suddenly lead to the alignment warning?
Is misalignment / an alignment problem even possible if I cast from base to a struct overlay2 * IF the actual type IS in fact a struct overlay2 with a nested struct base ? Please explain!
Appreciate any answers that can provide some insight
is there any danger to this?
Only if what base points to was not allocated as a struct overlay2 in the first place. If you're just smuggling in a pointer to struct overlay2 as a struct base*, and casting back internally, that should be fine (the address would actually be aligned correctly in the first place).
If yes, what is the danger? If not, does that mean the warning is a false positive and how can it be dealt with?
The danger occurs when you allocate something as something other than struct overlay2 then try to use the unaligned double inside it. For example, a union of struct base and char data[sizeof(struct overlay2)] would be the right size to contain a struct overlay2, but it might be four byte aligned but not eight byte aligned, causing the double (size 8, alignment 8) to be misaligned. On x86 systems, a misaligned field typically just slows your code, but on non-x86 misaligned field access can crash your program.
As for dealing with it, you can silence the warning (with some compiler-specific directives), or you can just force all your structures to be aligned identically, by adding alignas directives (available since C11/C++11):
#include <stdalign.h> // Add at top of file to get alignas macro
struct base {
alignas(double) enum OL_TYPE type;
};
why does adding the double suddenly lead to the alignment warning?
Because it's the only data type in any of the structures that requires eight byte alignment; without it, the structures can all be legally aligned to four bytes; with it, overlay2 needs eight byte alignment while the others only need four byte alignment.
is an alignment even possible if I cast from base to a struct overlay2 * IF the actual type IS in fact a struct overlay2 with a nested struct base ?
As noted above, if it was really pointing to something allocated as a struct overlay2 all along, you're safe.
It's not really a false positive, because the compiler doesn't know that your passed base* argument will be the address of an actual overlay2 structure (even though you do). However, if that condition is always going to be true, then no issue should arise due to inappropriate alignment.
However, to be completely sure (and to silence the warning), you could always make the base structure align to the requirements of a double, if that doesn't cause any other issues:
struct base {
_Alignas(double) enum OL_TYPE type;
};
Why does adding the double suddenly lead to the alignment warning
Because, in general, the alignment requirement of any structure must be at least that of the largest alignment requirement of all its members. Without such an assurance, an array of those structures would – by definition – result in misalignment of such members in any one one of two adjacent elements in the array.
Some processors require that pointers to an item of size S must be aligned to the next power-of-two greater than or equal to S, otherwise you get memory alignment faults.
If you take pointer to a variable (including aggregate) that's lesser alignment (lower power-of-two) and cast it to the type of a variable that has higher alignment requirements, you potentially set yourself up for those alignment faults.
If this is likely to be an issue, one option is to put all the item types that will be cast between into a union, always allocate that union, and then reference the type within the union.
That has the potential to get expensive on memory, so another possibility is to create a copy function that initialises a greater-alignment object from the contents of the lesser-alignment one, ensuring any additional values the larger one has will be set to sensible default values.
Finally, and since we're talking about linked lists anyway, the better option is to have a single struct type that comprises the linked items. It will also contain a void * or union thing * pointer to the actual data and an indication of what type of data the pointer refers to. This way you're pretty economical with memory, you don't have to worry about alignment issues, and you have the cleanest and most adaptable representation.

Unaligned memory acces with array

In a C program, having an array that is meant to work as a buffer FooBuffer[] for storing the contents of a data member of a struct like this:
struct Foo {
uint64_t data;
};
I was told that this line might cause unaligned access:
uint8_t FooBuffer[10] = {0U};
I have some knowledge that unaligned access depends on the alignment offset of the processor and in general, it consumes more read/write cycles. Under what circumstances would this cause unaligned memory access and how could I prevent it?
Edit:
A variable of type struct Foo would be stored in the buffer. Particularly, its member data would be split up into eight bytes that would be stored in the array FooBuffer. See attached code with some options for this.
#include <stdio.h>
#include <string.h>
typedef unsigned long uint64;
typedef unsigned char uint8;
struct Foo
{
uint64 data;
};
int main()
{
struct Foo foo1 = {0x0123456789001122};
uint8 FooBuffer[10] = {0U};
FooBuffer[0] = (uint8)(foo1.data);
FooBuffer[1] = (uint8)(foo1.data >> 8);
FooBuffer[2] = (uint8)(foo1.data >> 16);
FooBuffer[3] = (uint8)(foo1.data >> 24);
FooBuffer[4] = (uint8)(foo1.data >> 32);
FooBuffer[5] = (uint8)(foo1.data >> 40);
FooBuffer[6] = (uint8)(foo1.data >> 48);
FooBuffer[7] = (uint8)(foo1.data >> 56);
struct Foo foo2 = {0x9876543210112233};
uint8 FooBuffer2[10] = {0U};
memcpy(FooBuffer2, &foo2, sizeof(foo2));
return 0;
}
However, it is not clear how this process is done since a piece of privative software performs the operation. What would be the scenarios that could result in unaligned memory access after the "conversion"?
Defining either a structure such as struct Foo { uint64_t data; } or an array such as uint8_t FooBuffer[10]; and using them in normal ways will not cause an unaligned access. (Why did you use 10 for FooBuffer? Only 8 bytes are needed for this example?)
A method that novices sometimes attempt that can cause unaligned accesses is attempting to reinterpret an array of bytes as a data structure. For example, consider:
// Get raw bytes from network or somewhere.
uint8_t FooBuffer[10];
CallRoutineToReadBytes(FooBuffer,...);
// Reinterpret bytes as original type.
struct Foo x = * (struct Foo *) FooBuffer; // Never do this!
The problem here is that struct Foo has some alignment requirement, but FooBuffer does not. So FooBuffer could be at any address, but the cast to struct Foo * attempts to force it to an address for a struct Foo. If the alignment is not correct, the behavior is not defined by the C standard. Even if the system allows it and the program “works,” it may be accessing a struct Foo at an improperly aligned address and suffering performance problems.
To avoid this, a proper way to reinterpret bytes is to copy them into a new object:
struct Foo x;
memcpy(&x, FooBuffer, sizeof x);
Often a compiler will recognize what is happening here and, especially if struct Foo is not large, implement the memcpy in an efficient way, perhaps as two load-four-byte instructions or one load-eight-byte instruction.
Something you can do to help that along is ask the compiler to align FooBuffer by declaring it with the _Alignas keyword:
uint8_t _Alignas(Struct Foo) FooBuffer[10];
Note that that might not help if you need to take bytes from the middle of a buffer, such as from a network message that includes preceding protocol bytes and other data. And, even if it does give the desired alignment, never use the * (struct Foo *) FooBuffer shown above. It has more problems than just alignment, one of which is that the C standard does not guarantee the behavior of reinterpreting data like this. (A supported way to do it in C is through unions, but memcpy is a fine solution.)
In the code you show, bytes are copied from foo1.data to FooBuffer using bit shifts. This also will not cause alignment problems; expressions that manipulate data like this work just fine. But there are two issues with it. One is that it nominally manipulates individual bytes one by one. That is perfectly legal in C, but it can be slow. A compiler might optimize it, and there might be built-ins or library functions to assist with it, depending on your platform.
The other issue is that it puts the bytes in an order according to their position values: The low-position-value bytes are put into the buffer first. In contrast, the memcpy method copies the bytes in the order they are stored in memory. Which method you want to use depends on the problem you are trying to solve. To store data on one system and read it back later on the same system, the memcpy method is fine. To send data between two systems using the same byte ordering, the memcpy method is fine. However, if you want to send data from one system on the Internet to another, and the two systems do not use the same byte order in memory, you need to agree on an order to use in the network packages. In this case, it is common to use the arrange-bytes-by-position-value method. Again, your platform may have builtins or library routines to assist with this. For example, the htonl and ntohl routines are BSD routines that take a normal 32-bit unsigned integer and return it with its bytes arranged for network order or vice-versa.

Why is a double member in struct not aligned on 8 byte boundary?

This is about memory alignment. In code below, I expected that the offset of b inside the structure to be 8 (32-bit machine). See here. There by, making b always occur within a cache line. However, that's not the case. The member b in a global object of struct test1 seems to be aligned. I am not sure if its by chance or compiler is doing this deliberately.
I wanted to understand why compiler is not padding 4 bytes after a.
struct test1
{
int a;
double b;
}t1;
int main()
{
struct test1 *p = malloc(sizeof(struct test1));
printf("sizes int %d, float %d, double %d, long double %d\n", sizeof(int), sizeof(float), sizeof(double), sizeof(long double));
printf("offset of b %d\n",(int)&(t1.b)-(int)&(t1));
printf("\naddress of b (on heap) = %p, addr of b (on data seg) = %p\n",&(p->b), &(t1.b));
return 0;
}
The output is...
sizes int 4, float 4, double 8, long double 12
offset of b 4
address of b (on heap) = 0x804a07c, addr of b (on data seg) = 0x80497e0
I am using standard gcc compiler on ubuntu 10.04
According to the System V ABI for i386, page 28, double only gets 4 bytes alignment, but compilers are recommended to provide an option for 8 bytes as well. It appears this is what is implemented by GCC on Linux, the option being called -malign-double.
Another alternative is to use -m64 to get x86-64 object code, which is already the default on some systems including Mac OS X.
I expected that the offset of b inside the structure to be 8 (32-bit machine). See here
Your references explain why it could be advantageous to 8-align doubles. This doesn't amount to a guarantee that doubles will always be 8-aligned. If your sources say that they're always 8-aligned, and you observe an implementation in which they aren't, then your sources are wrong.
From the GCC man page:
Aligning
"double" variables on a two word boundary will produce code that
runs somewhat faster on a Pentium at the expense of more memory.
So GCC's stated reason for 4-alignment is to save memory. You can use -malign-double and -mno-align-double to control that, although of course you risk creating binary incompatibilities if you share structs between code compiled with conflicting options. For particular objects/struct members you can use GCC's own __attribute__((aligned)), _Alignas (in C11) or alignas (in C++11), all of which can take an integer constant to specify the required alignment.
There is no guarantee about alignment, at all, in ANSI C.
The alignment happens with automatic variables more than with anything declared on the heap. If you're on a POSIX OS, use memalign(3) to receive memory that you're sure is aligned. Malloc may return memory at any offset. You can use compiler directives like __attribute__ ((__packed__)) to put in your own alignments.

sizeof sideeffect and allocation location [duplicate]

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Why isn’t sizeof for a struct equal to the sum of sizeof of each member?
I can not understand why is it like this:
#include <stdio.h>
#include <stdlib.h>
typedef struct
{
char b;
int a;
} A;
typedef struct
{
char b;
} B;
int main() {
A object;
printf("sizeof char is: %d\n",sizeof(char));
printf("sizeof int is: %d\n",sizeof(int));
printf("==> the sizeof both are: %d\n",sizeof(int)+sizeof(char));
printf("and yet the sizeof struct A is: %d\n",sizeof(object));
printf("why?\n");
B secondObject;
printf("pay attention that the sizeof struct B is: %d which is equal to the "
"sizeof char\n",sizeof(secondObject));
return 0;
}
I think I explained my question in the code and there is no more need to explain. besides I have another question:
I know there is allocation on the: heap/static heap/stack, but what is that means that the allocation location is unknown, How could it be ?
I am talking about this example:
typedef struct
{
char *_name;
int _id;
} Entry;
int main()
{
Entry ** vec = (Entry**) malloc(sizeof(Entry*)*2);
vec[0] = (Entry *) malloc(sizeof (Entry));
vec[0]->_name = (char*)malloc(6);
strcpy (vec[0]->_name, "name");
vec[0]->_id = 0;
return 0;
}
I know that:
vec is on the stack.
*vec is on the heap.
*vec[0] is on the heap.
vec[0]->id is on the heap.
but :
vec[0]->_name is unknown
why ?
There is an unspecified amount of padding between the members of a structure and at the end of a structure. In C the size of a structure object is greater than or equal to the sum of the size of its members.
Take a look at this question as well as this one and many others if you search for CPU and memory alignment. In short, CPUs are happier if they access the memory aligned to the size of the data they are reading. For example, if you are reading a uint16_t, then it would be more efficient (on most CPUs) if you read at an address that is a multiple of 2. The details of why CPUs are designed in such a way is whole other story.
This is why compilers come to the rescue and pad the fields of the structures in such a way that would be most comfortable for the CPU to access them, at the cost of extra storage space. In your case, you are probably given 3 byte of padding between your char and int, assuming int is 4 bytes.
If you look at the C standard (which I don't have nearby right now), or the man page of malloc, you will see such a phrase:
The malloc() and calloc() functions return a pointer to the allocated memory
that is suitably aligned for any kind of variable.
This behavior is exactly due to the same reason I mentioned above. So in short, memory alignment is something to care about, and that's what compilers do for you in struct layout and other places, such as layout of local variables etc.
You're running into structure padding here. The compiler is inserting likely inserting three bytes' worth of padding after the b field in struct A, so that the a field is 4-byte aligned. You can control this padding to some degree using compiler-specific bits; for example, on MSVC, the pack pragma, or the aligned attribute on GCC, but I would not recommend this. Structure padding is there to specify member alignment restrictions, and some architectures will fault on unaligned accesses. (Others might fixup the alignment manually, but typically do this rather slowly.)
See also: http://en.wikipedia.org/wiki/Data_structure_alignment#Data_structure_padding
As to your second question, I'm unsure what you mean by the name is "unknown". Care to elaborate?
The compiler is free to add padding in structures to ensure that datatypes are aligned properly. For example, an int will be aligned to sizeof(int) bytes. So I expect the output for the size of your A struct is 8. The compiler does this, because fetching an int from an unaligned address is at best inefficient, and at worst doesn't work at all - that depends on the processor that the computer uses. x86 will fetch happily from unaligned addresses for most data types, but will take about twice as long for the fetch operation.
In your second code-snippet, you haven't declared i.
So vec[0]->_name is not unknown - it is on the heap, just like anything else you get from "malloc" (and malloc's siblings).

Are there any guarantees about C struct order?

I've used structs extensively and I've seen some interesting things, especially *value instead of value->first_value where value is a pointer to struct, first_value is the very first member, is *value safe?
Also note that sizes aren't guaranteed because of alignment, whats the alginment value based on, the architecture/register size?
We align data/code for faster execution can we tell compiler not to do this? so maybe we can guarantee certain things about structs, like their size?
When doing pointer arithmetic on struct members in order to locate member offset, I take it you do - if little endian + for big endian, or does it just depend on the compiler?
what does malloc(0) really allocate?
The following code is for educational/discovery purposes, its not meant to be of production quality.
#include <stdlib.h>
#include <stdio.h>
int main()
{
printf("sizeof(struct {}) == %lu;\n", sizeof(struct {}));
printf("sizeof(struct {int a}) == %lu;\n", sizeof(struct {int a;}));
printf("sizeof(struct {int a; double b;}) == %lu;\n", sizeof(struct {int a; double b;}));
printf("sizeof(struct {char c; double a; double b;}) == %lu;\n", sizeof(struct {char c; double a; double b;}));
printf("malloc(0)) returns %p\n", malloc(0));
printf("malloc(sizeof(struct {})) returns %p\n", malloc(sizeof(struct {})));
struct {int a; double b;} *test = malloc(sizeof(struct {int a; double b;}));
test->a = 10;
test->b = 12.2;
printf("test->a == %i, *test == %i \n", test->a, *(int *)test);
printf("test->b == %f, offset of b is %i, *(test - offset_of_b) == %f\n",
test->b, (int)((void *)test - (void *)&test->b),
*(double *)((void *)test - ((void *)test - (void *)&test->b))); // find the offset of b, add it to the base,$
free(test);
return 0;
}
calling gcc test.c followed by ./a.out
I get this:
sizeof(struct {}) == 0;
sizeof(struct {int a}) == 4;
sizeof(struct {int a; double b;}) == 16;
sizeof(struct {char c; double a; double b;}) == 24;
malloc(0)) returns 0x100100080
malloc(sizeof(struct {})) returns 0x100100090
test->a == 10, *test == 10
test->b == 12.200000, offset of b is -8, *(test - offset_of_b) == 12.200000
Update
this is my machine:
gcc --version
i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5666) (dot 3)
Copyright (C) 2007 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
uname -a
Darwin MacBookPro 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun 7 16:33:36 PDT 2011; root:xnu-1504.15.3~1/RELEASE_I386 i386
From 6.2.5/20:
A structure type describes a sequentially allocated nonempty set of member objects
(and, in certain circumstances, an incomplete array), each of which has an optionally
specified name and possibly distinct type.
To answer:
especially *value instead of value->first_value where value is a pointer to struct, first_value is the very first member, is *value safe?
see 6.7.2.1/15:
15 Within a structure object, the non-bit-field members and the units in which bit-fields
reside have addresses that increase in the order in which they are declared. A pointer to a
structure object, suitably converted, points to its initial member (or if that member is a
bit-field, then to the unit in which it resides), and vice versa. There may be unnamed
padding within a structure object, but not at its beginning.1
There may however be padding bytes at the end of the structure as also in-between members.
In C, malloc( 0 ) is implementation defined. (As a side note, this is one of those little things where C and C++ differ.)
[1] Emphasis mine.
I've used structs extensively and I've seen some interesting things, especially *value instead of value->first_value where value is a pointer to struct, first_value is the very first member, is *value safe?
Yes, *value is safe; it yields a copy of the structure that value points at. But it is almost guaranteed to have a different type from *value->first_value, so the result of *value will almost always be different from *value->first_value.
Counter-example:
struct something { struct something *first_value; ... };
struct something data = { ... };
struct something *value = &data;
value->first_value = value;
Under this rather limited set of circumstances, you would get the same result from *value and *value->first_value. Under that scheme, the types would be the same (even if the values are not). In the general case, the type of *value and *value->first_value are of different types.
Also note that sizes aren't guaranteed because of alignment, but is alignment always on register size?
Since 'register size' is not a defined C concept, it isn't clear what you're asking. In the absence of pragmas (#pragma pack or similar), the elements of a structure will be aligned for optimal performance when the value is read (or written).
We align data/code for faster execution; can we tell compiler not to do this? So maybe we can guarantee certain things about structs, like their size?
The compiler is in charge of the size and layout of struct types. You can influence by careful design and perhaps by #pragma pack or similar directives.
These questions normally arise when people are concerned about serializing data (or, rather, trying to avoid having to serialize data by processing structure elements one at a time). Generally, I think you're better off writing a function to do the serialization, building it up from component pieces.
When doing pointer arithmetic on struct members in order to locate member offset, I take it you do subtraction if little endian, addition for big endian, or does it just depend on the compiler?
You're probably best off not doing pointer arithmetic on struct members. If you must, use the offsetof() macro from <stddef.h> to handle the offsets correctly (and that means you're not doing pointer arithmetic directly). The first structure element is always at the lowest address, regardless of big-endianness or little-endianness. Indeed, endianness has no bearing on the layout of different members within a structure; it only has an affect on the byte order of values within a (basic data type) member of a structure.
The C standard requires that the elements of a structure are laid out in the order that they are defined; the first element is at the lowest address, and the next at a higher address, and so on for each element. The compiler is not allowed to change the order. There can be no padding before the first element of the structure. There can be padding after any element of the structure as the compiler sees fit to ensure what it considers appropriate alignment. The size of a structure is such that you can allocate (N × size) bytes that are appropriately aligned (e.g. via malloc()) and treat the result as an array of the structure.
Calling malloc(0) will return a pointer that may be safely passed to free() at least once. If the same value is returned by multiple malloc(0) calls, it may be freed once for each such call. Obviously, if it returns NULL, that could be passed to free() an unlimited number of times without effect. Every call to malloc(0) which returns non-null should be balanced by a call to free() with the returned value.
If you have an inner structure it is guaranteed to start on the same address as the enclosing one if that is the first declaration of the enclosing structure.
So *value and value->first is accessing memory at the same address (but using different types) in the following
struct St {
long first;
} *value;
Also, the ordering between memebers of the structure is guaranteed to be the same as the declaration order
To adjust alignment, you can use compiler specific directives or use bitfields.
The alignment of structure memebers are usually based on what's best to access the individual members on the target platform
Also, for malloc, it is possible it keeps some bookkeeping near the returned address, so even for zero-size memory it can return a valid address (just don't try to access anything via the returned address)
It is important to learn about the way that size of struct works. for example:
struct foo{
int i;
char c;
}
struct bar{
int i;
int j;
}
struct baz{
int i;
char c;
int j;
}
sizeof(foo) = 8 bytes (32 bit arch)
sizeof(bar) = 8 bytes
sizeof(baz) = 12 bytes
What this means is that struct sizes and offsets have to follow two rules:
1- The struct must be a multiple of it's first element (Why foo is 8 not 5 bytes)
2- A struct element must start on a multiple of itself. (In baz, int j could not start on 6, so bytes 6, 7, and 8 are wasted padding

Resources