I am using a library that has a function that takes an array of structs. That struct and function has the following layout:
struct TwoInt32s
{
int32_t a;
int32_t b;
};
void write(struct TwoInt32s *buffer, int len);
My initial tests suggest that an array of such structs has the same memory layout as an array of int32_t so I can do something like this:
int32_t *buffer = malloc(2 * len * sizeof(int32_t));
/* fill in the buffer */
write((struct TwoInt32s*)buffer, len);
However I'm wondering if this is universally true or not. Using an array of int32_t greatly simplifies my code.
EDIT: I forgot the sizeof
From what I read, C guarantees a few things about struct padding:
members will NOT be reordered
padding will only be added between members with different alignments or at the end of the struct
a pointer to a struct points to the same memory location as a pointer to its first member
each member is aligned in a manner appropriate for its type
there may be unnamed holes in the struct as necessary to achieve alignment
From this I can extrapolate that a and b have no padding between them. However it's possible that the struct will have padding at the end. I doubt this since it's word-aligned on both 32 and 64 bit systems. Does anyone have additional information on this?
The implementation is free to pad structs - there may be unused bytes in between a and b. It is guaranteed that the first member isn't offset from the beginning of the struct though.
Typically you manage such layout with a compiler-specific pragma, e.g:
#pragma pack(push)
#pragma pack(1)
struct TwoInt32s
{
int32_t a;
int32_t b;
};
#pragma pack(pop)
malloc allocates bytes. Why did you choose "2*len" ?
You could simply use "sizeof":
int32_t *buffer = malloc(len * sizeof(TwoInt32s));
/* fill in the buffer */
write((struct TwoInt32s*)buffer, len);
and as Erik mentioned, it would be a good practice to pack the struct.
It's safest to not cast, but convert -- i.e., create a new array and fill it with the values found in the struct, then kill the struct.
You could allocate structures but treat their members as a sort of virtual array:
struct TwoInt32s *buffer = malloc(len * sizeof *buffer);
#define BUFFER(i) (*((i)%2 ? &buffer[(i)/2].b : &buffer[(i)/2].a))
/* fill in the buffer, e.g. */
for (int i = 0; i < len * 2; i++)
BUFFER(i) = i;
Unfortunately, neither GCC nor Clang currently "get" this code.
Related
I have set up structs in this way.
When I print the sizeof(DATA), I get 16. Why is it 16? I thought it would be 8 as 4 for idx, and 4 for the ptr.
I allocate memory for the STUDENT struct and STATS struct array of size 50. Have I allocated enough memory?
typedef struct {
int idNum;
int classNum;
} STATS;
typedef struct {
STATS * stats;
int currGrade;
}STUDENT;
typedef struct {
STUDENT * ptr;
int idx;
} DATA;
//student_main.c
void function1()
{
DATA d;
func(&d);
}
//student.c
void func(DATA * d)
{
Student * s = malloc(sizeof(STUDENT));
d->ptr = s;
d->ptr->currGrade = 1;
STATS * arr = malloc(sizeof(STATS)* 50);
d->ptr->stats = arr;
d->ptr->stats[0].idNum = 1;
d->ptr->stats[0].classNum = 1;
}
I have set up structs in this way. When I print the sizeof(DATA), I
get 16. Why is it 16? I thought it would be 8 as 4 for idx, and 4 for
the ptr.
Because compiler (may) add padding between/behind struct elements to satisfy alignment constraints. You can read something about data structure alignment here.
In your case of DATA, its 16B because (if your machine is x64)
typedef struct
{
STUDENT * ptr; // 8B pointer
int idx; // 4B int
// 4B padding
} DATA;
I allocate memory for the STUDENT struct and STATS struct array of
size 50. Have I allocated enough memory?
It depends on your needs. You have allocated enough space for 50 structures. Remember to always use sizeof(type) while allocating dynamical memory.
The compiler will pad structures, for example to allow faster access to subsequent elements in 64 bit architectures.
You shouldn't ever need to worry about the numeric value of the size of a struct, as the compiler handles it. Use it only for comparisons, and ignore the absolute value (unless your hobby is reverse engineering the compiler)
It can be 16 for one of two reasons - either You have 32 bit machine and the compiler adds some "space" in Your structure to make the operations faster. You can stop this behavior if You want your structures to be of the exact size, by adding __attribute__ ((__packed__)) to the struct definition. Or You have 64 bit machine with 64 bit pointers and integers. Or a mix of both.
You have allocated enough memory. Doing malloc(x * sizeof(struct xxx)) gives You the confidence.
I recommend reading this excellent article: The Lost Art of C Structure Packing
Frankly, is such a code valid or does it invoke undefined behavior?
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
struct two_values
{
int some;
char value;
};
int main(void) {
int some = 5;
char value = 'a';
unsigned char *data = malloc(sizeof(struct two_values));
memcpy(data, &some, sizeof(int));
memcpy(data+sizeof(int), &value, sizeof(char));
struct two_values dest;
memcpy(&dest, data, sizeof(struct two_values));
printf("some = %d, value = %c\n", dest.some, dest.value);
return 0;
}
http://ideone.com/4JbrP9
Can I just put the binary representation of two struct field together and reinterpret this as the whole struct?
You had better to not disturb the internal compiler doings in your code, as it would lead you to incorrect code and undefined behaviour. You can switch compilers, or just updating the version of your favourite, and run into trouble.
The best way to solve the thing you show of having two variables and to store them properly in the struct fields is to use properly the types provided by C, and use a pointer typed to the proper type. If you use
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
struct two_values
{
int some;
char value;
};
int main(void) {
int some = 5;
char value = 'a';
/* next instead of unsigned char *data = malloc(sizeof(struct two_values)); */
struct two_values *data = malloc(sizeof(struct two_values));
/* next instead of memcpy(data, &some, sizeof(int)); */
data->some = some;
/* next instead of memcpy(data+sizeof(int), &value, sizeof(char)); */
data->value = value;
struct two_values dest;
/* next instead of memcpy(&dest, data, sizeof(struct two_values)); */
dest = *data;
printf("some = %d, value = %c\n", dest.some, dest.value);
return 0;
}
You'll avoid all compiler alignment issues. It is always possible to do it with the language operators & (address of) and * (points to) or -> (field of struct pointed to).
Anyway, if you prefer the memcpy approach (no idea of why, but you are on your way, anyway) you can substitute:
data->some = some;
...
data->value = value;
...
dest = *data;
by
memcpy(&data->some, &some, sizeof data->some);
...
memcpy(&data->value, &value, sizeof data->value);
...
memcpy(&dest, data, sizeof dest);
And that will take internally the alignments that the compiler could make by itself.
All compilers have defined some pragma, or keyword, to control alignment. This is also nonportable, as you can switch compilers and get to the issue of having to change the way you expressed things. C11 has some standard means to control for packed structs and use no alignment in the compiler. This is done mainly when you have to serialize some structure and don't want to deal with holes on it. Look at the C11 specs for that.
Serializing structs is not completely solved by just making them packed, as normally you have to deal with the serialized representations of integer, floating point or char data (which can or cannot coincide with the internal representation used by the compiler) so you again face the problem of being compiler agnostic and have to think twice before using externally the internal representation of data.
My recomendation anyway, is never trust how the compiler stores data internally.
The padding is determined by the compiler. The order is guaranteed. If you need something similar to your code above, I would recommend the offsetof-macro in <stddef.h>.
memcpy(data + offsetof(struct two_values, value), &value, sizeof(char));
Or without explicitly adding the offset at all:
memcpy(&data->value, &value, sizeof(char));
It depend on how your structure is aligned. You can check by verifying sizeof(two_values), if it comes 5(assuming sizeof int is 4), you probably are ok.
If its more than that it implies filler bytes are inserted in your structure to align each element of your structure at correct byte boundry
May I assume that struct fields are placed in order
Yes, this is guaranteed by the standard. C11 6.2.5/20:
a structure is a type consisting of a sequence of members, whose
storage is allocated in an ordered sequence
and with no padding?
No, you cannot assume this. C11 6.7.1/15:
Within a structure object, the non-bit-field members and the units in which bit-fields reside have addresses that increase in the order in which they are declared. /--/
There may be unnamed padding within a structure object, but not at its beginning.
Padding and alignment are implementation-defined behavior.
You are however guaranteed that two structs of the same type have the same padding. Copying from a struct to another struct of same type, as in your example, is safe and well-defined.
I'm programming for embedded, resource constrained devices in C.
I have a structure, like this:
typedef struct UjThread{
struct{
UInt32 runInstr;
UInt8* mailbox;
}appBucket;
struct{
UInt32 appId;
UInt32 numInstr;
UInt32 allocMem;
UInt32 eepromStartAddr;
}appContract;
UInt16 spBase; //we use an empty ascending stack
UInt16 spLimit; //also used for "isPtr"
UInt16 localsBase;
UInt32 stack[];
}UjThread;
I start a thread per object and allocate the needed memory (92 bytes for this structure, but I haven't shown all fields).
However, some objects won't use the internal appContract and appBucket structures but memory for those structures will still be allocated.
Is there a way to avoid this? To designate internal structures as optional or perhaps extract the size of those internal structures and subtract it from the memory allocation?
I could make two separate structures, one per type of object, but I'd prefer not to since I'd have to adjust my code everywhere to work with the two types of threads.
Besides the obvious - using two structs, I see only two other possibilities.
Either use a pointer to a separately allocated appContract, or if some of the data you need is mutually exclusive, use a union.
Consider this implementation of single inheritance that works in C.
Define a base struct that contains all the elements common to both objects. Notice that I've changed the type of the stack member to a pointer because that's going to have to be a separate allocation in this design.
typedef struct ThreadBase{
UInt16 spBase; //we use an empty ascending stack
UInt16 spLimit; //also used for "isPtr"
UInt16 localsBase;
UInt32 *stack;
}ThreadBase;
Then declare another struct that contains the base object as the first member and appends the extra stuff.
typedef struct ThreadExtra{
ThreadBase base;
struct{
UInt32 runInstr;
UInt8* mailbox;
}appBucket;
struct{
UInt32 appId;
UInt32 numInstr;
UInt32 allocMem;
UInt32 eepromStartAddr;
}appContract;
}ThreadExtra;
Now you can define a ThreadBase object for threads that only need the base stuff. And you can define a ThreadExtra object for the threads that need more. But you can cast the ThreadExtra object to ThreadBase because ThreadBase is the first member of ThreadExtra. So in general purpose code that doesn't deal with the ThreadExtra elements you can treat all the Thread objects as if they are ThreadBase objects.
If your optional fields are at the beginning of your struct, you can adjust the address of an allocated object, so the optional fields reside in unallocated memory. Use the offsetof macro to determine where the mandatory data starts:
offsetof(UjThread, spBase) // in bytes
Adjust the allocation size by this amount:
UjThread *newThread;
if (NoOptionalFields())
{
size_t sizeReduce = offsetof(UjThread, spBase);
size_t size = sizeof(UjThread) - sizeReduce;
newThread = (void*)((char*)malloc(size) - sizeReduce);
}
else
{
newThread = malloc(sizeof(UjThread));
}
To free the memory, don't forget to adjust the pointer back:
if (NoOptionalFields())
{
size_t sizeReduce = offsetof(UjThread, spBase);
free((char*)newThread + sizeReduce);
}
else
{
free(newThread);
}
BTW since you have a "flexible array member" in your struct, the actual size calculation is more complicated than in my example. But you get the idea - just subtract the size of the optional fields from both the allocation size and the resulting pointer.
If the stack had a fixed size, you could use the idiomatic C-style single inheritance:
typedef struct {
int a;
} Base;
typedef struct {
Base base;
int b;
} Derived;
void useBase(Base *);
void test(void) {
Base b;
Derived d;
useBase(&b);
useBase(&d.base); // variant 1
useBase((Base*)&d); // variant 2
}
Alas, the stack doesn't have a fixed size, so the somewhat idiomatic if unnecessarily shaky variant 2 won't work, but variant 1 will:
typedef struct {
int a[];
} Small;
typedef struct {
int b;
Small small;
} Large;
void useBase(Base *);
void test(void) {
Small s;
Large l;
useBase(&s);
useBase(&l.small);
}
I want the size of a C struct to be multiple of 16 bytes (16B/32B/48B/..).
It does not matter which size it gets to; it only needs to be multiple of 16 bytes.
How could I enforce the compiler to do that?
For Microsoft Visual C++:
#pragma pack(push, 16)
struct _some_struct
{
...
}
#pragma pack(pop)
For GCC:
struct _some_struct { ... } __attribute__ ((aligned (16)));
Example:
#include <stdio.h>
struct test_t {
int x;
int y;
} __attribute__((aligned(16)));
int main()
{
printf("%lu\n", sizeof(struct test_t));
return 0;
}
compiled with gcc -o main main.c will output 16. The same goes for other compilers.
The size of a C struct will depend on the members of the struct, their types and how many of them there are. There is really no standard way to force the compiler to make structs to be a multiple of some size. Some compilers provide a pragma that will allow you to set the alignment boundary however that is really a different thing. And there may be some that would have such a setting or provide such a pragma.
However if you insist on this one method would be to do memory allocation of the struct and to force the memory allocation to round up to the next 16 byte size.
So if you had a struct like this.
struct _simpleStruct {
int iValueA;
int iValueB;
};
Then you could do something like the following.
{
struct _simpleStruct *pStruct = 0;
pStruct = malloc ((sizeof(*pStruct)/16 + 1)*16);
// use the pStruct for whatever
free(pStruct);
}
What this would do is to push the size up to the next 16 byte size so far as you were concerned. However what the memory allocator does may or may not be to give you a block that is actually that size. The block of memory may actually be larger than your request.
If you are going to do something special with this, for instance lets say that you are going to write this struct to a file and you want to know the block size then you would have to do the same calculation used in the malloc() rather than using the sizeof() operator to calculate the size of the struct.
So the next thing would be to write your own sizeof() operator using a macro such as.
#define SIZEOF16(x) ((sizeof(x)/16 + 1) * 16)
As far as I know there is no dependable method for pulling the size of an allocated block from a pointer. Normally a pointer will have a memory allocation block that is used by the memory heap management functions that will contain various memory management information such as the allocated block size which may actually be larger than the requested amount of memory. However the format for this block and where it is located relative to the actual memory address provided will depend on the C compiler's run time.
This depends entirely on the compiler and other tools since alignment is not specified that deeply in the ISO C standard (it specifies that alignment may happen at the compilers behest but does not go into detail as to how to enforce it).
You'll need to look into the implementation-specific stuff for your compiler toolchain. It may provide a #pragma pack (or align or some other thing) that you can add to your structure defininition.
It may also provide this as a language extension. For example, gcc allows you to add attributes to a definition, one of which controls alignment:
struct mystruct { int val[7]; } __attribute__ ((aligned (16)));
You could perhaps do a double struct, wrapping your actual struct in a second one that can add padding:
struct payload {
int a; /*Your actual fields. */
float b;
char c;
double d;
};
struct payload_padded {
struct payload p;
char padding[16 * ((sizeof (struct payload) + 15) / 16)];
};
Then you can work with the padded struct:
struct payload_padded a;
a.p.d = 43.3;
Of course, you can make use of the fact that the first member of a structure starts 0 bytes from where the structure starts, and treat a pointer to struct payload_padded as if it's a pointer to a struct payload (because it is):
float d_plus_2(const struct payload *p)
{
return p->d + 2;
}
/* ... */
struct payload_padded b;
const double dp2 = d_plus_2((struct payload *) &b);
Hi I have the following scenario
#define CONSTANT 10
struct structA{
int var1;
int var2[CONSTANT];
};
main(){
structA *varA = NULL;
int i;
/* some C code */
varA = mmap(0,..);
for(i = 0; i < CONSTANT; i++){
varA.var2[i] = /* C code */ ;
}
/* C code */
}
Here the number of elements in var2 array of structA is constant and hence the varA.var2[i] can be directly referenced . But If the number CONSTANT is to be determined dynamically during runtime , how should the structure definition be modified?
The cleanest way would be to use a C99 flexible array member:
struct structA{
int var1;
int var2[];
};
Then you can just mmap(NULL, sizeof(int) * (num + 1), ...). Just be careful with flexible array members, sizeof behaves funny with them (doesn't include their size).
If you can't / won't use a flexible array member, you can use a pointer instead of an array and mmap memory separately for it.
The most common way to do this is:
struct structA{
int var1;
int var2[1];
};
It's not really an array with a single element, because you allocate more memory than the size of a structA.
The number CONSTANT cannot be determined dynamically at runtime, since it affects the structure size.
The usual trick for what you're trying to do (I'm not sure if it's strictly valid C but it's very common) is to create
struct structA{
int var1;
int var2[0];
};
and when you map or allocate, allocate (sizeof(structA) + sizeof(int) * CONSTANT) bytes.