I'm confused about how to access an array of structs.
simple case:
typedef struct node
{
int number;
struct node *left;
struct node *right;
} node;
node *nodeArray = malloc(sizeof(node));
nodeArray->number = 5;
So, that all makes sense. but the following doesn't work:
typedef struct node
{
int number;
struct node *left;
struct node *right;
} node;
node *nodeArray = malloc(511 * sizeof(node));
for(int i = 0; i < 511; i++)
{
nodeArray[i]->number = i;
}
However, nodeArray[i].number = i does seem to work can someone explain what's going on and also what's the difference between node *nodeArray = malloc(511 * sizeof(node)); and node (*nodeArray) = malloc(511 * sizeof(node));
In the first snippet, the following are all equivalent:
nodeArray->number = 5; // preferred
nodeArray[0].number = 5;
(*nodeArray).number = 5;
In the second snippet, the following are all equivalent:
(nodeArray + i)->number = i;
nodeArray[i].number = i; // preferred
(*(nodeArray + i)).number = i;
So, as you can see, there is a choice of three different syntaxes that all do the same thing. The arrow syntax (nodeArray->number) is preferred when dealing with a pointer to a single instance of the struct. The array indexing with dot notation (nodeArray[i].number) is preferred when dealing with a pointer to an array of structs. The third syntax (dereferencing the pointer and dot notation) is avoided by sensible programmers.
When you allocate an array like this
node* nodeArray = malloc(511*sizeof(node));
nodeArray is a pointer, getting a pointer to an individual struct node you just add an integer:
nodeArray + 1 would give a pointer to the second node
nodeArray + 1 can be written as &nodeArray[1]
so to dereference the pointer
*(nodeArray + 1).number or write nodeArray[1].number
May be the problem is caused by alignment:
Your node structure contains an integer and two pointers, its minimum storage size could be 12 bytes (on most 32-bit architectures) or 24 bytes (64-bit architectures) but the alignment constraints of the architecture may force each node to be aligned using another maximum storage size (with extra padding, which needs to be allocated too.
sizeof(type) just returns a minimum storage size (the extra allocated padding should not be accessible, even if this is not checked at runtime or by the compiler).
Solution: use calloc() which will also take into consideration the alignment constraints for each item in your array!
Replace:
node *nodeArray = malloc(511 * sizeof(node));
by:
node *nodeArray = calloc(511, sizeof(node));
and now your code is normally safe, the actually allocated size will include the necessary additional padding required by the underlying architecture.
Otherwise your code is not portable.
Note that some C/C++ compiler also provide a alignof(type) to get the correct alignment for the datatype (and it should be used for implementing void *calloc(size_t nitems, size_t size) in the C/C++ libraries).
Your sample code above may suffer from buffer overflows because you did not allocate enough space for the array before writing items in the loop.
You don't see the difference when you use simple types (you don't care about their alignment or where they are are isolately allocated, there's possibly extra padding allocated on the stack or in structures using them, which is not accessible, even if no padding is necessary when their storage is allocated inside physical registers; but even with "auto" or "register" allocation, the compiler may still allocate space on the stack for it, as a backing store that could be used to save the register when it is needed for something else or before performing an external function call, or method call in C++ and the function body is not inlined).
See the documentation of alignofand alignas declarators in C++11. There are many resources about them; for example:
https://en.cppreference.com/w/cpp/language/alignas
See as well the documentation of calloc()
(And don't be confused by the simplified 32-bit or 64-bit memory models used in Linux; even Linux uses now more precise memory models, taking into account alignment problems, as well as accessibility and performance problems, sometimes enforced by the underlying platform for good security reasons in order to reduce a surface of attacks that exists in the single/unified "flat" memory model for everything: segmented architectures are coming back in the computing industry, and C/C++ compilers had to adapt: C++11 replies to this problematic that otherwise would require costlier or inefficient solutions in the compiled code, severely limiting some optimizations such as cache management, efficiency of TLB stores, paging and virtualized memory, enforced security scopes for users/process/threads and so on).
Remember that each datatype has its own size and alignment and they are independent. The assumption that there's a single "size" to allocate for a datatype in an array is wrong (as well extra padding at end of the allocated array, after its last item, may not be allocated, and read/write access to padding areas may be restricted/enforced by the compiler or at runtime).
Now consider also the case of bitfields (datatypes declared as members of structures with an extra precision/size parameter): their sizeof() is not the true minimum as they can be packed more tightly (including arrays of booleans: sizeof() returns the minimum size of the datatype once it has been promoted to an integer and so when it has possibly been enlarged with extra padding or extension of the sign bit; usually the compiler enforces theses invalid accesses to padding bits by using bitmasking, shifts or rotations; but a processor may provide more convenient instructions to handle bits inside a word unit in memory or even in a register, so that your bitfields won't overflow and modify other surrounding bitfields or padding bits because of an arithmetic operation on their value).
As well your nodeArray[i] returns a reference to a node object, not a pointer, so nodeArray[i]->anything is invalid: you need to replace the -> by a ..
Related
I'm quit confused with the difference between flexible arrays and pointer as struct members. Someone suggested, struct with pointers need malloc twice. However, consider the following code:
struct Vector {
size_t size;
double *data;
};
int len = 20;
struct Vector* newVector = malloc(sizeof *newVector + len * sizeof*newVector->data);
printf("%p\n",newVector->data);//print 0x0
newVector->data =(double*)((char*)newVector + sizeof*newVector);
// do sth
free(newVector);
I find a difference is that the address of data member of Vector is not defined. The programmer need to convert to "find" the exactly address. However, if defined Vector as:
struct Vector {
size_t size;
double data[];
};
Then the address of data is defined.
I am wondering whether it is safe and able to malloc struct with pointers like this, and what is the exactly reason programmers malloc twice when using struct with pointers.
The difference is how the struct is stored. In the first example you over-allocate memory but that doesn't magically mean that the data pointer gets set to point at that memory. Its value after malloc is in fact indeterminate, so you can't reliably print it.
Sure, you can set that pointer to point beyond the part allocated by the struct itself, but that means potentially slower access since you need to go through the pointer each time. Also you allocate the pointer itself as extra space (and potentially extra padding because of it), whereas in a flexible array member sizeof doesn't count the flexible array member. Your first design is overall much more cumbersome than the flexible version, but other than that well-defined.
The reason why people malloc twice when using a struct with pointers could either be that they aren't aware of flexible array members or using C90, or alternatively that the code isn't performance-critical and they just don't care about the overhead caused by fragmented allocation.
I am wondering whether it is safe and able to malloc struct with pointers like this, and what is the exactly reason programmers malloc twice when using struct with pointers.
If you use pointer method and malloc only once, there is one extra thing you need to care of in the calculation: alignment.
Let's add one extra field to the structure:
struct Vector {
size_t size;
uint32_t extra;
double *data;
};
Let's assume that we are on system where each field is 4 bytes, there is no trailing padding on struct and total size is 12 bytes. Let's also assume that double is 8 bytes and requires alignment to 8 bytes.
Now there is a problem: expression (char*)newVector + sizeof*newVector no longer gives address that is divisible by 8. There needs to be manual padding of 4 bytes between structure and data. This complicates the malloc size calculation and data pointer offset calculation.
So the main reason you see 1 malloc pointer version less, is that it is harder to get right. With pointer and 2 mallocs, or flexible array member, compiler takes care of necessary alignment calculation and padding so you don't have to.
I have read articles and SO answers about how we can cutoff compiler added padding if we layout the structs in the decreasing order of size of independent fields.
eg: Instead of laying out (L1) the struct like this
typedef struct dummy {
char c1;
short s1;
int i1;
unsigned int ui1;
double d1;
} dummy_t;
create the layout (L2) like this
typedef struct dummy {
double d1;
unsigned int ui1;
int i1;
short s1;
char c1;
} dummy_t;
My question here is, are there scenarios where layout L2 has downsides when compared to L1 or any other layout ? Or can I take it into my design rules to always use L2 ? Also is the natural alignment rule
natural alignment = sizeof(T)
always true for primitive types ?
You should wonder about alignement/structure size only if you are running on a memory constrained system (you want to limit the size of each element). This makes sense only for systems/designs where you know you will store a lot of such structure and used storage space is an issue. On the other hand grouping the content of a structure logically can bring clarity to your code. I would always favor clarity against performance, especially at early design phases. Even more if you do not have performance related constraints.
About performance it might happen that accessing "random" content from L2 is worse than from L1 because memory cache alignement constraints can also come into play. (eg accessing in a row 2 elements of struct L1 can be slower or quicker than 2 elements of L2). For such optimization, profiling is the only way to know which layout is the best.
My question here is, are there scenarios where layout L2 has downsides when compared to L1 or any other layout ?
Sometimes you need to have members in a different order. Reasons for this may include:
The structure is part of a communications protocol and has to be sent byte-by-byte across a network or other communication device. In this case, the layout of the structure may be dictated by the protocol, and you will have to match it exactly (including padding or lack thereof).
The structure is part of a family of structures that share initial sequences of members, so those members must be placed first. This is sometimes done to allow structures to be handled in a “generic” way by operating on those initial members only.
The structure includes a buffer whose length will vary (by being allocated dynamically according to needs that arise during program execution). Such a buffer is implemented with a flexible array member, which must be the last member in the structure.
Also, there may be incidental effects of how members are ordered. For example, if member x happens to be used much more frequently than other members of the structure, putting it at the front might allow the compiler to access it with simpler address arithmetic, since its offset from the beginning of the structure will be zero. This is rarely a consideration in programming, but I mention it for completeness.
Aside such considerations, you are generally free to order members as you desire.
Also is the natural alignment rule natural alignment = sizeof(T) always true for primitive types ?
No. As an example, an eight-byte long might have an alignment require of one, two, four, or eight bytes.
… we can cutoff compiler added padding if we layout the structs in the decreasing order of size of independent fields.
This is not true for members that are aggregates (arrays, structures, and unions). Consider that a member char x[13]; is 13 bytes in size but only requires alignment of one byte. To minimize padding, order members in order of decreasing alignment requirement, not decreasing size.
IMO abstracting from the implementation details like caching (which is too broad to be discussed in the post answer) there is no difference.
The difference is if you place the variable sized (or zero sized) object at the end of the struct (example):
typedef struct
{
size_t size;
char data[];
}string;
string *create(size_t size)
{
string *ptr = malloc(sizeof(*ptr) + size);
if(ptr)
{
ptr -> size = size;
}
return ptr;
}
If course the logical member order (and potential packing) is required if struct will store some received binary data (for example communication packet header)
typedef struct Node {
int data;
struct Node *next;
} node;
pointer->next = (node*)malloc(sizeof(node));
How many bytes of memory are dynamically given to pointer->next in the above code. For (int*)malloc(sizeof(int)), 2 bytes are given. Likewise how many for node?
Malloc will dinamically assign the size of "node".
Node is a struct and the size of every struct depends on the size of every element inside the struct.
In this case, the size of node will be: size of int + size of struct Node*
(If the result is not multiple of 2, it will be padded for architecture reasons)
Your device has an architecture of 2 bytes, and for that reason, the size of the structs can only be 2, 4, 6, 8 etc...
The size of int depends on the target you are working on. Since your architecture is 16 bits, the size of int is 2 bytes.
About. the size of struct Node *, you need to know that EVERY pointer data types have exactly the same size, it doesn't matter the data type their are pointing to. And that size also depends on the architecture. Again, your architecture is 16 bits and that's why the size of struct node * is 2 bytes.
size of int = 2.
size of struct node * = 2
Total memory assigned by malloc = 2 + 2 = 4
First, a suggestion: rewrite
pointer->next=(node*)malloc(sizeof(node));
as
pointer->next = malloc( sizeof *pointer->next );
You don't need the cast (unless you're working on a pre-ANSI implementation, in which case God help you), and using the dereferenced target as the operand of sizeof means you don't have to specify the type, potentially saving you some maintenance heartburn.
Also, a little whitespace goes a long way (although you don't need to put whitespace around the function arguments - that's my style, some people don't like it, but it makes things easier for me to read).
How much bytes of memory is dynamically given to pointer->next
It will be at least as big as sizeof (int) plus sizeof (struct Node *), and potentially may be bigger; depending on your platform, it could be as small as 4 bytes or as large as 16. C allows for "padding" bytes between struct members to satisfy alignment requirements for the underlying architecture. For example, a particular architecture may require that all multi-byte objects be aligned on addresses that are multiples of 4; if your data member is only 2 bytes wide, then there will be 2 unused bytes between it and the next member.
Without knowing a lot about your system, we just can't tell you. You can take that same code and try it on multiple compilers, and you'll get different answers. You have to check yourself, using sizeof(node) or sizeof(struct Node) (I think either syntax works, but just in case).
I think I have a basic understanding of how skip lists work, but being new to them in addition to being a C-beginner has me confused on a few points, especially the initialization of the list. Here's the code I'm trying to follow:
#define MAXSKIPLEVEL 5
typedef struct Node {
int data;
struct Node *next[1];
} Node;
typedef struct SkipList {
Node *header;
int level;
} SkipList;
// Initialize skip list
SkipList* initList() {
SkipList *list = calloc(1, sizeof(SkipList));
if ((list->header = calloc(1, sizeof(Node) + MAXSKIPLEVEL*sizeof(Node*))) == 0) {
printf("Memory Error\n");
exit(1);
}
for (int i = 0; i < MAXSKIPLEVEL; i++)
list->header->next[i] = list->header;
return list;
}
I haven't done anything with arrays of pointers yet in C, so I think I'm getting a bit caught up with how they work. I have a few questions if someone would be kind enough to help me out.
First, I did sizeof(int) and got 4, sizeof(Node*) and got 8, so I expected sizeof(Node) to equal 12, but it ended up being 16, why is this? Same confusion with the size of SkipList compared to the sizes of its contents. I took the typedef and [1] out to see if either of them was the cause, but the size was still 16.
Second, why is the [1] in struct Node *next[1]? Is it needed for the list->header->next[i] later on? Is it okay that next[i] will go higher than 1? Is it just because the number of pointers for each node is variable, so you make it an array then increase it individually later on?
Third, why does list->header->next[i] = list->header initially instead of NULL?
Any advice/comments are greatly appreciated, thanks.
For your first question - why isn't the size of the struct the size of its members? - this is due to struct padding, where the compiler, usually for alignment reasons, may add in extra blank space between or after members of a struct in order to get the size up to a nice multiple of some fundamental size (often 8 or 16). There's no portable way to force the size of the struct to be exactly the size of its members, though most compilers have some custom switches you can flip to do this.
For your second question - why the [1]? - the idea here is that when you actually allocate one of the node structs, you'll overallocate the space so that the memory at the end of the struct can be used for the pointers. By creating an array of length one and then overallocating the space, you make it syntactically convenient to access this overallocated space as though it were a part of the struct all along. Newer versions of C have a concept called flexible array members that have supplanted this technique; I'd recommend Googling it and seeing if that helps out.
For your final question - why does list->header->next[i] initially point to list->header rather than NULL? - without seeing more of the code it's hard to say. Many implementations of linked list structures use some sort of trick like this to avoid having to special-case on NULL in the implementation, and it's entirely possible that this sort of trick is getting used here as well.
The sizeof number is 16 because of structure padding.
Many architectures either require or strongly prefer their pointers to be aligned on a certain boundary (e.g., 4-byte boundary, 8-byte boundary, etc.) They will either fail, or they will perform slowly, if pointers are "misaligned". Your C compiler is probably inserting 4 unused bytes in the middle of your structure so that your 8-byte pointer is aligned on an 8-byte boundary, which causes the structure size to increase by 4 bytes.
There is more explanation available from the C FAQ.
I practice in realization a memory manager in C.
I want the structure, that has a various length and self-described.
So, I peep at a POSIX textbook something, like that:
struct layout
{
uint32_t size; // array size in bytes, include space after the struct
uchar_t data[1];
};
// But, is next line correct?
layout *val = malloc (array_memory_in_bytes + sizeof (uint32_t) - 1);
// Where does a static array keep the pointer for using it?
If I have several these structures one-after-one in uninterrupted piece of memory, and I want be able to iterate through them. Can I write something, like that:
layout *val1 = pointer;
layout *val2 = val1 + val1.size + sizeof (val1.size);
Or can you recommend me a better approach?
The Standard C version of this is called flexible array member and it looks like:
struct layout
{
uint32_t size;
uchar_t data[];
};
// allocate one of these blocks (in a function)
struct layout *val = malloc( sizeof *val + number_of_bytes );
val->size = number_of_bytes;
The code val1->data + val1->size will get you a pointer one-past-the-end of the space you just malloc'd.
However you cannot iterate off the end of one malloc'd block and hope to hit another malloc'd block. To implement this idea you would have to malloc a large block and then place various struct layout objects throughout it, being careful about alignment.
In this approach, it's probably best to also store an index of where each struct layout is. In theory you could go through the list from the start each time, adding on size and then doing your alignment adjustment; but that would be slow and also it would mean you could not cope with a block in the middle being freed and re-"allocated".
If this is meant to be a drop-in replacement for malloc then there are in fact two alignment considerations:
alignment for struct layout
data must be aligned for any possible type
The simplest way to cope with this is to align struct layout for any possible type also. This could look like (note: #include <stdint.h> required):
struct layout
{
uint64_t size; // may as well use 64 bits since they're there
_Alignas(max_align_t) uchar_t data[];
};
An alternative might be to keep size at 32-bit and throw in a pragma pack to prevent padding; then you'll need to use some extra complexity to make sure that the struct layout is placed 4 bytes before a max_align_t-byte boundary, and so on. I'd suggest doing it the easy way first and get your code running; then later you can go back and try this change in order to save a few bytes of memory if you want.
Alternative approaches:
Keep each instance of a struct layout plus its trailing data in a separate allocation.
Change data be a pointer to malloc'd space; then you could keep all of the struct layout objects in an array.
The general idea will work, but that specific struct will only work if the most-severe boundary alignment case is an int.
A memory manager, particularly one that might be a back-end for an implementation of malloc(), must know what that worst-case boundary is. The actual start of data must be on that boundary in order to satisfy the general requirement that the allocated memory be suitably aligned for the storage of any data type.
The easiest way to get that done is to make the length allocation header described by the layout struct and the actual allocation sizes all multiples of that alignment unit.
No matter what, you can't describe the start of data as a struct member and have the size of that struct be the size of the header. C doesn't support zero-length fields. You should use something to put that array on boundary, and use the offsetof() macro from <stddef.h>.
Personally, I'd use a union, based on both old habits and occasional use of Visual C++ for C. But uint32_t is a C99 type and if you also have C11 support you can use _Alignas(). With that, your struct could look something like:
#define ALIGN_TYPE double /* if this is the worst-case type */
#define ALIGN_UNIT ((sizeof)(ALIGN_TYPE))
#define ALIGN_SIZE(n) (((size_t)(n) + ALIGN_UNIT - 1) & ~(ALIGN_UNIT-1))
typedef struct layout
{
size_t size; /* or use uint32_t if you prefer */
_Alignas(ALIGN_UNIT) char data[1];
} layout;
#define HEADER_SIZE (offsetof(layout, data))
That makes most everything symbolic except for the worst-case alignment type. You'd allocate the combined header plus data array with:
layout *ptr = (layout*) malloc(HEADER_SIZE + ALIGN_SIZE(number_of_bytes));
ptr->size = HEADER_SIZE;
The ALIGN_SIZE type really isn't a symbolic constant, though, unless C99/C11 changed the definition of sizeof. You can't use to compute ordinary array dimensions, for example. You can hard code a literal number, like 8 for a typical double, if that's a problem. Beware that long double has a problematical size (10 bytes) on many x86 implementations. If you're going to base the allocation unit on a type, then long double might not be your best choice.