So I'm trying to write a buffering library for the 64th time and I'm starting get into some pretty advanced stuff. Thought I'd ask for some proffesional input on this.
In my first header file I have this:
typedef struct StdBuffer { void* address; } StdBuffer;
extern void StdBufferClear(StdBuffer);
In another header file that #includes the first header file I have this:
typedef struct CharBuffer { char* address; } CharBuffer;
void (*CharBufferClear)(CharBuffer) = (void*) StdBufferClear;
Will declaring this function pointer void interfere with the call? They have matching by value signatures. I have never seen a function pointer declared void before, but its the only way to get it to compile cleanly.
Stackwise it should not make any difference at all from what I learned in assembler coding.
irrelevent OMG! I just said Stackwise on StackOverflow!
Hmm.. Looks like I've assumed too much here. Allow me to reclarify if I may. I don't care what 'type' of data is stored at the address. All that I am concerned with is the size of a 'unit' and how many units are at the address. Take a look at the interface agreement contract for the API if you will:
typedef struct StdBuffer {
size_t width; ///< The number of bytes that complete a data unit.
size_t limit; ///< The maximum number of data units that can be allocated for this buffer.
void * address; ///< The memory address for this buffer.
size_t index; ///< The current unit position indicator.
size_t allocated; ///< The current number of allocated addressable units.
StdBufferFlags flags;///< The API contract for this buffer.
} StdBuffer;
You see, memcpy, memmove and the like don't really care whats at an address all they want is the specifics which I'm clearly keeping track of here.
Have a look now at the first prototype to follow this contract:
typedef struct CharBuffer {
size_t width; ///< The number of bytes that complete a data unit.
size_t limit; ///< The maximum number of data units that can be allocated for this buffer.
char * address; ///< The memory address for this buffer.
size_t index; ///< The current unit position indicator.
size_t allocated; ///< The current number of allocated addressable units.
CharBufferFlags flags;///< The API contract for this buffer.
} CharBuffer;
As you an clearly see the data type is irrelevant in this context. You can say that C handles it differently depending on the case, but at the end of the day, an address is an address, a byte is byte and a long is a long for as long as we are dealing with memory on the same machine.
The purpose of this system when brought together is to remove all of this type based juggling C seems to be so proud of (and rightfully so...) Its just pointless for what I would like to do. Which is create a contract abiding prototype for any standard size of data (1, 2, 4, 8, sizeof(RandomStruct)) located at any address.
Having the ability to perform my own casting with code and manipulate that data with api functions that operate on specific length blocks of memory with specific length memory units. However, the prototype must contain the official data pointer type, because it just doesn't make sense for the end user to have to recast their data every time they would like to do something with that address pointer. It would not make sense to call it a CharBuffer if the pointer was void.
The StdBuffer is a generic type that is never EVER used except within the api itself, to manage all contract abiding data types.
The api that this system will incorporate is from my latest edition of buffering. Which is quite clearly documented here #Google Code I am aware that some things will need to change to bring this all together namely I won't have the ability to manipulate data directly from within the api safely without lots of proper research and opinion gathering.
Which just brought to my attention that I also need a Signed/Unsigned bit flag in the StdBufferFlags Members.
Perhaps the final piece to this puzzle is also in order for your perusal.
/** \def BIT(I)
\brief A macro for setting a single constant bit.
*
* This macro sets the bit indicated by I to enabled.
* \param I the (1-based) index of the desired bit to set.
*/
#define BIT(I) (1UL << (I - 1))
/** \enum StdBufferFlags
\brief Flags that may be applied to all StdBuffer structures.
* These flags determine the contract of operations between the caller
* and the StdBuffer API for working with data. Bits 1-4 are for the
* API control functions. All other bits are undefined/don't care bits.
*
* If your application would like to use the don't care bits, it would
* be smart not to use bits 5-8, as these may become used by the API
* in future revisions of the software.
*/
typedef enum StdBufferFlags {
BUFFER_MALLOCD = BIT(1), ///< The memory address specified by this buffer was allocated by an API
BUFFER_WRITEABLE = BIT(2), ///< Permission to modify buffer contents using the API
BUFFER_READABLE = BIT(3), ///< Permission to retrieve buffer contents using the API
BUFFER_MOVABLE = BIT(4) ///< Permission to resize or otherwise relocate buffer contents using the API
}StdBufferFlags;
This code requires a diagnostic:
void (*CharBufferClear)(CharBuffer) = (void*) StdBufferClear;
You're converting a void * pointer to a function pointer without a cast. In C, a void * pointer can convert to pointers to object types without a cast, but not to function pointer types. (In C++, a cast is needed to convert void * to object types also, for added safety.)
What you want here is just to cast between function pointer types, i.e.:
void (*CharBufferClear)(CharBuffer) = (void (*)(CharBuffer)) StdBufferClear;
Then you are still doing the same type punning because the functions are different types. You are trying to call a function which takes a StdBuffer using a pointer to a function which takes a CharBuffer.
This type of code is not well-defined C. Having defeated the type system, you're on your own, relying on testing, examining the object code, or obtaining some assurances from the compiler writers that this sort of thing works with that compiler.
What you learned in assembler coding doesn't apply because assembly languages have only a small number of rudimentary data types such as "machine address" or "32 bit word". The concept that two data structures with an identical layout and low-level representation might be incompatible types does not occur in assembly language.
Even if two types look the same at the low level (another example: unsigned int and unsigned long are sometimes exactly the same) C compilers can optimize programs based on the assumption that the type rules have not been violated. For instance suppose that A and B point to the same memory location. If you assign to an an object A->member, a C compiler can assume that the object B->member is not affected by this, if A->member and B->member have incompatible types, like one being char * and the other void *. The generated code keeps caching the old value of B->member in a register, even though the in-memory copy was overwritten by the assignment to A->member. This is an example of invalid aliasing.
The standard does not define the results of casting a function-pointer to void *.
Equally, converting between function pointers and then calling through the wrong one is also undefined behaviour.
There are some constructs which any standards-conforming C compiler are required to implement consistently, and there are some constructs which 99% of C compilers do implement consistently, but which standards-conforming compilers would be free to implement differently. Attempting to cast a pointer to a function which takes one type of pointer, into a pointer to a function which takes another type of pointer, falls into the latter category. Although the C standard specifies that a void* and a char* must be the same size, there is nothing that would require that they share the same bit-level storage format, much less the parameter-passing convention. While most machines allow bytes to be accessed in much the same way as words, such ability is not universal. The designer of an application-binary-interface [the document which specifies among other things how parameters are passed to routines] might specify that a char* be passed in a way which maximizes the efficiency of byte access, while a void* should be passed in a way that maximizes the efficiency of word access while retaining the ability to hold an unaligned byte address, perhaps by using a supplemental word to hold a zero or one to indicate LSB/MSB). On such a machine, having a routine that expects a void* called from code that expects to pass a char* could cause the routine to access arbitrary wrong data.
No, it doesn't matter what data type is used to store the data. It only matters the type C uses to read and write that data, and that the data is of sufficient size.
Related
I've recently been working on wifi sniffer using ESP8266, in order to sniff wifi packets there is a function called wifi_set_promiscuous_rx_cb(wifi_sniffer_packet_handler) it takes a callback function as a parameter and passes buffer pointer, which has the packet info and length of the packet as parameters to the callback function , wifi_sniffer_packet_handler(uint8_t *buff, uint16_t len) is the call back function. i am not understanding what these two statements are doing
const wifi_promiscuous_pkt_t *ppkt = (wifi_promiscuous_pkt_t *)buff;
const wifi_ieee80211_packet_t *ipkt = (wifi_ieee80211_packet_t *)ppkt->payload;
wifi_promiscuous_pk_t is a structure
typedef struct{
wifi_pkt_rx_ctrl_t rx_ctrl; /**< metadata header */
uint8_t payload[0]; /**< Data or management payload. Length of payload is described by rx_ctrl.sig_len. Type of content determined by packet type argument of callback. */
} wifi_promiscuous_pkt_t;
and wifi_ieee80211_packet_t is another structure
typedef struct
{
wifi_ieee80211_mac_hdr_t hdr;
uint8_t payload[2]; /* network data ended with 4 bytes csum (CRC32) */
} wifi_ieee80211_packet_t;
how the data in the *buff is assigned to these structures, are the above statements responsible for the assignment
i've seen many stackoverflow questions and many other threads regarding this but none of the posts clarified my doubt
The packet coming from the radio presumably has a format like this:
| metadata | mac hdr | mac payload |
This is stored initially as a uint8_t array pointed to by buff.
Now, you typecast the buffer into the first structure type:
const wifi_promiscuous_pkt_t *ppkt = (wifi_promiscuous_pkt_t *)buff;
This is what our buffer conceptually looks like now:
|<---------------------- buff ---------------------->| buff
|<-- rx_ctrl -->|<------- payload ------------------>| ppkt
The structure is kind of overlaid on top of the byte array and since the metadata is always a fixed size, ppkt->rx_ctrl will now contain this metadata header.
But what about payload? The declaration indicates payload to be a zero sized array. This is special syntax to tell the compiler that this is a 'placeholder' for an arbitrarily large array of user-managed memory. (Please note that this is an outdated GCC extension and might produce compile errors on most modern compilers. The post C99 approach is to simply leave out the array dimension like this: uint8_t payload[])
Hence ppkt->payload is now an array starting with the first byte beyond the metadata. Since C doesn't bother with bounds checking, you can access any byte of this payload with ppkt->payload[i]
Now, we need to get to the mac payload by parsing out the mac header, which is again of a fixed size. This is accomplished by:
const wifi_ieee80211_packet_t *ipkt = (wifi_ieee80211_packet_t *)ppkt->payload;
In this case, we discard the header (rx_ctrl) and overlay the wifi_ieee80211_packet_t struct on top of the payload portion of the buffer.
Looking at our buffer again, this is what has happened to it:
|<---------------------- buff ---------------------->| buff
|<-- rx_ctrl -->|<------- payload ------------------>| ppkt
|<-- hdr -->|<----- payload -------->| ipkt
Now you can easily get to the mac payload in the original packet with ipkt->payload.
This is a very common idiom used when parsing packets in networking protocol stacks.
PS: However, please note that typecasting between incompatible types is risky business. Firstly, you have to understand and account for the alignment of structure members. Also, depending on how your structures have been defined, this might result in violation of strict aliasing rules and hence, invoke undefined behaviour on certain compilers.
The goal of converting from a pointer to bytes that have been read from the network to a pointer to a structure is to interpret the bytes as that structure type. This relies on compiler features that are not required by the C standard.
The code (wifi_promiscuous_pkt_t *)buff converts the pointer buff to a pointer to the type wifi_promiscuous_pkt_t. This relies on buff being aligned as required for the structure,1 and it requires that the conversion produce a pointer to the same place in memory but with a different type.2
Then const wifi_promiscuous_pkt_t *ppkt = (wifi_promiscuous_pkt_t *)buff; defines a new pointer ppkt and initializes it to point to the same memory buff points to. The goal is to have references such as ppkt->rx_ctrl interpret the data as the data for the wifi_promiscuous_pkt_t structure instead of as the raw uint8_t bytes.
The C standard does not define the behavior here.3 Compilers may support this by defining the behavior themselves. Notably, the compiler needs to support aliasing memory: Accessing memory using a type other than the original type used to define or store data there.
For illustration, let’s say the type of rx_ctrl, wifi_pkt_rx_ctrl_t, is a four-byte int. Then, when these requirements above are met, ppkt->rx_ctrl_t will access the first four bytes at buff and interpret them as if they were an int.
(If those requirements are not met, then a C-standard way to reinterpret the bytes is to make a new object and copy the data into it: wifi_promiscuous_pkt_t temp; memcpy(temp, buff, sizeof temp + extra for the payload length);, after which temp.rx_ctrl will provide the int value. However, depending on circumstances and compiler quality, that can cause a lot of extra memory copying we would like to avoid. Accessing the memory directly is preferable, if it is supported. Another alternative is to read the data directly from the network into the target object instead of into a buffer of bytes.)
After this, const wifi_ieee80211_packet_t *ipkt = (wifi_ieee80211_packet_t *)ppkt->payload; does the same thing with ppkt->payload instead of buff. Once ppkt has been set up, ppkt->payload points to the bytes starting after the rx_ctrl member.4 This pointer is then converted to point to the type wifi_ieee80211_packet_t, and ipkt is initialized with the new pointer.
As before, the intent is that ipkt->hdr and ipkt->payload will refer to the data in the first structure’s payload area as if that data contained a wifi_ieee80211_packet_t structure. However, there is another problem here. As noted above, buff may pointed to well-aligned memory if it was allocated with malloc. But ipkt does not point to the same place as buff. It points to ppkt->payload, which is some number of bytes after the start because it follows the rx_ctrl member. So we do not know, from this code alone, that ipkt is properly aligned for a wifi_ieee80211_packet_t. Possibly it is, if the size of the rx_ctrl member is a multiple of the alignment requirement for the hdr member.
Without comments in the code speaking to this, this requirement may have been neglected. The code might work because rx_ctrl is an okay size for this to work, but it might generate an alignment trap. If not now, then possibly in the future when the types involved change.
Footnotes
1 If buff points to the start of memory allocated with malloc or another standard memory allocation routine, it will be correctly aligned for any fundamental type. Or, if it was defined as an array, correct alignment can be requested with the standard _Alignas keyword or possibly with a compiler extension. If it was merely defined as an ordinary array of uint8_t, is not guaranteed to be aligned as required for this use.
2 For general pointer-to-object conversions, the standard only requires that a pointer converted to another pointer-to-object type and then back to its original type points to the original object. It does not guarantee the pointer is otherwise usable in the other type. But this is common in C implementations.
3 In general, accessing an array of uint8_t using other types violates the rule in C 2018 6.5 7, which says that memory which has been defined with a certain type, or, for dynamic memory, assigned with a certain type, shall be accessed only by certain “matching” types or by a “character” (byte) type.
4 Formally, ppkt->payload designates an array, and that array is automatically converted to a pointer. Also, the array is declared to have zero elements. That is not defined by the C standard, but GCC and other compilers supported it as a way to allow variable length data at the end of a structure. Since C 1999, the standard way to do this is to declare the member with [] instead of [0].
When I used the structure:
|<-- rx_ctrl -->|<------- payload ------------------>|
I get a misalignment in the payload data by a shift of 2 bytes to the left.
When I replace rx_ctrl part with a field e.g. unsigned frame_ctrl:16 I do not see this shift. I do have to typecast frame_ctrl to rx_ctrl to get the details.
I assume this is because I use esp32-S2 which assumes teh rx_ctrl is 32 bits regardless of its definition as 16 bits.
I want to detect if a value of a void* type input function argument is an integer or a pointer. Currently, I'm passing the integer only (this is Midnight Commander source):
mc_event_raise (MCEVENT_GROUP_CORE,
"clipboard_file_from_ext_clip",
(void*)(intptr_t)clip_id);
However I would also like to sometimes pass a string (char *) in the same parameter, as extending the API would be difficult. I know that this is wrong, however, maybe there is a fully legal way to obtain such effect ? My ideas:
– the addresses are in general aligned to 4 or 8 bytes, so I've could skip such values from the clip_id (jump over them) and then decode the value? if (data % 8) then clip_id = decode_clip_id(data); ?
– maybe pointers cannot be as small as 10…20 and a simple check would be sufficient?
– other ideas…?
A problem like this can be solved with a structure containing a union and a tag.
struct EventData {
enum { EV_IS_INT, EV_IS_PTR } type;
union {
intptr_t v_int;
void * v_ptr;
};
};
So, you can then pass a pointer to this EventData, and the callback can decode whether to access v_int or v_ptr, based on the value of type.
struct EventData *ed = makeEventData (EV_IS_INT, clip_id);
mc_event_raise (MCEVENT_GROUP_CORE,
"clipboard_file_from_ext_clip",
ed);
If the event is completed asynchronously, you may need to create the event dynamically, and then the callback function would have to release the memory when the callback is done with the argument.
Windows does a lot of this hackery with their HANDLEs and resource IDs.
Your integer needs to have a defined valid range. Anything outside of that range is probably a pointer.
Almost all operating systems reserve low memory areas. You can probably assume that the 16-bit range 0 to 65535 is not a pointer. Beyond that gets iffy.
If your code needs to be portable to things like embedded RTOS or freestanding environments then you cannot assume anything about pointers.
In any case you will probably want some sort of function to create values to pass into your void* which will have an assert or abort to avoid accidentally creating invalid values.
I recently wrote a wrapper for LevelDB in C and stumbled about the following problem. The LevelDB function to store data in a database looks like this:
leveldb_put(leveldb_t* db, const leveldb_writeoptions_t* options, const char* key, size_t keylen, const char* val, size_t vallen, char** errptr);
For the key and value, they use a char*. That means I would have to cast arguments that aren't char pointers. This happens often because I often store structs in the database.
After thinking about this I decided to use a void* for key and data in my wrapper function. It then looks something like this:
int db_put(db_t db, void *key, size_t keylen, void *value, size_t valuelen)
{
char *k = (char*)key;
char *v = (char*)value;
/* Call leveldb_put() here with k and v as parameters. */
return 0;
}
This way I don't have to cast the arguments I pass to my db_put() function. I think this solution is more elegant, but I guess LevelDB knew what they were doing when they choose the char pointers.
Is there a reason not to use void* to pass arbitrary data to a function?
Is there a reason not to use void* to pass arbitrary data to a
function?
No. In fact, void * exists to facilitate passing arbitrary data without the need for ugly casting. That's why ptr-to-void was standardized. In C at least. C++ is a different beast.
At LevelDB they have to deal with historical code born with char * , or pre C89 compilers, or any other veiled reason causing refactoring-inertia. Their code would work with ptrs-to-void just as well.
Note that in your version of db_put the casts should be removed as they are redundant.
The current accepted answer exists only to flatter the OP; It's actually slightly invalid.
Consider that your arbitrary struct may (most likely) have padding bytes somewhere. The value of those padding bytes is indeterminate and may or may not be insignificant.
Consider what might happen if you put a struct as key that has padding bytes, and you then attempt to get the value for that key which is otherwise equal except for the padding bytes.
Consider also how you might handle pointer members, if you choose to do so in the future.
If you intend to use a struct as key, it would be a good idea to serialise it, so you can guarantee retrieval of the corresponding value without worrying about those padding bits.
Perhaps you could pass a function pointer telling your wrapper how to serialise the key into a string...
A void* can be considered a black box that holds a pointer. By holding it
in a void* you are effectively saying that you don't care or it contains at that point, so this will allow you to make any assumption about it.
Void* is a "true" generic pointer, and can be directly assign to any particular data type without using cast.
Meanwhile, a char* explicitly specify the type of the respective object. Initially there was no char* and char* was also used to represent generic pointers. When char* is used an explicit cast is required, however the usage of char* as a generic pointer is not recommanded, because it may create confusion, like it did back there when it was hard to tell if a char* contains a string or some generic data.
Also, is legal to perform arithmeticon a char*, but not on a void*.
The downside of using void*, is given by their main usage, they can hide the actual type of the data you're storing, which prevents the compiler and other stuff to detect type errors.
In your specific situation there is no problem in using void* instead of char*, so you can use void* without worries.
Edit: Updated and reformuled the answer to correct some wrong info
You should be serializing to some standard format like json instead of dealing with raw data like that. It looks very error prone unless you always assume that the arbitrary data is just a byte buffer. In which case I would use uint8_t pointer (which is an unsigned char*) and cast all data structures to it so that the routine just thinks that it is dealing with a byte buffer.
A note on void*: i almost never ever use them. Think carefully when you introduce void pointers because in most cases you can do away with the right way of doing things which takes advantage of the standard types to avoid future bugs. The only places where you should use void* is in places like malloc() where there is not really a better way.
I am writing a library for various mathematical computations in C. Several of these need some "scratch" space -- memory that is used for intermediate calculations. The space required depends on the size of the inputs, so it cannot be statically allocated. The library will typically be used to perform many iterations of the same type of calculation with the same size inputs, so I'd prefer not to malloc and free inside the library for each call; it would be much more efficient to allocate a large enough block once, re-use it for all the calculations, then free it.
My intended strategy is to request a void pointer to a single block of memory, perhaps with an accompanying allocation function. Say, something like this:
void *allocateScratch(size_t rows, size_t columns);
void doCalculation(size_t rows, size_t columns, double *data, void *scratch);
The idea is that if the user intends to do several calculations of the same size, he may use the allocate function to grab a block that is large enough, then use that same block of memory to perform the calculation for each of the inputs. The allocate function is not strictly necessary, but it simplifies the interface and makes it easier to change the storage requirements in the future, without each user of the library needing to know exactly how much space is required.
In many cases, the block of memory I need is just a large array of type double, no problems there. But in some cases I need mixed data types -- say a block of doubles AND a block of integers. My code needs to be portable and should conform to the ANSI standard. I know that it is OK to cast a void pointer to any other pointer type, but I'm concerned about alignment issues if I try to use the same block for two types.
So, specific example. Say I need a block of 3 doubles and 5 ints. Can I implement my functions like this:
void *allocateScratch(...) {
return malloc(3 * sizeof(double) + 5 * sizeof(int));
}
void doCalculation(..., void *scratch) {
double *dblArray = scratch;
int *intArray = ((unsigned char*)scratch) + 3 * sizeof(double);
}
Is this legal? The alignment probably works out OK in this example, but what if I switch it around and take the int block first and the double block second, that will shift the alignment of the double's (assuming 64-bit doubles and 32-bit ints). Is there a better way to do this? Or a more standard approach I should consider?
My biggest goals are as follows:
I'd like to use a single block if possible so the user doesn't have to deal with multiple blocks or a changing number of blocks required.
I'd like the block to be a valid block obtained by malloc so the user can call free when finished. This means I don't want to do something like creating a small struct that has pointers to each block and then allocating each block separately, which would require a special destroy function; I'm willing to do that if that's the "only" way.
The algorithms and memory requirements may change, so I'm trying to use the allocate function so that future versions can get different amounts of memory for potentially different types of data without breaking backward compatibility.
Maybe this issue is addressed in the C standard, but I haven't been able to find it.
The memory of a single malloc can be partitioned for use in multiple arrays as shown below.
Suppose we want arrays of types A, B, and C with NA, NB, and NC elements. We do this:
size_t Offset = 0;
ptrdiff_t OffsetA = Offset; // Put array at current offset.
Offset += NA * sizeof(A); // Move offset to end of array.
Offset = RoundUp(Offset, sizeof(B)); // Align sufficiently for type.
ptrdiff_t OffsetB = Offset; // Put array at current offset.
Offset += NB * sizeof(B); // Move offset to end of array.
Offset = RoundUp(Offset, sizeof(C)); // Align sufficiently for type.
ptrdiff_t OffsetC = Offset; // Put array at current offset.
Offset += NC * sizeof(C); // Move offset to end of array.
unsigned char *Memory = malloc(Offset); // Allocate memory.
// Set pointers for arrays.
A *pA = Memory + OffsetA;
B *pB = Memory + OffsetB;
C *pC = Memory + OffsetC;
where RoundUp is:
// Return Offset rounded up to a multiple of Size.
size_t RoundUp(size_t Offset, size_t Size)
{
size_t x = Offset + Size - 1;
return x - x % Size;
}
This uses the fact, as noted by R.., that the size of a type must be a multiple of the alignment requirement for that type. In C 2011, sizeof in the RoundUp calls can be changed to _Alignof, and this may save a small amount of space when the alignment requirement of a type is less than its size.
If the user is calling your library's allocation function, then they should call your library's freeing function. This is very typical (and good) interface design.
So I would say just go with the struct of pointers to different pools for your different types. That's clean, simple, and portable, and anybody who reads your code will see exactly what you are up to.
If you do not mind wasting memory and insist on a single block, you could create a union with all of your types and then allocate an array of those...
Trying to find appropriately aligned memory in a massive block is just a mess. I am not even sure you can do it portably. What's the plan? Cast pointers to intptr_t, do some rounding, then cast back to a pointer?
The latest C11 standard has the max_align_t type (and _Alignas specifier and _Alignof operator and <stdalign.h> header).
GCC compiler has a __BIGGEST_ALIGNMENT__ macro (giving the maximal size alignment). It also proves some extensions related to alignment.
Often, using 2*sizeof(void*) (as the biggest relevant alignment) is in practice quite safe (at least on most of the systems I heard about these days; but one could imagine weird processors and systems where it is not the case, perhaps some DSP-s). To be sure, study the details of the ABI and calling conventions of your particular implementation, e.g. x86-64 ABI and x86 calling conventions...
And the system malloc is guaranteed to return a sufficiently aligned pointer (for all purposes).
On some systems and targets and some processors giving a larger alignment might give performance benefit (notably when asking the compiler to optimize). You may have to (or want to) tell the compiler about that, e.g. on GCC using variable attributes...
Don't forget that according to Fulton
there is no such thing as portable software, only software that has been ported.
but intptr_t and max_align_t is here to help you....
Note that the required alignment for any type must evenly divide the size of the type; this is a consequence of the representation of array types. Thus, in the absence of C11 features to determine the required alignment for a type, you can just estimate conservatively and use the type's size. In other words, if you want to carve up part of an allocation from malloc for use storing doubles, make sure it starts at an offset that's a multiple of sizeof(double).
I have a linked list, which stores groups of settings for my application:
typedef struct settings {
struct settings* next;
char* name;
char* title;
char* desc;
char* bkfolder;
char* srclist;
char* arcall;
char* incfold;
} settings_row;
settings_row* first_profile = { 0 };
#define SETTINGS_PER_ROW 7
When I load values into this structure, I don't want to have to name all the elements. I would rather treat it like a named array -- the values are loaded in order from a file and placed incrementally into the struct. Then, when I need to use the values, I access them by name.
//putting values incrementally into the struct
void read_settings_file(settings_row* settings){
char* field = settings + sizeof(void*);
int i = 0;
while(read_value_into(field[i]) && i++ < SETTINGS_PER_ROW);
}
//accessing components by name
void settings_info(settings_row* settings){
printf("Settings 'profile': %s\n", settings.title);
printf("Description: %s\n", settings.desc);
printf("Folder to backup to: %s\n", settings.bkfolder);
}
But I wonder, since these are all pointers (and there will only ever be pointers in this struct), will the compiler add padding to any of these values? Are they guaranteed to be in this order, and have nothing between the values? Will my approach work sometimes, but fail intermittently?
edit for clarification
I realize that the compiler can pad any values of a struct--but given the nature of the struct (a struct of pointers) I thought this might not be a problem. Since the most efficient way for a 32 bit processor to address data is in 32 bit chunks, this is how the compiler pads values in a struct (ie. an int, short, int in a struct will add 2 bytes of padding after the short, to make it into a 32 bit chunk, and align the next int to the next 32 bit chunk). But since a 32 bit processor uses 32 bit addresses (and a 64 bit processor uses 64 bit addresses (I think)), would padding be totally unnecessary since all of the values of the struct (addresses, which are efficient by their very nature) are in ideal 32 bit chunks?
I am hoping some memory-representation / compiler-behavior guru can come shed some light on whether a compiler would ever have a reason to pad these values
Under POSIX rules, all pointers (both function pointers and data pointers) are all required to be the same size; under just ISO C, all data pointers are convertible to 'void *' and back without loss of information (but function pointers need not be convertible to 'void *' without loss of information, nor vice versa).
Therefore, if written correctly, your code would work. It isn't written quite correctly, though! Consider:
void read_settings_file(settings_row* settings)
{
char* field = settings + sizeof(void*);
int i = 0;
while(read_value_into(field[i]) && i++ < SETTINGS_PER_ROW)
;
}
Let's assume you're using a 32-bit machine with 8-bit characters; the argument is not all that significantly different if you're using 64-bit machines. The assignment to 'field' is all wrong, because settings + 4 is a pointer to the 5th element (counting from 0) of an array of 'settings_row' structures. What you need to write is:
void read_settings_file(settings_row* settings)
{
char* field = (char *)settings + sizeof(void*);
int i = 0;
while(read_value_into(field[i]) && i++ < SETTINGS_PER_ROW)
;
}
The cast before addition is crucial!
C Standard (ISO/IEC 9899:1999):
6.3.2.3 Pointers
A pointer to void may be converted to or from a pointer to any incomplete or object
type. A pointer to any incomplete or object type may be converted to a pointer to void
and back again; the result shall compare equal to the original pointer.
[...]
A pointer to a function of one type may be converted to a pointer to a function of another
type and back again; the result shall compare equal to the original pointer. If a converted
pointer is used to call a function whose type is not compatible with the pointed-to type,
the behavior is undefined.
In many cases pointers are natural word sizes, so the compiler is unlikely to pad each member, but that doesn't make it a good idea. If you want to treat it like an array you should use an array.
I'm thinking out loud here so there's probably many mistakes but perhaps you could try this approach:
enum
{
kName = 0,
kTitle,
kDesc,
kBkFolder,
kSrcList,
kArcAll,
kIncFold,
kSettingsCount
};
typedef struct settings {
struct settings* next;
char *settingsdata[kSettingsCount];
} settings_row;
Set the data:
settings_row myRow;
myRow.settingsData[kName] = "Bob";
myRow.settingsData[kDescription] = "Hurrrrr";
...
Reading the data:
void read_settings_file(settings_row* settings){
char** field = settings->settingsData;
int i = 0;
while(read_value_into(field[i]) && i++ < SETTINGS_PER_ROW);
}
It's not guaranteed by the C standard. I've a sneaking suspicion, that I don't have time to check right now either way, that it guarantees no padding between the char* fields, i.e. that consecutive fields of the same type in a struct are guaranteed to be layout-compatible with an array of that type. But even if so, you're on your own between the settings* and the first char*, and also between the last char* and the end of the struct. But you could use offsetof to deal with the first issue, and I don't think the second affects your current code.
However, what you want is almost certainly guaranteed by your compiler, which somewhere in its documentation will set out its rules for struct layout, and will almost certainly say that all pointers to data are word sized, and that a struct can be the size of 8 words without additional padding. But if you want to write highly portable code, you have to use only the guarantees in the standard.
The order of fields is guaranteed. I also don't think you'll see intermittent failure - AFAIK the offset of each field in that struct will be consistent for a given implementation (meaning the combination of compiler and platform).
You could assert that sizeof(settings*) == sizeof(char*) and sizeof(settings_row) == sizeof(char*)*8. If both those hold, there is no room for any padding in the struct, since fields are not allowed to "overlap". If you ever hit a platform where they don't hold, you'll find out.
Even so, if you want an array, I'd be inclined to say use an array, with inline accessor functions or macros to get the individual fields. Whether your trick works or not, it's even easier not to think about it at all.
Although not a duplicate, this probably answers your question:
Why isn't sizeof for a struct equal to the sum of sizeof of each member?
It's not uncommon for applications to write an entire struct into a file and read it back out again. But this suffers from the possibility that one day the file will need to be read back on another platform, or by another version of the compiler that packs the struct differently. (Although this can be dealt with by specially-written code that understands the original packing format).
Technically, you can rely only on the order; the compiler could insert padding. If different pointers were of different size, or if the pointer size wasn't a natural word size, it might insert padding.
Practically speaking, you could get away with it. I wouldn't recommend it; it's a bad, dirty trick.
You could achieve your goal with another level of indirection (what doesn't that solve?), or by using a temporary array initialized to point to the various members of the structure.
It's not guaranteed, but it will work fine in most cases. It won't be intermittent, it will either work or not work on a particular platform with a particular build. Since you're using all pointers, most compilers won't mess with any padding.
Also, if you wanted to be safer, you could make it a union.
You can't do that the way you are trying. The compiler is allowed to pad any and all members of the struct. I do not believe it is allowed to reorder the fields.
Most compilers have an attribute that can be applied to the struct to pack it (ie to turn it into a collection of tightly packed storage with no padding), but the downside is that this generally affects performance. The packed flag will probably allow you to use the struct the way you want, but it may not be portable across various platforms.
Padding is designed to make field access as efficient as possible on the target architecture. It's best not to fight it unless you have to (ie, the struct goes to a disk or over a network.)
It seems to me that this approach creates more problems than it solves.
When you read this code six months from now, will you still be aware of all the subtleties of how the compiler pads a struct?
Would someone else, who didn't write the code?
If you must use the struct, use it in the canonical way and just write a function which
assigns values to each field separately.
You could also use an array and create macros to give field names to indices.
If you get too "clever" about optimizing your code, you will end up with slower code anyway, since the compiler won't be able to optimize it as well.