I'm creating some functions that work with a struct that simulates an ethernet header.
typedef struct ethernet_hdr_ {
dir_mac_t dst_mac;
dir_mac_t src_mac;
short type;
char payload[MAX_SIZE_PAYLOAD];
unsigned int FCS;
} ethernet_hdr_t;
#define ETH_HDR_SIZE_EXCL_PAYLOAD (sizeof(ethernet_hdr_t) - sizeof(((ethernet_hdr_t *)0)->payload))
I need to define the next function.
static inline ethernet_hdr_t * ALLOC_ETH_HDR_WITH_PAYLOAD(char *pkt, unsigned int pkt_size)
According to this assignment, which I already did, but I want to know whether there is something wrong with my solution.
The above API must encapsulate the existing DATA into the payload of
ethernet header, i.e. above API must return a pointer to the ethernet
hdr the payload of which carries the data of size pkt_size pointed by
pkt in above diagram.
The layout of data with new ethernet hdr must look like given in the
image Q3 attached in the instruction of this assignment. Initialize
all the fields of the new ethernet hdr (including FCS) exclusing
payload to zero.
This is Q3.
This is how I did it.
static inline ethernet_hdr_t * ALLOC_ETH_HDR_WITH_PAYLOAD(char *pkt, unsigned int pkt_size) {
ethernet_hdr_t *ethernet_hdr = calloc(1, sizeof(ethernet_hdr_t));
memcpy(ethernet_hdr->payload, pkt, pkt_size);
return ethernet_hdr;
}
However, I found the official solution to this assignment, and it is very different from mine.
static inline ethernet_hdr_t * ALLOC_ETH_HDR_WITH_PAYLOAD(char *pkt, unsigned int pkt_size) {
char *temp = calloc(1, pkt_size);
memcpy(temp, pkt, pkt_size);
ethernet_hdr_t *eth_hdr = (ethernet_hdr_t *)(pkt - ETH_HDR_SIZE_EXCL_PAYLOAD);
memset((char *)eth_hdr, 0, ETH_HDR_SIZE_EXCL_PAYLOAD);
memcpy(eth_hdr->payload, temp, pkt_size);
free(temp);
return eth_hdr;
}
Clearly, my function is much simpler but I think it is missing something. So, I'm wondering whether both solutions are correct, but even if they are, maybe the second one is better.
In your case you are creating a new object to store the packet header and payload (frame) data, setting the header to zeros and copying the payload from pkt to the new frame data object payload member.
In the "official" solution, the program assumes that pkt points to the payload member of an object of type ethernet_hdr_t. Then, in a very complicated fashion, it attempts to do the same thing as your code, but using the original object of which pkt is a member.
Overall, I would say that the "official" solution is the problematic for several reasons:
It assumes that the compiler will arrange the contents of struct ethernet_hdr_ in a specific way, but this is actually defined by the implementation. This is due to the macro ETH_HDR_SIZE_EXCL_PAYLOAD using the object and member payload sizes to attempt to locate payload within the struct. A safer way might be to use the standard library offsetof macro to determine the offset of member payload within the struct.
Assuming that pkt is a member of another object type is not a good programming practice. For example, assume that the payload contents were copied to another object, which is not a member of ethernet_hdr_t prior to calling ALLOC_ETH_HDR_WITH_PAYLOAD. This would result in a memory access violation. The function should only attempt to access the objects provided as parameters and not objects that these objects "may" be members of.
The use of temp in this function does not make much sense. The payload data is effectively copied to temp then copied back to pkt indirectly through the object pointed to by eth_hdr.
The object generating the inputs to ALLOC_ETH_HDR_WITH_PAYLOAD is being updated by the function and the returned pointer will point to the same object. This could lead to some confusion by anyone trying to use the function.
In the case of your code, it accomplishes the same thing as the "official" answer, but creates a new object to contain the data. You will need to be sure that the memory is properly deallocated once the object is no longer needed, but when comparing the two, I would argue that your code is more correct in that it will not result in undefined, implementation-defined or confusing behavior.
Related
I've recently been working on wifi sniffer using ESP8266, in order to sniff wifi packets there is a function called wifi_set_promiscuous_rx_cb(wifi_sniffer_packet_handler) it takes a callback function as a parameter and passes buffer pointer, which has the packet info and length of the packet as parameters to the callback function , wifi_sniffer_packet_handler(uint8_t *buff, uint16_t len) is the call back function. i am not understanding what these two statements are doing
const wifi_promiscuous_pkt_t *ppkt = (wifi_promiscuous_pkt_t *)buff;
const wifi_ieee80211_packet_t *ipkt = (wifi_ieee80211_packet_t *)ppkt->payload;
wifi_promiscuous_pk_t is a structure
typedef struct{
wifi_pkt_rx_ctrl_t rx_ctrl; /**< metadata header */
uint8_t payload[0]; /**< Data or management payload. Length of payload is described by rx_ctrl.sig_len. Type of content determined by packet type argument of callback. */
} wifi_promiscuous_pkt_t;
and wifi_ieee80211_packet_t is another structure
typedef struct
{
wifi_ieee80211_mac_hdr_t hdr;
uint8_t payload[2]; /* network data ended with 4 bytes csum (CRC32) */
} wifi_ieee80211_packet_t;
how the data in the *buff is assigned to these structures, are the above statements responsible for the assignment
i've seen many stackoverflow questions and many other threads regarding this but none of the posts clarified my doubt
The packet coming from the radio presumably has a format like this:
| metadata | mac hdr | mac payload |
This is stored initially as a uint8_t array pointed to by buff.
Now, you typecast the buffer into the first structure type:
const wifi_promiscuous_pkt_t *ppkt = (wifi_promiscuous_pkt_t *)buff;
This is what our buffer conceptually looks like now:
|<---------------------- buff ---------------------->| buff
|<-- rx_ctrl -->|<------- payload ------------------>| ppkt
The structure is kind of overlaid on top of the byte array and since the metadata is always a fixed size, ppkt->rx_ctrl will now contain this metadata header.
But what about payload? The declaration indicates payload to be a zero sized array. This is special syntax to tell the compiler that this is a 'placeholder' for an arbitrarily large array of user-managed memory. (Please note that this is an outdated GCC extension and might produce compile errors on most modern compilers. The post C99 approach is to simply leave out the array dimension like this: uint8_t payload[])
Hence ppkt->payload is now an array starting with the first byte beyond the metadata. Since C doesn't bother with bounds checking, you can access any byte of this payload with ppkt->payload[i]
Now, we need to get to the mac payload by parsing out the mac header, which is again of a fixed size. This is accomplished by:
const wifi_ieee80211_packet_t *ipkt = (wifi_ieee80211_packet_t *)ppkt->payload;
In this case, we discard the header (rx_ctrl) and overlay the wifi_ieee80211_packet_t struct on top of the payload portion of the buffer.
Looking at our buffer again, this is what has happened to it:
|<---------------------- buff ---------------------->| buff
|<-- rx_ctrl -->|<------- payload ------------------>| ppkt
|<-- hdr -->|<----- payload -------->| ipkt
Now you can easily get to the mac payload in the original packet with ipkt->payload.
This is a very common idiom used when parsing packets in networking protocol stacks.
PS: However, please note that typecasting between incompatible types is risky business. Firstly, you have to understand and account for the alignment of structure members. Also, depending on how your structures have been defined, this might result in violation of strict aliasing rules and hence, invoke undefined behaviour on certain compilers.
The goal of converting from a pointer to bytes that have been read from the network to a pointer to a structure is to interpret the bytes as that structure type. This relies on compiler features that are not required by the C standard.
The code (wifi_promiscuous_pkt_t *)buff converts the pointer buff to a pointer to the type wifi_promiscuous_pkt_t. This relies on buff being aligned as required for the structure,1 and it requires that the conversion produce a pointer to the same place in memory but with a different type.2
Then const wifi_promiscuous_pkt_t *ppkt = (wifi_promiscuous_pkt_t *)buff; defines a new pointer ppkt and initializes it to point to the same memory buff points to. The goal is to have references such as ppkt->rx_ctrl interpret the data as the data for the wifi_promiscuous_pkt_t structure instead of as the raw uint8_t bytes.
The C standard does not define the behavior here.3 Compilers may support this by defining the behavior themselves. Notably, the compiler needs to support aliasing memory: Accessing memory using a type other than the original type used to define or store data there.
For illustration, let’s say the type of rx_ctrl, wifi_pkt_rx_ctrl_t, is a four-byte int. Then, when these requirements above are met, ppkt->rx_ctrl_t will access the first four bytes at buff and interpret them as if they were an int.
(If those requirements are not met, then a C-standard way to reinterpret the bytes is to make a new object and copy the data into it: wifi_promiscuous_pkt_t temp; memcpy(temp, buff, sizeof temp + extra for the payload length);, after which temp.rx_ctrl will provide the int value. However, depending on circumstances and compiler quality, that can cause a lot of extra memory copying we would like to avoid. Accessing the memory directly is preferable, if it is supported. Another alternative is to read the data directly from the network into the target object instead of into a buffer of bytes.)
After this, const wifi_ieee80211_packet_t *ipkt = (wifi_ieee80211_packet_t *)ppkt->payload; does the same thing with ppkt->payload instead of buff. Once ppkt has been set up, ppkt->payload points to the bytes starting after the rx_ctrl member.4 This pointer is then converted to point to the type wifi_ieee80211_packet_t, and ipkt is initialized with the new pointer.
As before, the intent is that ipkt->hdr and ipkt->payload will refer to the data in the first structure’s payload area as if that data contained a wifi_ieee80211_packet_t structure. However, there is another problem here. As noted above, buff may pointed to well-aligned memory if it was allocated with malloc. But ipkt does not point to the same place as buff. It points to ppkt->payload, which is some number of bytes after the start because it follows the rx_ctrl member. So we do not know, from this code alone, that ipkt is properly aligned for a wifi_ieee80211_packet_t. Possibly it is, if the size of the rx_ctrl member is a multiple of the alignment requirement for the hdr member.
Without comments in the code speaking to this, this requirement may have been neglected. The code might work because rx_ctrl is an okay size for this to work, but it might generate an alignment trap. If not now, then possibly in the future when the types involved change.
Footnotes
1 If buff points to the start of memory allocated with malloc or another standard memory allocation routine, it will be correctly aligned for any fundamental type. Or, if it was defined as an array, correct alignment can be requested with the standard _Alignas keyword or possibly with a compiler extension. If it was merely defined as an ordinary array of uint8_t, is not guaranteed to be aligned as required for this use.
2 For general pointer-to-object conversions, the standard only requires that a pointer converted to another pointer-to-object type and then back to its original type points to the original object. It does not guarantee the pointer is otherwise usable in the other type. But this is common in C implementations.
3 In general, accessing an array of uint8_t using other types violates the rule in C 2018 6.5 7, which says that memory which has been defined with a certain type, or, for dynamic memory, assigned with a certain type, shall be accessed only by certain “matching” types or by a “character” (byte) type.
4 Formally, ppkt->payload designates an array, and that array is automatically converted to a pointer. Also, the array is declared to have zero elements. That is not defined by the C standard, but GCC and other compilers supported it as a way to allow variable length data at the end of a structure. Since C 1999, the standard way to do this is to declare the member with [] instead of [0].
When I used the structure:
|<-- rx_ctrl -->|<------- payload ------------------>|
I get a misalignment in the payload data by a shift of 2 bytes to the left.
When I replace rx_ctrl part with a field e.g. unsigned frame_ctrl:16 I do not see this shift. I do have to typecast frame_ctrl to rx_ctrl to get the details.
I assume this is because I use esp32-S2 which assumes teh rx_ctrl is 32 bits regardless of its definition as 16 bits.
Suppose I am given a (void*) ptr (my basic understanding is, it represents a pointer to a region of unknown data type) passed through the parameter of a function. I am trying to figure out how to access and check if a struct exists a few addresses behind.
To clarify, I am working with a big char array (not malloced) and the ptr passed into the function should point to an address of an unspecified data type within the array. Located before this data is a struct for which I am trying to access.
void function(void *ptr)
{
void *structPtr = (void*)((void*)ptr - sizeof(struct block));
}
Would this work to get me a pointer to the address of the struct located behind the initial "ptr"? And if so, how could I check if it is the block struct?
Apologizes in advance, I know this code is not specific as I am fairly new to the concepts entirely but also, I am in the process of coming up with an algorithm and not yet implementing it. Any references to possibly useful information are much appreciated.
what you are trying to do is risky as you must be sure that you address a correct place in memory. Usually, we add some magic number in struct block so that we can test here that we are not going anywhere.
This pattern is generally used in memory allocators,
have a look to https://sourceware.org/glibc/wiki/MallocInternals for an example.
The usual way of writing this is something like:
...function(void *ptr) {
struct block *hdr = (struct block *)ptr - 1;
relying on pointer arithmetic automatically scaling by the size of the pointed at type. As long as all the pointers passed in here were originally created by taking a pointer to a valid struct block and adding 1 to it (to get the memory after it), you should be fine.
I want to use libuv to send files over TCP.
My problem is with uv_read_start for reading the content of the file.
To do that, I first send the size of the file, so I make a static malloc(sizeof(uint64_t)) in the alloc_cb to read the size.
After that I have to make malloc(size) to allocate the necessary amount of memory to fill all the file in memory (I absolutely want to have the file like this in memory).
But I don't know how to pass the argument size to the alloc_cb (I don't want to use global variable).
Is there a way to do that without global variable?
Perhaps a way totaly different to do the same thing?
Thanks in advance.
I see that handle->data contain a void* so we can put in exactly what we want.
Before calling uv_read_start store the size in the stream handle stream->data = (size_t*)malloc(sizeof(size_t)); and *(size_t *)(stream->data) = size;, and reuse it in alloc_cb by reading size_t rsize = *(size_t*)(handle->data);.
After that you must free this buffer.
Any other ideas?
As OP mentioned, using the void * field is an option.
The other is to use inheritance (if using C++) or inheritance-style hackery in C.
For C++ : we use inheritance.
struct my_struct: public uv_handle_t {
size_t size;
};
You can pass a pointer to this structure to alloc_cb (since my_struct inherits from handle), and then cast it back to type my_struct:
my_struct * s = reinterpret_cast<my_struct*>(handle);
For C : we don't have inheritance, so we need a bit of a hack:
struct my_struct {
uv_handle_t handle;
size_t size;
};
Then call alloc_cb after casting a pointer to an object of type my_struct to uv_handle_t ((uv_handle_t*) s), and cast back to my_struct inside alloc_cb:
my_struct * s = (my_struct*) handle;
The C method is explained in more details (though in a different context) in the book An introduction to libuv, release 2, at paragraph 8.4 "Passing data to worker thread" (see here).
So I'm trying to write a buffering library for the 64th time and I'm starting get into some pretty advanced stuff. Thought I'd ask for some proffesional input on this.
In my first header file I have this:
typedef struct StdBuffer { void* address; } StdBuffer;
extern void StdBufferClear(StdBuffer);
In another header file that #includes the first header file I have this:
typedef struct CharBuffer { char* address; } CharBuffer;
void (*CharBufferClear)(CharBuffer) = (void*) StdBufferClear;
Will declaring this function pointer void interfere with the call? They have matching by value signatures. I have never seen a function pointer declared void before, but its the only way to get it to compile cleanly.
Stackwise it should not make any difference at all from what I learned in assembler coding.
irrelevent OMG! I just said Stackwise on StackOverflow!
Hmm.. Looks like I've assumed too much here. Allow me to reclarify if I may. I don't care what 'type' of data is stored at the address. All that I am concerned with is the size of a 'unit' and how many units are at the address. Take a look at the interface agreement contract for the API if you will:
typedef struct StdBuffer {
size_t width; ///< The number of bytes that complete a data unit.
size_t limit; ///< The maximum number of data units that can be allocated for this buffer.
void * address; ///< The memory address for this buffer.
size_t index; ///< The current unit position indicator.
size_t allocated; ///< The current number of allocated addressable units.
StdBufferFlags flags;///< The API contract for this buffer.
} StdBuffer;
You see, memcpy, memmove and the like don't really care whats at an address all they want is the specifics which I'm clearly keeping track of here.
Have a look now at the first prototype to follow this contract:
typedef struct CharBuffer {
size_t width; ///< The number of bytes that complete a data unit.
size_t limit; ///< The maximum number of data units that can be allocated for this buffer.
char * address; ///< The memory address for this buffer.
size_t index; ///< The current unit position indicator.
size_t allocated; ///< The current number of allocated addressable units.
CharBufferFlags flags;///< The API contract for this buffer.
} CharBuffer;
As you an clearly see the data type is irrelevant in this context. You can say that C handles it differently depending on the case, but at the end of the day, an address is an address, a byte is byte and a long is a long for as long as we are dealing with memory on the same machine.
The purpose of this system when brought together is to remove all of this type based juggling C seems to be so proud of (and rightfully so...) Its just pointless for what I would like to do. Which is create a contract abiding prototype for any standard size of data (1, 2, 4, 8, sizeof(RandomStruct)) located at any address.
Having the ability to perform my own casting with code and manipulate that data with api functions that operate on specific length blocks of memory with specific length memory units. However, the prototype must contain the official data pointer type, because it just doesn't make sense for the end user to have to recast their data every time they would like to do something with that address pointer. It would not make sense to call it a CharBuffer if the pointer was void.
The StdBuffer is a generic type that is never EVER used except within the api itself, to manage all contract abiding data types.
The api that this system will incorporate is from my latest edition of buffering. Which is quite clearly documented here #Google Code I am aware that some things will need to change to bring this all together namely I won't have the ability to manipulate data directly from within the api safely without lots of proper research and opinion gathering.
Which just brought to my attention that I also need a Signed/Unsigned bit flag in the StdBufferFlags Members.
Perhaps the final piece to this puzzle is also in order for your perusal.
/** \def BIT(I)
\brief A macro for setting a single constant bit.
*
* This macro sets the bit indicated by I to enabled.
* \param I the (1-based) index of the desired bit to set.
*/
#define BIT(I) (1UL << (I - 1))
/** \enum StdBufferFlags
\brief Flags that may be applied to all StdBuffer structures.
* These flags determine the contract of operations between the caller
* and the StdBuffer API for working with data. Bits 1-4 are for the
* API control functions. All other bits are undefined/don't care bits.
*
* If your application would like to use the don't care bits, it would
* be smart not to use bits 5-8, as these may become used by the API
* in future revisions of the software.
*/
typedef enum StdBufferFlags {
BUFFER_MALLOCD = BIT(1), ///< The memory address specified by this buffer was allocated by an API
BUFFER_WRITEABLE = BIT(2), ///< Permission to modify buffer contents using the API
BUFFER_READABLE = BIT(3), ///< Permission to retrieve buffer contents using the API
BUFFER_MOVABLE = BIT(4) ///< Permission to resize or otherwise relocate buffer contents using the API
}StdBufferFlags;
This code requires a diagnostic:
void (*CharBufferClear)(CharBuffer) = (void*) StdBufferClear;
You're converting a void * pointer to a function pointer without a cast. In C, a void * pointer can convert to pointers to object types without a cast, but not to function pointer types. (In C++, a cast is needed to convert void * to object types also, for added safety.)
What you want here is just to cast between function pointer types, i.e.:
void (*CharBufferClear)(CharBuffer) = (void (*)(CharBuffer)) StdBufferClear;
Then you are still doing the same type punning because the functions are different types. You are trying to call a function which takes a StdBuffer using a pointer to a function which takes a CharBuffer.
This type of code is not well-defined C. Having defeated the type system, you're on your own, relying on testing, examining the object code, or obtaining some assurances from the compiler writers that this sort of thing works with that compiler.
What you learned in assembler coding doesn't apply because assembly languages have only a small number of rudimentary data types such as "machine address" or "32 bit word". The concept that two data structures with an identical layout and low-level representation might be incompatible types does not occur in assembly language.
Even if two types look the same at the low level (another example: unsigned int and unsigned long are sometimes exactly the same) C compilers can optimize programs based on the assumption that the type rules have not been violated. For instance suppose that A and B point to the same memory location. If you assign to an an object A->member, a C compiler can assume that the object B->member is not affected by this, if A->member and B->member have incompatible types, like one being char * and the other void *. The generated code keeps caching the old value of B->member in a register, even though the in-memory copy was overwritten by the assignment to A->member. This is an example of invalid aliasing.
The standard does not define the results of casting a function-pointer to void *.
Equally, converting between function pointers and then calling through the wrong one is also undefined behaviour.
There are some constructs which any standards-conforming C compiler are required to implement consistently, and there are some constructs which 99% of C compilers do implement consistently, but which standards-conforming compilers would be free to implement differently. Attempting to cast a pointer to a function which takes one type of pointer, into a pointer to a function which takes another type of pointer, falls into the latter category. Although the C standard specifies that a void* and a char* must be the same size, there is nothing that would require that they share the same bit-level storage format, much less the parameter-passing convention. While most machines allow bytes to be accessed in much the same way as words, such ability is not universal. The designer of an application-binary-interface [the document which specifies among other things how parameters are passed to routines] might specify that a char* be passed in a way which maximizes the efficiency of byte access, while a void* should be passed in a way that maximizes the efficiency of word access while retaining the ability to hold an unaligned byte address, perhaps by using a supplemental word to hold a zero or one to indicate LSB/MSB). On such a machine, having a routine that expects a void* called from code that expects to pass a char* could cause the routine to access arbitrary wrong data.
No, it doesn't matter what data type is used to store the data. It only matters the type C uses to read and write that data, and that the data is of sufficient size.
I would like some advice on safe ways to deal with struct's when the size of certain members are not known at code time.
For example I have a Struct Named "Channel". This struct has a member name "AudioSourceOBJ" which is a pointer to an an array of other struct type named "AudioSource". I wont know how many AudioSources I will have per channel until the program is run. I deal with that like this.
channel object
struct channelobj {
AudioUnitSampleType *leftoutput;
AudioUnitSampleType *rightoutput;
AudioSourceOBJ *audioSource;
};
audiosource
struct audiosourceobj {
AudioUnitSampleType *leftoutput;
AudioUnitSampleType *rightoutput;
};
creation of variable sized structs
void createInputs(ChannelOBJ channel,int numAudioInputs)
{
channel->audioSource=(AudioSourceOBJ *)malloc(numAudioInputs * sizeof(AudioSourceOBJ));
for (int i=0;i<numAudioInputs;i++)
{
AudioSourceOBJ obj;
obj=newAudioSourceOBJ();
channel->audioSource[i]=obj;
}
}
I think this is o.k?
The problem I am now facing is that even though I can assign memory for the correct number of audio objects in my channel struct, the leftoutput and rightoutput arrays in the audiosource struct will not be set until later in the program. They will be filled with an undermined amount of data, and are likely to change in size and content throughout the lifetime of the application.
Will I have to completely re malloc the channel containing the audiosource every time I want to make changes to a single audio object?
What is a safe way to do this or is there a better approach?
"Will I have to completely re malloc the channel containing the audiosource every time I want to make changes to a single audio object?"
No. You could for example replace the left output of the ith audio source like this:
free(channel->audioSource[i].leftoutput);
channel->audioSource[i].leftoutput = malloc(newSize * sizeof(AudioUnitSampleType));
Or even:
AudioUnitSampleType *tmp = realloc(channel->audioSource[i].leftoutput,
newSize * sizeof(*tmp));
if (tmp == 0) { /* handle the error */ }
channel->audioSource[i].leftoutput = tmp;
By the way, if you don't post real code, it's possible that answers will contain errors due to errors in your examples.
There seems to be some confusion in your code between pointers and objects, for example the channel parameter is of type ChannelOBJ, then you use it as if it's a pointer. Is this an error, or is ChannelOBJ a typedef for struct channelobj*? It's generally better not to conceal that something is a pointer using a typedef.
If AudioUnitSampleType is likewise a pointer type, then my first code snippet above is incomplete, since it would then also be necessary to free the old objects pointed to by the elements of the array, and allocate new ones. The second one needs to free old ones or allocate new ones according to whether the size is being increased or decreased.
No, you won't have to resize the allocated block of AudioSourceObj structs. leftoutput and rightoutput are merely pointers of a fixed size (not variable-sized arrays) and can be assigned an address by doing a separate malloc:
channel->audioSource[i].leftoutput = malloc(5 * sizeof(AudioUnitSampleType));