I am doing some network programming, and I use some struct to describe my frame header like that:
struct my_frame_header {
uint16_t field1;
uint16_t field2;
};
And so, when I have a buffer frame I can do something like that:
uint8_t buffer[BUFFER_SIZE];
struct my_frame_header *frame_header = (struct my_frame_header *)buffer;
my_read(buffer, BUFFER_SIZE);
I can now access to the header field like that:
ntohl(frame_header->field1);
Now, my question is: What is the most elegant way to access the data after the structure? (i.e. to get a pointer at the beginning of the data part)
Well, if I understood your question correctly, you can do something like
uint8_t * data = (uint8_t *)buffer + sizeof (struct my_frame)
then, data will be a pointer to the next uint8_t element after the header.
uint8_t buffer[BUFFER_SIZE];
struct my_frame_header *frame_header = (struct my_frame_header *)buffer;
This is wrong, the base address of buffer can be unaligned for my_frame_header.
Take look to Memory access and alignment
On the other hand:
The block that malloc gives you is guaranteed to be aligned so that it
can hold any type of data.
Then, you can use malloc in order to skip this problem:
uint8_t *buffer = malloc(BUFFER_SIZE);
Related
I have a serial communication, where I will serialize a binary file and send it to a microcontroller.
My question is how to store this file.
I thinked to use the structure:
typedef struct
{
uint8_t id; // protocol
size_t data_size; // to help know the file
uint8_t data[1024]; // data bytes
} RECEIVED_FILE_t;
The file is a binary and can vary up to 50Kb, how would it be better to store it in this structure, reference it with a pointer or a vector? Which would be more efficient?
With a uint8_t data[1024] i can store up to 1MB file? (1 byte * 1024 = 1024 Bytes = 1MB) Is correct?
I can change this structure.
You probably want something like this:
typedef struct
{
uint8_t id;
size_t data_size;
uint8_t *pdata; // pointer to data bytes
} RECEIVED_FILE_t;
Once you have the size, allocate memory and read the data there.
Pseudo code:
RECEIVED_FILE_t file;
...
int data_size = ReadSize(...); // get the data size somehow
file.data_size = data_size; // put the size into the struct
file.pdata = malloc(data_size); // allocate memory for the data
ReadData(file.data_size, file.pdata); // read data_size bytes and put them
// in the memory where pdata points to
I have no idea how you get the data size and how you actually read the data. ReadSize and ReadData are just there to illustrate how you code could work.
I have the following struct definition:
typedef struct mb32_packet_t {
union {
struct {
uint16_t preamble;
uint8_t system_id;
uint8_t message_id;
uint8_t reserved;
uint32_t paylen;
};
uint8_t header[9];
};
uint8_t *payload;
uint16_t checksum;
} __attribute__((packed)) mb32_packet_t;
Now I would like to have another union, so that I can get an uint8_t body[] pointer to the entire packet object. Something like this:
typedef struct mb32_packet_t {
union {
struct {
union {
struct {
uint16_t preamble;
uint8_t system_id;
uint8_t message_id;
uint8_t reserved;
uint32_t paylen;
};
uint8_t header[9];
};
uint8_t *payload;
uint16_t checksum;
};
uint8_t body[?];
};
} __attribute__((packed)) mb32_packet_t;
The problem is that the payload field size is dynamically determined at runtime. Is there another way to accomplish this other than making payload fixed sized?
I basically want to send objects of this type through a network socket, so I need a uint8_t pointer that points to an object of this type. At the time of sending the object, I know the size of the entire object in bytes.
Introduction
The question is unclear, so I will discuss three apparent possibilities.
Fixed-length header followed by variable-length payload
A typical way to define a packet for a networking or messaging service is to have a fixed-length header followed by a variable-length payload. In modern C, the variable-length payload may be defined using a flexible array member, which is an array with no dimension at the end of a structure:
typedef struct
{
uint16_t preamble;
uint8_t system_id;
uint8_t message_id;
uint8_t reserved;
uint32_t paylen;
uint8_t payload[];
} mb32_packet_t;
Memory for such a structure is allocated use the base size provided by sizeof plus additional memory for the payload:
mb32_packet_t *MyPacket = malloc(sizeof *MyPacket + PayloadLength);
When you pass such an object to a routine that requires a char * or uint8_t * or similar type for its argument, you can simply convert the pointer:
SendMyMessage(…, (uint8_t *) MyPacket,…);
That cast, (uint8_t *) MyPacket, provides the pointer to the first byte of the packet requested in the question. There is no need to wedge another member into the structure or layer on a union or other declaration.
Prior to the introduction of flexible array members in C 1999, people would use one of two workarounds to create structures with variable amounts of data. One, they might just define a member array with one element and adjust the space calculations accordingly:
typedef struct
{
…
unsigned char payload[1];
} mb32_packet_t;
mb32_packet_t *MyPacket = malloc(sizeof *MyPacket + PayloadLength - 1);
Technically, that violated the C standard, since the structure contained an array of only one element even though more space was allocated for it. However, compilers were not as aggressive in their analysis of program semantics and their optimization as they are now, so it generally worked. So you may still see old code using that method.
Two, GCC had its own pre-standard implementation of flexible array members, just using an array dimension of zero instead of omitting a dimension:
typedef struct
{
…
unsigned char payload[0];
} mb32_packet_t;
Again, you may see old code using that, but new code should use the standard flexible array member.
Fixed-length header with pointer to variable-length payload
The payload-after-header form shown above is the form of packet I would most expect in a messaging packet, because it matches what the hardware has to put “on the wire” when sending bytes across a network: It writes the header bytes followed by the data bytes. So it is convenient to have them arranged that way in memory.
However, your code shows another option: The data is not in the packet but is pointed to by a pointer in the packet, with uint8_t *payload;. I would suspect that is a mistake, that the network or messaging service really wants a flexible array member, but you show it followed by another member, uint16_t checksum. A flexible array member must be the last member in a structure, so the fact that there is another member after the payload suggests this definition with a pointer may be correct for the messaging service you are working with.
However, if that is the case, it is not possible to get a pointer to the complete packet object, because the object is in two pieces. One contains the header, and the other, at some unrelated location in memory, contains the data.
As above, you can produce a uint8_t * pointer to the start of the packet with (uint8_t) MyPacket. If the messaging system knows about the pointer in the structure, that should work. If you have mistaken what the packet structure must be, it will fail.
Fixed-length header followed by fixed-length payload space
Code elsewhere on Stack Overflow shows a struct mb32_packet_t with a fixed amount of space for a payload:
typedef struct mb32_packet_t {
uint8_t compid;
uint8_t servid;
uint8_t payload[248];
uint8_t checksum;
} __attribute__((packed)) mb32_packet_s;
In this form, the packet is always a fixed size, although the amount of space used for the payload could vary. Again, you would obtain a uint8_t * pointer to the packet by a cast. There is no need for a special member for that.
This is possible, but not with a struct or union, because all parts of a struct or union need to have a known size. You can still use a struct for the header.
Because the body starts at a known location, there's a trick you can use to access it as if it was part of the structure. You can declare it with no size at all (a "flexible array member") or as 0 bytes (a GCC extension that predates the standard). The compiler will not allocate any space for it, but it will still let you use the name to refer to the end of the struct. The trick is that you can malloc extra bytes after the end of the struct, and then use body to refer to them.
typedef struct mb32_packet_t {
union {
struct {
uint16_t preamble;
uint8_t system_id;
uint8_t message_id;
uint8_t reserved;
uint32_t paylen;
};
uint8_t header[9];
};
uint8_t body[]; // flexible array member
} __attribute__((packed)) mb32_packet_t;
// This is not valid. The body is 0 bytes long, so the write is out of bounds.
mb32_packet_t my_packet;
my_packet.body[0] = 1;
// This is valid though!
mb32_packet_t *my_packet2 = malloc(sizeof(*my_packet2) + 50);
my_packet2->body[49] = 1;
// Alternative way to calculate size
mb32_packet_t *my_packet3 = malloc(offsetof(mb32_packet_t, body[50]));
my_packet3->body[49] = 1;
The flexible array member must be last. To access the checksum, you will need to allocate an extra 2 bytes, and use pointer arithmetic. Fortunately, this is just for the checksum, and not the entire header.
mb32_packet_t *my_packet = malloc(sizeof(*my_packet) + body_size + 2);
uint16_t *pchecksum = (uint16_t*)&my_packet.body[body_size];
// or
uint16_t *pchecksum = (uint16_t*)(my_packet.body + body_size);
After you fill in the header, body and checksum, then because they are contiguous in memory, a pointer to the header is also a pointer to the entire packet object.
I usually do it this way:
typedef struct
{
size_t payload_size;
double x;
char y[45];
/* another members */
unsigned char payload[];
}my_packet_t;
or if your compiler does not support FAMs
typedef struct
{
size_t payload_size;
double x;
char y[45];
/* another members */
unsigned char payload[0];
}my_packet_t;
So it the payload can be at the end of the header structure
I'm trying to get into Socket programming and came across an article at https://www.tenouk.com/Module43a.html I'm having difficulty understanding as how a char array is cast into struct pointer
char buffer[PCKT_LEN];
struct ipheader *ip = (struct ipheader *) buffer;
//some code here
ip->iph_ihl = 5;
ip->iph_ver = 4;
ip->iph_tos = 16;
As per my understanding, pointer ip will now hold the address of buffer and values for members of struct ipheader will now be stored in buffer. Please help understanding the same. If I'm right, then how would we be able to print values stored in buffer?
You understanding is correct. The pointer *ip will point to buffer. char buffer[PCKT_LEN] is an array of size sizeof(char) * PCKT_LEN. Since a char is usually 1 byte long it is just a chunk of memory of PCKT_LEN bytes. PCKT_LEN is defined to be 8192
The amount of bytes needed to store a struct ipheader is much less than this. Try int a = sizeof(ipheader) and use a debugger to see the value assigned to a. For me it is 24 bytes, but it could be slightly different for you. This means that buffer can hold much more data than the struct ipheader needs. I haven't looked to deeply into the code, and I don't know much about socket programming. But one use for this could be to augment buffer with additional data outside of the struct. Since you know struct ipheader takes up sizeof(ipheader) bytes you will have sizeof(char)*8192 - sizeof(ipheader) left to augment the array.
Edit:
Upun further inspection, this is kinda what is happening:
struct ipheader *ip = (struct ipheader *) buffer;
struct udpheader *udp = (struct udpheader *) (buffer + sizeof(struct ipheader));
It tries to store the ip header at the beginning of the buffer, then it augments that same buffer with an udp header. By using buffer + sizeof(struct ipheader)
it makes sure that it stores the udp header after ipheader by offsetting buffer by sizeof(struct ipheader) bytes. Basically struct ipheader *ip points to the beginning of the buffer and struct udpheader *udp points to buffer + sizeof(struct ipheader). I hope this makes sense. Obviously there is still a lot of space left over in buffer so you could potentially augment it even further.
how a char array is cast into struct pointer
You can't do that safely. The code invokes undefined behavior:
char buffer[PCKT_LEN];
struct ipheader *ip = (struct ipheader *) buffer;
//some code here
ip->iph_ihl = 5;
ip->iph_ver = 4;
ip->iph_tos = 16;
That code violates the strict aliasing rule. That basically means memory that isn't a certain type of object can't be treated as being that type of object, with the exception that any non-char object can be treated as an array of char.
That's not what's happening in the posted code. In the posted code, a char array is being treated as if it were a struct ipheader.
The memory is not a struct ipheader - it's an array of char - so the code violates strict aliasing.
The casting from char * to struct ipheader * can also result in an improperly aligned object and violate 6.3.2.3 Pointers, paragraph 7:
A pointer to an object type may be converted to a pointer to a different object type. If the resulting pointer is not correctly aligned for the referenced type, the behavior is undefined. ...
Code such as you've found here is unfortunately all too common as the x86-based machines that are the most common platform widely used by programmers are very forgiving of misaligned accesses, so such code tends to "work".
See Structure assignment in Linux fails in ARM but succeeds in x86 for an example of a platform where it doesn't work.
Given the below simple code, where you have process_payload is given a pointer to the payload portion of the packet, how do you access the header portion? Ideally the caller should simply give a pointer to full packet from beginning, but there are cases where you don't have the beginning of the message and need to work backwards to get to the header info. I guess this question becomes a understanding of walking through the memory layout of a struct.
The header computes to 8 bytes with sizeof operation. I assume Visual C++ compiler added 3 bytes padding to header.
The difference between pptr and pptr->payload is decimal 80 (not sure why this value??) when doing ptr arith (pptr->payload - pptr). Setting ptr = (struct Packet*)(payload - 80) works but seems more a hack. I don't quite understand why subtracting sizeof(struct header) doesn't work.
Thanks for any help you can give.
struct Header
{
unsigned char id;
unsigned int size;
};
struct Packet
{
struct Header header;
unsigned char* payload;
};
void process_payload(unsigned char* payload);
int main()
{
struct Packet* pptr = (struct Packet*)malloc(sizeof(struct Packet));
pptr->payload = (unsigned char*)malloc(sizeof(unsigned char)*10);
process_payload(pptr->payload);
return 1;
}
// Function needs to work backwards to get to header info.
void process_payload(unsigned char* payload)
{
// If ptr is correctly setup, it will be able to access all the fields
// visible in struct Packet and not simply payload part.
struct Packet* ptr;
// This does not work when intuitively it should?
ptr = (struct Packet*)(payload - sizeof(struct Header));
}
It's because in main you allocate two pointers, and pass the second pointer to the process_payload function. The two pointers are not related.
There are two ways of solving this problem, where both include a single allocation.
The first solution is to used so called flexible arrays, where you have an array member last in the structure without any size:
struct Packet
{
struct Header header;
unsigned char payload[];
};
To use it you make one allocation, with the size of the structure plus the size of the payload:
struct Packet *pptr = malloc(sizeof(struct Packet) + 10);
Now pptr->payload is handled like a normal pointer pointing to 10 unsigned characters.
Another solution, which is a mix of your current solution and the solution with flexible arrays, is to make one allocation and make the payload pointer to point to the correct place in the single allocated memory block:
struct Packet
{
struct Header header;
unsigned char *payload;
};
// ...
struct Packet *pptr = malloc(sizeof(struct Packet) + 10);
pptr->payload = (unsigned char *) ((char *) pptr + sizeof(struct Packet);
Note that in this case, to get the Packet structure from the payload pointer, you have to use sizeof(Packet) instead of only sizeof(Header).
Two things to note about the code above:
I don't cast the result of malloc
sizeof(char) (and also the size of unsigned char) is specified to always be one, so no need for sizeof
Ok I am putting the whole struct here, its specification of a protocol named openflow that is implemented in some of industrial switches , so the struct is like:
struct ofp_packet_in {
struct ofp_header header;
uint32_t buffer_id; /* ID assigned by datapath. */
uint16_t total_len; /* Full length of frame. */
uint16_t in_port; /* Port on which frame was received. */
uint8_t reason; /* Reason packet is being sent (one of OFPR_*) */
uint8_t pad;
uint8_t data[0]; /* Ethernet frame, halfway through 32-bit word,
so the IP header is 32-bit aligned. The
amount of data is inferred from the length
field in the header. Because of padding,
offsetof(struct ofp_packet_in, data) ==
sizeof(struct ofp_packet_in) - 2. */
};
OFP_ASSERT(sizeof(struct ofp_packet_in) == 20);
now I have to fill up some data in the last field that is -uint8_t data[0] which can be varied and info is gathered from the length field inside the header. I have to construct a packet in, and for that data has to be put in. Please take a look.
You'll need to use a dynamic allocation and copy the contents.
Something like:
#include <stdlib.h>
#include <string.h>
void foo(void) {
struct some_struct *container = malloc(sizeof(struct some_struct) + 100);
if (!container) {
// handle out-of-memory situation
}
memcpy(container->data, some_data, 100);
}
You cannot do this. It won't fit! The array in the struct is 0 chars long and you're trying to stuff a 100-char array into it.
If, for some reason, you are certain that the memory after the struct is available, e.g. you just malloc'ed it like this:
some_struct *foo = (some_struct*)malloc(sizeof(some_struct) + 100);
Then you can do this:
memcpy(foo->data, some_data, 100);
It's hideous, and probably still undefined behaviour, but I've seen APIs that require this (Windows?).
you can't.
You defined the size of some_struct.data to be 0, meaning it cannot hold any items.
If all you want is to copy max. 100 items into it, then you can define the size statically:
struct some_struct {
char data[100]; // some_struct.data has room for up to 100 characters
};