Ok I am putting the whole struct here, its specification of a protocol named openflow that is implemented in some of industrial switches , so the struct is like:
struct ofp_packet_in {
struct ofp_header header;
uint32_t buffer_id; /* ID assigned by datapath. */
uint16_t total_len; /* Full length of frame. */
uint16_t in_port; /* Port on which frame was received. */
uint8_t reason; /* Reason packet is being sent (one of OFPR_*) */
uint8_t pad;
uint8_t data[0]; /* Ethernet frame, halfway through 32-bit word,
so the IP header is 32-bit aligned. The
amount of data is inferred from the length
field in the header. Because of padding,
offsetof(struct ofp_packet_in, data) ==
sizeof(struct ofp_packet_in) - 2. */
};
OFP_ASSERT(sizeof(struct ofp_packet_in) == 20);
now I have to fill up some data in the last field that is -uint8_t data[0] which can be varied and info is gathered from the length field inside the header. I have to construct a packet in, and for that data has to be put in. Please take a look.
You'll need to use a dynamic allocation and copy the contents.
Something like:
#include <stdlib.h>
#include <string.h>
void foo(void) {
struct some_struct *container = malloc(sizeof(struct some_struct) + 100);
if (!container) {
// handle out-of-memory situation
}
memcpy(container->data, some_data, 100);
}
You cannot do this. It won't fit! The array in the struct is 0 chars long and you're trying to stuff a 100-char array into it.
If, for some reason, you are certain that the memory after the struct is available, e.g. you just malloc'ed it like this:
some_struct *foo = (some_struct*)malloc(sizeof(some_struct) + 100);
Then you can do this:
memcpy(foo->data, some_data, 100);
It's hideous, and probably still undefined behaviour, but I've seen APIs that require this (Windows?).
you can't.
You defined the size of some_struct.data to be 0, meaning it cannot hold any items.
If all you want is to copy max. 100 items into it, then you can define the size statically:
struct some_struct {
char data[100]; // some_struct.data has room for up to 100 characters
};
Related
I define a struct inside a header file which looks similar to:
#define BUFSIZE (500)
typedef struct container {
int file_descriptor;
char file_content[BUFSIZE];
}container;
Now, I want to have a variable BUFSIZE, such that the total characters that file_content can hold, reflects that. How could I make file_content a variable-sized character buffer array? It can be around 30-50KB. If I initialize it with 50KB, I will get the stack overflow error.
You'll need to learn dynamic memory usage, and "malloc" the necessary memory.
Example:
#define BUFSIZE (50*1024) // 50KBytes.
int main(void)
{
container my_container; // Error: Stack Overflow
container* correct_container = NULL;
correct_container = malloc(sizeof(container)); // No StackOverflow: should succeed.
correct_container->file_descriptor = get_fd();
memset(correct_container->file_content, 0, BUFSIZE);
// After usage is done.
free(correct_container);
}
A struct may have a flexible array member as its last member. In your case it would look like this:
typedef struct container {
int file_descriptor;
char file_content[];
}container;
The size of the struct is (roughly speaking) the size has the last element been omitted. You can then dynamically allocate an instance of this struct with as much room as needed for the last member. For example:
container *mycontainer = malloc(sizeof(*mycontainer) + 50000);
Also, because the size of the struct can vary in this way, it may not be a member of an array or of another struct or union.
I have the following struct definition:
typedef struct mb32_packet_t {
union {
struct {
uint16_t preamble;
uint8_t system_id;
uint8_t message_id;
uint8_t reserved;
uint32_t paylen;
};
uint8_t header[9];
};
uint8_t *payload;
uint16_t checksum;
} __attribute__((packed)) mb32_packet_t;
Now I would like to have another union, so that I can get an uint8_t body[] pointer to the entire packet object. Something like this:
typedef struct mb32_packet_t {
union {
struct {
union {
struct {
uint16_t preamble;
uint8_t system_id;
uint8_t message_id;
uint8_t reserved;
uint32_t paylen;
};
uint8_t header[9];
};
uint8_t *payload;
uint16_t checksum;
};
uint8_t body[?];
};
} __attribute__((packed)) mb32_packet_t;
The problem is that the payload field size is dynamically determined at runtime. Is there another way to accomplish this other than making payload fixed sized?
I basically want to send objects of this type through a network socket, so I need a uint8_t pointer that points to an object of this type. At the time of sending the object, I know the size of the entire object in bytes.
Introduction
The question is unclear, so I will discuss three apparent possibilities.
Fixed-length header followed by variable-length payload
A typical way to define a packet for a networking or messaging service is to have a fixed-length header followed by a variable-length payload. In modern C, the variable-length payload may be defined using a flexible array member, which is an array with no dimension at the end of a structure:
typedef struct
{
uint16_t preamble;
uint8_t system_id;
uint8_t message_id;
uint8_t reserved;
uint32_t paylen;
uint8_t payload[];
} mb32_packet_t;
Memory for such a structure is allocated use the base size provided by sizeof plus additional memory for the payload:
mb32_packet_t *MyPacket = malloc(sizeof *MyPacket + PayloadLength);
When you pass such an object to a routine that requires a char * or uint8_t * or similar type for its argument, you can simply convert the pointer:
SendMyMessage(…, (uint8_t *) MyPacket,…);
That cast, (uint8_t *) MyPacket, provides the pointer to the first byte of the packet requested in the question. There is no need to wedge another member into the structure or layer on a union or other declaration.
Prior to the introduction of flexible array members in C 1999, people would use one of two workarounds to create structures with variable amounts of data. One, they might just define a member array with one element and adjust the space calculations accordingly:
typedef struct
{
…
unsigned char payload[1];
} mb32_packet_t;
mb32_packet_t *MyPacket = malloc(sizeof *MyPacket + PayloadLength - 1);
Technically, that violated the C standard, since the structure contained an array of only one element even though more space was allocated for it. However, compilers were not as aggressive in their analysis of program semantics and their optimization as they are now, so it generally worked. So you may still see old code using that method.
Two, GCC had its own pre-standard implementation of flexible array members, just using an array dimension of zero instead of omitting a dimension:
typedef struct
{
…
unsigned char payload[0];
} mb32_packet_t;
Again, you may see old code using that, but new code should use the standard flexible array member.
Fixed-length header with pointer to variable-length payload
The payload-after-header form shown above is the form of packet I would most expect in a messaging packet, because it matches what the hardware has to put “on the wire” when sending bytes across a network: It writes the header bytes followed by the data bytes. So it is convenient to have them arranged that way in memory.
However, your code shows another option: The data is not in the packet but is pointed to by a pointer in the packet, with uint8_t *payload;. I would suspect that is a mistake, that the network or messaging service really wants a flexible array member, but you show it followed by another member, uint16_t checksum. A flexible array member must be the last member in a structure, so the fact that there is another member after the payload suggests this definition with a pointer may be correct for the messaging service you are working with.
However, if that is the case, it is not possible to get a pointer to the complete packet object, because the object is in two pieces. One contains the header, and the other, at some unrelated location in memory, contains the data.
As above, you can produce a uint8_t * pointer to the start of the packet with (uint8_t) MyPacket. If the messaging system knows about the pointer in the structure, that should work. If you have mistaken what the packet structure must be, it will fail.
Fixed-length header followed by fixed-length payload space
Code elsewhere on Stack Overflow shows a struct mb32_packet_t with a fixed amount of space for a payload:
typedef struct mb32_packet_t {
uint8_t compid;
uint8_t servid;
uint8_t payload[248];
uint8_t checksum;
} __attribute__((packed)) mb32_packet_s;
In this form, the packet is always a fixed size, although the amount of space used for the payload could vary. Again, you would obtain a uint8_t * pointer to the packet by a cast. There is no need for a special member for that.
This is possible, but not with a struct or union, because all parts of a struct or union need to have a known size. You can still use a struct for the header.
Because the body starts at a known location, there's a trick you can use to access it as if it was part of the structure. You can declare it with no size at all (a "flexible array member") or as 0 bytes (a GCC extension that predates the standard). The compiler will not allocate any space for it, but it will still let you use the name to refer to the end of the struct. The trick is that you can malloc extra bytes after the end of the struct, and then use body to refer to them.
typedef struct mb32_packet_t {
union {
struct {
uint16_t preamble;
uint8_t system_id;
uint8_t message_id;
uint8_t reserved;
uint32_t paylen;
};
uint8_t header[9];
};
uint8_t body[]; // flexible array member
} __attribute__((packed)) mb32_packet_t;
// This is not valid. The body is 0 bytes long, so the write is out of bounds.
mb32_packet_t my_packet;
my_packet.body[0] = 1;
// This is valid though!
mb32_packet_t *my_packet2 = malloc(sizeof(*my_packet2) + 50);
my_packet2->body[49] = 1;
// Alternative way to calculate size
mb32_packet_t *my_packet3 = malloc(offsetof(mb32_packet_t, body[50]));
my_packet3->body[49] = 1;
The flexible array member must be last. To access the checksum, you will need to allocate an extra 2 bytes, and use pointer arithmetic. Fortunately, this is just for the checksum, and not the entire header.
mb32_packet_t *my_packet = malloc(sizeof(*my_packet) + body_size + 2);
uint16_t *pchecksum = (uint16_t*)&my_packet.body[body_size];
// or
uint16_t *pchecksum = (uint16_t*)(my_packet.body + body_size);
After you fill in the header, body and checksum, then because they are contiguous in memory, a pointer to the header is also a pointer to the entire packet object.
I usually do it this way:
typedef struct
{
size_t payload_size;
double x;
char y[45];
/* another members */
unsigned char payload[];
}my_packet_t;
or if your compiler does not support FAMs
typedef struct
{
size_t payload_size;
double x;
char y[45];
/* another members */
unsigned char payload[0];
}my_packet_t;
So it the payload can be at the end of the header structure
I am doing some network programming, and I use some struct to describe my frame header like that:
struct my_frame_header {
uint16_t field1;
uint16_t field2;
};
And so, when I have a buffer frame I can do something like that:
uint8_t buffer[BUFFER_SIZE];
struct my_frame_header *frame_header = (struct my_frame_header *)buffer;
my_read(buffer, BUFFER_SIZE);
I can now access to the header field like that:
ntohl(frame_header->field1);
Now, my question is: What is the most elegant way to access the data after the structure? (i.e. to get a pointer at the beginning of the data part)
Well, if I understood your question correctly, you can do something like
uint8_t * data = (uint8_t *)buffer + sizeof (struct my_frame)
then, data will be a pointer to the next uint8_t element after the header.
uint8_t buffer[BUFFER_SIZE];
struct my_frame_header *frame_header = (struct my_frame_header *)buffer;
This is wrong, the base address of buffer can be unaligned for my_frame_header.
Take look to Memory access and alignment
On the other hand:
The block that malloc gives you is guaranteed to be aligned so that it
can hold any type of data.
Then, you can use malloc in order to skip this problem:
uint8_t *buffer = malloc(BUFFER_SIZE);
Given the below simple code, where you have process_payload is given a pointer to the payload portion of the packet, how do you access the header portion? Ideally the caller should simply give a pointer to full packet from beginning, but there are cases where you don't have the beginning of the message and need to work backwards to get to the header info. I guess this question becomes a understanding of walking through the memory layout of a struct.
The header computes to 8 bytes with sizeof operation. I assume Visual C++ compiler added 3 bytes padding to header.
The difference between pptr and pptr->payload is decimal 80 (not sure why this value??) when doing ptr arith (pptr->payload - pptr). Setting ptr = (struct Packet*)(payload - 80) works but seems more a hack. I don't quite understand why subtracting sizeof(struct header) doesn't work.
Thanks for any help you can give.
struct Header
{
unsigned char id;
unsigned int size;
};
struct Packet
{
struct Header header;
unsigned char* payload;
};
void process_payload(unsigned char* payload);
int main()
{
struct Packet* pptr = (struct Packet*)malloc(sizeof(struct Packet));
pptr->payload = (unsigned char*)malloc(sizeof(unsigned char)*10);
process_payload(pptr->payload);
return 1;
}
// Function needs to work backwards to get to header info.
void process_payload(unsigned char* payload)
{
// If ptr is correctly setup, it will be able to access all the fields
// visible in struct Packet and not simply payload part.
struct Packet* ptr;
// This does not work when intuitively it should?
ptr = (struct Packet*)(payload - sizeof(struct Header));
}
It's because in main you allocate two pointers, and pass the second pointer to the process_payload function. The two pointers are not related.
There are two ways of solving this problem, where both include a single allocation.
The first solution is to used so called flexible arrays, where you have an array member last in the structure without any size:
struct Packet
{
struct Header header;
unsigned char payload[];
};
To use it you make one allocation, with the size of the structure plus the size of the payload:
struct Packet *pptr = malloc(sizeof(struct Packet) + 10);
Now pptr->payload is handled like a normal pointer pointing to 10 unsigned characters.
Another solution, which is a mix of your current solution and the solution with flexible arrays, is to make one allocation and make the payload pointer to point to the correct place in the single allocated memory block:
struct Packet
{
struct Header header;
unsigned char *payload;
};
// ...
struct Packet *pptr = malloc(sizeof(struct Packet) + 10);
pptr->payload = (unsigned char *) ((char *) pptr + sizeof(struct Packet);
Note that in this case, to get the Packet structure from the payload pointer, you have to use sizeof(Packet) instead of only sizeof(Header).
Two things to note about the code above:
I don't cast the result of malloc
sizeof(char) (and also the size of unsigned char) is specified to always be one, so no need for sizeof
edit: a better way of phrasing this: What's the correct [modern] way to ensure that a struct is a specific size in bytes?
just spending a relaxing saturday afternoon debugging a legacy codebase, and having a bit of trouble figuring this out. The compiler error I get is this:
INC/flx.h:33: error: dereferencing pointer to incomplete type
the code at line 33 looks like this
typedef struct flx_head {
FHEAD_COMMON;
LONG frames_in_table; /* size of index */
LONG index_oset; /* offset to index */
LONG path_oset; /* offset to flipath record chunk */
/* this will insure that a Flx_head is the same size as a fli_head but won't
* work if there is < 2 bytes left (value <= 0) */
PADTO(sizeof(Fli_head),flx_head,flxpad); /* line 33 is this one */
} Flx_head;
well okay so I can see that the struct is referring to itself to pad it out somehow. But I don't know an alternative way of doing what PADTO does without the self reference.
here's what PADTO is defined as
#define MEMBER(struc,field) \
((struc*)NULL)->field
/* returns offset of field within a given struct name,
* and field name ie: OFFSET(struct sname,fieldname) */
#define OFFSET(struc,field) \
(USHORT)((ULONG)((PTR)&MEMBER(struc,field)-(PTR)NULL))
/* offset to first byte after a field */
#define POSTOSET(struc,field) \
(OFFSET(struc,field)+sizeof(MEMBER(struc,field)))
/* macro for defining pad sizes in structures can not define a pad of
* less than two bytes one may use pname for the offset to it but
* sizeof(struc->pname) will not be valid
*
* struct sname {
* char fld1[64];
* PADTO(68,sname,pname);
* };
* will make:
*
* struct sname {
* char fld1[64];
* UBYTE pname[1];
* UBYTE __pname[3];
* };
*/
#define PADTO(sz,struc,padfld) \
UBYTE padfld[1];UBYTE __##padfld[(sz)-OFFSET(struct struc,padfld)-1]
here is FHEAD_COMMON
#define FHEAD_COMMON \
CHUNKID_FIELDS;\
USHORT frame_count;\
USHORT width;\
USHORT height;\
USHORT bits_a_pixel;\
SHORT flags;\
LONG speed;\
USHORT unused;\
Fli_id id;\
USHORT aspect_dx;\
USHORT aspect_dy;\
UBYTE commonpad[38] /* should be total of 80 bytes (48 for unique) */
and flihead
typedef struct fli_head {
FHEAD_COMMON;
LONG frame1_oset;
LONG frame2_oset;
UBYTE padfill[40];
} Fli_head;
this is Autodesk animator pro. what I am working on is the "reference" implementation for the FLI file format- which you can see a spec for here:
http://www.compuphase.com/flic.htm
Incidentally, I'm pretty sure that what the /source code/ there refers to as "flx" is actually what that webpage there calls "flc" , not what it calls "flx"
update:
better source for format info http://drdobbs.com/architecture-and-design/184408954
It isn't pretty, but one possibility is to define another identical structure and use its size to determine the padding for the one you actually want to use:
#define FLX_HEAD \
FHEAD_COMMON;\
LONG frames_in_table; /* size of index */ \
LONG index_oset; /* offset to index */ \
LONG path_oset /* offset to flipath record chunk */
struct flx_head_unpadded {
FLX_HEAD;
};
typedef struct flx_head {
FLX_HEAD;
char __flxpad[sizeof(Fli_head)-sizeof(struct flx_head_unpadded)];
} Flx_head;
I suppose the answer depends on what you're trying to achieve. In most cases, the correct, modern way to pad a struct is not to. The only situation I can think of where it's legitimate to pad a struct is when you have a library interface where the caller creates objects of a structure type and passes pointers to them to the library, and where you want to leave room to add additional fields to the structure without breaking the ABI. In this case, I would start out with something like char pad[256]; and change it to char pad[256-3*sizeof(long)]; or similar as you add fields (making sure to avoid internal padding when you add fields).
Define it in a union with a byte/char array of the desired size?
I can think quickly of some scenarios where this is needed:
1) Compatibility with old software that uses flat binary files to store data, (as in OP).
2) Interaction with drivers and/or hardware
3) Forcing structs to be an exact multiple of the cache line size to prevent false sharing in inter-thread comms.
If you only want to achieve specific size you can use (for sure working in GCC):
typedef union {
struct {
FHEAD_COMMON;
LONG frames_in_table; /* size of index */
LONG index_oset; /* offset to index */
LONG path_oset; /* offset to flipath record chunk */
};
uint8_t __padding[128];
} Flx_head;
void test() {
Flx_head boo;
boo.frames_in_table= 0;
}
I am not sure if this is modern enough. If youc compiler does not support anonymous struct (the one inside union) it will get "messy".
Also you must remember that struct is now padded, but not aligned to specific data size.
thanks everyone. not sure who to award the green checkmark to, since I found this solution as a result of everyone kind of hinting and pointing in the right direction. After looking at the problem, it struck me that the struct just needs to be exactly 128 bytes. I "hand parsed" the macro, cross referencing with the spec and ended up with this:
typedef struct flx_head {
FHEAD_COMMON;
LONG frames_in_table; /* size of index */
LONG index_oset; /* offset to index */
LONG path_oset; /* offset to flipath record chunk */
UBYTE flxpad[36];
} Flx_head;
which is 128-(80+4+4+4) = 36