I have a serial communication, where I will serialize a binary file and send it to a microcontroller.
My question is how to store this file.
I thinked to use the structure:
typedef struct
{
uint8_t id; // protocol
size_t data_size; // to help know the file
uint8_t data[1024]; // data bytes
} RECEIVED_FILE_t;
The file is a binary and can vary up to 50Kb, how would it be better to store it in this structure, reference it with a pointer or a vector? Which would be more efficient?
With a uint8_t data[1024] i can store up to 1MB file? (1 byte * 1024 = 1024 Bytes = 1MB) Is correct?
I can change this structure.
You probably want something like this:
typedef struct
{
uint8_t id;
size_t data_size;
uint8_t *pdata; // pointer to data bytes
} RECEIVED_FILE_t;
Once you have the size, allocate memory and read the data there.
Pseudo code:
RECEIVED_FILE_t file;
...
int data_size = ReadSize(...); // get the data size somehow
file.data_size = data_size; // put the size into the struct
file.pdata = malloc(data_size); // allocate memory for the data
ReadData(file.data_size, file.pdata); // read data_size bytes and put them
// in the memory where pdata points to
I have no idea how you get the data size and how you actually read the data. ReadSize and ReadData are just there to illustrate how you code could work.
Related
I'm working on a small project that communicates with a FPGA board. It sends data to FPGA and sometimes the FPGA will send data back. So my colleague and I are going to design a simple communication protocol.
We decide to add a checksum field to our protocol. But we have different ideas about where to put it. I suggest putting it at the head of a packet and he prefers putting it at tail.
To put it in code, here is my suggestion:
typedef struct {
uint32_t magic_number;
uint16_t frame_type;
uint16_t length;
uint16_t checksum;
uint8_t data[];
}blah_blah;
And here is his:
typedef struct {
uint32_t magic_number;
uint16_t frame_type;
uint16_t length;
uint8_t data[]; //let's assume it is a valid syntax
uint16_t checksum;
}blah_blah;
My point is that I can declare the struct easily and directly cast the last data field to another struct that I want, iff the variable length part is at the end of the packet. And his argument is FPGA reads and writes data by clock, so it's more convenient to update checksum each clock and finally put or check it at the end of the packet.
My knowledge about digital design is very limited, so I don't know which one is a better idea. Where should I put the checksum field?
Edit: Somebody reminds me about strict aliasing when casting a pointer. Even if I use memcpy to avoid casting, it is still easier for me to get checksum in the head. If the checksum is in the tail then I have to cast the buffer to a const char* then add the length offset and then cast it to a const uint16_t* to get it.
For the embedded device, it depends on the hardware. If you're doing it a byte at a time via interrupts, probably better at the end. (you calculate it as you go reading/writing RX/TX registers and then tack it on, or check it, depending on whether you're reading or writing.)
If its DMA driven and two separate blocks, or only one of it is DMA driven, then it probably doesn't matter. You'll end up calculating the checksum/crc after all the data has reached you.
For the FPGA, however, it's almost certainly easier for them to 'tack on' the checksum at the end after the last byte has gone out, rather than pre-calculate the checksum on the packet, then insert it in the middle, then send out the data.
There are a number of ways to do this.
Except for the fact that uint8_t data[]; must be at the end of the struct.
Thus, as I suggested above, don't put the checksum inside the struct. You could even leave the data out of the struct [or not].
As I suggested in my top comment, a buffer could look like:
struct|data|csum
And, checksumming all of that, could sum to zero if the packet is valid.
Here's some sample code for one way that I've seen used (e.g. SDLC, etc.). It is crude and only checks some errors. It allow for the data payload to be in a different buffer.
typedef struct {
uint32_t magic_number;
uint16_t frame_type;
uint16_t length;
uint8_t data[];
} blah_blah;
#define CSUMSEED 0x1234
#define MAGIC 0xDEADBEEF
uint16_t
calcsum(uint16_t csum,const void *ptr,uint16_t length)
{
uint8_t *data = ptr;
for (; length != 0; --length, ++data)
csum += *data;
return csum;
}
void
sendone(int socket,uint16_t type,const void *data,uint16_t length)
{
blah_blah hdr;
uint16_t csum = CSUMSEED;
hdr.magic_number = MAGIC;
hdr.frame_type = type;
hdr.length = length;
csum = calcsum(csum,&hdr,sizeof(hdr));
csum = calcsum(csum,data,length);
// convert csum to "inverse" ???
csum = ~csum;
// could use writev here instead ...
write(socket,&hdr,sizeof(hdr));
write(socket,data,length);
write(socket,&csum,sizeof(csum));
}
void
recvone(int socket,blah_blah *hdr,void *data)
// data -- must be large enough to hold _maximum_ payload
{
uint16_t csum = CSUMSEED;
ssize_t length;
// get the header struct
length = read(socket,hdr,sizeof(*hdr));
if (length != sizeof(*hdr))
error();
// magic number must match
if (hdr.magic != MAGIC)
error()
// get checksum for header
csum = calcsum(csum,hdr,sizeof(*hdr));
// get the data
length = read(socket,data,hdr->length + sizeof(uint16_t));
if (length != (hdr->length + sizeof(uint16_t))
error();
// get remaining checksum (including the trailing/appended CRC)
csum = calcsum(data,data,length);
if (csum != 0)
error();
}
You could adapt this to embed the data at the end of the struct if you wish [as shown in the struct definition] with some slight modifications.
Here's a slight modification that appends the checksum. The receiver compares the checksum gotten from read with the calculated checksum:
typedef struct {
uint32_t magic_number;
uint16_t frame_type;
uint16_t length;
uint8_t data[];
} blah_blah;
#define CSUMSEED 0x1234
#define MAGIC 0xDEADBEEF
uint16_t
calcsum(uint16_t csum,const void *ptr,uint16_t length)
{
uint8_t *data = ptr;
for (; length != 0; --length, ++data)
csum += *data;
return csum;
}
void
sendone(int socket,uint16_t type,const void *data,uint16_t length)
{
blah_blah hdr;
uint16_t csum = CSUMSEED;
hdr.magic_number = MAGIC;
hdr.frame_type = type;
hdr.length = length;
csum = calcsum(csum,&hdr,sizeof(hdr));
csum = calcsum(csum,data,length);
// could use writev here instead ...
write(socket,&hdr,sizeof(hdr));
write(socket,data,length);
write(socket,&csum,sizeof(csum));
}
void
recvone(int socket,blah_blah *hdr,void *data)
// data -- must be large enough to hold _maximum_ payload
{
uint16_t csum = CSUMSEED;
ssize_t length;
// get the header struct
length = read(socket,hdr,sizeof(*hdr));
if (length != sizeof(*hdr))
error();
// magic number must match
if (hdr.magic != MAGIC)
error()
// get checksum for header
csum = calcsum(csum,hdr,sizeof(*hdr));
// get the data
length = read(socket,data,hdr->length);
if (length != hdr->length)
error();
// get data checksum
csum = calcsum(data,data,length);
// get message checksum
uint16_t csum2;
length = read(socket,&csum2,sizeof(csum2));
if (length != sizeof(csum2))
error();
// validate the checksum
if (csum2 != csum)
error();
}
I have a 1-byte pragma packed struct in C which I want to copy into a byte array for serialization purpose to be sent over a serial port.
#pragma pack(push, 1)
typedef struct {
uint8_t ck_a;
uint8_t ck_b;
} UBXChecksum_t ;
#pragma pack(pop)
What is the best way of serializing it into a byte array, should I just use memcpy()?
void writeStructToArray(const void* inStruct,
const uint16_t inLenStruct,
uint8_t* const outArray)
{
memcpy(outArray, inStruct, inLenStruct);
}
or better use byte-by-byte copying doing pointer typecasting?
void writeStructToArray(const void* inStruct,
const uint16_t inLenStruct,
uint8_t* const outArray)
{
for(uint16_t i = 0; i < inLenStruct; i++)
{
outArray[i] = ((uint8_t*)inStruct)[i];
}
}
As Kamil Cuk commented, your two proposals are nearly the same with some possible speed difference.
Another option would be to use a union:
typedef struct {
uint8_t ck_a;
uint8_t ck_b;
} UBXChecksum_t ;
union convert {
UBXChecksum_t checksum;
char buffer[sizeof UBXChecksum_t];
};
UBXChecksum_t checksum;
union convert converter;
converter.checksum = checksum;
passArrayToSomeFunction(converter.buffer, sizeof(converter.buffer));
You don't have to copy the data to convert it to an array. You could pass a pointer to the structure (if necessary casted to char* or void*) and the structure size to a function that sends the data to the serial port. Example:
typedef struct {
uint8_t ck_a;
uint8_t ck_b;
} UBXChecksum_t ;
int sendData(void *buf, size_t size);
UBXChecksum_t checksum;
/* ... */
int rc = sendData(&checksum, sizeof(checksum));
All these variants send the structure's internal representation as binary data. Normally "serializing" is understood as a way to convert the data into a platform-independent format.
Sending binary data structures works if the receiving system is of the same type and using the same compiler. You might get problems when the receiving system uses different byte order or different data type sizes.
In your case you have a structure of two uint8_t values, so the size is fixed and the byte order is not a problem.
It is OK to send binary data if the requirement for the structure is to match a specified binary data protocol and you are prepared to handle the byte order if necessary.
memcpy() will not consider endiannsess of the system. so if Sender is big endian and receiver is little endian then then will be a conflict in the receiver for the structure variable value.
With the second method you know how the byte stream is prepared at sender so at the receiving end also it can receive accordingly to make sure of the proper structure variable value.
If the endianness of the systems is same and endianness is not a concern then both the method will serve the purpose and memcpy() will be faster compare to the assigning the byte value in a loop.
I am doing some network programming, and I use some struct to describe my frame header like that:
struct my_frame_header {
uint16_t field1;
uint16_t field2;
};
And so, when I have a buffer frame I can do something like that:
uint8_t buffer[BUFFER_SIZE];
struct my_frame_header *frame_header = (struct my_frame_header *)buffer;
my_read(buffer, BUFFER_SIZE);
I can now access to the header field like that:
ntohl(frame_header->field1);
Now, my question is: What is the most elegant way to access the data after the structure? (i.e. to get a pointer at the beginning of the data part)
Well, if I understood your question correctly, you can do something like
uint8_t * data = (uint8_t *)buffer + sizeof (struct my_frame)
then, data will be a pointer to the next uint8_t element after the header.
uint8_t buffer[BUFFER_SIZE];
struct my_frame_header *frame_header = (struct my_frame_header *)buffer;
This is wrong, the base address of buffer can be unaligned for my_frame_header.
Take look to Memory access and alignment
On the other hand:
The block that malloc gives you is guaranteed to be aligned so that it
can hold any type of data.
Then, you can use malloc in order to skip this problem:
uint8_t *buffer = malloc(BUFFER_SIZE);
I am very new to C/C++, so I know that this question is probably trivial but I don't understand how to proceed. I have allocated blocks of memory that are 512 bytes each like so (Sector is a struct of size 512 bytes):
char* block = (char *) malloc (sizeof(Sector));
I have another struct that is 128 bytes in size:
typedef struct inode {
int fileSize;
int fileType;
int* blockPointer[30];
} inodeFile;
What I want to do is overlay this struct on the block. I need each block to contain 4 inode structs, but each struct may not have any values associated with it until later on in the program. So for example:
((*inodeFile) block)->fileSize = 10;
If I am understanding correctly, this is setting the first 4 bytes of the pointer block to fileSize. I would continue to do this for each field of the struct.
My question is, how can I do this for 4 different inodes? And since there are 4 different inodes, how can I return the values of each field for the different structs? It seems like simply saying block->fileSize wouldn't work because there can be up to four different file sizes.
You can use a union to overlay the blocks on the sectors. Because the fields are of int type or pointer or size 512 there shouldn't be an alignment problem so long as the natural size is 32-bit.
typedef struct {
int fileSize;
int fileType;
int* blockPointer[30];
} inode;
typedef union {
unsigned char bytes[512];
inode block[4];
} sector;
int main()
{
sector thisec;
thisec.bytes[511] = 0;
thisec.block[0].fileSize = 10;
return 0;
}
Ok I am putting the whole struct here, its specification of a protocol named openflow that is implemented in some of industrial switches , so the struct is like:
struct ofp_packet_in {
struct ofp_header header;
uint32_t buffer_id; /* ID assigned by datapath. */
uint16_t total_len; /* Full length of frame. */
uint16_t in_port; /* Port on which frame was received. */
uint8_t reason; /* Reason packet is being sent (one of OFPR_*) */
uint8_t pad;
uint8_t data[0]; /* Ethernet frame, halfway through 32-bit word,
so the IP header is 32-bit aligned. The
amount of data is inferred from the length
field in the header. Because of padding,
offsetof(struct ofp_packet_in, data) ==
sizeof(struct ofp_packet_in) - 2. */
};
OFP_ASSERT(sizeof(struct ofp_packet_in) == 20);
now I have to fill up some data in the last field that is -uint8_t data[0] which can be varied and info is gathered from the length field inside the header. I have to construct a packet in, and for that data has to be put in. Please take a look.
You'll need to use a dynamic allocation and copy the contents.
Something like:
#include <stdlib.h>
#include <string.h>
void foo(void) {
struct some_struct *container = malloc(sizeof(struct some_struct) + 100);
if (!container) {
// handle out-of-memory situation
}
memcpy(container->data, some_data, 100);
}
You cannot do this. It won't fit! The array in the struct is 0 chars long and you're trying to stuff a 100-char array into it.
If, for some reason, you are certain that the memory after the struct is available, e.g. you just malloc'ed it like this:
some_struct *foo = (some_struct*)malloc(sizeof(some_struct) + 100);
Then you can do this:
memcpy(foo->data, some_data, 100);
It's hideous, and probably still undefined behaviour, but I've seen APIs that require this (Windows?).
you can't.
You defined the size of some_struct.data to be 0, meaning it cannot hold any items.
If all you want is to copy max. 100 items into it, then you can define the size statically:
struct some_struct {
char data[100]; // some_struct.data has room for up to 100 characters
};