copying struct 8 bytes at at time - c

I'm working in the arduino environment. I have a struct defined as follows. The struct will ultimately be encrypted and sent wirelessly over a radio link layer. It's 32bytes long.
struct SENSORTYPE{
int sensor1:8;
int sensor2:8;
int sensor3:8;
int sensor4:8;
};
struct SENSOR{
float sensor1;
float sensor2;
float sensor3;
float sensor4;
};
struct HEADER{
byte type;
short id;
short to;
short from;
byte version;
long _buff;
SENSORTYPE sensortype;
SENSOR sensor;
};
HEADER header;
I have an XTEA encryption/decryption routine that's defined as follows and is verified to work. It operates on two 32bit blocks at at time.
void xteaEncrypt( unsigned long v[2])
void xteaDecrypt(unsigned long v[2])
What I'm trying to do is run header through xteaEncrypt. I'm getting tripped up on converting my struct to the two 32bit numbers. The following is what I have so far.
#define BLOCK_SIZE 8
header.type = 1; //test value
header._buff = 1; //test value
byte data[BLOCK_SIZE]; //8 byte buffer to encrypt/decrypt
byte buff[32]; //32 byte buffer to put encrypted/decrypted data into
for (uint32_t i = 0; i < 4; i++){ //4 times (4 * 8 = 32)
memcpy(data, &header+(i*BLOCK_SIZE), BLOCK_SIZE); //copy 8 bytes from header struct into data
xteaEncrypt((uint32_t*)data); //encrypt data
memcpy(&buff+(i*8), data, BLOCK_SIZE); //put encrypted data into the new buffer
}
memcpy(&header, &buff, sizeof(header)); //copy into original header for convenience
//now decrypt it back
for (uint32_t i = 0; i < 4; i++){
memcpy(data, &header+(i*BLOCK_SIZE), BLOCK_SIZE);
xteaDecrypt((uint32_t*)data);
memcpy(&buff+(i*8), data, BLOCK_SIZE);
}
memcpy(&header, &buff, sizeof(header));
After encryption header.type = 0xee and header._buff = C0010000. After decryption, header.type = 1 and _buff still = C0010000 so it would seem there is an error in my memcpy'ing but I can't find it. Any help would be greatly appreciated. This one has been particularly hard to debug for me. If I'm going about this completely wrong let me know and point me in the right direction.

You're getting tripped up by pointer arithmetic. Consider the snippet &header+(i*BLOCK_SIZE). What you evidently expect to happen is that you'll get some address, let's call it addr, and then add some small number to it to calculate a new address, i.e.
finalAddress = addr + (i * 8);
But because &header is a pointer to a struct HEADER, the actual calculation the compiler does is
finalAddress = addr + (sizeof(struct HEADER) * i * 8);
The result is an address that's well beyond the end of the header for any i greater than 0. The same thing is happening with &buff+(i*8), since &buff is a pointer to 32 bytes.
To solve the problem, I recommend using intermediate variables that are char *, e.g.
char *headAddress = (char *)&header;
char *buffAddress = (char *)&buff;
for (uint32_t i = 0; i < 4; i++)
{
memcpy(data, headAddress+(i*BLOCK_SIZE), BLOCK_SIZE);
xteaEncrypt((uint32_t*)data);
memcpy(&buffAddress+(i*BLOCK_SIZE), data, BLOCK_SIZE);
}

Related

Convert uint8_t array to structure in C

I'm beginner and I have a problem. I have this structure:
typedef struct {
char data[26];
int index;
Placa_baza pb; // char nume_placa[10], int index_placa;
} PC;
And a structure vector:
static PC computers[5] = { ... };
I need to have a vector of type uint8_t pc[5*sizeof(computers)] instead of the structure vector.
Is it well declared that way? :
uint8_t pc[5*sizeof(computers)]
How can I convert (cast) vector uint8_t pc[5*sizeof(computers)] to PC?
To use the uint8_t pointer to address the structure, how should it be written?
Thank you in advance.
Your pc array, which could serve as a backup for the PC data is too large: it is sufficient to define it as:
uint8_t pc[sizeof(computers)];
Or possibly:
uint8_t pc[5 * sizeof(PC)];
You can then copy computers to pc with:
memcpy(pc, computers, sizeof pc);
You could also use a pointer to access the pc array as an array of PC:
PC *p = (PC *)pc; // Don't do this!
Note however that this has undefined behavior as the byte array pc might not be properly aligned to access members of the PC structure, especially the index member and using such a pointer is a violation of the strict aliasing rule. It would be much better to define pc as PC pc[5]; and access this array via a uint8_t pointer of so required.
WARNING: Below program is just demonstration purpose, it may not behave same way with all compilers/systems. You can use it to test your compilers or systems behavior and modify accordingly.
In the below program am copying the contents from the structure computers to unit8_t.
as you can see its not easy and not portable, because we need to extract the data as per the boundaries of memory, allocated for variables.
#include <stdio.h>
#include <stdint.h>
#include <string.h>
typedef struct
{
char nume_placa[10];
int index_placa;
}Placa_baza;
typedef struct {
char data[26];
int index;
Placa_baza pb;
}PC;
int main()
{
printf("sizeof(int) = %zu\n", sizeof(int));
printf("sizeof(Placa_baza) = %zu\n", sizeof(Placa_baza));
printf("sizeof(PC) = %zu\n", sizeof(PC));
static PC computers[3] = { {"data1",1,"comp1", 0}, {"data2",2,"comp2", 1}, {"data3",3,"comp3", 2} };
printf("sizeof(computers) = %zu\n\n", sizeof(computers));
for(int i =0; i<3; i++)
printf("data = %s, index =%d, pb.nume_placa =%s, pb.index_placa =%d\n",
computers[i].data,
computers[i].index,
computers[i].pb.nume_placa,
computers[i].pb.index_placa
);
uint8_t uint8_t_pc[sizeof(computers)] = {0};
// for copying the contents from pc (uint8_t), used same variable names as that of structures
/* typedef struct { */
char data[26];
int index;
/* Placa_baza pb;
} PC; */
/* typedef struct
{ */
char nume_placa[10];
int index_placa;
/* }Placa_baza;
*/
printf("\n sizeof(uint8_t_pc) = %zu\n", sizeof(uint8_t_pc));
memcpy(uint8_t_pc,computers,sizeof(computers));
int count = 0;
uint8_t* temp = uint8_t_pc;
printf("\n **From uint8_t memory ***\n");
while(count < 3) {
memcpy(data, temp, 26);
// since there is a padding of 2 bytes , so extract from 28
memcpy(&index, temp+28, 4);
memcpy(nume_placa, temp+32, 10);
//again there is a padding of 2 bytes
memcpy(&index_placa, temp+44, 4);
printf("data = %s, index = %d, nume_placa =%s , index_placa =%d\n", data, index, nume_placa, index_placa);
temp = temp+sizeof(computers[0]);
count++;
}
return 0;
}
Output:
sizeof(int) = 4
sizeof(Placa_baza) = 16
sizeof(PC) = 48
sizeof(computers) = 144
data = data1, index =1, pb.nume_placa =comp1, pb.index_placa =0
data = data2, index =2, pb.nume_placa =comp2, pb.index_placa =1
data = data3, index =3, pb.nume_placa =comp3, pb.index_placa =2
sizeof(uint8_t_pc) = 144
**From uint8_t memory ***
data = data1, index = 1, nume_placa =comp1 , index_placa =0
data = data2, index = 2, nume_placa =comp2 , index_placa =1
data = data3, index = 3, nume_placa =comp3 , index_placa =2
online source
Update:
Indeed we can use offsetof to get the offset of any member of the structure, so the statements inside while can also be replaced by below statments.
memcpy(data, temp+offsetof(PC, data), sizeof(computers[count].data));
memcpy(&index, temp+offsetof(PC, index), sizeof index);
memcpy(nume_placa, temp+offsetof(PC, pb.nume_placa), sizeof computers[count].pb.nume_placa);
memcpy(&index_placa, temp+offsetof(PC, pb.index_placa), sizeof index_placa);

How do i optimize the size of socket message i am sending , without missing any data?

My question is like this
below is the structure which i have to populate and send it through socket
struct Mystruct
{
int numofarray1elements;
array1[50];
int numofarray2elements;
array2[25];
};
Here size of 1 array1 member is 1024 bytes i.e total size of array1 = 50*1024 = 51200 bytes
size of 1 member of array2 is say 500 so total size of array2 = 12500 bytes
whenever i use send api of socket(unix domain socket) i have to send 51200+12500 +4+4 = 63708 bytes
Problem is i have to send entire size of structure even if i have very less
numofarray1elements, and numofarray2elements
this leads to performance issue
where in almost cases my original data can be less than 10kb but i end up sending 63k everytime
i cannot keep dynamic arrays as its socket message
i have already otimized my data strutures , array1 must have max 50 elements
array2 must have max 25 elements.
now is there any way that i can send exact data which i have populated?
please provide some method if any
Thanks
Actually the way for doing it is to have messages of variable length. One of the methods is to use a single array with undefined size as the last element of the struct. Depending on the types of the messages it could be represented by messages or by bytes, e.g.
struct Mystruct
{
int numofarray1elements;
int numofarray2elements;
char array[];
};
The size of your struct can be calculated as the size of static fields plus sizes needed for actual payload as this:
int packetSize = (sizeof(struct Mystruct) + n1 * sizeof(el1) + n2 * sizeof(el2));
now you can use it to allocate the struct and send the packet in a single operation.
struct Mystruct *packet = malloc(packetSize);
// assign packet fields
...
write(fd, packet, packetSize);
Here is a simple example which emulates a version of write/read. It will work if writer and reader have the same endian order. It also assumes that the packet size is sent separately an is known to the reader.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
struct Mystruct {
int numofarray1elements;
int numofarray2elements;
char payload[];
};
struct Element1 {
int len;
char name[30];
};
struct Element2 {
char name[20];
int len;
};
// reader emulation
void readData(int packetSize, char *dataIn) {
union {
char data[packetSize];
struct Mystruct packet;
} dataUnion;
int i;
struct Element1 *e1 = NULL;
struct Element2 *e2 = NULL;
memcpy(dataUnion.data, dataIn, packetSize);
printf("Read data e1 size is %d, e2 size is %d\n",
dataUnion.packet.numofarray1elements, dataUnion.packet.numofarray2elements);
e1 = malloc(sizeof(struct Element1) * dataUnion.packet.numofarray1elements);
e2 = malloc(sizeof(struct Element2) * dataUnion.packet.numofarray2elements);
memcpy(e1, dataUnion.packet.payload, sizeof(struct Element1) * dataUnion.packet.numofarray1elements);
memcpy(e2, dataUnion.packet.payload + sizeof(struct Element1) * dataUnion.packet.numofarray1elements,
sizeof(struct Element2) * dataUnion.packet.numofarray2elements);
for (i = 0; i < dataUnion.packet.numofarray1elements; i++) {
printf("e1[%d].len = %d, name = %s\n", i, e1[i].len, e1[i].name);
}
for (i = 0; i < dataUnion.packet.numofarray2elements; i++) {
printf("e2[%d].len = %d, name = %s\n", i, e2[i].len, e2[i].name);
}
}
void main() {
struct Element1 e1[4];
struct Element2 e2[8];
int i;
int packetSize;
struct Mystruct *packet = NULL;
for (i = 0; i < 4; i++) {
sprintf(e1[i].name, "e1:%d", i);
e1[i].len = i;
}
for (i = 0; i < 8; i++) {
sprintf(e2[i].name, "e2:%d", i);
e2[i].len = i;
}
// emulated write data
packetSize = (sizeof(struct Mystruct) + sizeof(e1) + sizeof(e2));
packet = malloc(packetSize);
packet->numofarray1elements = 4;
packet->numofarray2elements = 8;
memcpy(packet->payload, &e1, sizeof(e1));
memcpy(packet->payload + sizeof e1, &e2, sizeof(e2));
// here you do write data, e.g. write(socFd, packet, packetSize);
// emulate read data
readData(packetSize, (char*)packet);
}
Instead of using structures, use TLV mechanism. So for your solution: you can use Type , , Count of Type, Length , Value.
Define types which is known at both receiver and sender side
Define your message structure as
Type taking 2 bytes, Count-of-type taking 2 or 4 bytes , Length taking 4 bytes and Value.
This is extensible as you can add any number of types in future as long as the type is known at both side. At receiver side if type is not known, they can ignore that TLV.
You should not send structs over network protocols (or store them in files, etc). You need to serialize them.
But to solve your problem, simply change your send code:
//this is what your code looks like, I assume:
write(sockFd, myStructVariable, sizeof(struct MyStruct));
to:
//be aware of writev(2) if you want to send these in one system call at once, or copy them into one buffer
write(sockFd, myStructVariable.numofarray1elements, sizeof(int));
write(sockFd, myStructVariable.array1, sizeof(MyArray1) * myStructVariable.numofarray1elements);
write(sockFd, myStructVariable.numofarray2elements, sizeof(int));
write(sockFd, myStructVariable.array2, sizeof(MyArray2) * myStructVariable.numofarray2elements);
or something similar
Then change your receive code:
read(sockFd, myStructVariable.numofarray1elements, sizeof(int));
read(sockFd, myStructVariable.array1, sizeof(MyArray1) * myStructVariable.numofarray1elements);
read(sockFd, myStructVariable.numofarray2elements, sizeof(int));
read(sockFd, myStructVariable.array2, sizeof(MyArray2) * myStructVariable.numofarray2elements);
of course, you could also leave the array and counts defined locally and send those instead. Make sure you check for errors, short reads, etc
Take a look at flatbuffers as well, this lets you write structures that come with serialization functions

How to convert bytes stored inside a buffer to a variable?

So I'm reading from a file descriptor which contains an int variable in its raw byte format.
So I'm doing:
char buffer[sizeof(int)];
ssize_t sizeOfFile = read(sock_fd, buffer, sizeof(int));
int extractedInt = ???;
How can I convert that buffer to an integer? I was thinking of memcpy but was wondering if there are better ways.
You could read directly an integer
int extractedInt;
ssize_t sizeOfFile = read(sock_fd, &extractedInt, sizeof(int));
read will read the size of an int bytes, and store them into extractedInt.
If your int is actually a string in a file you want to convert to an int, the procedure is a bit different.
#define SIZE 20
char buffer[SIZE]; // ensure there is enough space for a string containing an integer
ssize_t sizeOfFile = read(sock_fd, buffer, SIZE);
int extractedInt = atoi(buffer); // convert string to integer
I can guess from your code that you're reading from the network. This is then not portable to just read a int from the buffer, in your network protocol you chose a certain endianness but you cannot expect that all the platforms where your program will run to have the same, so it will lead to bad convertions.
And other proposed solutions of asking read to return an int will lead to the same problem.
So in your case, I can only advice to iterate through your array and compute the integer by progressively placing the bytes at the right place depending on the endianness of the platform.
You can detect the endianness of the build target platform by using the macro __BYTE_ORDER__in GCC.
There is an example for network data that is big endian:
// construct an `int` with the bytes in the given buffer
// considering the buffer contains the representation
// of an int in big endian
int buffer_to_int(char* buffer, int buffer_size) {
int result = 0;
int i;
char sign = buffer[0] & 0x80;
char * res_bytes = (char*)&result; // this pointer allows to access the int bytes
int offset = sizeof(int) - buffer_size;
if( sign != 0 )
sign = 0xFF;
if( offset < 0 ) {
// not representable with a `int` type
// we chose here to return the closest representable value
if( sign == 0 ) { //positive
return INT_MAX;
} else {
return INT_MIN;
}
}
#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
for(i=0; i<buffer_size; i++) {
res_bytes[i] = buffer[buffer_size-i-1]; // invert the bytes
}
for(i=0; i<offset; i++){
res_bytes[buffer_size+i] = sign;
}
#else
// same endianness, so simply copy bytes using memcpy
memcpy(&result + offset, buffer, buffer_size);
for(i=0; i<offset; i++){
res_bytes[i] = sign;
}
#endif
return result;
}

How to read a BMP file into a grid of pixels in C?

I'm trying to read in a given BMP file and store it in an image. I'm confused about the syntax since I'm struggling to understand the .h files given to me. Here's how I'm going about reading in the image:
BMPImage * readImage(FILE * fp) {
// FILL IN
BMPHeader * hp = malloc(sizeof(BMPHeader);
Pixel * p = malloc(sizeof(Pixel));
p -> pixels = malloc(p -> height_px * sizeof(Pixel *));
for(int i = 0; i < p -> height_px; i++){
p -> pixels[i] = malloc(p -> width_px * sizeof(Pixel));
}
for (i = 0; i < p -> height_px; i++){
for(int j = 0; j < p -> width_px; j++){
Pixel px = fread(hp, sizeof(Pixel), 1, fp);
p -> pixels[i][j] = px;
}
}
return p;
}
Here's the .h file:
typedef struct __attribute__((packed)) BMPHeader { // Total: 54 bytes
uint16_t type; // Magic identifier: 0x4d42
uint32_t size; // File size in bytes
uint16_t reserved1; // Not used
uint16_t reserved2; // Not used
uint32_t offset; // Offset to image data in bytes from beginning of file (54 bytes)
uint32_t dib_header_size; // DIB Header size in bytes (40 bytes)
int32_t width_px; // Width of the image
int32_t height_px; // Height of image
uint16_t num_planes; // Number of color planes
uint16_t bits_per_pixel; // Bits per pixel
uint32_t compression; // Compression type
uint32_t image_size_bytes; // Image size in bytes
int32_t x_resolution_ppm; // Pixels per meter
int32_t y_resolution_ppm; // Pixels per meter
uint32_t num_colors; // Number of colors
uint32_t important_colors; // Important colors
} BMPHeader;
typedef struct __attribute__((packed)) Pixel {
uint8_t blue;
uint8_t green;
uint8_t red;
uint8_t alpha;
} Pixel;
typedef struct BMPImage {
BMPHeader header;
int norm_height; //normalized height
Pixel * * pixels;
} BMPImage;
How should I correct my reading method?
Given how you're struggling with structs, lists, and basic I/O, reading in a BMP is probably over your head right now. I'd suggest trying to populate a simpler struct. If this is for a production code, please use an existing library.
I'm going to do some basic fixups so your code at least compiles, hopefully you can take it from there.
BMPImage *readImage(FILE * fp) {
// Allocate space for the image.
// This also covers BMPHeader, since it's not a pointer.
BMPImage *bmp = malloc(sizeof(BMPImage));
BMPHeader *bmph = &(bmp->header);
The basic unit is the BMPImage struct. That contains the BMPHeader as well as a pointer to list of Pixels. The BMPHeader is not a pointer, the actual memory is contained in the BMPImage struct, so malloc(sizeof(BMPImage)) also allocates memory for the BMPHeader. I've taken a pointer to the BMPHeader to make it easier to work with, otherwise the code is sprinkled with bmp.header->height_x.
I'll leave populating the BMPHeader to you. Once you've done that...
// Allocate space for the pixels.
bmp->pixels = malloc( bmph->height_px * sizeof(Pixel *) );
for(int i = 0; i < bmph->height_px; i++){
bmp->pixels[i] = malloc(bmph->width_px * sizeof(Pixel));
}
I think you've basically got the memory allocation right. Your problem was trying to use the Pixel struct when you should have been using BMPImage. While the height and width comes from the BMPHeader.
That isn't really how you should be reading in a BMP since the size of the pixels are variable according to the bits_per_pixel header. Your structs only support the 8bpp format. It also might be compressed. I'm going to assume this is an exercise and not production code, so I'll avoid going further into the details of BMP.
// Read in each pixel
for (int i = 0; i < bmph->height_px; i++){
for(int j = 0; j < bmph->width_px; j++){
Pixel px;
if( fread(&px, sizeof(Pixel), 1, fp) < 1 ) {
fprintf(stderr, "Error while reading bmp: %s", strerror(errno));
return NULL;
}
bmp->pixels[i][j] = px;
}
}
Again, you had the BMPImage and BMPHeader mixed up with Pixel. Furthermore, fopen does not return the structure to be read. Instead, you're responsible for allocating the necessary memory (a theme in C). fopen returns the number of items read. It's very important to do error checking on all file operations, else you'll be left with difficult to understand garbage.
I suspect you can do this much simpler by reading in all the pixels in one big chunk, but I don't know the details of the BMP format.
This is by no means 100% correct, I'm not even sure it's 80% correct, but it'll at least clean up the worst of the confusion.
Here is untested code.
void readImage(FILE * fp, BMPImage *bpp) {
Pixel pixel;
for (i = 0; i < p -> height_px; i++){
for(int j = 0; j < p -> width_px; j++){
fread(&pixel, sizeof(Pixel), 1, fp);
bpp->pixels[i][j] = pixel;
}
}
int
main()
{
BMPHeader *hp = malloc(sizeof(* hp));
FILE *fp;
int i;
... read header info and stored into hp->header
... open fp file
hp->pixels = malloc(bpp->header.width_px * sizeof(Pixel *));
for( i=0; i<hp->header.width_px; i++ )
hp->pixels[i] = malloc(bpp->header.height_px * sizeof(Pixel));
//Send in pointers.
readImage(fp, hp);
... need to free up memory.
}
.
First, need to follow composite data structure.
Second, need to free up memory since it looks like it will need a lot of memory.
It is a good practice to allocate/free up memory in one area.

Converting Char buffer into an array of shorts

I have a char * buffer that is filled by an API function. I need to take the data that is contained with that pointer, cast it to unsigned shorts and translate it into network (htons()) format to send it over UDP. Here is my code (not all though for a few reasons)
The code below will work but that data on the other side is bad (not shorts or network translated)
char * pcZap;
while(1)
{
unsigned short *ps;
unsigned short short_buffer[4096];
write_reg(to start xfer);
return_val = get_packet(fd, &pcZap, &uLen, &uLob);
check_size_of_uLen_and_uLob(); //make sure we got a packet
// here I need to chage pcZap to (unsigned short *) and translate to network
sendto(sockFd,pcZap,size,0,(struct sockaddr *)Server_addr,
sizeof(struct sockaddr));
return_val = free_packet(fd, pcZap);
thread_check_for_exit();
}
Any help would be appreciated. Thank you.
Assuming you have 4080 bytes in your buffer that are composed of 16-bit samples, that would mean you have 2040 total 16-bit samples in the 4080 bytes of your buffer (16-bytes are reserved for the header). Therefore you can do the following:
#define MAXBUFSIZE 4096
#define MAXSHORTSIZE 2040
unsigned char pcZap[MAXBUFSIZE];
unsigned ushort[MAXSHORTSIZE];
//get the value of the returned packed length in uLen, and the header in uLob
unsigned short* ptr = (unsigned short*)(pcZap + uLob);
for (int i=0; i < ((uLen - uLob) / 2); i++)
{
ushort[i] = htons(*ptr++);
}
Now your ushort array will be composed of network-byte-order unsigned short values converted from the original values in the pcZap array. Then, when you call sendto(), make sure to use the values from ushort, not the values from pcZap.
If your array of chars is null terminated then you can simply do:
for (int i=0; i<strlen(CHAR_ARRAY); i++)
short_buffer[i] = (unsigned short) CHAR_ARRAY[i];
If the array isn't null terminated then you'll need to figure out how long it is exactly and then replace strlen(CHAR_ARRAY) with that value.
If all you need to do is convert a chunk of bytes, representing short ints in host endian to network endian, you do this:
size_t i;
size_t len = uLen - 16 - uLob;
size_t offset = uLob + 16;
if(len % 2 != 0) {
..error not a multiple of 16 bit shorts...
}
//now, if you're on a little endian host (assuming the shorts in
//pcZap is laid out as the host endian...), just swap around the bytes
//to convert the shorts to network endian.
for(i = 0; i < len; i+=2) {
//swap(&pcZap[offset + i],&pcZap[offset + i + 1]);
char tmp = pcZap[offset + i];
pcZap[offset + i] = pcZap[offset + i + 1];
pcZap[offset + i + 1] = tmp;
}
//if you're on a big endian host, forget the above loop, the data
//is already in big/network endian layout.
//and just send the data.
if(sendto(sockFd,pcZap + offset,len,0,(struct sockaddr *)&Server_addr,
sizeof Server_addr) == -1) {
perror("sendto");
}
Note that your code had sizeof(struct sockaddr) in the sendto() call, which is wrong, you want it to be the actual size of Server_addr.

Resources