My struct looks like this:
struct tlv_msg
{
uint8_t datatype; //type of data
/* data stored in a union */
union{
int32_t s32val; /* int */
int64_t s64val; /* long long */
uint32_t u32val; /* unsigned int */
uint64_t u64val; /* unsigned long long */
char* strval; /* string */
unsigned char* binval; /* any binary data */
};
uint32_t bytelen; /* no. bytes of union/data part */
};
This struct uses a union to hold some different data types. I have an alloc function which allocates memory for the struct on the heap. Am I correct in thinking that if I am allocating for an integral type (ie the first four types above in union) I only need to allocate as follows:
tlv_msg* msg = malloc(sizeof(tlv_msg));
sizeof(tlv_msg) returns 24. I presume this is enough bytes to hold the largest data type in the union plus the other data members. (not sure why 24 - can someone explain?).
But if the data type to be stored is a pointer type, eg char* then I then need to also do this:
msg->strval = (char*)malloc(sizeof(string_length+1);
That would make sense to me and that seems to work but just wanted to check.
That's perfectly right.
That said, you may want to create helper functions, to help you dealing with this.
For instance:
tlv_msg * new_tlv_msg( void );
/* There, you need to free struct members, if applicable */
void delete_tlv_msg( tlv_msg * msg );
/* Here you may copy your string, allocating memory for it */
tlv_msg_set_strval( tlv_msg * msg, char * str );
Implementation may be (basic, of course)
tlv_msg * new_tlv_msg( void )
{
return calloc( sizeof( tlv_msg ), 1 );
}
void delete_tlv_msg( tlv_msg * msg )
{
if( msg->strval != NULL )
{
free( msg-strval );
}
free( msg );
}
tlv_msg_set_strval( tlv_msg * msg, char * str )
{
if( msg->strval != NULL )
{
free( msg-strval );
}
msg->strval = strdup( str );
}
Yes, you are correct about having to perform two memory allocation steps, the first for the struct and the second for the character string.
Unless this is an embedded system where memory space is at a premium, one way to get around this is to decide on a maximum string size. Yes, that does waste memory, if, for example, you only usually have 10 character or fewer strings and allocate for say 25 characters.
#define WORKING_BUF_LEN 1024
struct tlv_msg
{
uint8_t datatype; //type of data
/* data stored in a union */
union{
int32_t s32val; /* int */
int64_t s64val; /* long long */
uint32_t u32val; /* unsigned int */
uint64_t u64val; /* unsigned long long */
char strval[WORKING_BUF_LEN={0}; /* string */
unsigned char* binval; /* any binary data */
};
uint32_t bytelen; /* no. bytes of union/data part */
};
You could also do your own memory management to avoid fragmenting the heap if you plan on having many of these structs and hence many char* pointers, but that requires a lot of work. You'd overwrite new with a macro and assign pre-allocated storage to your pointer and then do storage allocation book keeping. Don't do it unless you have to.
Related
i'm relatively newer in C and i have some problems to memcpy a struct in a buffer;
I have this struct:
`typedef struct {
uint8_t size_of_payload;
uint8_t command;
unsigned char* payload;
}__attribute__((packed)) mystruct``
and i want to copy in a tx_buffer[SIZE] the values of this struct, including the value pointed by payload.
payload point to an array of char with a size equals to the size_of_payload .
memcpy(&tx_buffer,&mystruct,sizeof(mystruct));
copies only the value of payload address.
It is possible do this?
Well it isn't possible to do it in one simple call. You will have to do it 'manually'.
memcpy can only copy one memory region to an other. Your source memory region isn't structured the way you want it at the destination, so will have to do it manually.
Let's first copy the first two fields
memcpy(&tx_buffer+0*sizeof(uint8_t),&mystruct.size_of_payload, 1*sizeof(uint8_t));
memcpy(&tx_buffer+1*sizeof(uint8_t),&mystruct.command, 1*sizeof(uint8_t));
What we do here, is copy the first element to position zero of tx_buffer, and the second to position 1 byte after that.
For the string we don't know the length. We could find it and do a memcpy, but there is a convenient alternative:
strcpy(&tx_buffer+2*sizeof(uint8_t), mystruct.payload)
Use a flexible array member. You'll have much simpler code:
typedef struct {
uint8_t size_of_payload;
uint8_t command;
uint8_t payload[];
} mystruct;
Allocate a structure:
mystruct *allocMyStruct( uint_8 command, uint8_t size, uint8_t *payload )
{
mystruct *m = malloc( sizeof( *m ) + size );
m->size_of_payload = size;
m->command = command;
memcpy( m->payload, payload, size );
return( m )
}
Copy:
mystruct *m = ...
.
.
.
// copy *ALL* the fields and the payload
memcpy( txBuffer, m, sizeof( *m ) + m->size_of_payload );
Free a structure:
mystruct *m = ...
.
.
.
free( m );
The only drawback to a flexible array member is you can't allocate them as local or static variables.
You can't do
mystruct m;
when you're using flexible array members.
But compared to having a uint_t *payload pointer in the structure that pretty much has to be allocated dynamically, in reality there's no flexibility lost.
There's no reason not to use a flexible array member in a case like this.
This whole setup seems a bit strange to me. Why the need to "flatten" the structure? Or if the need is justified, why was this structure layout chosen? Anyway, will try to exemplify (what I consider) the easiest way (which from other criteria wouldn't be the best).
Since the structure contains a member which is a pointer (a level of indirection), it can't be done in one go. In this particular case (as the pointer is the last member), the approach is to:
Copy the contents of the struct till the pointer member
Copy the contents of the dereferenced pointer member (based on the size member)
Note that for more complex structures, with alternating pointer and regular members, you'd need one memcpy call for the 1st member(s) of the same type + one additional call for each consecutive member types (normal -> pointer, pointer -> normal, pointer -> pointer).
code00.c
#include <stddef.h>
#include <stdio.h>
#include <stdint.h>
#include <string.h>
#define BUF_DIM 0xFF // Make sure it's large enough
typedef struct {
uint8_t size_of_payload;
uint8_t command;
unsigned char *payload;
} __attribute__((packed)) Data;
size_t dataSize(const Data *pData)
{
if (pData == NULL)
return 0;
return offsetof(Data, payload) + pData->size_of_payload;
}
size_t copyData(const Data *pData, uint8_t *pBuf)
{
size_t sz = dataSize(pData);
if (sz == 0)
return 0;
size_t ofs = offsetof(Data, payload);
memcpy(pBuf, pData, ofs);
memcpy(pBuf + ofs, pData->payload, pData->size_of_payload);
return sz;
}
int main()
{
unsigned char array[] = { '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'A', 'B', 'c', 'd' };
Data data = { sizeof(array), 123, array };
uint8_t buf[BUF_DIM] = { 0 };
copyData(&data, buf);
size_t ds = dataSize(&data);
printf("Size: %ld\nBuf:\n", ds);
for (size_t i = 0; i < ds; ++i)
printf("0x%02X(%c) ", buf[i], buf[i]);
printf("\nDone.\n");
return 0;
}
Output:
(qaic-env) [cfati#cfati-5510-0:/mnt/e/Work/Dev/StackOverflow/q071451221]> ~/sopr.sh
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###
[064bit prompt]> ls
main00.c
[064bit prompt]> gcc -o main00.exe main00.c
[064bit prompt]> ./main00.exe
Size: 16
Buf:
0x0E() 0x7B({) 0x30(0) 0x31(1) 0x32(2) 0x33(3) 0x34(4) 0x35(5) 0x36(6) 0x37(7) 0x38(8) 0x39(9) 0x41(A) 0x42(B) 0x63(c) 0x64(d)
Done.
I am trying to copy a byte array to my struct, then serialize my struct to a byte array again.
But, after I serialize my struct array, I cant get my data value (0x12, 0x34, 0x56) again, instead i get some rubbish data.
What is wrong here?
#pragma pack(push, 1)
typedef struct {
uint8_t length;
uint8_t *data;
} Tx_Packet;
#pragma pack(pop)
static void create_tx_packet(uint8_t *packet, uint8_t *src, int length);
int main(void)
{
uint8_t packet[32];
uint8_t data[] = { 0x12, 0x34, 0x56 };
create_tx_packet(packet, data, 3);
//i check using debugger, i cant get the data value correctly
//but i could get length value correctly
return 0;
}
static void create_tx_packet(uint8_t *packet, uint8_t *src, int length)
{
Tx_Packet *tx_packet = malloc(sizeof(*tx_packet ));
tx_packet->length = length;
tx_packet->data = (uint8_t *)malloc(length);
memcpy(tx_packet->data, src, length);
memcpy(packet, tx_packet, sizeof(*tx_packet));
}
Right now, your create_tx_packet() function copies a Tx_Packet struct created in the function to a uint8_t array. That struct contains the length and a pointer to the data, but not the data itself. It's actually not necessary to use the struct as an intermediate step at all, particularly for such a simple packet, so you could instead do:
static void create_tx_packet(uint8_t *packet, uint8_t *src, int length)
{
*packet = length; /* set (first) uint8_t pointed to by packet to the
length */
memcpy(packet + 1, src, length); /* copy length bytes from src to
the 2nd and subsequent bytes of
packet */
}
You still need to make sure packet points to enough space (at least length + 1 bytes) for everything (which it does). Since the version above doesn't dynamically allocate anything, it also fixes the memory leaks in your original (which should have freed tx_packet->data and tx_packet before exiting).
--
If you do want to use a struct, you can (since the data is at the end) change your struct to use an array instead of a pointer for data -- then extra space past the size of the struct can be used for the data, and accessed through the data array in the struct. The struct might be:
typedef struct {
uint8_t length;
uint8_t data[];
} Tx_Packet;
and the function becomes (if a temporary struct is used):
static void create_tx_packet(uint8_t *packet, uint8_t *src, int length)
{
/* allocate the temporary struct, with extra space at the end for the
data */
Tx_Packet *tx_packet = malloc(sizeof(Tx_Packet)+length);
/* fill the struct (set length, copy data from src) */
tx_packet->length = length;
memcpy(tx_packet->data, src, length);
/* copy the struct and following data to the output array */
memcpy(packet, tx_packet, sizeof(Tx_Packet) + length);
/* and remember to free our temporary struct/data */
free(tx_packet);
}
Rather than allocate a temporary struct, though, you could also use struct pointer to access the byte array in packet directly and avoid the extra memory allocation:
static void create_tx_packet(uint8_t *packet, uint8_t *src, int length)
{
/* Set a Tx_Packet pointer to point at the output array */
Tx_Packet *tx_packet = (Tx_Packet *)packet;
/* Fill out the struct as before, but this time directly into the
output array so we don't need to allocate and copy so much */
tx_packet->length = length;
memcpy(tx_packet->data, src, length);
}
If you use memcpy(packet, tx_packet, sizeof(*tx_packet)); you are copying the memory representation of tx_Packet into packet, starting with tx_packet->length.
Additionally when mallocating tx_packet that size should be sizeof(*packet)+sizeof(uint8_t) (length of packet plus length field)
And again when copying the tx_packet back to packet you are writing out of the boundaries of packet.
EDIT:
I forgot to mention that depending on your compiler memory alignment parameter you could get any length for the fields (including tx_packet->length) to accelerate memory operation. On 32bits machine it could be 4 and padded with rubbish.
When you serialize your struct with
memcpy(packet, tx_packet, sizeof(*tx_packet));
you're copying the length and the pointer to the data, but not the data itself. You'll probably need two memcpy calls: one of sizeof(uint8_t) to copy the length field, and one of length to copy the data.
This line:
Tx_Packet *tx_packet = malloc(sizeof(*packet));
only allocates one byte for the packet header, which you then immediately write off the end of, causing undefined behavior. You probably meant
Tx_Packet *tx_packet = malloc(sizeof(*tx_packet));
I'm looking at some example code and trying to figure this out but am stuck. I'm just trying to create a buffer of 10 ints and have my shared memory pointers point to that. Can someone help me understand what this code is actually doing and where I went wrong?
int shmem_id; /* shared memory identifier */
int *shmem_ptr[BUFSIZE]; /* pointer to shared segment */
key_t key = 4455; /* a key... */
int size = 2048; /* 2k memory */
int flag = 1023; /* permissions */
char keystr[10];
sprintf (keystr, "%d", key);
shmem_id = shmget (key, size, flag); /* create a shared memory segment */
shmem_ptr = shmat (shmem_id, (void *) NULL, 1023);
In reality I want it to a buffer of 10 struct items.
typedef struct widget{
char color[10];
};
Your declaration:
int *shmem_ptr[BUFSIZE];
declares shmem_ptr to be an array of pointers to integers. You just want a pointer to integers, so it should be:
int *shmem_ptr;
If the memory points to widget structures, you can do:
typedef struct widget {
char color[10];
} widget;
widget *shmem_ptr;
You don't need to declare the length when declaring a pointer. The length is specified when the shared memory block is created, not in the program that attaches to it.
The structure I am speaking of is the very last one. I get segmentation faults when I try to use it, and when I use sizeof to get its size 218369176 is returned.
typedef struct
{
unsigned long a1; /* Last structure in group. */
unsigned long a2; /* Next structure in group. */
char rc; /* Representing character. */
short st; /* Type of structure (purpose). */
short pl; /* Privilege level required to alter. */
short vt; /* Type of value (short, char, int, long, float, double, void*). */
union
{
short s;
char c;
int i;
long l;
float f;
double d;
void* p;
} un; /* Union containing values to be stored. */
} index_struct; /* Structure composing a table tree. */
typedef struct
{
unsigned long sr; /* Script return value. */
unsigned long ir; /* Interpreter return value. */
unsigned long lc; /* Execution counter (text division interpreter stopped at). */
short ai; /* Action identifier (current status of interpretation). */
short pr; /* Script privilege information. */
char st[65536 /* Change SCRIPT_TEXT_SIZE with this. */]; /* Segment containing script text. */
index_struct lt[65536 /* Change LOCAL_TREE_SIZE with this. */]; /* Segment containing local tree. */
} script_struct; /* Structure containing script state information and variables. */
typedef struct
{
unsigned long us; /* Number of unjoined scripts. */
unsigned long sn; /* Number of running scripts. */
short es; /* Environment status. */
script_struct sl[100 /* Change MAX_NUMBER_SCRIPTS with this. */]; /* Segment containing script list. */
index_struct gt[65536 /* Change GLOBAL_TREE_SIZE with this. */]; /* Segment containing global tree. */
} environment_struct;
EDIT: by popular request, here is the entire source code file.
/*
* BY: Charles Edwin Swain the 3rd.
* LANGUAGES: C and (if I ever comment out certain sections of code) x86 Assembly.
*/
#include <stdio.h>
/*
#include <stdint.h>
const uint8_t CPUID_UNSPECIFIED = 0;
const uint8_t CPUID_SUPPORTED = 1;
const uint8_t CPUID_UNSUPPORTED = 2;
typedef struct
{
uint32_t maximum_standard_level;
uint32_t raw_vendorid[4];
uint32_t raw_processortypeORfamilyORmodelORstepping;
uint32_t num_extendedfamily;
uint32_t num_extendedmodel;
uint32_t num_type;
uint32_t num_family;
uint32_t
uint32_t raw_brandidORCLUFLUSHORCPUcountORAPICID;
uint32_t raw_featureflags_A;
uint32_t raw_featureflags_B;
uint8_t features[64];
} CPUID_struct;
*/
/* These constants are associated with certain hard coded limits, and all must be the same to ensure proper functionality. */
const unsigned long SCRIPT_TEXT_SIZE = 65536; /* Size of segment containing script text. */
const unsigned long GLOBAL_TREE_SIZE = 65536; /* Size of segment composing global tree. */
const unsigned long LOCAL_TREE_SIZE = 65536; /* Size of segments composing local trees. */
const unsigned long MAX_NUMBER_SCRIPTS = 100; /* Maximum number of running scripts in an environment. */
typedef struct
{
unsigned long a1; /* Last structure in group. */
unsigned long a2; /* Next structure in group. */
char rc; /* Representing character. */
short st; /* Type of structure (purpose). */
short pl; /* Privilege level required to alter. */
short vt; /* Type of value (short, char, int, long, float, double, void*). */
union
{
short s;
char c;
int i;
long l;
float f;
double d;
void* p;
} un; /* Union containing values to be stored. */
} index_struct; /* Structure composing a table tree. */
typedef struct
{
unsigned long sr; /* Script return value. */
unsigned long ir; /* Interpreter return value. */
unsigned long lc; /* Execution counter (text division interpreter stopped at). */
short ai; /* Action identifier (current status of interpretation). */
short pr; /* Script privilege information. */
char st[65536 /* Change SCRIPT_TEXT_SIZE with this. */]; /* Segment containing script text. */
index_struct lt[65536 /* Change LOCAL_TREE_SIZE with this. */]; /* Segment containing local tree. */
} script_struct; /* Structure containing script state information and variables. */
typedef struct
{
unsigned long us; /* Number of unjoined scripts. */
unsigned long sn; /* Number of running scripts. */
short es; /* Environment status. */
script_struct sl[100 /* Change MAX_NUMBER_SCRIPTS with this. */]; /* Segment containing script list. */
index_struct gt[65536 /* Change GLOBAL_TREE_SIZE with this. */]; /* Segment containing global tree. */
} environment_struct; /* Structure containing environment state information and global tree. */
/*
* Function definition and calling conventions follow:
*
* - All non-interpreter functions should be called through a wrapper function.
* - This wrapper function's address is specified through the p field of an index_struct in the global tree.
* - The return items of the function are specified through the global tree, under 'f_retu'.
* - The arguments to the function are specified through the global tree, under 'f_argv'.
* - The number of arguments to the function are specified through the global tree, under 'f_argc'.
* - Before calling the wrapper function, these fields and the environment status are appropriately set.
* - The wrapper function takes a pointer to the segment containing the global tree as an argument (outside the interpreter).
* - The wrapper function sorts through the arguments and calls the appropriate function it is wrapping.
* - Once this function returns, it sets any actual interpreter buffers accordingly.
* - What is meant by the above is that the wrapper function will use temporary buffers in the call to the function, then transfer data over according to global tree arguments.
* - Once the wrapper function returns, the calling (interpreter) code should copy all data from the return to an appropriate location and wipe all involved tables (for security).
* - This entire state is uninterruptable by interruption code from the moment the interpreter begins the call to after the interpreter finishes wiping data.
* - The above does not include signals, and only describes with regard to the interpreter auto returning after interpreting some input.
*
*/
/* Creates a fresh interpreter environment. */
int ecreate(environment_struct* environment)
{
if (environment == NULL)
{
return -1;
}
else
{
if (environment->es != 0)
{
return -2;
}
else
{
environment->us = 0;
environment->sn = 0;
environment->es = 1;
return 0;
}
}
}
/* Cleans up and shuts down an interpreter environment. */
int edestroy(environment_struct* environment)
{
if (environment == NULL)
{
return -1;
}
else
{
if (environment->es == 0)
{
return -2;
}
else
{
environment_struct environment_B;
*environment = environment_B;
return 0;
}
}
}
/* Main function. */
int main(int argc, char** argv)
{
/* This is where the sizeof is. Works fine when code behind next comment is not commented in.*/
printf("%lu\n", sizeof(environment_struct));
/* Next comment. */
environment_struct environment;
int r_ecreate, r_edestroy;
r_ecreate = ecreate(&environment);
r_edestroy = edestroy(&environment);
printf("%d, %d\n", r_ecreate, r_edestroy);
return 0;
}
index_struct will have a size of 24 bytes on your average 32-bit system (or 32 bytes on your average 64-bit system).
script_struct will have a size of 1,638,416 bytes (1.6 MB) on your average 32-bit system (or 2,162,720 bytes (2.16 MB) on your average 64-bit system).
environment_struct will have a size of 165,414,476 (165.4 MB) on your average 32-bit system (or 218,369,176 bytes (218.3 MB) on your average 64-bit system (which is the size you're seeing)).
That is an insanely large size for a struct, and very well might crash your system (particularly if you use it on the stack). If you allocate several environment_structs (on the heap), you could very well run out of memory.
So yeah, they're too large. Waaay too large.
Edit: Yup, you're declaring a environment_struct on the stack. A struct that big is insane for the stack.
Just a small addendum to Cornstalks' answer. On a 64-bit Intel system, the sizes would be, according to my calculations:
sizeof(index_struct) = 32
sizeof(script_struct) = 2162716
sizeof(environment_struct) = 218368770
Therefore, the result you get from sizeof are correct.
In your code, you are putting environment_struct on the stack. The stack space is usually fixed and quite limited - on my system it's just 8 MiB. If you really want to use such a gigantic structure, you should allocate the memory for it with malloc(). A better approach, which would also remove the rather arbitrary limit of 65536 scripts, would be to store a linked list or array of scripts allocated with malloc(), instead of reserving a fixed amount of space.
I have a predefined struct to use :
typedef struct somestruct_s {
int s;
union {
unsigned char *ptr;
unsigned char l_ptr[sizeof(char *)];
};
}somestruct_t, *somestruct;
It contains union to reduce memory usage.
I know the size can vary due to m32 and m64 compilation (pointer size).
My question is how to "use" that struct for my precise assignment. The purpose of this struct is to implement basic bit operations, the s variable contains the size of the bitmap in bytes. If the bitmap can fit inside of memory occupied by pointer to bitmap then we allocate her there. Im writing some bitmap operations on it, but i can't really get the struct or how to operate on it.
I cannot understand what your problem is. You have to have one major function which will return correct pointer to bitmap depending on pointer size:
unsigned char* somestruct_get_bitmap(somestruct_t* ths) {
if( sizeof(char*) > ths->s )
return ths->ptr;
return ths->l_ptr;
}
all other functions must use this function to get the correct pointer to bitmap. Also you need the constructor/destructor pair to initialize/deinitialize bitmap pointer in the correct way (of cause I am showing the simplest example supposing that you have null-terminated bitmaps):
unsigned char* somestruct_init(somestruct_t* ths, unsigned char* ptr) {
ths->s = strlen(ptr) + 1;
if( sizeof(char*) > ths->s ) {
ths->ptr = strdup(ptr);
return;
}
strcpy(ths->l_ptr, ptr);
}
unsigned char* somestruct_destroy(somestruct_t* ths) {
if( sizeof(char*) > ths->s ) {
free(ths->ptr);
return;
}
}