I'm trying to share a struct similar to the following example:
typedef struct {
int *a;
int b;
int c;
} example;
I'm trying to share this struct between processes, the problem that I find is that when I initialize 'a' with malloc, I won't be able to access the array from within the second process.
Is it possible to add this dynamic array to the memory mapped file?
You can have it as
typedef struct {
int b;
int c;
int asize; // size of "a" in bytes - not a number of elements
int a[0];
} example;
/* allocation of variable */
#define ASIZE (10*sizeof(int))
example * val = (example*)malloc(sizeof(example) + ASIZE);
val->asize = ASIZE;
/* accessing "a" elements */
val->a[9] = 125;
the trick is zero sized a array at the end of the structure and malloc larger then size of structure by actual size of a.
You can copy this structure to mmapped file. You should copy sizeof(example)+val->asize bytes. On the other side, just read asize and you know how many data you should read - so read sizeof(example) bytes, realloc and read additional asize bytes.
Related
I'm doing some programming for an old system which has unreliable dynamic memory allocation functions (malloc, calloc, realloc, etc...).
I started toying with the idea of loading a file without using any of those functions. Designate a pointer with a memory address on which I would get the file content loaded, this pointer would increase as I iterate through the file copying the content I need to my struct variables or assigning the pointers of my internal structs to this new address. Any new file would be loaded at the end address of the previous one.
for a simple structs likes this:
typedef struct A
{
int a;
int b;
int c;
}A;
A a;
I copy the content with a simple memcpy and then increase the pointer
memcpy(&a, pointer_to_the_file, sizeof(A));
pointer_to_the_file += sizeof(A)
This worked fine. For structs on which before I would use some dynamic allocations like:
typedef struct B
{
A* p_a;
A* p_b;
}B;
B b;
I assign the pointers directly:
b->p_a = (B *)pointer_to_the_file;
//then increase the pointer
pointer_to_the_file += sizeof(B) * num_of_p_a_elements;
This also seemed to work OK.
But it was when I move to more complex structures that I started having some problems:
typedef struct C
{
A p_a_1;
A* p_a_2;
B** p_b;
}C;
C *p_c;
Here I don't know how to approach it correctly. I did:
p_c = (C *)pointer_to_the_file;
But I did not get the expected results for p_a_1, although the address of p_c was pointing at the right location where the correct values for p_a_1 were located. I tried using C p_c[0]; and then the values of p_a_1 were correct, but I don't know why that is working so I don't want to build it upon this and then find out later on it causes issues.
Also, I cannot wrap my head about how could I assign p_c->p_b (a non square 2d matrix) using the [0] trick.
Is this at all possible (reading structs like this from a file without using dnyamic memory alloc functions)? If so, what would be the best approach to do it?
I see two routes for this:
Either you use some global variables and implement you own memory allocation (and maybe even replace the one for malloc and co. by linking two your implementation).
Or you a globale buffer to read chunks of the files into memory, like:
uint8_t buffer[1024];
int read_chunk(int fd) {
int bytes_read = 0;
while (bytes_read < 1024) {
ssize_t res = read(fd, &buffer + bytes_read, 1024 - bytes_read);
if (res < 0) return -1;
else if (res == 0) break;
else {
bytes_read += res;
}
}
return bytes_read;
}
and then with a function with the signature
void parse(uint8_t* data, C* p_c);
you can parse you file in stack allocated mempory like.
C p_cs[2];
read_chunk(fd);
for (int i = 0; i < 2; i++) {
parse(&buffer + (sizeof(C)*i), &p_cs[i]);
}
But imho you should go with the first suggestion if possible.
I am working on a micro controller, so, no malloc. Actually, I want to create a memory manager, so I am kinda implementing the malloc function for later use and using the BLOCK strategy to get it, like FreeRTOS does.
typedef struct BLOCK {
unsigned char used; // If 1 block is used, if 0 it's free to use
unsigned long offset; // Offset of the starting block
unsigned long size; // Size of this block
struct BLOCK * next; // Pointer to the next block
} BLOCK_t;
#define MAX_PROGRAMS 3
#define BLOCKS_NEEDED (MAX_PROGRAMS * 2) + 1
BLOCK_t blocks[BLOCKS_NEEDED]; // Not allocating 7 * sizeof(BLOCK)
This BLOCK is a linked list and I want to create (allocate) a fixed amount of them and set the first blocks[0] only. The next ones will be created in execution time, when memory is allocated.
Thanks in advance.
EDIT: In case the title is not clear enough, I want to compiler to assign some memory space to my array (fixed location and size) but I don't want to initialize it with data because I will get the data in run-time, so I want an array of 7 BLOCKs with empty data. The code above shows my attempt to do it, I declared the array, but I assume that declaring an array doesn't give you the space needed. How can I achieve this ? How can I get the compiler to give me that space ?
EDIT 2: This would be tha Java code to do it.
private static int MAX_PROGRAMS = 3;
private static int BLOCKS_NEEDED = (MAX_PROGRAMS * 2) + 1:
Block myBlockList[] = new Block[BLOCKS_NEEDED];
This get the space for myBlockList even though the list is empty and each item is uninitialized, but I have the space already.
All you want to do is allocate memory automatically on the stack.
#include <stdio.h>
#define blockcontent_size 1024
#define blockcount 3
typedef struct
{
unsigned char used;
unsigned long offset;
unsigned long size;
unsigned data[blockcontent_size];
} BLOCK;
BLOCK blocks[blockcount];
int main()
{
printf("memory available in one block %u\n", sizeof(blocks[0].data));
printf("memory used for one block %u\n", sizeof(BLOCK));
printf("memory used for all blocks %u\n", sizeof(blocks));
return 0;
}
You actually do not need a linked list, you can just use the index.
Is this close to what you are asking?
#LPs quote:
Using c writing BLOCK_t blocks[BLOCKS_NEEDED]; you are declaring the array and sizeof(BLOCK_t)*BLOCKS_NEEDED bytes are occupied by the array.
So my statement :
BLOCK_t blocks[BLOCKS_NEEDED]; // Not allocating 7 * sizeof(BLOCK)
was false, it actually does allocate the space.
I am having some trouble starting my program. I'm new to this. I have done some research and found some resources, but I have trouble applying it to the code. It is mostly based on pointers and structures.
I mainly need help with learning how to store the data in the struct, and how to initialize everything.
The program is supposed to find the frequency of characters in a file. I need to use dynamic memory allocation. And use a dynamically allocated array of pointers to store the characters and frequencies. I am supposed to use malloc() to allocate the array and realloc() to increase the size of the array to insert more elements. But I have no idea how to do this.
The program uses these functions-
• charInCFStruct:which returns the index of charfreq struct which has the char c stored in its member variable next. If none of the charfreq structs contains c then it has to return -1.
• printCFStruct: Prints the contents of all of the charfreq structs.
• freeCFStruct: Frees all of the charfreq structs and then frees the pointer
to the structs.
Down below is what I know is right so far. I thought it would be easier to start again from there. I am not asking for code exactly, just some help with the topics I need to do this, and a push in the right direction. Thank you!
#include <stdio.h>
#include <stdlib.h>/*
* struct for storing a char and the number of times it appears in a provided text */
struct charfreq
{
int count;
char next;
};
typedef struct charfreq charfreq;
/*
* Returns the index of charfreq struct which has the char c stored in its member variable next.
* If none of the charfreq structs contains c then it returns -1.
*/
int charInCFStruct(charfreq **cfarray, int size, char c){
}
/*
* Prints the contents of all of the charfreq structs.
*/
void printCFStruct(charfreq **cfarray, int size){
}
/*
* Frees all of the charfreq structs and then frees the pointer to the structs.
*/
void freeCFStruct(charfreq **cfarray, int size){
}
int main(void)
{
charfreq **cfarray;
FILE *inputfile;
char next;
int size = 10; /* used initial value of 10 but any positive number should work */
int i = 0;
int pos;
/* open file to read from */
inputfile = fopen("chars.txt", "r");
if(inputfile == NULL)
printf("chars.txt could not be opened. Check that the file is in the same directory as you are running this code. Ensure that its name is chars.txt.\n\n");
/* allocate space for pointers to char frequency structs */
cfarray = (charfreq**)malloc(size*sizeof(charfreq*));
/* read in chars until the end of file is reached */
while(fscanf(inputfile, "%c", &next) != EOF){
/* fill in code to fill structs with data being read in */
/* call to increase space after changing size */
cfarray = realloc(cfarray,size*sizeof(charfreq*));
}
/* print out char frequency structs */
printCFStruct(cfarray,i);
/* free all char frequency structs */
freeCFStruct(cfarray,i);
/* close the file we opened earlier */
fclose(inputfile);
return 0;
}
Today I was working on a problem of moving an array of smaller structs directly into an array of larger structs (arrayNew) (essentially upgrading the smaller structs to store more information). The smaller structs needed to be read from a HDD in one single read operation into the array of new 'upgraded' larger structs, a function would be called to do the 'upgrading'. Also all the new fields in the structs that were read from the hard drive would be set to '0'.
Other more simple solutions that I tried were:
Creating a local array of the old structures (arrayOld), loading the structures from the HDD into it then simply looping through the empty array of the new structures (arrayNew) and manually moving each structs contents from arrayOld into arrayNew. (e.g. arrayNew[i].x = arrayOld[i].x; )
The problem with this is that in my case the arrays I was working with were very large and too large for the stack ( about 1mb for each array) causing a segmentation fault the instant the upgrading function was called.
Another viable solution was to create a dynamic array of the old structures (arrayDy) and load the old structures into arrayDy and then again manually moving each structs contents from arrayDy into arrayNew. (e.g. arrayNew[i].y = arrayDy[i].y; ) This addressed the issue of running out of stack memory.
After implementing the second solution. I decided to experiment and develop a solution that uses no dynamically allocated memory and loads the array of old structures from the HHD directly into the larger array of larger structs arrayNew in one read operation and manipulate the contents of arrayNew in memory to pad out the missing values that are there due to the array being bigger.
I will post my solution below in a scaled down version of what I implemented, using the following structs for my example:
typedef struct INNER_STRUCT_ {
int i_item1;
int i_item2;
char i_item3;
} INNER_STRUCT;
typedef struct SMALL_STRUCT_ {
int item1;
char item2;
INNER_STRUCT item3;
} SMALL_STRUCT;
typedef struct BIG_STRUCT_ {
int item1;
char item2;
INNER_STRUCT item3;
INNER_STRUCT item4;
} BIG_STRUCT;
Yes, this is possible - you can use union for that. C99 standard makes a special guarantee that can be used to implement your requirement:
6.5.2.3-5: One special guarantee is made in order to simplify the use of unions: if a union contains several structures that share a common initial sequence (see below), and if the union object currently contains one of these structures, it is permitted to inspect the common initial part of any of them anywhere that a declaration of the complete type of the union is visible.
Your structA_ and structB_ do share a common initial sequence, so creating a union and accessing the structs through it would do the trick:
union {
structA a;
structB b;
} u;
memset(&u.b, 0, sizeof(structB)); // Zero out the bigger structB
loadFromHdd(&u.a); // Load structA part into the union
// At this point, u.b is valid, with its structA portion filled in
// and structB part zeroed out.
Note that you cannot do it to an array (unless, of course, you make an array of unions). Each structA needs to be loaded individually into the union, from which it could then be read as structB.
The method I propose and used as a solution basically loads the smaller structs for the HDD ( a file in this case) into the array of new larger structs and then rearranges the block of memory so that each field can be accessed properly. The code to illustrate this is below, and is an mcve.
#include <stdio.h>
#include <string.h>
typedef struct INNER_STRUCT_ {
int i_item1;
int i_item2;
char i_item3;
} INNER_STRUCT;
typedef struct SMALL_STRUCT_ {
int item1;
char item2;
INNER_STRUCT item3;
} SMALL_STRUCT;
typedef struct BIG_STRUCT_ {
int item1;
char item2;
INNER_STRUCT item3;
INNER_STRUCT item4;
/*
Note that the big struct is exactly the same as the small
struct with one extra field - Key to this method working
is the fact that the extension to the struct is appended
at the end, in an array of the structs will be placed one
after the other in memory with no gaps*/
} BIG_STRUCT;
void printSmallStruct (SMALL_STRUCT *inStruct, int count) {
// Print everything inside given small struct
printf("\n\n Small struct %d, item1: %d \n",count,inStruct->item1);
printf(" Small struct %d, item2: %c \n",count,inStruct->item2);
printf(" Small struct %d, item3.i_item1: %d \n",count,inStruct->item3.i_item1);
printf(" Small struct %d, item3.i_item2: %d \n",count,inStruct->item3.i_item2);
printf(" Small struct %d, item3.i_item3: %c \n",count,inStruct->item3.i_item3);
}
void printBigStruct (BIG_STRUCT *inStruct, int count) {
// Print everything inside given big struct
printf("\n\n Big struct %d, item1: %d \n",count,inStruct->item1);
printf(" Big struct %d, item2: %c \n",count,inStruct->item2);
printf(" Big struct %d, item3.i_item1: %d \n",count,inStruct->item3.i_item1);
printf(" Big struct %d, item3.i_item2: %d \n",count,inStruct->item3.i_item2);
printf(" Big struct %d, item3.i_item3: %c \n",count,inStruct->item3.i_item3);
printf(" Big struct %d, item4.i_item1: %d \n",count,inStruct->item4.i_item1);
printf(" Big struct %d, item4.i_item1: %d \n",count,inStruct->item4.i_item2);
printf(" Big struct %d, item4.i_item1: %c \n",count,inStruct->item4.i_item3);
}
int main() {
SMALL_STRUCT smallStructArray[5]; // The array of small structs that we will write to a file then read
BIG_STRUCT loadedBigStructArray[5]; // The large array of structs that we will read the data from the file into
int i; // Counter that we will use
FILE *pfile; // pointer to our file stream
void *secondary_ptr; // void pointer that we will use to 'chop' memory into the size we want
/* Fill the array of structs (smallStructArray) */
for (i = 0; i < 5; i++) {
/* We fill each field with different data do we can ID that the right data is in the right fields */
smallStructArray[i].item1 = 111;
smallStructArray[i].item2 = 'S';
INNER_STRUCT* temp = &smallStructArray[i].item3;
temp->i_item1 = 777;
temp->i_item2 = 999;
temp->i_item3 = 'I';
}
/* Write the contents of smallStructArray to binary file then display it */
pfile = fopen("test.dat","wb");
if (pfile!=NULL){
for (i = 0; i < 5; i++) {
fwrite(&smallStructArray[i],sizeof(SMALL_STRUCT),1,pfile);
}
fclose(pfile);
}
else{
printf("Unable to open file!");
return 1;
}
for (i = 0; i < 5; i++) {
printSmallStruct(&smallStructArray[i],i);
}
/* Clear array of big structs using memset */
memset(&loadedBigStructArray[0],0,sizeof(loadedBigStructArray));
/* Here we read from the smallStructArray that was aved to file into the loadedBigStructArray */
pfile = fopen("test.dat","rb");
if (pfile !=NULL){
/*
He we pass fread the following: size_t fread(void *args1, size_t args2, size_t args3, FILE *args4)
args1 - a pointer to the beginning of a block of memory, in our case the beginning of the
array loadedBigStructArray.
args2 - the size of the ammout of bytes we wish to read, in our case the size of a SMALL_STRUCT,
the size one of the elments in the array saved to the file.
args3 - the ammount of elements to read, in our case five (which is the number of elements the
array saved to the file has.
args4 - a pointer to a FILE that specifies our input stream.
Essentially what fread will do here is read a block of bytes the size of the array we saved to
the file (smallStructArray) into the array in memory loadedBigStructArray from the
beggining of loadedBigStructArray. Fig 1 illustrates what this will look like in memory.
*/
fread(&loadedBigStructArray,sizeof(SMALL_STRUCT),5,pfile);
fclose(pfile);
}
else{
printf("Unable to open file!");
return 1;
}
/*
Due to the way the array on the file has been read into the array in memory, if we try
to access the data in loadedBigStructArray only the first 5 values will be valid, due to
the memory not being in the order we want. We need to re-arrange the data in loadedBigStructArray
*/
/*
Here we use a void pointer to point to the beggining of the loadedBigStructArray.
we will use this pointer to 'chop' the data loadedBigStructArray into SMALL_STRUCT
sized 'chunks' we can read from.
Due to the way pointers and arrays work in C we can cast the void pointer to any type we want
and get a chunk of memory that size begginnig from the pointer and its off set.
E.g. : int temp = ((int *)void_ptr)[i];
This example above will give us an integer 'temp' that was taken from memory beggining from position
void_ptr in memory and its offset i. ((int *)void_ptr) casts the pointer to type int and [i] dereferances
the pointer to location i.
*/
secondary_ptr = &loadedBigStructArray;
/*
Not we are going through the array backwards so that we can rearange the data with out overwriting
data in a location that has data which we havent moved yet. As the bottom end of the loadedBigStructArray
is essentially empty we can shift data down that way.
*/
for (i = 5; i > -1; i=i-1) {
SMALL_STRUCT temp = ((SMALL_STRUCT *)secondary_ptr)[i]; // dereference pointer to SMALL_STRUCT [i] inside loadedBigStructArray call it 'temp'
/*
Now that we have dereferenced a pointer a given SMALL_STRUCT inside loadedBigStructArray called 'temp'
we can use temp to move the data inside temp to its corresponding position in loadedBigStructArray
which rearragnes the data.
*/
loadedBigStructArray[i].item1 = temp.item1;
loadedBigStructArray[i].item2 = temp.item2;
loadedBigStructArray[i].item3.i_item1 = temp.item3.i_item1;
loadedBigStructArray[i].item3.i_item2 = temp.item3.i_item2;
loadedBigStructArray[i].item3.i_item3 = temp.item3.i_item3;
/* We then fill the new field to be blank */
loadedBigStructArray[i].item4.i_item1 = 0;
loadedBigStructArray[i].item4.i_item2 = 0;
loadedBigStructArray[i].item4.i_item3 = '0';
}
/* Print our new structures */
for (i = 0; i < 5; i++) {
printBigStruct(&loadedBigStructArray[i],i);
}
return 0;
}
Visualization of technique:
When fread does the single read operation of the array saved on disk into the array in memory due to it being smaller it will take up the first potion of the array in memory but the 'bottom' section could be anything, if we try to access the data in the new array with the current handles we have on the data we will either get inaccurate information or a bad piece of memory. We have to rearrange this data before we can use any of our handles on the structs in the array.
I am trying to make a struct for a circular buffer that contains an array of type "quote." However, the quote array must start out at a size of 10. I am trying to figure out if I declare the size of 10 in my .h file or in my .c file. My two files are as follows:
.h file:
typedef struct{
unsigned int time;
double rate;
}quote;
typedef struct{
unsigned int testNum;
quote quoteBuffer[];
}cbuf;
cbuf* cbuf_init();
.c file:
cbuf* cbuf_init(){
cbuf *buffer = (cbuf *)calloc(1,sizeof(cbuf));
buffer->testNum = 1;
quote newQuote = {1,1.00};
buffer->quoteBuffer[1] = newQuote;
return buffer;
}
These are obviously just test values, however if I wanted to specifically make the quote array in the cbuf struct start out at a size of 10, would I declare that in the .h file as:
typedef struct{
unsigned int testNum;
quote quoteBuffer[10];
}cbuf;
or in the .c file some other way?
There are two ways of having dynamic arrays in structures. The obvious is of course to have it as a pointer, and dynamically allocate (or reallocate) when needed.
The other is to have an array of size 1, and then allocate a larger size than the structure, to accommodate for the array:
typedef struct {
unsigned int testNum;
quote quoteBuffer[1];
} cbuf;
cbuf *cbuf_init(const size_t num_quotes) {
/* Allocate for the `cbuf` structure, plus a number of `quote`
* structures in the array
*/
cbuf *buffer = malloc(sizeof(cbuf) + (num_quotes - 1) * sizeof(quote));
/* Other initialization */
return buffer;
}
/* If more quotes are needed after initial allocation, use this function */
cbuf *cbuf_realloc(cbuf *buffer, const size_t new_num_quotes) {
buffer = realloc(buffer, sizeof(cbuf) + (new_num_quotes - 1) * sizeof(quote));
/* Other initialization */
return buffer;
}
Now you can use the array as a normal array:
cbuf *buffer = cbuf_init();
buffer->quoteBuffer[5].time = 123;
Note: I only allocate extra space for 9 quote structures, but state that I allocate ten. The reason is that the cbuf structure already contains one quote structure in its array. 1 + 9 = 10. :)
Note 2: I put the quote array in the cbuf structure with one entry already in it for backwards compatibility. Having an array without a size in the structure is quite new in the C world.
you can also do this if you want 10 quotes in a cbuf but a statically allocated like quote buffer[10] would work too:
cbuf* cbuf_init(int numQuotes)
{
cbuf *b = calloc(1, sizeof(cbuf) + numQuotes * sizeof(quote));
return b;
}
If you want a statically sized circular buffer then your could declare the size in the header file. Using a #define for the buffer size will make the code more readable and maintainable, as you'll reference the size elsewhere in your code.
If you want the circular buffer to be growable then define the size in your C file. You'll then have to take care of tracking the size and destroying the memory that you will have to allocate dynamically.
In your example, I think you need to allocate more room for your quote structs...
cbuf *buffer = (cbuf *)calloc(1,sizeof(cbuf) + NUM_QUOTES*sizeof(struct quote));
---------------------------------
The reason for this is that in your struct def...
quote quoteBuffer[];
... quoteBuffer doesn't add size to the struct. quoteBuffer will point to one byte past the end of the struct, hence the need to allocate memory for the struct + memory for the array.
EDIT: Daniel Fischer's comment (thanks Daniel) - quoteBuffer may, in some cases, add size to the struct if it introduces padding. The reason is that the compiler will probably strive to get the most optimal alignment for quoteBuffer. For example, ints normally aligned of 4-byte boundaries. E.g. a struct like:
struct {
char a;
int b;
}
is probably changed by compiler to
struct {
char a;
char pad[3]; // compiler adds padding behind the scenes
int b; // align b on a 4-byte boundary
}
This probs doesn't apply in your case as your struct leaves quoteBuffer[] on a 4 byte boundary.
The reason that the compiler does this is two fold.
1. On some architectures (not so common nowadays I think?), unaligned accesses aren't supported.
2. Aligned accesses are more efficient, even if architecture allows non-aligned accesses as it is one memory read as opposed to two memory reads plus a manipulation.