C - 2D Dynamic Array (Double Pointer) - Shared Memory - c

I've got 2 processes (Client and Server) that are communicating through shared memory.
I need to create a 2D Array that is Dynamic (based on parameters). The array is stored in a struct and then written to the shared segment.
I can write the array to the shared memory, but cannot retrieve it from the other process.
Client Code:
struct shared_use_st {
int written_by_you;
int **PID_PRI_array;
};
/* Prepare Dynamic 2D array */
data_store = malloc(/*ROWS*/ 5 * sizeof(int*));
for(i=0;i<5; i++)
data_store[i] = malloc(/*COLS*/ 2 * sizeof(int));
/* Prepare Dynamic 2D array - Shared Memory Seg */
shared_stuff->PID_PRI_array = malloc(/*ROWS*/ 5 * sizeof(int*));
for(i=0;i<5; i++)
shared_stuff->PID_PRI_array[i] = malloc(/*COLS*/ 2 * sizeof(int));
/* Write PID and PRI to data_store array */
data_store[0][0] = pid;
data_store[0][1] = 1;
data_store[1][0] = 12345;
data_store[1][1] = 2;
data_store[2][0] = 12346;
data_store[2][1] = 3;
data_store[3][0] = 12347;
data_store[3][1] = 4;
data_store[4][0] = 12348;
data_store[4][1] = 5;
for(i=0;i<5;i++){
for(x=0;x<=1;x++){
shared_stuff->PID_PRI_array[i][x] = data_store[i][x];
}
}
Server Code:
for(i=0;i<5;i++){
printf("PID: %d, PRI:%d\n", shared_stuff->PID_PRI_array[i][0], shared_stuff->PID_PRI_array[i][1]);
}
I get a "Segmentation Fault" error.
Thanks.

Even if your shared_stuff object is in shared memory, you are not writing the array to shared memory. You are allocating space with malloc, writing data to that space, and then putting pointers to that space into shared_stuff. malloc allocates space within the current process‘ normal address space, not in a shared memory segment you have created. You need to write the array contents to the shared memory.
Presuming there is enough space for the array within the shared memory segment, you will have to manage the addresses yourself, not using malloc. (If there is not enough space, you must make the shared memory segment larger or convey the information in pieces over time.)
You can place a variable-length array within the shared memory segment as follows.
First, define a structure that contains all the “management” information you need, such as the array sizes:
struct StuffStruct
{
size_t NumberOfRows, NumberOfColumns;
… Other information as desired.
};
Create a pointer to that structure and set it to point to the shared memory segment:
struct StuffStruct *Stuff = shm; // shm contains the address from shmat, performed previously.
Create a pointer to an array with the desired number of columns and set it to point into the shared memory segment after the initial structure:
int (*data_store)[NumberOfColumns] = (int (*)[NumberOfColumns]) ((char *) Stuff + sizeof *Stuff);
(Note for C purists: Yes, the C standard does not guarantee what happens when you do pointer arithmetic like this. However, any implementation providing shared memory support must provide support for this sort of pointer arithmetic.)
Note that sizeof *Stuff + NumberOfRows * NumberOfColumns * size(int) must be no greater than the size of the shared memory segment. Otherwise you will overrun the shared memory segment in the next step.
For the next step, fill the array with data: Assign values to the elements of data_store as for a normal two-dimensional array.
In the server, set Stuff the same way. Then, after the client has written the shared memory segment, read the numbers of rows and columns from Stuff. Then set data_store the same way. Then read from data_store.

Related

C how to free sub memory?

I allocate a big memory , char* test= malloc(10000000); , then I put value on this memory , and do some work for each value.
What I want is , each 1000 index, I want to release all the memory until it.
For ex.
for(long i=0; i<10000000;i++)
DoSomeWork(test[i]);
if(i%1000==0)
releaseMemory(i-1000,i);
How can I do it in c?
I know that free can only free all of my allocate, but I don't want to wait to the end of work the free all the memory.
I want each 1000 works free all the 1000 back
I must to allocate all the memory in the begining of program.
What you want can be achieved by allocating the program in smaller chunks.
You have to adjust your algorithm to handle a bunch of small sub-arrays which you then can release after use.
In this case, it might be useful to allocate the chunks in reversed direction to give the libc the chance to release the freed memory to the underlying OS.
Let me enhance a bit here:
Assume you want an array with 10000000 (10 million) entries. Instead of allocating it as one chunk as depicted in the question, it could be possible to have
#define CHUNKSIZE 10000
#define ENTRYSIZE 8
#define NUM_CHUNKS 1000
void test(void)
{
void** outer_array = malloc(NUM_CHUNKS * sizeof(void*))
for (int i = 0; i < NUM_CHUNKS; i++) {
void * chunk = malloc(CHUNKSIZE * ENTRYSIZE);
outer_array[NUM_CHUNKS - 1 - i] = chunk;
// allocate them in reverse order
}
// now, set item #123456
size_t item_index = 123456;
// TODO check if the index is below the maximum
size_t chunk_index = item_index / CHUNKSIZE;
size_t index_into_chunk = item_index % CHUNKSIZE;
void * item_address = &outer_array[chunk_index][index_into_chunk * ENTRY_SIZE];
// after having processed one chunk, you can free it:
free(outer_array[0]);
outer_array[0] = NULL;
}
There are (roughly) two possibilities how a program can enhance the heap in order to allocate memory:
It can obtain a completely new memory block from the OS, indepedent from the "main address space". Then it can use it for allocation and return it to the OS as soon as it is free()d. This happens in some allocators if the allocation size is above a certain threshold.
It can enhance the program address space. Then, the new memory is added at the end. After free()ing the last memory block, the program address space can be reduced again. This happens in some allocators if the allocation size is below a certain threshold.
This way, your program's memory footprint decreases over time.

Freeing memory in mex code

I am writing a mex code and I am thinking that I am not using memory very efficiently. Here is what I do. I am allocating memory for variable called out like
out=mxMalloc(sizeof(double)*N);
plhs[0]=mxCreateDoubleMatrix(1,N,mxREAL);
set(plhs[0],out);
The problem is that I am not freeing memory that I allocate for variable out because if I do I will receive segmentation error since I will use it in Matlab. So can anyone suggest me a better technique to do what above commands are doing ? Is there anyway to free memory for out or to avoid defining matrix of length 1X N?
Thanks.
You don't need to allocate the array with mxMalloc. mxCreateDoubleMatrix already allocates the array. Once you have your pointer to this data (obtained using mxGetPr), you can then fill the array with the necessary values.
double *out;
// Allocate memory for the first output
plhs[0] = mxCreateDoubleMatrix(1,N,mxREAL);
// Get the pointer to the output data
out = mxGetPr(plhs[0]);
// Run your algorithm here to populate out with the data you need
If for some reason you do need to create out in some other way, you want to copy the contents of that separate array into the output prior to freeing the memory.
double *out;
double *realout;
// Allocate data to use internally
out = mxMalloc(sizeof(double) * N);
// Initialize the array that will be returned to MATLAB
plhs[0] = mxCreateDoubleMatrix(1, N, mxREAL);
realout = mxGetPr(plhs[0]);
// Now copy all values to the MATLAB output
for ( j = 0; j < N; j++ )
realout[j] = out[j];
// Now you can free up memory for out
mxFree(out)

Segmentation fault when trying to access a struct array

As part of an assignment, I have to deal with three structs. There is some larger table within a file, FileHeader, that is made up of SectionHeader structs. Hdr is made up of an array of these structs laid out in contiguous memory. As a result, I should be able to access the array by typecasting the location of the table in memory.
typedef struct {
unsigned int offset; // offset in bytes from start of file to section headers
unsigned short headers; // count of section headers in table
} FileHeader;
typedef struct {
unsigned int name;
unsigned int type;
} SectionHeader;
I am supposed to: Use the offset and headers fields from the FileHeader (hdr) to identify the location and length of the section header table. I have assumed the start of the file is &hdr.
So I did this, but it is giving me a seg-fault. What is the proper way to access this array?
int header_location = hdr.offset;
int header_length = hdr.headers;
SectionHeader *sec_hdrs = (SectionHeader *) &hdr + header_location;
SectionHeader sec_hdr;
for (int i = 0; i < header_length; i++) {
sec_hdr = sec_hdrs[i];
if (sec_hdr.type == SHT_SYMTAB) break;
}
Try this: ElfSectionHeader *sec_hdrs = (ElfSectionHeader *)((unsigned char *) &hdr + header_location);
Your orinal code &hdr + header_location would offset the pointer by sizeof(hdr) * header_location which is not your intention.
You declared sec_hdrs as a pointer to SectionHeader. It is not an array and it can't be indexed. Your compiler should raise a warning.
Try this:
SectionHeader hdrs[header_length]
int header_location = hdrs[0].offset;
int header_length = hdrs[0].headers;
SectionHeader *sec_hdrs = hdrs + header_location;
SectionHeader sec_hdr;
for (int i = 0; i < header_length; i++) {
sec_hdr = sec_hdrs[i];
if (sec_hdr.type == SHT_SYMTAB) break;
}
Here is a visualization of the memory with an initial offset followed by SectionHeader's placed in contiguous memory.
header_location | sizeof(SectionHeader)| sizeof(SectionHeader) | sizeof(SectionHeader)
vijairaj makes a very valid point about a possible bug in your code.
Your original code &hdr + header_location would offset the pointer by
sizeof(hdr) * header_location which is not your intention.
This is a valid diagnosis and you should investigate how pointer arithmetic works. We increment the address by the size of its type. Once you are sure that *sec_hdrs is pointing to the correct place, rerun your program. If the segfault persists, try my next piece of debugging advice.
Yes, on other questions here, I have seen that you might have to malloc first. But I do not understand why that is necessary if you have a pointer to the array if you know that it is in contiguous memory, and also how to do this.
Just because we know something is in contiguous memory does not mean it is safe from being overwritten or reused by our program. That is the point of malloc - to protect certain blocks of memory from being overwritten. If you access unallocated memory, you run the risk of accessing sensitive data, overwriting program-dependent data, or storing data that will get overwritten. This is why a segfault will occur and this is why you need to malloc.
Ensure that you malloc enough space:
malloc(header_location + header_length * sizeof(SectionHeader))
This line of code is saying, "Please allocate contiguous memory for one offset and n SectionHeader's". The malloc call will return a pointer to the start of that memory block (&hdr) and then you may access anything within that block of memory.
Perhaps include the code that is providing you with &hdr? Hope this is helpful!

Pointers inside shared memory segment

I've been trying this for hours, and google all the things I kind think of, but I'm going crazy.
I have a struct:
typedef struct {
int rows;
int collumns;
int* mat;
char* IDs_row;
} mem;
I don't know the sizes of the int* (a Matrix) and char* untill later.
When I do, I create the shared memory like this:
mem *ctrl;
int size = (2 + ((i-1)*num_cons))*sizeof(int) + i*26*sizeof(char); //I have the real size now
shmemid = shmget(KEY, size, IPC_CREAT | 0666);
if (shmemid < 0) {
perror("Ha fallado la creacion de la memoria compartida.");
exit(1);
}
ctrl = (mem *)shmat(shmemid, 0, 0);
if (ctrl <= (mem *)(0)) {
perror("Ha fallado el acceso a memoria compartida");
exit(2);
}
No problem here. Then I give a value to ctrl->rows and collumns, and assign 0 to all the matrix.
But after that, I write something in the char* and bam, segmentation fault.
Debugging the program I saw that both pointers, mat and IDs_row where null. How do I give them the correct values inside the shared memory segment??
I tried removing the char* pointer, just to give it a try, and then the segmentation fault error was in the other program that connected to said shared memory and just checked the values inside the matrix (checking ->rows and ->collumns was succesfull)
First of all, putting absolute pointers in shared memory segments is terrible terible idea - those pointers would only be valid in the process that filled in their values. Shared memory segments are not guaranteed to attach at the same virtual address in every process. On the contrary - they attach where the system deems it possible when shmaddr == NULL is specified on call to shmat(). You could specify the same virtual address when calling shmat() but it is up to you to ensure that nothing else is mapped on that memory region in all participating processes. This is hard to do in a portable manner. What you would most like to do is to either:
1) Allocate one big shared memory segment that accomodates both the mem structure and the two data arrays. Then you should put not absolute pointers but rather pointers relative to the beginning of the memory block and then adjust on usage.
2) Allocate three different shared memory segments but instead of putting pointers, put the shared memory IDs as returned by shmget():
typedef struct {
int rows;
int collumns;
int mat_id;
int IDs_row_id;
} mem;
When you need to access the matrix or the IDs array just attach to the shared memory ID stored in the corresponding field.
Pay attention though that using the same KEY in subsequent invocations of shmget() will not produce the expected result unless KEY == IPC_PRIVATE. It is best to use a the fixed key value for the shared memory block with the descriptor (of type mem) and IPC_PRIVATE for the other two memory blocks otherwise the three calls will actually return the same shared memory block - the first one will create it and the next two will simply return its ID since a block with that key already exists.
ctrl = (mem *)shmat(shmemid, 0, 0);
This only assigns valid memory to the ctrl pointer, not to ctrl->mat or ctrl->IDs_row.
You probably want:
mem *ctrl;
shmemid = shmget(KEY, sizeof(ctrl), IPC_CREAT | 0666);
//allocate memory for the structure
ctrl = (mem *)shmat(shmemid, 0, 0);
//allocate memory for the int*
shmemid = shmget(KEY,((i-1)*num_cons))*sizeof(int), IPC_CREAT | 0666);
ctrl->mat = (int*)shmat(shmemid, 0, 0);
//allocate memory for the char*
shmemid = shmget(KEY,i*26*sizeof(char), IPC_CREAT | 0666);
ctrl->IDs_row = (char*)shmat(shmemid,0,0);

realloc inside a realloc

In C can you have realloc inside realloc? For example, a struct inside a struct when you need to malloc both of them and realloc both of them. If yes can someone please provide a simple example?
Thank you in advance.
Your question is not dreadfully clear, but...
Yes, a given dynamically allocated structure (for example, an array of structures) can itself contain pointers to allocated data (such as various other arrays of allocated structures), and you can reallocate the various parts independently.
However, the system will not call realloc() for you while you are reallocating one of the structures; you would have to separately program the various resizing operations.
Example nested data structures:
struct line { char *info; size_t length; };
struct section { size_t num_lines; struct line *lines; };
You could allocate an array of sections, and reallocate that array when needed. Each section contains an array of lines, and each of those arrays of lines can be independently reallocated too.
Hence:
size_t num_sections = 0;
size_t max_sections = 0;
struct section *sections = 0;
if (num_sections == max_sections)
{
size_t new_max = (max_sections + 1) * 2;
struct section *new_sections;
if (sections == 0)
new_sections = malloc(new_max * sizeof(*new_sections));
else
new_sections = realloc(sections, new_max * sizeof(*new_sections));
if (new_sections == 0)
...out of memory error...
sections = new_sections;
max_sections = new_max;
}
struct section *section = &sections[num_sections++]; // Newly available section
section->num_lines = 0;
section->lines = 0;
return section;
(I'm assuming C99 - with variable declarations where I want them.)
A similar process applies for the array of lines within a section, except the section structure doesn't have separate values for the number of allocated lines and the number of lines actually in use. Each line also has its own allocated memory for the string of characters, of course...

Resources