This question already has answers here:
C - shared memory - dynamic array inside shared struct
(5 answers)
Closed 5 years ago.
I am working on a server with forking which stores a map(k->v) of strings in shared memory. I want to do it very simplistic but am lost in pointers and in what exactly I need to copy. So I extracted the relevant code which looks like this:
struct key_value {
char key[32];
char value[32];
};
struct key_value **map;
int *map_size;
int shmid = shmget(IPC_PRIVATE, sizeof(struct key_value**), 0600);
map = (struct key_value**) shmat(shmid, 0, 0);
int shmid_size = shmget(IPC_PRIVATE, sizeof(int), 0600);
map_size = (int*) shmat(shmid_size, 0, 0);
*map_size = 0;
//the above happens before fork()
char *c = "abc";
int shmid_struct = shmget(IPC_PRIVATE, sizeof(struct key_value*), 0600);
struct key_value *entry = (struct key_value*) shmat(shmid_struct, 0, 0);
*entry->key = *c;
printf("%s\n", map[0]->key);
//smhdt's & shmctl's
So what I want is to copy that string "abc" from *c into the map so into shared memory. Clearly, I do not yet fully understand pointers and structs so am hoping someone can clear it up. I currently get a segfault 'somewhere in main' (thanks gdb...).
Note that I am ok with the map having a fixed max_size for now (though would be great if dynamic).
EDIT: it's been pointed out in an answer that having a char* in the struct is difficult, so to use char[x] instead. Have updated the code to reflect that, but still not working.
Structures which contain pointers cannot be safely stored in shared memory, as the pointers are meaningless outside the process that created them. Even if the shared memory region is mapped at the same address in each process (which is true if the memory was mapped before a fork(), but may be false in other scenarios), pointers into non-shared memory will not behave properly, as each process may have different data at that address.
If you want to store strings in shared memory, you will need to store them as explicit character arrays, e.g.
struct key_value {
char key[32];
char value[32];
};
or use another scheme, such as storing an offset into a string table in the shared memory region.
Generally speaking, though, shared memory is not a good tool for inter-process communication. If your application depends on being able to share data in memory, threading is probably a better approach.
Related
I have a data structure that acts as a buffer. It's two structures that have a char *color within them. I am able to store and retrieve the correct color out of each producer process by themselves, but the consumer does not seem to see anything in the color variable, though it knows it's not null. How can this be?
typedef struct {
char *color;
struct timeval time;
} data_struct;
typedef struct{
data_struct buf1;
data_struct buf2;
} buffer;
In the producer I store:
ptr->buf1.color = color;
gettimeofday (&tval, NULL);
ptr->buf1.time = tval;
And I am able to then print the color it just stored and it does work within the same process. But the consumer uses:
printf("%s\t", ptr->buf1.color);
struct timeval tval = ptr->buf1.time;
printf("%ld\n", (tval.tv_sec)*1000000L + tval.tv_usec);
And he just ends up printing a blank area, then a tab, then the time in seconds. How is it accessing the same shared memory but not seeing my string?? This approximately identical code works in the threaded version just fine. Thanks for your help!
Update with shared memory segments.
int shmem_id;
buffer *shmem_ptr;
key_t key = 7484;
int size = 2048;
int flag = 1023;
char keystr[10];
/* create a shared memory segment */
shmem_id = shmget (key, size, flag);
shmem_ptr = shmat (shmem_id, (void *) NULL, 1023);
shmem_ptr->buf1.color = NULL;
shmem_ptr->buf2.color = NULL;
sprintf (keystr, "%d", key);
Even though buf may be in shared memory, that does not guarantee that color is also pointing to something in shared memory. My guess is that color is pointing at some address that is only visible to the producer process and not the consumer process. To fix this, you could define the color field as a fixed length char array, and use
strncpy(ptr->buf1.color, color, BUFLEN);
ptr->buf1.color[BUFLEN-1] = '\0';
to set the color.
I think the reason that you are not malloc the struct and buffer properly as they are not protected and thus it would be over written by any thing. Try to do things like these:
typedef struct {
char *color;
struct timeval time;
} data_struct;
data_struct *temp=(data_struct*)malloc(sizeof(data_struct));
temp->color = (char *)malloc(strlen(color)+1);
strcpy(temp->color, color);
I think doing these would not generate problem. But remember to free the allocated memory first of string then structure.
free(temp->color);
free(temp);
It doesn't work because you have one process which stores a pointer, and a separate process which reads that pointer. But these two processes do not share the same address space, so the pointer is meaningless when read. You could solve this a few ways:
Use threads instead of processes; this way the producer and consumer will have a single address space and pointers from one will be valid in the other.
Store the text directly in the struct, without a pointer indirection (i.e. char color[30]).
Store an "offset pointer" in the struct, and store the character data somewhere else in the shared memory region (a sort of pool allocator).
Make an array of fixed strings in the producer and consumer and just store the index into that lookup table. This works if the strings are known in advance.
am using C on OSx and using shmget() to get memory to be used between two different programs i
have a structure of 2 int values
does this mean if i want a size of 27 instead it will have to be 54 and how would this work with the pointers used for the structure or any help would be greatly appreciated
No, you need to use sizeof on the struct. You can't guarantee how the implementation will pad the values inside the struct, so you don't assume the size of them. Also, if the size changes later due to adding member(s) to the struct, the memory allocation will still work correctly.
Then use that value from sizeof to multiply by the number of instances of the struct you intend to use in the shared memory area.
Assume you have a struct like this:
typedef struct {
int aCount;
int bCount;
int cCount;
} data_t;
This below function will create shared memory for the specified size.
int shared_memory_create(size_t memory_size)
{
int shm_key = shmget(IPC_PRIVATE, memory_size, IPC_CREAT | IPC_EXCL | 0666);
if(shm_key < 0) {
perror("Failed to Create Shared Memory Key");
exit(0);
}
return shm_key;
}
If you want to create shared memory to accommodate 27 elements, call the function shared_memory_create as shown below:
shm_key = shared_memory_create(27*sizeof(data_t));
This question already has answers here:
Writing and reading (fwrite - fread) structures with pointers
(3 answers)
Closed 8 years ago.
gcc (GCC) 4.7.0
c89
Hello,
I have the following structure that I am trying to fwrite and fread.
However, because my device and resource are pointers. The fwrite will read the pointer values and not the data. I cannot use a array for the device or resource. Only pointers as they have to be dynamically allocated.
I allocate all memory for the structure elements before I write. Not shown here as I want to keep the snippet short. Nor is free'ing.
In my fread function, I allocate the memory for the device and resource so that the fread will read into these memory locations. However, this will not work.
What is the best way to do this?
Many thanks for any advice,
struct data {
int id;
int set;
char *device;
char *resource;
};
struct database {
struct data **db_data;
size_t database_rows;
size_t database_data_size;
};
int database_write(FILE *fp, const struct database *db)
{
rewind(fp);
if(fwrite(*db->db_data, sizeof(struct data), 1, fp) == -1) {
return DATABASE_ERROR;
}
return 0;
}
struct database* database_read(FILE *fp, size_t db_rows, size_t db_data_size)
{
struct database *db = NULL;
size_t i = 0;
db = malloc(sizeof(struct database));
db->database_rows = db_rows;
db->database_data_size = db_data_size;
db->db_data = malloc(sizeof(struct data) * db_rows);
for(i = 0; i < db_rows; i++) {
db->db_data[i] = malloc(sizeof(struct data));
db->db_data[i]->device = malloc(db_data_size);
db->db_data[i]->resource = malloc(db_data_size);
}
rewind(fp);
if(fread(*db->db_data, sizeof(struct data), 1, fp) == -1) {
return NULL;
}
return db;
}
You seem to have answered your own question, fread and fwrite just look at what's in memory and put that in the file. This works great if you're writing things that don't have pointers (e.g. big arrays of numbers). It's not designed to write structs with pointers.
If this file has a format, you need to do what the format says. If you're making up a format as you go, then you should write each member one by one into the file. You will need some sort of buffer to read into (you may need to resize this if you don't have a maximum length specification). Also, your database_write function will need to be changed quite a bit as well.
If device and resource can have variable length you should write down the size of device and then the data. Do the same for resource.
When you read them back you can read the size, then allocate memory and finally read the value.
You have yourself described you problem. fwrite will write the address and not the value.
May be you can use a field for the length of device and resource in your structure "struct data".
Create a wrapper for fread() and fwrite() which reads/writes this length.
In this wrapper you can memcpy devices, resource in a temporary buffer and use fwrite() on it.
This is a simple and very basic solution.
While sending packets in networks, you will generally see a structures containing char pointers. The first 4/8 bytes store the length of the data and the remaining bytes contain the actual data.
User reading the packet, first reads the beginning 4/8 bytes. Depending on this, read() call is issued to read the remaining data.
You may refer
Is the "struct hack" technically undefined behavior?
I've been trying this for hours, and google all the things I kind think of, but I'm going crazy.
I have a struct:
typedef struct {
int rows;
int collumns;
int* mat;
char* IDs_row;
} mem;
I don't know the sizes of the int* (a Matrix) and char* untill later.
When I do, I create the shared memory like this:
mem *ctrl;
int size = (2 + ((i-1)*num_cons))*sizeof(int) + i*26*sizeof(char); //I have the real size now
shmemid = shmget(KEY, size, IPC_CREAT | 0666);
if (shmemid < 0) {
perror("Ha fallado la creacion de la memoria compartida.");
exit(1);
}
ctrl = (mem *)shmat(shmemid, 0, 0);
if (ctrl <= (mem *)(0)) {
perror("Ha fallado el acceso a memoria compartida");
exit(2);
}
No problem here. Then I give a value to ctrl->rows and collumns, and assign 0 to all the matrix.
But after that, I write something in the char* and bam, segmentation fault.
Debugging the program I saw that both pointers, mat and IDs_row where null. How do I give them the correct values inside the shared memory segment??
I tried removing the char* pointer, just to give it a try, and then the segmentation fault error was in the other program that connected to said shared memory and just checked the values inside the matrix (checking ->rows and ->collumns was succesfull)
First of all, putting absolute pointers in shared memory segments is terrible terible idea - those pointers would only be valid in the process that filled in their values. Shared memory segments are not guaranteed to attach at the same virtual address in every process. On the contrary - they attach where the system deems it possible when shmaddr == NULL is specified on call to shmat(). You could specify the same virtual address when calling shmat() but it is up to you to ensure that nothing else is mapped on that memory region in all participating processes. This is hard to do in a portable manner. What you would most like to do is to either:
1) Allocate one big shared memory segment that accomodates both the mem structure and the two data arrays. Then you should put not absolute pointers but rather pointers relative to the beginning of the memory block and then adjust on usage.
2) Allocate three different shared memory segments but instead of putting pointers, put the shared memory IDs as returned by shmget():
typedef struct {
int rows;
int collumns;
int mat_id;
int IDs_row_id;
} mem;
When you need to access the matrix or the IDs array just attach to the shared memory ID stored in the corresponding field.
Pay attention though that using the same KEY in subsequent invocations of shmget() will not produce the expected result unless KEY == IPC_PRIVATE. It is best to use a the fixed key value for the shared memory block with the descriptor (of type mem) and IPC_PRIVATE for the other two memory blocks otherwise the three calls will actually return the same shared memory block - the first one will create it and the next two will simply return its ID since a block with that key already exists.
ctrl = (mem *)shmat(shmemid, 0, 0);
This only assigns valid memory to the ctrl pointer, not to ctrl->mat or ctrl->IDs_row.
You probably want:
mem *ctrl;
shmemid = shmget(KEY, sizeof(ctrl), IPC_CREAT | 0666);
//allocate memory for the structure
ctrl = (mem *)shmat(shmemid, 0, 0);
//allocate memory for the int*
shmemid = shmget(KEY,((i-1)*num_cons))*sizeof(int), IPC_CREAT | 0666);
ctrl->mat = (int*)shmat(shmemid, 0, 0);
//allocate memory for the char*
shmemid = shmget(KEY,i*26*sizeof(char), IPC_CREAT | 0666);
ctrl->IDs_row = (char*)shmat(shmemid,0,0);
I want (or need) to do something along the lines of
char **my_array = malloc(1000*64);
strcpy(arr[0], "test");
While I know that arr[0] isn't pointing to a separate piece of allocated memory, I thought one should be able to copy a string into it like this (yet it segs). This works
arr[0] = "test";
However that is not going to work, for my actual goal is to do this in shared memory.
shm_array = shmget(IPC_PRIVATE, 1000 * 64, IPC_CREAT | 0644);
my_array = (char**) shmat(shm_array, (void**)0, 0);
Actually my question could be rephrased to: "How do you create an array of strings in shared memory?". I tried creating 1000 separate "string" shared memory segments, but apart of that it did not work it also seems wrong. Moreover, I thought one should simply be able to write into a big shared memory segment using relative pointer offsets.
You could just create one single piece of memory and write to specific offsets:
char * const buf = malloc(HUGE);
strcpy(buf + offset1, "hello");
strcpy(buf + offset2, "world");
It'd probably be better to use strncpy and pass a size of HUGE - offset along to make sure you don't run over the end. Managing the offsets is your own responsibility. Or you can use strncat, if efficiency doesn't matter so much.
It looks like you are looking for a 2D array 1000 by 64. If this is indeed the case, you can do this:
struct shared_1000_by_64 {
char strings[1000][64];
};
struct shared_1000_by_64 *shared = malloc(sizeof(struct shared_1000_by_64));
for (int i = 0 ; i != 1000 ; i++) {
strcpy(shared->strings[i], "test");
}
This uses the standard trick of preventing the array from decaying into a pointer.
If you want to dynamically allocate an array of 64-character arrays, then use a pointer to an array rather than a pointer to a pointer:
char (*my_array)[64] = malloc(1000 * sizeof my_array[0]);
strcpy(my_array[0], "test");
or for the shared-memory case,
shm_array = shmget(IPC_PRIVATE, 1000 * sizeof my_array[0], IPC_CREAT | 0644);
my_array = shmat(shm_array, NULL, 0);