noob alert with C here.
I have a struct as such
typedef struct {
char* name
} info;
And storing an array (size 10) of this struct (in another struct called table) in a shared memory object using the shm_open call:
int fd = shm_open("someName", O_CREAT | O_EXCL | O_RDWR, S_IRWXU);
if (fd < 0) {
fd = shm_open("someName", O_CREAT | O_RDWR, S_IRWXU);
if (fd < 0) {
printf("ERROR: Could not open shared memory space\n");
return -1;
}
}
(*tables) = mmap(NULL, sizeof(table), PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
ftruncate(fd, sizeof(table));
close(fd);
However, the issue that I have is that later through the code such a scenario happens:
If process B runs this and puts some text, say "foo" in the name field of an element of the table array, process A does not have access to that char*.
All it sees is a memory address, but not the actual content of the char* as opposed to if it wrote the content itself. However, I would also like to note that if I replace char* with a fixed size char[], say char name[20] instead of char *name, then this issue does not occur.
I would like to know whether there is any way around this, and if not, why is it so?
Thank you!
When using shared memory for IPC, all of the data to be shared, must be located in shared memory. It's that simple, there's no way around it. What you can do however for some larger, more variable chunks of data, is simply allocate a dedicated shared memory chunk for that data, and provide its name via the master table. Another option in your case is to arrange for the shared memory to be sufficiently larger than your info struct, such that name is just an offset from that address, to where the name data resides. Then address of data is `&name + *name'.
Related
I have a structure A of the form described below. My goal is to create the structure in one process and share it across to a different process. A->buffer depends upon user_size and value provided and hence I cannot have a pre-allocated buffer. Knowing that sharing pointers with shared memory is hard across processes since pointer value in one process will no longer be valid in the other process, I tried the following approach.
I wanted to allocate the buffer from shared memory, from the writing process so that other processes can map to the same shared memory and obtain a valid pointer address to the shared memory.
Conceptually, calling mmap on the same shared memory fd descriptor should point to the same location in shared memory, however on the reading process, I'm able to get a valid pointer and valid another_value.
Can someone clarify this? Parts of the code below.
struct A {
int another_variable;
void *buffer;
}
/* the size (in bytes) of shared memory object */
const int SIZE = 4096;
/* name of the shared memory object */
const char *name = "OS";
/* shared memory file descriptor */
int shm_fd;
/* create the shared memory object */
shm_fd = shm_open(name, O_CREAT | O_RDWR, 0666);
/* configure the size of the shared memory object */
ftruncate(shm_fd, SIZE);
a_data = mmap(NULL, sizeof(*A), PROT_READ | PROT_WRITE, MAP_SHARED,
shm_fd, 0);
if (a_data == MAP_FAILED) {
printf("ERROR: mmap failed for a_data\n");
exit(EXIT_FAILURE);
}
a_data->buffer = mmap(NULL, user_size, PROT_WRITE,
MAP_SHARED , shm_fd, 0);
if (a_data->buffer == MAP_FAILED) {
printf("ERROR: mmap failed for a_data buffer\n");
exit(EXIT_FAILURE);
}
memcpy(a_data->buffer, "test", strlen("test"));
I wanted to write some function void* share(void*, int) that should set up shared memory to share the data at the pointer.
My first attempt looked like (without checks etc.):
void* share(void *toBeShared, int size) {
int fd = shm_open(SHM_NAME, O_CREAT | O_RDWR | O_EXCL, S_IRUSR | S_IWUSR);
ftruncate(fd, size);
return mmap(toBeShared, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
}
but this does not seem to work as I would like it. The second attempt was something like:
void* share(void *toBeShared, int size) {
void *mem = NULL;
int fd = shm_open(SHM_NAME, O_CREAT | O_RDWR | O_EXCL, S_IRUSR | S_IWUSR);
ftruncate(fd, size);
mem = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0)
memcpy(mem, toBeShared, size);
return mem;
}
and this does work, but I need to copy the entire data, which I would like to avoid.
Therefore my question: is there a way to share memory that has already been allocated (if possible without having to copy too much around) and if yes, how could it be done?
Thanks in advance.
PS: I've seen more of these questions (e.g. here and here), but there are no answers given in there.
edit:
how I would like to use it:
typedef struct {
char *name;
int status;
} MyTask;
int main(int argc, char** argv) {
MyTask* taskList = NULL, sharedTaskList = NULL;
int length = 0;
...
readFile(&taskList, &length, ...);
sharedTaskList = share(taskList, length * sizeof(MyTask));
// or maybe even better: without needing to assign it to new variable
for(i = 0; i < NR_WORKERS; i++) {
switch(pid = fork()) {
//etc...
}
}
...
return 0;
}
How to share existing memory?
Don't share existing memory. Get some (small amount of) "fresh" shared memory and use (i.e. fill or read) it later.
Assuming you are on Linux, read shm_overview(7).
I guess that some of your functions might fail. You should test against failure each call, e.g.
int fd = shm_open(SHM_NAME, O_CREAT | O_RDWR | O_EXCL, S_IRUSR | S_IWUSR);
if (fd<0) {perror("shm_open"); exit(EXIT_FAILURE);};
and so on. Perhaps use also strace(1)
is there a way to share memory that has already been allocated
Short answer, no! (or not easily, and not in a portable way). You conventionally do the opposite: obtain some shared segment of known size, and use some pointers into it. (the same shared segment might have different virtual addresses in different processes, e.g. because of ASLR).
You could use mmap(2) with MAP_FIXED on some already used virtual address space subsegment (that would overwrite & replace the mapping with a new one, not share an existing mapping!), but I would suggest to avoid that. Notice that the virtual address space is managed in multiples of pages, so there is no way to share some data which is not page aligned. So your share function is impossible unless both toBeShared and size are page-aligned. You could consider the Linux specific mremap(2)
In other words, your applications should first allocate some shared memory and then put/use some data inside the obtained shared segment, not try to share some existing unshared virtual memory range. So you probably want to code some void* get_my_shared_memory(); (assuming the size is a compile time constant, and you call that function once per process, and its resulting virtual address would often vary from one process to another)
In practice, memory is a finite resource, and shared memory is a scarce and very limited resource. On most systems, you'll be able to share a few dozens of megabytes only... So sharing an arbitrary large amount of memory is unreasonable.
Perhaps your entire application might just use some server, e.g. some database server à la PostGreSQL, to share information, by making requests to that server (and using ACID properties of DBMS). Or you could organize it as a monitoring process exchanging messages (e.g. URL to be processed) -on pipes or sockets or fifos- with slave processes. But we don't know what kind of application are you coding.
BTW, sharing memory is not enough. You need to synchronize your processes.
I'm having trouble accessing a global struct pointer that I'm initalizing with mmap. Attempting to access members of the struct in functions outside of the one it is declared in throw segfaults.
the struct:
typedef struct foo {
uint32_t size;
bar_t array[0];
} foo_t;
the initialization:
foo_t* foo; // global
static void* init_function(...) {
fd = open(filename, O_CREAT | O_WRONLY, 0644);
write(...);
lseek(...);
write(...);
foo = mmap(0, BIG_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
close(fd);
foo->size = 0;
}
what causes the segfault:
static int another_function(...) {
if (foo->size == 0) {...} //foo->size causes it
}
BIG_SIZE is a defined value that should be adequately large for my needs.
Any ways, I apologize for the (...)s, but this is where the issues are showing up. I've looked into mmap docs and variable length methods with no luck. Thanks in advance!
You don't show us everything, but it seems that you just assign your global variable, but that you don't initialize the contents of the mapped region. You should at least give it a
foo->size = 0 somewhere, if the file is newly created, or alternatively you should use ftruncate instead of your fseek write sequence to warrant that the blocks (including size) are 0 filled.
Firstly check if mmap really succeeded:
foo = mmap(...);
if ( MAP_FAILED == foo )
{
//no memory for me...
}
And if I remember correctly, the file must be at least BIG_SIZE long, otherwise mmap will fail - but I may be wrong on that part...
I've got this struct:
typedef struct ip_row {
sem_t row_lock;
char row_name[NAME_SIZE];
char row_address4[NAME_SIZE]; // IPv4 address name
char row_address6[NAME_SIZE]; // IPv6 address name
} ip_row_t;
I would like to use the struct multiple times in a shared memory file.I have verified that for exactly one use, it works.
int shmfd; //Shared memory file descriptor
struct ip_row *data;
/*...creating shared memory and map...*/
shmfd = shm_open(shared_memory, O_CREAT | O_RDWR, S_IRUSR | S_IWUSR | S_IRGRP );
ftruncate(shmfd,FILESIZE);
data = (ip_row_t*)mmap(0, FILESIZE, PROT_READ | PROT_WRITE, MAP_SHARED, shmfd,0);
/*...getting to user inputs, this stuff is in a loop...*/
strcpy(data->row_name,ipstr1);
strcpy(data->row_address6,ipstr2);
strcpy(data->row_address4,ipstr3);
When I run the loop again, the writing starts at the beginning of the shared memory file, overwriting what was there before. How can I move the offset so I can support more entries into the shared memory? I've tried these two:
lseek(shmfd,sizeof(struct ip_row_t),SEEK_CUR); //segfault when we write agian
data = (ip_row_t*)mmap(0, FILESIZE, PROT_READ | PROT_WRITE, MAP_SHARED, shmfd,sizeof(struct ip_row_t)); //also segfaults when we try to read
Any advice would be greatly appreciated.
`
You should use pointer arithmetic on struct ip_row *data; to reach "further" in your mmaped shared memory since mmap returns nothing but a pointer to a memory region of FILESIZE, which happens to be mirrored into shared memory.
For example use data[0] to access first copy, data[1] to access second etc.
lseek on shared memory object is unspecified and your second mmap causes segfault because you are trying to mmap FILESIZE bytes to a FILESIZE memory region that has been allocated to you but at an sizeof(struct ip_row_t) offset, thus effectively going outside the memory you are allowed to access, besides, offset has to be multiple of getpagesize(), which in this case it probably is not.
I have a process that dived itself with fork. I need to create a region of memory (a matrix) for the result of the computation of each process. How can I do this? Everything I tried or I can use but it's not shared between processes or I can't use (not sure if shared or not). Someone knows what I can use? It can be something simple and without any security. The simpler the better.
I tried shmget but it's not sharing and I couldn't get how to use mmap to allocate or use it correctly. I tried other estranges things, but nothing. Any tips?
Some tries:
segment_id = shmget(IPC_PRIVATE, (sizeof(int) * linhas_mat1 * colunas_mat2) , S_IRUSR|S_IWUSR);
matriz_result = (int **) shmat(segment_id, NULL, 0);
Forks after that. Each process can use the matriz_result normally as a matrix, but the memory is not shared. Each one has one like a local variable.
segment_id = shm_open("/myregion", O_CREAT | O_RDWR, S_IRUSR | S_IWUSR);
matriz_result = mmap(NULL, (sizeof(int) * linhas_mat1 * colunas_mat2), PROT_READ | PROT_WRITE, MAP_SHARED, segment_id, 0);
Tried this with mmap, but I don't know if it's right. I'm not good with such low level programming and I couldn't find any good example on how to use it correctly.
declarations:
int segment_id is;
int **matriz_result;
int createMemShare(){
//File descriptor declaration:
int fd;
//We want to open the file with readwrite,create it, and empty it if it exists
//We want the user to have permission to read and write from it
fd = open(MEMSHARENAME, O_RDWR| O_CREAT | O_TRUNC, S_IRUSR| S_IWUSR );
if(fd <= 0){
puts("Failed in creating memory share .");
return -1;
}
//Move the file pointer and write an empty byte, this forces the file to
//be of the size we want it to be.
if (lseek(fd, MEMSHARESIZE - 1, SEEK_SET) == -1) {
puts("Failed to expand the memory share to the correct size.");
return -1;
}
//Write out 1 byte as said in previous comment
write(fd, "", 1);
//Memory share is now set to use, send it back.
return fd;
}
//Later on...
int memShareFD = mmap(NULL, MEMSHARESIZE, PROT_READ, MAP_SHARED, fd, 0);
//And to sync up data between the processes using it:
//The 0 will invalidate all memory so everything will be checked
msync(memshareFD,0,MS_SYNC|MS_INVALIDATE);
you can try the above function to create a shared memory space. Essentially all you need to do is treat it like any other file once you've made it. The code example on the man page is pretty complete and worth a look into: check it out here
Edit:
You'd probably be better off using shm_open as Jens Gustedt suggested in the comments. It's simple to use and simpler than making the file yourself with the function I've written above.