I have a process that dived itself with fork. I need to create a region of memory (a matrix) for the result of the computation of each process. How can I do this? Everything I tried or I can use but it's not shared between processes or I can't use (not sure if shared or not). Someone knows what I can use? It can be something simple and without any security. The simpler the better.
I tried shmget but it's not sharing and I couldn't get how to use mmap to allocate or use it correctly. I tried other estranges things, but nothing. Any tips?
Some tries:
segment_id = shmget(IPC_PRIVATE, (sizeof(int) * linhas_mat1 * colunas_mat2) , S_IRUSR|S_IWUSR);
matriz_result = (int **) shmat(segment_id, NULL, 0);
Forks after that. Each process can use the matriz_result normally as a matrix, but the memory is not shared. Each one has one like a local variable.
segment_id = shm_open("/myregion", O_CREAT | O_RDWR, S_IRUSR | S_IWUSR);
matriz_result = mmap(NULL, (sizeof(int) * linhas_mat1 * colunas_mat2), PROT_READ | PROT_WRITE, MAP_SHARED, segment_id, 0);
Tried this with mmap, but I don't know if it's right. I'm not good with such low level programming and I couldn't find any good example on how to use it correctly.
declarations:
int segment_id is;
int **matriz_result;
int createMemShare(){
//File descriptor declaration:
int fd;
//We want to open the file with readwrite,create it, and empty it if it exists
//We want the user to have permission to read and write from it
fd = open(MEMSHARENAME, O_RDWR| O_CREAT | O_TRUNC, S_IRUSR| S_IWUSR );
if(fd <= 0){
puts("Failed in creating memory share .");
return -1;
}
//Move the file pointer and write an empty byte, this forces the file to
//be of the size we want it to be.
if (lseek(fd, MEMSHARESIZE - 1, SEEK_SET) == -1) {
puts("Failed to expand the memory share to the correct size.");
return -1;
}
//Write out 1 byte as said in previous comment
write(fd, "", 1);
//Memory share is now set to use, send it back.
return fd;
}
//Later on...
int memShareFD = mmap(NULL, MEMSHARESIZE, PROT_READ, MAP_SHARED, fd, 0);
//And to sync up data between the processes using it:
//The 0 will invalidate all memory so everything will be checked
msync(memshareFD,0,MS_SYNC|MS_INVALIDATE);
you can try the above function to create a shared memory space. Essentially all you need to do is treat it like any other file once you've made it. The code example on the man page is pretty complete and worth a look into: check it out here
Edit:
You'd probably be better off using shm_open as Jens Gustedt suggested in the comments. It's simple to use and simpler than making the file yourself with the function I've written above.
Related
noob alert with C here.
I have a struct as such
typedef struct {
char* name
} info;
And storing an array (size 10) of this struct (in another struct called table) in a shared memory object using the shm_open call:
int fd = shm_open("someName", O_CREAT | O_EXCL | O_RDWR, S_IRWXU);
if (fd < 0) {
fd = shm_open("someName", O_CREAT | O_RDWR, S_IRWXU);
if (fd < 0) {
printf("ERROR: Could not open shared memory space\n");
return -1;
}
}
(*tables) = mmap(NULL, sizeof(table), PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
ftruncate(fd, sizeof(table));
close(fd);
However, the issue that I have is that later through the code such a scenario happens:
If process B runs this and puts some text, say "foo" in the name field of an element of the table array, process A does not have access to that char*.
All it sees is a memory address, but not the actual content of the char* as opposed to if it wrote the content itself. However, I would also like to note that if I replace char* with a fixed size char[], say char name[20] instead of char *name, then this issue does not occur.
I would like to know whether there is any way around this, and if not, why is it so?
Thank you!
When using shared memory for IPC, all of the data to be shared, must be located in shared memory. It's that simple, there's no way around it. What you can do however for some larger, more variable chunks of data, is simply allocate a dedicated shared memory chunk for that data, and provide its name via the master table. Another option in your case is to arrange for the shared memory to be sufficiently larger than your info struct, such that name is just an offset from that address, to where the name data resides. Then address of data is `&name + *name'.
i can check entry point of my binary with "$readelf cbinary -a" and through the code. But how to check its entry point virtual adr when binary is mmaped and then jump there?
int fd;
int PageSize;
char *fileName = "/home/dssiam/workspace_eclipse/hello/src/cprog";
if ((PageSize = sysconf(_SC_PAGE_SIZE)) < 0) {
perror("sysconf() Error=");
}
if ((fd = open(fileName, O_RDWR, S_IXUSR | S_IXGRP | S_IXOTH)) == -1)
{
perror("err open file:");
exit(1);
}
else
{
fd = open(fileName, O_RDWR, S_IXUSR | S_IXGRP | S_IXOTH);
}
void *address;
int len;
off_t my_offset = 0;
len = PageSize*3; //Map one page
address = mmap(NULL, len, PROT_WRITE, MAP_SHARED, fd, my_offset);
if (address == MAP_FAILED)
{
perror("mmap error. ");
}
lseek(fd, 24, SEEK_SET);
unsigned long entry_point;
read(fd, &entry_point, sizeof(entry_point)); //IT RETURN entry point adr of my binary at "/home/dssiam/workspace_eclipse/hello/src/cprog" but not in VM
printf("entry: 0x%lx\n", entry_point);
close(fd);
void *ptr = (void *)0x80484b0; // 0x80484b0 - entry_point vaddress
goto *ptr; //no jump here
so i can jump to the start of my main program, but i cant jump to the binary "cprog" stored at my hdd and mmaped region too.
any help would be appreciated.
The code has lots of mistakes (wrong mmap protection, wrong mmap start address, arbitrary pagesize, C standard specifically prohibits this kind of computed goto) but the biggest problem is that this method simply will not work, except maybe for the most basic cases.
You cannot just mmap a single function from elf file into the memory and expect it to work -- you will need to perform relocations for relocatable code, and even for PIC (position independent code), you still need to create GOT.
I am going to guess that what you really want to dynamically load complied files, so use a standard way to do this: compile your file into .so dynamic library, then use dlopen/dlsym to access functions from the file.
I am using shared memory for communication between two different process. I am creating shared memory of 16 MB size. I am trying to attach two different parts of the shared memory. One for writing and other for reading. Even though it maps to different memory address but when one is modified other also gets changed. I must be doing something wrong. Below is the code snippet where I am attaching to multiple shared memory location.
void createCommPool ()
{
CommSet set1;
int shmid1;
int fd1;
int r;
void * ptr;
void * ptr_res;
umask (0);
fd1 = open(SHARED_MEMORY0, O_CREAT | O_TRUNC | O_RDWR, 0777);
if (fd1 == -1)
error_and_die("open");
r = ftruncate(fd1, region_size);
if (r != 0)
error_and_die("ftruncate");
ptr = mmap(0, sizeof(struct operation_st), PROT_READ | PROT_WRITE,
,MAP_SHARED,fd1,sizeof(struct operation_st));
if (ptr == MAP_FAILED)
error_and_die("mmap");
close(fd1);
set1.shm_addr = ptr;
fd1 = open(SHARED_MEMORY0, O_RDWR, 0777);
if (fd1 == -1)
error_and_die("open");
fprintf(stderr,"The value of the file descriptor:%d\n",fd1);
if (lseek(fd1,sizeof(struct operation_st),SEEK_SET)<0)
{
fprintf(stderr,"could not perform lseek\n");
perror("lseek");
}
ptr_res = mmap(0,sizeof(struct operation_st), PROT_READ| PROT_WRITE,
MAP_SHARED,fd1,0);
if (ptr_res == MAP_FAILED)
error_and_die("mmap2");
close(fd1);
set1.shm_addr_res = ptr_res;
}
For data in shared memory, avoid the influence of bytes alignment with pack:
#pragma pack(1)
your shared memory code
#pragma unpack
lseek does not have any effect on the mapping of the shared memory. The offset parameter should be used in order to map to the different part of the shared memory. The offset should be in multiples of page size.
I wanted to write some function void* share(void*, int) that should set up shared memory to share the data at the pointer.
My first attempt looked like (without checks etc.):
void* share(void *toBeShared, int size) {
int fd = shm_open(SHM_NAME, O_CREAT | O_RDWR | O_EXCL, S_IRUSR | S_IWUSR);
ftruncate(fd, size);
return mmap(toBeShared, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
}
but this does not seem to work as I would like it. The second attempt was something like:
void* share(void *toBeShared, int size) {
void *mem = NULL;
int fd = shm_open(SHM_NAME, O_CREAT | O_RDWR | O_EXCL, S_IRUSR | S_IWUSR);
ftruncate(fd, size);
mem = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0)
memcpy(mem, toBeShared, size);
return mem;
}
and this does work, but I need to copy the entire data, which I would like to avoid.
Therefore my question: is there a way to share memory that has already been allocated (if possible without having to copy too much around) and if yes, how could it be done?
Thanks in advance.
PS: I've seen more of these questions (e.g. here and here), but there are no answers given in there.
edit:
how I would like to use it:
typedef struct {
char *name;
int status;
} MyTask;
int main(int argc, char** argv) {
MyTask* taskList = NULL, sharedTaskList = NULL;
int length = 0;
...
readFile(&taskList, &length, ...);
sharedTaskList = share(taskList, length * sizeof(MyTask));
// or maybe even better: without needing to assign it to new variable
for(i = 0; i < NR_WORKERS; i++) {
switch(pid = fork()) {
//etc...
}
}
...
return 0;
}
How to share existing memory?
Don't share existing memory. Get some (small amount of) "fresh" shared memory and use (i.e. fill or read) it later.
Assuming you are on Linux, read shm_overview(7).
I guess that some of your functions might fail. You should test against failure each call, e.g.
int fd = shm_open(SHM_NAME, O_CREAT | O_RDWR | O_EXCL, S_IRUSR | S_IWUSR);
if (fd<0) {perror("shm_open"); exit(EXIT_FAILURE);};
and so on. Perhaps use also strace(1)
is there a way to share memory that has already been allocated
Short answer, no! (or not easily, and not in a portable way). You conventionally do the opposite: obtain some shared segment of known size, and use some pointers into it. (the same shared segment might have different virtual addresses in different processes, e.g. because of ASLR).
You could use mmap(2) with MAP_FIXED on some already used virtual address space subsegment (that would overwrite & replace the mapping with a new one, not share an existing mapping!), but I would suggest to avoid that. Notice that the virtual address space is managed in multiples of pages, so there is no way to share some data which is not page aligned. So your share function is impossible unless both toBeShared and size are page-aligned. You could consider the Linux specific mremap(2)
In other words, your applications should first allocate some shared memory and then put/use some data inside the obtained shared segment, not try to share some existing unshared virtual memory range. So you probably want to code some void* get_my_shared_memory(); (assuming the size is a compile time constant, and you call that function once per process, and its resulting virtual address would often vary from one process to another)
In practice, memory is a finite resource, and shared memory is a scarce and very limited resource. On most systems, you'll be able to share a few dozens of megabytes only... So sharing an arbitrary large amount of memory is unreasonable.
Perhaps your entire application might just use some server, e.g. some database server à la PostGreSQL, to share information, by making requests to that server (and using ACID properties of DBMS). Or you could organize it as a monitoring process exchanging messages (e.g. URL to be processed) -on pipes or sockets or fifos- with slave processes. But we don't know what kind of application are you coding.
BTW, sharing memory is not enough. You need to synchronize your processes.
I have a C program that generates large amounts of data in memory, and I need to share one particular section of this data in memory, so that another process can have read access to it.
I'm attempting to use mmap to do this, but I'm not having much success. Here is my code:
//Code above generates a pointer to the memory section I need to share, named addr
if (infoBlock->memory_size == 1073741824) { //This is the data block I need to share
int err, fd;
fd = open("/tmp/testOutput", (0_RDWR | 0_CREAT), S_IWUSR);
if (fd < 0) {
perror("Couldn't create output file\n");
goto failedExit;
}
unsigned *p = mmap(addr, 1073741824, PROT_READ, (MAP_SHARED | MAP_FIXED), fd, 0);
if (!p) {perror("mmap failed"); goto failedExit; }
printf("p is now: %p\n", p); //This should point to the shared mapping
printf("%u\n", *p); //Try to print out some data from the mapping
}
After running the program, I can see the file /tmp/testOutput is there, but it's size is 0. I'm not sure if that's a normal thing with memory mappings, as it's not technically a file. Also all of the output within my program points to the same memory address.
I can also see the memory map present within the /proc/PID/maps, with a reference to /tmp/testOutput.
Everything seems to run, however when it comes to dereferencing the pointer, the program exits, I'm assuming this is because I've done the mapping wrong, and the pointer is pointing to something it shouldn't be.
If anyone can spot what I'm doing wrong, or can offer some advice, it would be greatly appreciated.
Thanks!
You've mapped the storage associated with that file (or tried to) into your process, and you've insisted that it be mapped at an address you're already using for something else (presumably, addr was allocated somehow).
You don't say whether p actually does have the address you requested, and as suspectus points out, your error checking is broken.
Your Confusion:
You can't associate arbitrary heap or other process memory pages with a file after the fact. You have to allocate them in the filesystem, and then map them. (There is a way to associate them with a UNIX pipe using vmsplice, although it isn't exactly what you asked for).
Note the MMAP_FIXED flag will just replace the page which was occupied by your data, with the new pages associated with the file. Without that flag, the address hint would be ignored and the mapping placed elsewhere.
The Solution:
ftruncate the file to your desired size before mapping it (this allocates storage in the filesystem)
map it and then populate it
fix your mmap error checking
If you can't change your allocation scheme, the best you can manage is to copy your process-local memory into the mapping, in which case you might as well just write it to the file.
The ideal case would look something like this:
void *alloc_mmap(const char *filename, size_t length)
{
int fd;
fd = open(filename, (0_RDWR | 0_CREAT), S_IWUSR);
if (fd < 0) {
perror("Couldn't create output file\n");
return NULL;
}
if (ftruncate(fd, length)) {
perror("Couldn't grow output file\n");
close(fd);
return NULL;
}
void *p = mmap(NULL, length, PROT_READ, MAP_SHARED, fd, 0);
if (p == -1) {
perror("mmap failed");
close(fd);
return NULL;
}
close(fd);
return p;
}
// now you've allocated your memory, you can populate it and it will be
// reflected in the file
Here is an extract from the mmap man page.
On success, mmap() returns a pointer to the mapped area. On error, the value
MAP_FAILED (that is, (void *) -1) is returned, and errno is set appropriately.
On success, munmap() returns 0, on failure -1, and errno is set (probably to
EINVAL).
The test for success should be changed to test for -1 return value of mmap. Then check the errno
value. HTH.