The following is code I've used to create a memory mapped file:
fid = open(filename, O_CREAT | O_RDWR, 0660);
if ( 0 > fid )
{
throw error;
}
/* mapped offset pointer to data file */
offset_table_p = (ubyte_2 *) shmat(fid, 0, SHM_MAP);
/* Initialize table */
memset(offset_table_p, 0x00, (table_size + 1) * 2);
say table_size is around 2XXXXXXXX bytes.
During debug, I've noticed it fails while attempt to initializing the 'offset table pointer',
Can anyone provide me some inputs on why it's failing during intilalization?
is there any possibilities that my memory map file was not created with required table size?
As far as I can tell from reading documentation, you are doing it completely wrong.
Either use open() and mmap() or use shmget() and shmat().
If you use open() you will need to make the file long enough first. Use ftruncate() for that.
First things first:
Examine the file both before and after the open() call. If on Linux, you can use the code:
char paxbuff[1000]; // at start of function
sprintf (paxbuff,"ls -al %s",filename);
system (paxbuff);
fid = open(filename, O_CREAT | O_RDWR, 0660); // this line already exists.
system (paxbuff);
Then, after you call shmat(), check the return values and size thus:
offset_table_p = (ubyte_2 *) shmat(fid, 0, SHM_MAP); // already exists.
printf ("ret = %p, errno = %d\n",offset_table_p,errno);
printf ("sz = %d\n",table_size);
That should be enough to work out the problem.
Related
noob alert with C here.
I have a struct as such
typedef struct {
char* name
} info;
And storing an array (size 10) of this struct (in another struct called table) in a shared memory object using the shm_open call:
int fd = shm_open("someName", O_CREAT | O_EXCL | O_RDWR, S_IRWXU);
if (fd < 0) {
fd = shm_open("someName", O_CREAT | O_RDWR, S_IRWXU);
if (fd < 0) {
printf("ERROR: Could not open shared memory space\n");
return -1;
}
}
(*tables) = mmap(NULL, sizeof(table), PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
ftruncate(fd, sizeof(table));
close(fd);
However, the issue that I have is that later through the code such a scenario happens:
If process B runs this and puts some text, say "foo" in the name field of an element of the table array, process A does not have access to that char*.
All it sees is a memory address, but not the actual content of the char* as opposed to if it wrote the content itself. However, I would also like to note that if I replace char* with a fixed size char[], say char name[20] instead of char *name, then this issue does not occur.
I would like to know whether there is any way around this, and if not, why is it so?
Thank you!
When using shared memory for IPC, all of the data to be shared, must be located in shared memory. It's that simple, there's no way around it. What you can do however for some larger, more variable chunks of data, is simply allocate a dedicated shared memory chunk for that data, and provide its name via the master table. Another option in your case is to arrange for the shared memory to be sufficiently larger than your info struct, such that name is just an offset from that address, to where the name data resides. Then address of data is `&name + *name'.
When I first made this project last semester, the code worked fine. Now I get a bus error when the mmapped memory to share between processes is being written to and I'm not sure why it is not working anymore.
Account_Info *mapData()
{
int fd;
//open/create file with read and write permission and check return value
if ((fd = open("accounts", O_RDWR|O_CREAT, 0644)) == -1)
{
perror("Unable to open account list file.");
exit(0);
}
//map data to be shared with different processes
Account_Info *accounts = mmap((void*)0, (size_t) 100*(sizeof(Account_Info)), PROT_WRITE,
MAP_SHARED, fd, 0);
int count= 0;
//loop to initialize values of Account_Info struct
while (count != 20)
{
//bus error occurs here
accounts[count].CurrBalance= 0;
accounts[count].flag = 0;
int i = 0;
while (i != 100)
{
//place NULL terminator into each element of AccName
accounts[count].AccName[i]= '\0';
i++;
}
count++;
}
close(fd);
return accounts;
}
A documented cause for SIGBUS with mmap is
Attempted access to a portion of the buffer that does not correspond to the file (for example, beyond the end of the file, including the case where another process has truncated the file).
My guess is that the accounts file didn't exist, so open with O_CREAT created it. But it has zero size, so any attempt to read or write through the mapping will fault. You need to fill the file with enough zeroes (or something else) to cover the mapping, for example using ftruncate.
You will get SIGBUS if you attempt to write past the mapped region of the file.
Chances are pretty good that your backing store file accounts is truncated/too short. (e.g.) if the file has space for 10 struct entries and you write to the 11th, you'll get SIGBUS
Do an fstat to get st_size and compare this against the length parameter you're giving to mmap
You may want to consider using ftruncate to extend the file before doing mmap
I have a process that dived itself with fork. I need to create a region of memory (a matrix) for the result of the computation of each process. How can I do this? Everything I tried or I can use but it's not shared between processes or I can't use (not sure if shared or not). Someone knows what I can use? It can be something simple and without any security. The simpler the better.
I tried shmget but it's not sharing and I couldn't get how to use mmap to allocate or use it correctly. I tried other estranges things, but nothing. Any tips?
Some tries:
segment_id = shmget(IPC_PRIVATE, (sizeof(int) * linhas_mat1 * colunas_mat2) , S_IRUSR|S_IWUSR);
matriz_result = (int **) shmat(segment_id, NULL, 0);
Forks after that. Each process can use the matriz_result normally as a matrix, but the memory is not shared. Each one has one like a local variable.
segment_id = shm_open("/myregion", O_CREAT | O_RDWR, S_IRUSR | S_IWUSR);
matriz_result = mmap(NULL, (sizeof(int) * linhas_mat1 * colunas_mat2), PROT_READ | PROT_WRITE, MAP_SHARED, segment_id, 0);
Tried this with mmap, but I don't know if it's right. I'm not good with such low level programming and I couldn't find any good example on how to use it correctly.
declarations:
int segment_id is;
int **matriz_result;
int createMemShare(){
//File descriptor declaration:
int fd;
//We want to open the file with readwrite,create it, and empty it if it exists
//We want the user to have permission to read and write from it
fd = open(MEMSHARENAME, O_RDWR| O_CREAT | O_TRUNC, S_IRUSR| S_IWUSR );
if(fd <= 0){
puts("Failed in creating memory share .");
return -1;
}
//Move the file pointer and write an empty byte, this forces the file to
//be of the size we want it to be.
if (lseek(fd, MEMSHARESIZE - 1, SEEK_SET) == -1) {
puts("Failed to expand the memory share to the correct size.");
return -1;
}
//Write out 1 byte as said in previous comment
write(fd, "", 1);
//Memory share is now set to use, send it back.
return fd;
}
//Later on...
int memShareFD = mmap(NULL, MEMSHARESIZE, PROT_READ, MAP_SHARED, fd, 0);
//And to sync up data between the processes using it:
//The 0 will invalidate all memory so everything will be checked
msync(memshareFD,0,MS_SYNC|MS_INVALIDATE);
you can try the above function to create a shared memory space. Essentially all you need to do is treat it like any other file once you've made it. The code example on the man page is pretty complete and worth a look into: check it out here
Edit:
You'd probably be better off using shm_open as Jens Gustedt suggested in the comments. It's simple to use and simpler than making the file yourself with the function I've written above.
I have a C program that generates large amounts of data in memory, and I need to share one particular section of this data in memory, so that another process can have read access to it.
I'm attempting to use mmap to do this, but I'm not having much success. Here is my code:
//Code above generates a pointer to the memory section I need to share, named addr
if (infoBlock->memory_size == 1073741824) { //This is the data block I need to share
int err, fd;
fd = open("/tmp/testOutput", (0_RDWR | 0_CREAT), S_IWUSR);
if (fd < 0) {
perror("Couldn't create output file\n");
goto failedExit;
}
unsigned *p = mmap(addr, 1073741824, PROT_READ, (MAP_SHARED | MAP_FIXED), fd, 0);
if (!p) {perror("mmap failed"); goto failedExit; }
printf("p is now: %p\n", p); //This should point to the shared mapping
printf("%u\n", *p); //Try to print out some data from the mapping
}
After running the program, I can see the file /tmp/testOutput is there, but it's size is 0. I'm not sure if that's a normal thing with memory mappings, as it's not technically a file. Also all of the output within my program points to the same memory address.
I can also see the memory map present within the /proc/PID/maps, with a reference to /tmp/testOutput.
Everything seems to run, however when it comes to dereferencing the pointer, the program exits, I'm assuming this is because I've done the mapping wrong, and the pointer is pointing to something it shouldn't be.
If anyone can spot what I'm doing wrong, or can offer some advice, it would be greatly appreciated.
Thanks!
You've mapped the storage associated with that file (or tried to) into your process, and you've insisted that it be mapped at an address you're already using for something else (presumably, addr was allocated somehow).
You don't say whether p actually does have the address you requested, and as suspectus points out, your error checking is broken.
Your Confusion:
You can't associate arbitrary heap or other process memory pages with a file after the fact. You have to allocate them in the filesystem, and then map them. (There is a way to associate them with a UNIX pipe using vmsplice, although it isn't exactly what you asked for).
Note the MMAP_FIXED flag will just replace the page which was occupied by your data, with the new pages associated with the file. Without that flag, the address hint would be ignored and the mapping placed elsewhere.
The Solution:
ftruncate the file to your desired size before mapping it (this allocates storage in the filesystem)
map it and then populate it
fix your mmap error checking
If you can't change your allocation scheme, the best you can manage is to copy your process-local memory into the mapping, in which case you might as well just write it to the file.
The ideal case would look something like this:
void *alloc_mmap(const char *filename, size_t length)
{
int fd;
fd = open(filename, (0_RDWR | 0_CREAT), S_IWUSR);
if (fd < 0) {
perror("Couldn't create output file\n");
return NULL;
}
if (ftruncate(fd, length)) {
perror("Couldn't grow output file\n");
close(fd);
return NULL;
}
void *p = mmap(NULL, length, PROT_READ, MAP_SHARED, fd, 0);
if (p == -1) {
perror("mmap failed");
close(fd);
return NULL;
}
close(fd);
return p;
}
// now you've allocated your memory, you can populate it and it will be
// reflected in the file
Here is an extract from the mmap man page.
On success, mmap() returns a pointer to the mapped area. On error, the value
MAP_FAILED (that is, (void *) -1) is returned, and errno is set appropriately.
On success, munmap() returns 0, on failure -1, and errno is set (probably to
EINVAL).
The test for success should be changed to test for -1 return value of mmap. Then check the errno
value. HTH.
While trying to resolve some debugging issues , I added some printf-s to my code :
I used that code :
struct PipeShm
{
int init;
sem_t sema;
...
...
}
struct PipeShm * sharedPipe = NULL;
func2:
int func2()
{
if (!sharedPipe)
{
int myFd = shm_open ("/myregion", O_CREAT | O_TRUNC | O_RDWR, 0666);
if (myFd == -1)
error_out ("shm_open");
// allocate some memory in the region in the size of the struct
int retAlloc = ftruncate (myFd, sizeof * sharedPipe);
if (retAlloc < 0) // check if allocation failed
error_out("ftruncate");
// map the region and shared in with all the processes
sharedPipe = mmap (NULL, sizeof * sharedPipe,PROT_READ | PROT_WRITE,MAP_SHARED , myFd, 0);
if (sharedPipe == MAP_FAILED) // check if the allocation failed
error_out("mmap");
// put initial value
int value = -10;
// get the value of the semaphore
sem_getvalue(&sharedPipe->semaphore, &value);
if (sharedPipe->init != TRUE) // get in here only if init is NOT TRUE !
{
if (!sem_init (&sharedPipe->semaphore, 1, 1)) // initialize the semaphore to 0
{
sharedPipe->init = TRUE;
sharedPipe->flag = FALSE;
sharedPipe->ptr1 = NULL;
sharedPipe->ptr2 = NULL;
sharedPipe->status1 = -10;
sharedPipe->status2 = -10;
sharedPipe->semaphoreFlag = FALSE;
sharedPipe->currentPipeIndex = 0;
printf("\nI'm inside the critical section! my init is: %d\n" , sharedPipe->init);
}
else
perror ("shm_pipe_init");
printf("\nI'm out the critical section! my init is: %d\n" , sharedPipe->init);
}
}
return 1; // always successful
}
With that main :
int main()
{
int spd, pid, rb;
char buff[4096];
fork();
func2();
return 0;
}
And got this :
shm_pipe_mkfifo: File exists
I'm inside the critical section! my init is: 1
I'm out the critical section! my init is: 1
Output:hello world!
I'm inside the critical section! my init is: 1
I'm out the critical section! my init is: 1
It seems that the shared memory is not so shared , why ?
The segment is shared between all processes due to MAP_SHARED | MAP_ANONYMOUS , so why both processes have the same before and after values ?
It seems that each process has its own semaphore even though it was shared between them , so what went wrong ?
Thanks
Since you use the MAP_ANONYMOUS flag to mmap, the myFd argument is ignored, and you create two independent shared memory chunks, one in each process, which have no relation to each other.
MAP_ANONYMOUS
The mapping is not backed by any file; its contents are initial‐
ized to zero. The fd and offset arguments are ignored; however,
some implementations require fd to be -1 if MAP_ANONYMOUS (or
MAP_ANON) is specified, and portable applications should ensure
this. The use of MAP_ANONYMOUS in conjunction with MAP_SHARED
is only supported on Linux since kernel 2.4.
If you get rid of MAP_ANONYMOUS you'll then only have one shared memory chunk, but you then have the problem of not calling sem_init. On Linux with NPTL it will actually work, as clearing a sem_t to all 0 bytes (the initial state here) is equivalent to sem_init(&sema, anything, 0); (NPTL ignores the pshared flag), but that's not portable to other systems.
Per Karoly's comment on another answer, there's also a race condition due O_TRUNC on the open call. If the second thread calls open after the first thread has already started modifying the semaphore, that TRUNC will clobber the semaphore state. Probably the best solution is to move the code creating, opening, and mmaping the shared memory to a different function that is called BEFORE calling fork.
edit
To fix the O_TRUNC problem, you can't have more than one process calling shm_open with O_TRUNC. But if you just get rid of the O_TRUNC, then you have the startup problem that if the shared memory object already exists (from a previous run of the program), it may not be in a predictable state. On possibility is to split off the beginning of func2:
main() {
func1();
fork();
func2();
}
func1() {
int myFd = shm_open ("/myregion", O_CREAT | O_TRUNC | O_RDWR, 0666);
if (myFd == -1)
error_out ("shm_open");
// allocate some memory in the region in the size of the struct
int retAlloc = ftruncate (myFd, sizeof *sharedPipe);
if (retAlloc < 0) // check if allocation failed
error_out("ftruncate");
// map the region and shared in with all the processes
sharedPipe = mmap (NULL, sizeof *sharedPipe, PROT_READ|PROT_WRITE, MAP_SHARED, myFd, 0);
if (sharedPipe == MAP_FAILED) // check if the allocation failed
error_out("mmap");
}
func2() {
// put initial value
int value = -10;
// get the value of the semaphore
sem_getvalue(&sharedPipe->semaphore, &value);
:
Alternately you could keep the same code (just get rid of O_TRUNC) and add a cleanup before the fork:
main() {
shm_unlink("/myregion");
fork();
func2();
In all cases you'll still have problem if you run multiple copies of your program at the same time.
A few thoughts...
I think this is a fundemental misunderstanding of how POSIX semaphores work. I don't see a call to sem_init or sem_open. You shouldn't be able to use them across processes without doing so much more explicitly than you've done.
I'm not so fresh on the implementation of mmap on Linux and how MAP_ANONYMOUS might affect this, but in general writes to mapped regions can't really be instantaneous. The manpage on linux.die says:
MAP_SHARED
Share this mapping. Updates to the mapping are visible to other processes that map this file, and are carried through to the underlying file. The file may not actually be updated until msync(2) or munmap() is called.
The reason for this is that your memory access gets trapped in a page fault and at that point the kernel will fill contents from the file descriptor, then let you do your write in RAM, then at some later point the kernel will flush back to the file descriptor.