If I create shared memory variables in C, where are they placed (heap / stack / data section / ...)? I am using 64 bit Ubuntu with gcc-4.8 and the compilerflag -m32 for 32bit and this code:
segment_id = shmget (IPC_PRIVATE, shared_segment_size,
IPC_CREAT | IPC_EXCL | S_IRUSR | S_IWUSR);
shared_memory = (char*) shmat (segment_id, 0, 0);
printf ("shared memory attached at address |%10p\n", shared_memory);
I get the following output for this:
shared memory attached at address |0xf76de000
However, valgrind gives this:
shared memory attached at address | 0x4039000
Why? Do shared variables have anything to do with the extern keyword? If another process wants to use the data, can he choose where to attach the memory?
Think of shared memory as process-independent (take that with a grain of salt) memory chunks that are managed by the kernel. When you call shmat(), think of it as a mmap()-like operation that creates a mapping between the address space of the current process and the shared segment. In fact, you can force the address to be a particular value with the second argument to shmat(). But if you do not, it is automatically generated. The actual value will depend on a number of factors, the current memory mapping of the process in particular.
Thus you have one mapping when you run the process standalone, and another when using valgrind. For all intents and purposes you can consider this a heap address - it is not going to be a part of your stack space. But better, just forget the stack and heap distinction and think of it as some memory you got that you can play with as long as you do it by the rules (do not overwrite the boundaries, respect permissions),
Related
I'm working on a project in C that uses shared memory for IPC on a Linux system. However, I'm a little bit confused about memory management in these segments. I'm using the POSIX API for this project.
I understand how to create the shared segments, and that these persist until a reboot if you fail to properly remove them with shm_unlink(). Additionally, I understand how to do the actually mapping & unmapping with mmap and munmap respectively. However, the usage of these operations and how it affects the stored data in these shared segments is confusing me.
Here is what I'm trying to properly understand:
Lets say I create a segment using shm_open() with the O_CREAT flag. This gives me a file descriptor that I've named msfd in the below example. Now I have a struct that I map into that address space with the following:
mystruct* ms = (mystruct*)mmap(NULL, sizeof(mystruct), PROT_READ | PROT_WRITE, MAP_SHARED, msfd, 0);
//set the elements of the struct here using ms->element = X as usual
Part 1)
Here's where my confusion beings. Lets say that this process is now done accessing that location since it was just setting data for another process to read. Do I still call munmap()?
I want the other process to still have access to all of this data that the current process has set. Normally, you wouldn't call free() on a malloc'ed pointer until its use is no longer needed permanently. However, I understand that when this process exits the unmapping happens automatically anyway. Is the data persisted inside the segment, or does that segment just get reserved with it's allotted size and name?
Part 2)
We're now in the process of the other application that needs to access and read from that shared segment. I understand that we now open that segment with shm_open() and then perform the same mapping operation with mmap(). Now we have access to the structure in that segment. When we call munmap() from this process (NOT the one that created the data) it "unlinks" us from that pointer, however the data is still accessible. Does this assume that process 1 (the creator) has NOT called munmap()?
Is the data persisted inside the segment,
Yes.
does that segment just get reserved with it's allotted size and name?
Also yes.
Does this assume that process 1 (the creator) has NOT called munmap()?
No.
The shared memory gets created via shm_create() (as being taken from available OS memory) and from this moment on it carries whichever content had been written into until it is given back to the OS via shm_unlink().
shm_create() and shm_open() act system oriented, in terms of the (shared) memory being a system (not process) specific resource.
mmap() and unmap() act process oriented, that is map and unmap the system resource shared memory into/out-of the process' address space.
I have a call to mmap() which I try to map 64MB using MAP_ANONYMOUS as follows:
void *block = mmap(0, 67108864, PROT_READ, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (block == MAP_FAILED)
exit(1);
I understand that to actually own the memory, I need to hit that block of memory. I want to add some sort of 0's or empty strings to actually own the memory. How would I do that? I tried the following, but that obviously segfaults (I know why it does):
char *temp = block;
for (int i = 0; i < 67108864; i++) {
*temp = '0';
temp++;
}
How would I actually gain ownership of that block by assigning something in that block?
Thanks!
Your process already owns the memory, but what I think you want is to make it resident. That is, you want the kernel to allocate physical memory for the mmaped region.
The kernel allocates a virtual memory area (VMA) for the process, but this just specifies a valid region and doesn't actually allocate physical pages (or frames as they are also sometimes called). To make the kernel allocate entries in the page table, all you need to do is force a page fault.
The easiest way to force a page fault is to touch the memory just like you're doing. Though, because your page size is almost certainly 4096 bytes, you really only need to read one byte every 4096 bytes thereby reducing the amount of work you actually need to do.
Finally, because you are setting the pages PROT_READ, you will actually want to read from each page rather than try to write.
Your question is not very well formulated. I don't understand why you think the process is not owning its memory obtained thru mmap?
Your newly mmap-ed memory zone has only PROT_READ (so you can just read the zeros inside) and you need that to be PROT_READ|PROT_WRITE to be able to write inside.
But your process already "owns" the memory once the mmap returned.
If the process has pid 1234, you could sequentially read (perhaps with cat /proc/1234/maps in a different terminal) its memory map thru /proc/1234/maps; from inside your process, use /proc/self/maps.
Maybe you are interested in memory overcommit; there is a way to disable that.
Perhaps the mincore(2), msync(2), mlock(2) syscalls are interesting you.
Maybe you want the MAP_POPULATE or MAP_LOCKED flag of mmap(2)
I actually don't understand why you say "own the memory" in your question, which I don't understand very well. If you just want to disable memory overcommit, please tell.
And you might also mmap some file segment. I believe there is no possible overcommit in that case. But I would just suggest to disable memory overcommit in your entire system, thru /proc/sys/vm/overcommit_memory.
mmap() can be optionally supplied with a fixed location to place the map. I would like to mmap a file and then have it available to a few different programs at the same virtual address in each program. I don't care what the address is, just as long as they all use the same address. If need be, the address can be chosen by one of them at run time (and communicated with the others via some other means).
Is there an area of memory that Linux guarantees to be unused (by the application and by the kernel) that I can map to? How can I find one address that is available in several running applications?
Not really, no. With address space randomisation on modern linux systems it is very hard to guarantee anything about what addresses may or may not be used.
Also, if you're thinking of using MAP_FIXED then be aware that you need to be very careful as it will cause mmap to unmap anything that may already be mapped at that address which is generally a very bad thing.
I really think you will need to find another solution to your problem...
Two processes can map a shared memory block to the same virtual address using shm_open() and mmap(); that is, the virtual address returned from mmap() can be the same for both processes. I found that Linux will by default give different virtual addresses to different processes for the same piece of shared memory, but that using mmap() with MAP_FIXED will force Linux to supply the same virtual address to multiple processes.
The process that creates the shared memory block must store the virtual address somewhere, either within the shared memory, in a file, or with some other method so that another process can determine the original virtual address. The known virtual address is then used in the mmap() call, along with the MAP_FIXED flag.
I was able to use the shared memory to do this. When doing so, the "golden" virtual address is stored within the shared memory block; I made a structure that contains a number of items, the address being one of them, and initialized it at the beginning of the block.
A process that wants to map that shared memory must execute the mmap() function twice; once to get the "golden" virtual address, then to map the block to that address using the MAP_FIXED flag.
Interestingly, I'm working with an embedded system running a 2.6 kernel. It will, by default, supply the same virtual address to all mmap() calls to a given file descriptor. Go figure.
Bob Wirka
You could look into doing a shared memory object using shmget(), shmat(), etc. First have the process that obtains the right to initialize the shared memory object read in your file and copy it into the shared memory object address space. Now any other process that simply gets a return shared memory ID value can access the data in the shared memory space. So for instance, you could employ some type initialization scheme like the following:
#include <sys/shm.h>
#define KEYVALUE 1000 //arbitrary value ... just needs to be shared between your processes
int file_size
//read your file and obtain its size in bytes;
//try to create the shared memory object
int shared_mem_id;
void* shared_mem_ptr = NULL;
if ((shared_mem_id = shmget(KEYVALUE, file_size, S_IRUSR | S_IWUSR IPC_CREAT | IPC_EXCL)) == -1)
{
if (errno == EEXIST)
{
//shared memory segment was already created, so just get its ID value
shared_mem_id = shmget(KEYVALUE, file_size, S_IRUSR | S_IWUSR);
shared_mem_ptr = shmat(shared_mem_id, NULL, 0)
}
else
{
perror("Unable to create shared memory object");
exit(1);
}
}
else
{
shared_mem_ptr = shmat(shared_mem_id, NULL, 0);
//copy your file into shared memory via the shared_mem_ptr
}
//work with the shared data ...
The last process to use the shared memory object, will, just before destroying it, copy the modified contents from shared memory back into the actual file. You may also want to allocate a structure at the beginning of your shared memory object that can be used for synchronization, i.e., there would be some type of "magic number" that the initializing process will set so that your other processes will know that the data has been properly initialized in the shared memory object before accessing it. Alternatively you could use a named semaphore or System V semaphore to make sure that no process tries to access the shared memory object before it's been initialized.
I want to know the full detail of the address space layout of a multithreaded Linux Process for both 64 bit and 32 bit. Link to any article that describes it will be appreciated. And note that I need to know full details, not just an overview, because I will be directly dealing with it. So I need to know for example, where are the thread stacks located, the heap, thread private data etc...
Thread stacks are allocated with mmap at thread start (or even before - you can set the stack space in pthread_attrs). TLS data is stored in the beginning of thread's stack. Size of thread's stacks is fixed, typically it is from 2 to 8 MB. Stack size of each thread can't be changed while the thread is live. (First thread - running main - is still uses main stack at the end of address space and this stack may grow and shrink.) Heap and code is shared between all threads. Mutexes can be anywhere in data section - it is just a struct.
The mmap of thread's stack is not fixed at any address:
Glibc sources
mem = mmap (NULL, size, prot,
MAP_PRIVATE | MAP_ANONYMOUS | MAP_STACK, -1, 0);
PS modern GCC allows threads stack to be unlimited with SplitStacks feature
For a uni assignment I need to create a circular list of up to 10 file names, and then store these in a shared memory area, so that 2 child processes can read/write to the list (using a semaphore to control access). Trouble is, that I am a total C novice and I feel loss and despair because its totally out of my depth. I need some help in "filling in the holes" of my knowledge.
Right now, I am simply focussing on it one problem at a time, and presently, I am only trying to get my circular list into the shared memory area.
So far I have:
typedef struct FILE
{
struct FILE *f_link; /* forward link for linked list */
char name[255]; /* name of the file */
} FILE_entry;
As my struct which will hold the reference to the file that comes next (f_link). this way I can just call ->f_link to get the next item in the list, and where the 10th element will simply have its f_link directed back to the 1st. My reason for doing this is so that I can simply traverse the list without an iterator (and never have to check for the end of the list as I would with an array).
I also know that I need to use shmget to get the memory area, and I understand it, I pass shmget a key, a size and a flag (which i dont get) and it returns an identifier in the type of an int.
So my question is 2 fold. How do I store my linked list INTO the shared memory area - and how do I access it FROM the shared memory area?
shmget just reserves some amount of shared memory - like creating a fixed-size file on disk. The flags are a permission mask (like the mode parameter for open) in the low 9 bits plus some extra flags, IPC_CREAT and IPC_EXCL, corresponding to O_CREAT and O_EXCL for open. To actually access that memory, you need to map it into the address space of your process ("attach" it - analogous to mmap for files). This is done using shmat (which returns a pointer). You then need to allocate your FILE structures from that pointer. The whole process looks something like this:
int id;
FILE_entry *entries;
id = shmget(key, N * sizeof(FILE_entry), IPC_CREAT | 0644);
entries = (FILE_entry *) shmat(id, NULL, 0);
// you can now access entries as if it was a N-element array.
// to turn it into a circular list, link the entries appropriately.
After mapping, you can work with it like regular memory - since it is regular memory. That's the whole point!
EDIT: One important caveat I forgot to mention. Putting a linked list into a shared memory segment like this will only work if all involved processes map it to the same address! So you either need to do that (using the second argument of shmat) or switch from pointers to offsets relative to the base address of the shared memory range. That means turning your next field from a pointer to a ptrdiff_t and adding the base address of the mapped memory range whenever you load it (and subtracting it whenever you store it).
You call shmat(identifier, NULL, 0), getting back a pointer to the location the shared memory has been mapped to in your process. Either create your object(s) at that location, or memcpy/memmove them to that location.
You'll probably find Beej's Unix IPC guide useful, it contains a section specifically on shared memory.
After you get the key from shmget, you have to use it to get the actual pointer via shmat. Here is some sample:
int shmid;
key_t key;
FILE* shm;
shmid = shmget(key, sizeof(FILE) * 10, IPC_CREAT | 0666);
shm = shmat(shmid, NULL, 0);
This way, you will be able to access the shared memory using the shm pointer. In this links you can see the manpage for shmget and shmat:
http://linux.die.net/man/2/shmget
http://linux.die.net/man/2/shmat
And see http://www.cplusplus.com for additional reference. It contains C reference also.
I don't know if I was clear enough, so, give some comments and I'll try to enlighten it for you, if necessary.
EDIT: In this site you can find a really simple example: http://simplestcodings.blogspot.com/2010/08/ipc-shared-memory-implementation-in-c.html
For the flag you need to specify at least IPC_CREATE if you want to allocate a new shared memory segment, otherwise, it will look of ran existing segemnt and fail if it does not find one.
Secondly as the shared memory segment is a contiguous block of memory you need to be able to store all 10 of your FILE_entry structures together (or allocate 10 shared memory segments -- yuck!).
So you really need to shmeget enough memory for an array of at least 10 FILE structures.
Lastly FILE and FILE_entry are really bad names! Use something less generic like MY_FILE_REF and MyFileRefEntry.