I'm having trouble accessing a global struct pointer that I'm initalizing with mmap. Attempting to access members of the struct in functions outside of the one it is declared in throw segfaults.
the struct:
typedef struct foo {
uint32_t size;
bar_t array[0];
} foo_t;
the initialization:
foo_t* foo; // global
static void* init_function(...) {
fd = open(filename, O_CREAT | O_WRONLY, 0644);
write(...);
lseek(...);
write(...);
foo = mmap(0, BIG_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
close(fd);
foo->size = 0;
}
what causes the segfault:
static int another_function(...) {
if (foo->size == 0) {...} //foo->size causes it
}
BIG_SIZE is a defined value that should be adequately large for my needs.
Any ways, I apologize for the (...)s, but this is where the issues are showing up. I've looked into mmap docs and variable length methods with no luck. Thanks in advance!
You don't show us everything, but it seems that you just assign your global variable, but that you don't initialize the contents of the mapped region. You should at least give it a
foo->size = 0 somewhere, if the file is newly created, or alternatively you should use ftruncate instead of your fseek write sequence to warrant that the blocks (including size) are 0 filled.
Firstly check if mmap really succeeded:
foo = mmap(...);
if ( MAP_FAILED == foo )
{
//no memory for me...
}
And if I remember correctly, the file must be at least BIG_SIZE long, otherwise mmap will fail - but I may be wrong on that part...
Related
I have 2 or more processes accessing to a shared memory.
I want to create a global variable in each process and then mapping the address of this variable to the shared memory using mmap API with MAP_FIXED flag.
So that, when reading/ writing to this shared memory, I just need to access to the global variable ( the same way as we share global variable between threads, but here I would like to shared global variable between processes).
I define the global variable in each process as below:
typedef struct // This struct define the shared memory area
{
int data1;
int data2;
// ...
} SharedMemory;
// the following attribute (supported by GCC) make the start address of the variable aligned to 4KB (PAGE_SIZE)
__attribute__((aligned(4096))) SharedMemory gstSharedMemory; // shared global variable
int giOtherVar = 10; // Another normal global variable
Then using mmap to map the shared memory to this global variable:
void* lpShmAddr = mmap(&gstSharedMemory,
sizeof(gstSharedMemory),
PROT_READ | PROT_WRITE,
MAP_SHARED | MAP_FIXED,
iFd, // File descriptor to the shared memory
0);
However, if sizeof(gstSharedMemory) is not multiple of PAGE_SIZE (4kb), and because the OS will always round up the map size to multiple of page size, all the bytes in the rounded-up region are initialized to 0.
And it may cause the data of other global variable (for example: giOtherVar) to become Zero if their address is within this rounded-up region.
To overcome this situation, I use a byte array to backup the rounded-up region and recover it as below:
unsigned char byBkupShm[PAGE_SIZE] = { 0 } ;
memcpy(&gbyBkupShm[0],
((unsigned char*)&gstSharedMemory+ sizeof(gstSharedMemory)),
PAGE_SIZE - (sizeof(gstSharedMemory)% PAGE_SIZE));
void* lpShmAddr = mmap(&gstSharedMemory,
sizeof(gstSharedMemory),
PROT_READ | PROT_WRITE,
MAP_SHARED | MAP_FIXED,
iFd, // File descriptor to the shared memory
0);
memcpy( ((unsigned char*)&gstSharedMemory+ sizeof(gstSharedMemory)),
&byBkupShm[0],
PAGE_SIZE - (sizeof(gstSharedMemory)% PAGE_SIZE));
And finally, I access to shared memory like this:
// Write to shared memory:
gstSharedMemory.data1 = 5;
// Read from shared memory;
printf("%d", gstSharedMemory.data1);
My question is: Is there any potential problem with this implementation?
Editted:
Thank to #None and his idea, I define a macro as below to make my struct aligned and rounded up to PAGE_SIZE, but at the same time, still provide the actual size of the struct if I need:
#define PAGE_SIZE (4 * 1024) // Page Size: 4KB
#define SHM_REG __attribute__((aligned(PAGE_SIZE))) // Aligned to 4KB boundary
#define DEFINE_SHM( structName_, shmSizeVar_, structContent_) \
typedef struct SHM_REG structContent_ structName_; \
int shmSizeVar_ = sizeof(struct structContent_);
// Using
DEFINE_SHM(
MySharedMemory, // Struct Name of shared memory
giSizeOfMySharedMemory, // Global Variable
{
int a;
int b;
char c;
}
);
printf("Rounded Size: %d\n", sizeof(MySharedMemory)); // = 4096
printf("Acutal Size: %d\n", giSizeOfMySharedMemory); // = 12
Make sure the shared memory structure is both aligned to and sized as a multiple of page size:
#include <stdlib.h>
#ifndef PAGE_SIZE
#define PAGE_SIZE 4096
#endif
typedef struct __attribute__((aligned (PAGE_SIZE))) {
/*
* All shared memory members
*/
} SharedMemory;
At run time, before mapping the shared memory, verify it first:
SharedMemory blob;
if (PAGE_SIZE != sysconf(_SC_PAGESIZE)) {
ABORT("program compiled for a different page size");
} else
if (sizeof blob % PAGE_SIZE) {
ABORT("blob is not sized properly");
} else
if ((uintptr_t)(&blob) % PAGE_SIZE) {
ABORT("blob is not aligned properly");
} else
if (MAP_FAILED == mmap(...)) {
ABORT("could not map shared memory over blob");
}
While this is a hack, at least this way it is safe, in Linux.
Yes, the potential problem is that giOtherVar is now shared as well, because the entire page is shared.
The normal way to do this is to not use MAP_FIXED, and let mmap choose a location for you, and store the pointer. You say you cannot do this because it would be a massive code change.
There's probably a way to use a linker script to force gstSharedMemory to be on a page by itself, but linker scripts are tricky.
You could add 4096 bytes of padding to SharedMemory so it's always bigger than 4096 bytes, and then not share the last page (which could overlap other global variables).
You could add an unused 4096 byte array after gstSharedMemory and hope that the compiler will put it right after gstSharedMemory.
You could make the next variable also 4096-byte aligned and hope the compiler doesn't decide to put other variables in the gap.
... or you could just use the pointer design, then #define gstSharedMemory (*gpstSharedMemory) so that you don't have to change all your code.
#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <sys/types.h>
#include <unistd.h>
typedef struct shared_data_s {
int a;
int b;
} __attribute__((aligned(4096))) shared_data_t;
shared_data_t g_data;
shared_data_t *g_data_ptr;
void child()
{
printf("child set g_data to 1/2\n");
g_data.a = 1;
g_data.b = 2;
sleep(2);
printf("child visit g_data a=%d, b=%d\n", g_data.a, g_data.b);
}
void parent()
{
sleep(1);
printf("parent visit g_data a=%d, b=%d\n", g_data.a, g_data.b);
g_data.a = 10;
g_data.b = 20;
sleep(3);
}
int main(int argc, char *argv[])
{
g_data_ptr = mmap(&g_data, sizeof(g_data), PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS | MAP_FIXED, -1, 0);
printf("%s: size=%d, &g_data=%p, g_data_ptr=%p\n", __func__, sizeof(g_data), &g_data, g_data_ptr);
pid_t pid;
pid = fork();
if (pid == 0) {
child();
} else {
parent();
}
return 0;
}
I have unnamed semaphores that I am placing in shared memory in one process following the method that I found here on SO
In P0:
/* addr is a pointer to the base of the shared memory area */
sem_t *sem = (sem_t*) addr;
void *usableSharedMemory = (char*) addr + sizeof(sem_t)
sem_init(sem, 1, 0);
In P1:
if ((addr = mmap(NULL, SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0)) == MAP_FAILED) {
exit(EXIT_FAILURE);
}
sem_t *my_sem = (sem_t*) addr;
...
sem_post(my_sem);
...
sem_wait(my_sem);
If I call sem_getvalue(my_sem) before and after a post or wait, the value of the semaphore does not change. I have prints like this to debug:
int v = 0;
v = sem_getvalue(rsem, &v);
printf("BEFORE-POST:%d\n", v);
sem_post(rsem);
v = sem_getvalue(rsem, &v);
printf("AFTER-POST:%d\n", v);
Before and after the call to sem_post, the semaphore value is zero (0).
I haven't used this kind of semaphore before, but I see a number of things one could trip over.
I'm not excited about the funky pointer math that the other SO post was suggesting, and I can't tell from here whether both processes are actually talking to the same chunk of shared memory.
A good idea in this case is to avoid the pointer math and use a struct to overlay the shared memory segment so you have a clear organization, and add a magic number so everybody can tell if they're getting a valid segment or some random bogus memory:
#define MYSHM_MAGIC 12345987 // can be anything random-ish
struct mysharedmem {
int magicvalue;
sem_t MySemaphore;
void *UsableMemory;
};
This structure overlays whatever your shared memory segment is and allows you to use consistent and more readable access methods, especially if you add additional variables that all sides agree on. The code that creates the segment should initialize it:
// in P1 ?
struct mysharedmem *myshm = mmap(NULL, SIZE, ...);
myshm->magic = MYSHM_MAGIC;
sem_init(&myshm->MySemaphore, 1, 0);
...
and then in the other process, once you obtain the shared address, actually ask the segment "did you come from the place I think?" by checking the magic number. The value of the number doesn't matter, only that both sides agree.
// In P0 ?
struct mysharedmem *myshm = addr; // obtained somehow
if (myshm->magic != MYSHM_MAGIC)
{
error - we don't see the special magic number
}
.. do stuff
How does P0 obtain a handle to the shared memory segment created by the P1?
As part of my C application for Linux, I wrote a small library to allow different processes to communicate through shared memory areas.
I am having lots of alerts for potential out-of-bounds pointers and potentially uninitialized variables contained in the memory areas I create this way.
Consider the function I wrote to create a shared memory area:
int LayOutShm (const char *shm_key, size_t memory_size, void ** new_memory)
{
int fd_value = 0;
void * pshm = NULL;
int return_value = MEM_MGMT_SHM_ERROR;
/* Force new_buffer pointer to NULL just like we initialize return_value to MEM_MGMT_SHM_ERROR */
*new_memory = NULL;
/* Shared object is created with O_EXCL to prevent two namesake areas from being created*/
fd_value = shm_open(shm_key, O_CREAT | O_RDWR | O_EXCL , S_IRWXU);
if(fd_value > 0){
if(ftruncate(fd_value, memory_size) == 0){
/* Here is where we get the pointer to the created memory area */
pshm = mmap(NULL, memory_size, PROT_READ | PROT_WRITE, MAP_SHARED, fd_value, 0);
if(pshm != MAP_FAILED){
return_value = MEM_MGMT_SHM_OK;
memset(pshm,0,memory_size); /* Initialize the memory area */
*new_memory = pshm; /* Pass the pointer back to the caller */
}
}
}
return return_value;
}/*LayOutShm*/
Now, this is some piece of code were I get warnings:
#define SHM_KEY "my_shm_key"
typedef struct{
pthread_mutex_t shm_mutex;
int shm_var1;
char shm_var2;
union{
int shm_union_var1;
char shm_union_var2;
}shm_union
}t_shm_area;
static t_shm_area * shm_area = NULL;
static int ShmInitialization(void)
{
int return_value = -1;
(void)LayOutShm(SHM_KEY, sizeof(t_shm_area), (void**)&shm_area);
/*Check for errors in shared memory creation*/
if (shm_area == NULL){
syslog(LOG_ERR | LOG_USER, "Error laying out shared memory segment\n");
}
else{
shm_area->var1 = 0; /* This assignment gets flagged with a potential out-of-bounds runtime error */
shm_area->shm_union.shm_union_var2 = 0; /* This assignment gets flagged with a potential out-of-bounds runtime error */
/*Create empty attributes structure*/
pthread_mutexattr_t mutex_attributes;
/*Initialize attributes structures with default values*/
(void)pthread_mutexattr_init(&mutex_attributes);
/*Set attributes structure with shared memory value*/
(void)pthread_mutexattr_setpshared(&mutex_attributes, PTHREAD_PROCESS_SHARED);
/*settype mutex PTHREAD_MUTEX_ERRORCHECK*/
(void)pthread_mutexattr_settype(&mutex_attributes, PTHREAD_MUTEX_ERRORCHECK);
/*Initialize the mutex with all the previous attributes*/
(void)pthread_mutex_init(&shm_area->shm_mutex, &mutex_attributes);
return_value = 0;
}
return return_value;
}/*ShmInitialization*/
This is the first time I try my hand at double pointers, so I wouldn't be surprised if I were screwing things up with either that void ** in the declaration or the way I cast the input double pointer to void.
A previous implementation of the function returned the pointer directly and didn't generate these problems, but I was asked to change it so that we can return a status code (even if we don't use them for now).
After that area is created, any locally declared variable I pass to the library to get values from the shared memory gets "tainted" with warnings from potential "out-of-bounds" runtime errors or "variable might not be initialized" warnings. The library has already been tested with several different types of data structures (some of them as big as 15kB) and data integrity and performance are satisfying.
Any idea why am I getting these warnings?
Thank you very much, best regards!
The generic pointer type in C is void*. There is however no generic pointer-to-pointer type void**. So the cast (void**)&shm_area is a cast between non-compatible pointer types. Technically this is undefined behavior (strict aliasing violation), so anything can happen.
To fix this, use a temporary void* for the parameter passing:
void* vptr = shm_area;
LayOutShm(...&vptr);
shm_area = vptr;
noob alert with C here.
I have a struct as such
typedef struct {
char* name
} info;
And storing an array (size 10) of this struct (in another struct called table) in a shared memory object using the shm_open call:
int fd = shm_open("someName", O_CREAT | O_EXCL | O_RDWR, S_IRWXU);
if (fd < 0) {
fd = shm_open("someName", O_CREAT | O_RDWR, S_IRWXU);
if (fd < 0) {
printf("ERROR: Could not open shared memory space\n");
return -1;
}
}
(*tables) = mmap(NULL, sizeof(table), PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
ftruncate(fd, sizeof(table));
close(fd);
However, the issue that I have is that later through the code such a scenario happens:
If process B runs this and puts some text, say "foo" in the name field of an element of the table array, process A does not have access to that char*.
All it sees is a memory address, but not the actual content of the char* as opposed to if it wrote the content itself. However, I would also like to note that if I replace char* with a fixed size char[], say char name[20] instead of char *name, then this issue does not occur.
I would like to know whether there is any way around this, and if not, why is it so?
Thank you!
When using shared memory for IPC, all of the data to be shared, must be located in shared memory. It's that simple, there's no way around it. What you can do however for some larger, more variable chunks of data, is simply allocate a dedicated shared memory chunk for that data, and provide its name via the master table. Another option in your case is to arrange for the shared memory to be sufficiently larger than your info struct, such that name is just an offset from that address, to where the name data resides. Then address of data is `&name + *name'.
I am trying to complete an exercise that is done with system calls and need to allocate memory for a struct *. My code is:
myStruct * entry = (myStruct *)mmap(0, SIZEOF(myStruct), PROT_READ|PROT_WRITE,
MAP_ANONYMOUS, -1, 0);
To clarify, I cannot use malloc() but can use mmap(). I was having no issues with this on Windows in Netbeans, now however I'm compiling and running from command line on Ubuntu I am getting "Segmentation Fault" each time I try to access it.
Is there a reason why it will work on one and not the other, and is mmap() a valid way of allocating memory in this fashion? My worry was I was going to be allocating big chunks of memory for each mmap() call initially, now I just cannot get it to run.
Additionally, the error returned my mmap is 22 - Invalid Argument (I did some troubleshooting while writing the question so the error check isn't in the above code). Address is 0, the custom SIZEOF() function works in other mmap arguments, I am using MAP_ANONYMOUS so the fd and offsetparameters must -1 and 0 respectively.
Is there something wrong with the PROT_READ|PROT_WRITE sections?
You need to specify MAP_PRIVATE in your flags.
myStruct * entry = (myStruct *)mmap(0, SIZEOF(myStruct),
PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
From the manual page:
The flags argument determines whether updates to the mapping are
visible to other processes mapping the same region, and whether
updates are carried through to the underlying file. This behavior is
determined by including exactly one of the following values in flags:
You need exactly one of the flags MAP_PRIVATE or MAP_SHARED - but you didn't give either of them.
A complete example:
#include <sys/mman.h>
#include <stdio.h>
typedef struct
{
int a;
int b;
} myStruct;
int main()
{
myStruct * entry = (myStruct *)mmap(0, sizeof(myStruct),
PROT_READ|PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (entry == MAP_FAILED) {
printf("Map failed.\n");
}
else {
entry->a = 4;
printf("Success: entry=%p, entry->a = %d\n", entry, entry->a);
}
return 0;
}
(The above, without MAP_PRIVATE of course, is a good example of what you might have provided as a an MCVE. This makes it much easier for others to help you, since they can see exactly what you've done, and test their proposed solutions. You should always provide an MCVE).
The man page for mmap() says that you must specify exactly one of MAP_SHARED and MAP_PRIVATE in the flags argument. In your case, to act like malloc(), you'll want MAP_PRIVATE:
myStruct *entry = mmap(0, sizeof *entry,
PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
(I've also made this more idiomatic C by omitting the harmful cast and matching the sizeof to the actual variable rather than its type).