Allocating pointers from existing shared memory region - POSIX C API - c

Question

I have a structure A of the form described below. My goal is to create the structure in one process and share it across to a different process. A->buffer depends upon user_size and value provided and hence I cannot have a pre-allocated buffer. Knowing that sharing pointers with shared memory is hard across processes since pointer value in one process will no longer be valid in the other process, I tried the following approach.
I wanted to allocate the buffer from shared memory, from the writing process so that other processes can map to the same shared memory and obtain a valid pointer address to the shared memory.
Conceptually, calling mmap on the same shared memory fd descriptor should point to the same location in shared memory, however on the reading process, I'm able to get a valid pointer and valid another_value.
Can someone clarify this? Parts of the code below.
struct A {
int another_variable;
void *buffer;
}
/* the size (in bytes) of shared memory object */
const int SIZE = 4096;
/* name of the shared memory object */
const char *name = "OS";
/* shared memory file descriptor */
int shm_fd;
/* create the shared memory object */
shm_fd = shm_open(name, O_CREAT | O_RDWR, 0666);
/* configure the size of the shared memory object */
ftruncate(shm_fd, SIZE);
a_data = mmap(NULL, sizeof(*A), PROT_READ | PROT_WRITE, MAP_SHARED,
shm_fd, 0);
if (a_data == MAP_FAILED) {
printf("ERROR: mmap failed for a_data\n");
exit(EXIT_FAILURE);
}
a_data->buffer = mmap(NULL, user_size, PROT_WRITE,
MAP_SHARED , shm_fd, 0);
if (a_data->buffer == MAP_FAILED) {
printf("ERROR: mmap failed for a_data buffer\n");
exit(EXIT_FAILURE);
}
memcpy(a_data->buffer, "test", strlen("test"));