Can anyone help me make a Shared Memory Segment in C - c

I need to make a shared memory segment so that I can have multiple readers and writers access it. I think I know what I am doing as far as the semaphores and readers and writers go...
BUT I am clueless as to how to even create a shared memory segment. I want the segment to hold an array of 20 structs. Each struct will hold a first name, an int, and another int.
Can anyone help me at least start this? I am desperate and everything I read online just confuses me more.
EDIT: Okay, so I do something like this to start
int memID = shmget(IPC_PRIVATE, sizeof(startData[0])*20, IPC_CREAT);
with startData as the array of structs holding my data initialized and I get an error saying
"Segmentation Fault (core dumped)"

The modern way to obtain shared memory is to use the API, provided by the Single UNIX Specification. Here is an example with two processes - one creates a shared memory object and puts some data inside, the other one reads it.
First process:
#include <stdio.h>
#include <unistd.h>
#include <sys/mman.h>
#include <fcntl.h>
#define SHM_NAME "/test"
typedef struct
{
int item;
} DataItem;
int main (void)
{
int smfd, i;
DataItem *smarr;
size_t size = 20*sizeof(DataItem);
// Create a shared memory object
smfd = shm_open(SHM_NAME, O_RDWR | O_CREAT, 0600);
// Resize to fit
ftruncate(smfd, size);
// Map the object
smarr = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, smfd, 0);
// Put in some data
for (i = 0; i < 20; i++)
smarr[i].item = i;
printf("Press Enter to remove the shared memory object\n");
getc(stdin);
// Unmap the object
munmap(smarr, size);
// Close the shared memory object handle
close(smfd);
// Remove the shared memory object
shm_unlink(SHM_NAME);
return 0;
}
The process creates a shared memory object with shm_open(). The object is created with an initial size of zero, so it is enlarged using ftruncate(). Then the object is memory mapped into the virtual address space of the process using mmap(). The important thing here is that the mapping is read/write (PROT_READ | PROT_WRITE) and it is shared (MAP_SHARED). Once the mapping is done, it can be accessed as a regular dynamically allocated memory (as a matter of fact, malloc() in glibc on Linux uses anonymous memory mappings for larger allocations). Then the process writes data into the array and waits until Enter is pressed. Then it unmaps the object using munmap(), closes its file handle and unlinks the object with shm_unlink().
Second process:
#include <stdio.h>
#include <sys/mman.h>
#include <fcntl.h>
#define SHM_NAME "/test"
typedef struct
{
int item;
} DataItem;
int main (void)
{
int smfd, i;
DataItem *smarr;
size_t size = 20*sizeof(DataItem);
// Open the shared memory object
smfd = shm_open(SHM_NAME, O_RDONLY, 0600);
// Map the object
smarr = mmap(NULL, size, PROT_READ, MAP_SHARED, smfd, 0);
// Read the data
for (i = 0; i < 20; i++)
printf("Item %d is %d\n", i, smarr[i].item);
// Unmap the object
munmap(smarr, size);
// Close the shared memory object handle
close(smfd);
return 0;
}
This one opens the shared memory object for read access only and also memory maps it for read access only. Any attempt to write to the elements of the smarr array would result in segmentation fault being delivered.
Compile and run the first process. Then in a separate console run the second process and observe the output. When the second process has finished, go back to the first one and press Enter to clean up the shared memory block.
For more information consult the man pages of each function or the memory management portion of the SUS (it's better to consult the man pages as they document the system-specific behaviour of these functions).

Related

Confused about mmap() return - two different pointers?

Currently trying to understand how memory mapping works in Linux (or in general, really), and I'm following with this one example of Shared Memory in POSIX systems from Operating System Concepts. The two files are as follows:
Producer file
// This is the producer file for the Shared memory object
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <fcntl.h>
#include <sys/shm.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
#include <errno.h>
#include <sys/mman.h>
int main()
{
// ---------------- PRODUCER ESTABLISHES SHARED MEMORY OBJECT AND WRITES TO IT ----------------
// Specifying the size in bytes of the shared memory object
const int SIZE = 4096;
// Name of the shared memory space
const char *name = "OS";
// The actual strings to write to shared memory
const char *message_0 = "Hello";
const char *message_1 = "World!";
// Shared memory file descriptor will be stored in this
int fd;
// Pointer to the shared memory object will be stored in this
char *ptr;
// Checking error
int errnum;
// Create a shared memory object. This opens (establishes a connection to) a shared memory object.
fd = shm_open(name, O_CREAT | O_RDWR,0666);
if (fd == -1)
{
errnum = errno;
fprintf(stdout, "Value of errno: %d\n", errno);
perror("Error printed by perror");
fprintf(stdout, "Error opening file: %s\n", strerror(errnum));
return 1;
}
// Configure the size of the shared-memory object to be 4096 bytes
ftruncate(fd, SIZE);
// Memory map the shared memory object
ptr = (char *) mmap(0, SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
printf("Ptr is: %p\n", ptr);
if (ptr == MAP_FAILED)
{
errnum = errno;
fprintf(stdout, "Value of errno in ptr: %d\n", errno);
perror("Error printed by perror");
fprintf(stdout, "Error opening file: %s\n", strerror(errnum));
return 1;
}
// Write to the shared memory object
sprintf(ptr, "%s", message_0);
ptr += strlen(message_0);
sprintf(ptr, "%s", message_1);
ptr += strlen(message_1);
return 0;
}
Consumer file
// This is the consumer file for the Shared memory object, in which it reads what is in the memory object OS
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <fcntl.h>
#include <sys/shm.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
#include <errno.h>
#include <sys/mman.h>
int main()
{
// Size in bytes of shared memory object
const int SIZE = 4096;
// Name of the shared memory space
const char *name = "OS";
// Shared memory file descriptor will be stored in this
int fd;
// Pointer to the shared memory object will be stored in this
char *ptr;
// Checking error
int errnum;
// Open the shared memory object
fd = shm_open(name, O_RDWR, 0666);
// If error in shm_open()
if (fd == -1)
{
errnum = errno;
fprintf(stdout, "Value of errno: %d\n", errno);
perror("Error printed by perror");
fprintf(stdout, "Error opening file: %s\n", strerror(errnum));
return 1;
}
// Memory-map the shared memory object
ptr = (char *) mmap(0, SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
printf("Ptr is: %p\n", ptr);
if (ptr == MAP_FAILED)
{
errnum = errno;
fprintf(stdout, "Value of errno in ptr: %d\n", errno);
perror("Error printed by perror");
fprintf(stdout, "Error opening file: %s\n", strerror(errnum));
return 1;
}
// Read from the shared memory object
printf("%s\n", (char *) ptr);
// Remove the shared memory object (delete it)
shm_unlink(name);
return 0;
}
When I print the pointer to the shared memory object (printf("Ptr value: %p\n, ptr)), I get two different values for the consumer and producer files. Why does this happen?
As I understand, the pointer ptr points at the shared memory object, which is in the physical memory. This physical memory is just shared amongst two processes, by mapping it onto their address space. However, this would require that the pointer point to the same address in physical memory, no? Or is it pointing to the virtual memory (i.e. the address space of the processes)? If so, does that itself point to the physical memory?
Thanks!
Each process has its own virtual memory address space. The mappings from virtual memory to physical memory are, in general, different for each process. While the shared memory may be in physical memory at address 0x1000, the producer process may have it mapped into virtual memory at address 0x7000, and the consumer process may have it mapped into virtual memory at address 0x4000.
Further, if the shared memory is swapped out of memory for some reason, the system could later reload it to a different physical address, say 0x13000, and update the mappings in the processes so that it appears at the same addresses as before in each of the producer and consumer processes.
Producer and consumer are two different processes, so they have their own virtual memory space each. When you attach the shared segment, you specify the kernel that you have no preference on where (in your unallocated virtual address space) to put it, so you normally get different pointers because most probably both processes have different memory maps. Just think on an scenario where one process has allocated the memory map at, say, address A, and the other process has that address A occupied by some other thing it has allocated (e.g. dynamic memory for the heap, or a different shared library) It is clear that the kernel cannot allocate two different things in the same (virtual) address range (or even overlap some other mapping), so it uses a differen place, and returns a different pointer.
Just don't worry about it, as both virtual addresses map to the same phisical address (this is sure, as it is a shared memory segment), and the pointers finally end pointing to the same thing. The kernel is normally not aware of where it has placed that same segment in a different process, it doesn't need to do. This means that it is highly improbable that you get both pointers equal, contrary to what you though.
As none of these processes knows about the virtual address space of the other, there's no conflict, but never pass the address to the other process to use, because an address in a virtual address space has absolutely no meaning in the virtual address space of another, different process.
A process has normally no means (and no possibility) to know how the kernel assings the memory and builds the map of virtual to physicall addresses. Even for itself (the kernel also runs in it's virtual memory space, and only deals with the tables that map ---these tables are in the virtual address space of the kernel, but not in the virtual address space of any user process, so this is the reason that makes the kernel capable of changing the mappings but it is not possible for other processes)
It is impossible for a process to know where in actual memory a virtual address points to. That is possible for some administrator processes through a device (and a special device is needed for this) /dev/mem. But if you try to look in there without the actual mapping you'll get a mess of pages some belonging to a process, some to another, with no apparent structure, or you cannot also know what does it mean the contents of those pages.
There's another device /dev/kmem that maps to the kernel virtual address space, and allows a process (e.g. a debugger) to get access to the mapping table there.... but this is a very dangerous bend, as you can easily crash your system by tweaking there. Think that you cannot normally stop the kernel (this stops the system) or if you can do, you will stop essential parts of the system that should be running.

Mapping existing memory (data segment) to another memory segment

As the title suggests, I would like to ask if there is any way for me to map the data segment of my executable to another memory so that any changes to the second are updated instantly on the first. One initial thought I had was to use mmap, but unfortunately mmap requires a file descriptor and I do not know of a way to somehow open a file descriptor on my running processes memory. I tried to use shmget/shmat in order to create a shared memory object on the process data segment (&__data_start) but again I failed ( even though that might have been a mistake on my end as I am unfamiliar with the shm API). A similar question I found is this: Linux mapping virtual memory range to existing virtual memory range? , but the replies are not helpful.. Any thoughts are welcome.
Thank you in advance.
Some pseudocode would look like this:
extern char __data_start, _end;
char test = 'A';
int main(int argc, char *argv[]){
size_t size = &_end - &__data_start;
char *mirror = malloc(size);
magic_map(&__data_start, mirror, size); //this is the part I need.
printf("%c\n", test) // prints A
int offset = &test - &__data_start;
*(mirror + offset) = 'B';
printf("%c\n", test) // prints B
free(mirror);
return 0;
}
it appears I managed to solve this. To be honest I don't know if it will cause problems in the future and what side effects this might have, but this is it (If any issues arise I will try to log them here for future references).
Solution:
Basically what I did was use the mmap flags MAP_ANONYMOUS and MAP_FIXED.
MAP_ANONYMOUS: With this flag a file descriptor is no longer required (hence the -1 in the call)
MAP_FIXED: With this flag the addr argument is no longer a hint, but it will put the mapping on the address you specify.
MAP_SHARED: With this you have the shared mapping so that any changes are visible to the original mapping.
I have left in a comment the munmap function. This is because if unmap executes we free the data_segment (pointed to by &__data_start) and as a result the global and static variables are corrupted. When at_exit function is called after main returns the program will crash with a segmentation fault. (Because it tries to double free the data segment)
Code:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdint.h>
#define _GNU_SOURCE 1
#include <unistd.h>
#include <sys/mman.h>
extern char __data_start;
extern char _end;
int test = 10;
int main(int argc, char *argv[])
{
size_t size = 4096;
char *shared = mmap(&__data_start, 4096, PROT_READ | PROT_WRITE, MAP_FIXED | MAP_ANONYMOUS | MAP_SHARED, -1, 0);
if(shared == (void *)-1){
printf("Cant mmap\n");
exit(-1);
}
printf("original: %p, shared: %p\n",&__data_start, shared);
size_t offset = (void *)&test - (void *)&__data_start;
*(shared+offset) = 50;
msync(shared, 4096, MS_SYNC);
printf("test: %d :: %d\n", test, *(shared+offset));
test = 25;
printf("test: %d :: %d\n", test, *(shared+offset));
//munmap(shared, 4096);
}
Output:
original: 0x55c4066eb000, shared: 0x55c4066eb000
test: 50 :: 50
test: 25 :: 25

How to create Posix shared memory to work with integer

here is a bit of background on what I am trying to do for this class assignment. I am going to have 10 total programs that are going using semaphores to be synchronized to accessing a shared memory construct that is going to hold an Integer. This integer will hold the amount of text that has currently been read. How exactly am I am going to take this setup from my textbook for strings and convert it for use with Integers?
The 9 of the programs will only increment the value of the shared memory and then one of the programs will just read the shared memory value for later use. "Producer" is going to be the 9 programs that will increment the value of the shared memory. "Consumer" is going to be the single program that will read the value later on to do separate operations.
The "Consumer" program will be initializing the shared memory and setting the int value to 0.
Below is the "Producer" program that will build and write to the shared memory construct.
#include <stdio.h>
#include <stlib.h>
#include <string.h>
#include <fcntl.h>
#include <sys/shm.h>
#include <sys/stat.h>
int main()
{
/* the size (in bytes) of shared memory object */
const int SIZE 4096;
/* name of the shared memory object */
const char *name = "OS";
/* strings written to shared memory */
const char *message 0 = "Hello";
const char *message 1 = "World!";
/* shared memory file descriptor */
int shm fd;
/* pointer to shared memory obect */
void *ptr;
/* create the shared memory object */
shm fd = shm open(name, O CREAT | O RDRW, 0666);
/* configure the size of the shared memory object */
ftruncate(shm fd, SIZE);
/* memory map the shared memory object */
ptr = mmap(0, SIZE, PROT WRITE, MAP SHARED, shm fd, 0);
/* write to the shared memory object */
sprintf(ptr,"%s",message 0);
ptr += strlen(message 0);
sprintf(ptr,"%s",message 1);
ptr += strlen(message 1);
return 0;
}
Now below is the "Consumer" program that will read from the shared memory
#include <stdio.h>
#include <stlib.h>
#include <fcntl.h>
#include <sys/shm.h>
#include <sys/stat.h>
int main()
{
/* the size (in bytes) of shared memory object */
const int SIZE 4096;
/* name of the shared memory object */
const char *name = "OS";
/* shared memory file descriptor */
int shm fd;
/* pointer to shared memory obect */
void *ptr;
/* open the shared memory object */
shm fd = shm open(name, O RDONLY, 0666);
/* memory map the shared memory object */
ptr = mmap(0, SIZE, PROT READ, MAP SHARED, shm fd, 0);
/* read from the shared memory object */
printf("%s",(char *)ptr);
/* remove the shared memory object */
shm unlink(name);
return 0;
}
My idea of how to change this is to end up having the following changes for the Producer
change SIZE to 4 bytes for the size of Int
read the value of the int from shared memory assign that value to shmValue
shmValue then gets incremented
shmValue then gets written back to shared memory
My idea of how to change the Consumer program
change SIZE to 4 bytes for the size of Int
initialize the shared memory to int with a value of 0
reads shared memory to shmValue
then uses shmValue for later operations.
Again, guys, I would very much appreciate the help in dealing with this issue that I am having in understanding how to use this technology. Thank you for the help in advance!!

copy whole of a file into memory using mmap

i want to copy whole of a file to memory using mmap in C.i write this code:
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <errno.h>
int main(int arg, char *argv[])
{
char c ;
int numOfWs = 0 ;
int numOfPr = 0 ;
int numberOfCharacters ;
int i=0;
int k;
int pageSize = getpagesize();
char *data;
float wsP = 0;
float prP = 0;
int fp = open("2.txt", O_RDWR);
data = mmap((caddr_t)0, pageSize, PROT_READ, MAP_SHARED, fp,pageSize);
printf("%s\n", data);
exit(0);
}
when i execute the code i get the Bus error message.
next, i want to iterate this copied file and do some thing on it.
how can i copy the file correctly?
2 things.
The second parameter of mmap() is the size of the portion of file you want to make visible in your address space. The last one is the offset in the file from which you want the map. This means that as you have called mmap() you will see only 1 page (on x86 and ARM it's 4096 bytes) starting at offset 4096 in your file. If your file is smaller than 4096 bytes, then there will be no mapping and mmap() will return MAP_FAILED (i.e. (caddr_t)-1). You didn't check the return value of the function so the following printf() dereferences an illegal pointer => BUS ERROR.
Using a memory map with string functions can be difficult. If the file doesn't contain binary 0. It can happen that these functions then try to access past the mapped size of the file and touch unmapped memory => SEGFAULT.
To open a memory for a file, you have to know the size of the file.
struct stat filestat;
if(fstat(fd, &filestat) !=0) {
perror("stat failed");
exit(1);
}
data = mmap(NULL, filestat.st_size, PROT_READ, MAP_SHARED, fp, 0);
if(data == MAP_FAILED) {
perror("mmap failed");
exit(2);
}
EDIT: The memory map will always be opened with a size that is a multiple of the pagesize. This means that the last page will be filled with 0 up to the next multiple of the pagesize. Often programs using memory mapped files with string functions (like your printf()) will work most of the time, but will suddenly crash when mapping a file whith a size exactly a multiple of the page size (4096, 8192, 12288 etc.). The often seen advice to pass to mmap() a size bigger than real file size works on Linux but is not portable and is even in violation of Posix, which explicitly states that mapping beyond the file size is undefined behaviour. The only portable way is to not use string functions on memory maps.
The last parameter of mmap is the offset within the file, where the part of file mapped to memory starts. It shall be 0 in your case
data = mmap(NULL, pageSize, PROT_READ, MAP_SHARED, fp,0);
If your file is shorter than pageSize, you will not be able to use addresses beyond the end of file. To use the full size, you shall expand the size to pageSize before calling mmap. Use something like:
ftruncate(fp, pageSize);
If you want to write to the memory (file) you shall use flag PROT_WRITE as well. I.e.
data = mmap(NULL, pageSize, PROT_READ|PROT_WRITE, MAP_SHARED, fp,0);
If your file does not contain 0 character (as end of string) and you want to print it as a string, you shall use printf with explicitly specified maximum size:
printf("%.*s\n", pageSize, data);
Also, of course, as pointed by #Jongware, you shall test result of open for -1 and mmap for MAP_FAILED.

How does one keep an int and an array in shared memory in C?

I'm attempting to write a program in which children processes communicate with each other on Linux.
These processes are all created from the same program and as such they share code.
I need them to have access to two integer variables as well as an integer array.
I have no idea how shared memory works and every resource I've searched has done nothing but confuse me.
Any help would be greatly appreciated!
Edit: Here is an example of some code I've written so far just to share one int but it's probably wrong.
int segmentId;
int sharedInt;
const int shareSize = sizeof(int);
/* Allocate shared memory segment */
segmentId = shmget(IPC_PRIVATE, shareSize, S_IRUSR | S_IWUSR);
/* attach the shared memory segment */
sharedInt = (int) shmat(segmentId, NULL, 0);
/* Rest of code will go here */
/* detach shared memory segment */
shmdt(sharedInt);
/* remove shared memory segment */
shmctl(segmentId, IPC_RMID, NULL);
You are going to need to increase the size of your shared memory. How big an array do you need? Whatever value it is, you're going to need to select it before creating the shared memory segment - dynamic memory isn't going to work too well here.
When you attach to shared memory, you get a pointer to the start address. It will be sufficiently well aligned to be used for any purpose. So, you can create pointers to your two variables and array along these lines (cribbing some of the skeleton from your code example) - note the use of pointers to access the shared memory:
enum { ARRAY_SIZE = 1024 * 1024 };
int segmentId;
int *sharedInt1;
int *sharedInt2;
int *sharedArry;
const int shareSize = sizeof(int) * (2 + ARRAY_SIZE);
/* Allocate shared memory segment */
segmentId = shmget(IPC_PRIVATE, shareSize, S_IRUSR | S_IWUSR);
/* attach the shared memory segment */
sharedInt1 = (int *) shmat(segmentId, NULL, 0);
sharedInt2 = sharedInt1 + 1;
sharedArry = sharedInt1 + 2;
/* Rest of code will go here */
...fork your child processes...
...the children can use the three pointers to shared memory...
...worry about synchronization...
...you may need to use semaphores too - but they *are* complex...
...Note that pthreads and mutexes are no help with independent processes...
/* detach shared memory segment */
shmdt(sharedInt1);
/* remove shared memory segment */
shmctl(segmentId, IPC_RMID, NULL);
From your comment it seems you're using IPC_PRIVATE, and that definitely looks wrong ("private" kinds of suggest it's not meant for sharing, no?-). Try something like:
#include <sys/ipc.h>
#include <sys/shm.h>
...
int segid = shmget((key_t)0x0BADDOOD, shareSize, IPC_CREAT);
if (segid < 0) { /* insert error processing here! */ }
int *p = (int*) shmat(segid, 0, 0);
if (!p) { /* insert error processing here! */ }
This guide looks useful: http://www.cs.cf.ac.uk/Dave/C/node27.html. It includes some example programs.
There are also Linux man pages online.
Shared memory is just a segment of memory allocated by one process, with a unique id, and the other process will also make the allocation, with the same id, and the size of the memory is the sizeof the structure that you are using, so you would have a structure with 2 integers and an integer array.
Now they both have a pointer to the same memory, so the writes of one will overwrite whatever else was there, and the other has immediate access to it.

Resources