I got a question about shared memory and segmentation fault.
I thought that it would be fine to use huge size of memory.
When I checked Shmmax, i found the huge memory can be allocated.
under data is the result of $ipcs -lm
------ Shared Memory Limits --------
max number of segments = 4096
max seg size (kbytes) = 18014398509465599
max total shared memory (kbytes) = 18014398442373116
min seg size (bytes) = 1
#include <stdio.h>
#include <sys/shm.h>
#include <sys/ipc.h>
#include <sys/types.h>
#include <stdlib.h>
#include <unistd.h>
#define ARRAY_SIZE 40000000
int main(int argc, char *argv[]){
int shmid;
void *shared_memory = (void *)0;
shmid = shmget((key_t)1234, sizeof(float), IPC_CREAT|0666);
if (shmid == -1)
{
perror("shmget failed : ");
exit(0);
}
shared_memory = (float *)shmat(shmid, NULL, 0);
if (shared_memory == (void *)-1)
{
perror("shmat failed : ");
exit(0);
}
static float *testx;
testx = (float *)shared_memory;
int k = 0;
for(k;k<400;k++){
testx[k] = 1.12;
}
for(k;k<40000000;k++){
testx[k] = 1.12;
}
}
the program can run the first for loop which has small amount of size
the problem, however, is the second loop with 40,000,000 size
any suggestion what should i edit to run this code?
The reason for your SEGFAULT is that you haven't created enough size segment with shmget.
The argument you passed to shmget as size is sizeof(float) which is just enough to store 1 float.
What you need to do is call shmget like this -
shmget((key_t)1234, sizeof(float)*40000000, IPC_CREAT|0666);
Then you can use all the memory correctly.
The reason that the smaller loop of 400 worked is because shmget creates segments that are multiple of PAGE_SIZE.
So even when you passed sizeof(float), it allocated atleast 1 page which was enough to hold 400 floats but not 40000000.
I hope that clears the confusion.
Related
The following code will generate errno 12 cannot allocate memory
#include <sys/mman.h>
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <memory.h>
#include <errno.h>
int main()
{
char* p;
for (size_t i = 0; i < 0x10000; i++)
{
char* addr = (char*)0xAAA00000000uL - i * 0x2000;
p = mmap(addr, 0x1000,
PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (p != addr) {
printf("%lu %d\n", i, errno);
getchar();
return 1;
}
memset(p, 'A' + (i % 26), 0x1000);
}
return 0;
}
The output is 65510 12 on my machine.
However, if we change size of each page from 0x1000 to 0x2000, the allocation will be successful, even if it is using more memory.
The only difference I think is the number of continuous pages, is there a limitation on this? If so, how to set it to unlimited?
It seems that setting /proc/sys/vm/max_map_count to a larger number solves the problem.
Reference: How much memory could vm use
#include <stdio.h>
#include <sys/mman.h>
#include <unistd.h>
int main(int argc, char *argv[])
{
int* ptr = (int *) mmap(0x4096, sizeof(int)*1024, 1100, MAP_PRIVATE, 0, 0);
fprintf(stdout, "%p\n", ptr);
if (ptr == MAP_FAILED)
{
fprintf(stderr, "Could not mmap\n");
return 1;
}
return 0;
}
My code is printing could not mmap, I cannot figure out what am I doing wrong. My memory page's size is 4096 bytes so my address input is page size aligned too. What I want to do is allocate a memory space for 1024 integers using mmap on my specified address.
As pointed by several people in the comments, I should be using MAP_ANONYMOUS, and haven't specified the memory location correctly. Using the following parameters worked,
mmap((void*) 0x0804a000, sizeof(int)*1024, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_FIXED, 0, 0);
To create a new and empty shared memory segment I can do something like:
int shmid = shmget(IPC_PRIVATE, nbytes, IPC_CREAT | 0777);
int* new_array = (int*)shmat(shmid, NULL, 0);
Now suppose int* array owns a chunk of pre-existing data. Something like
int* array = malloc(2 * sizeof(int));
array[0] = 42;
array[1] = 24;
I'd like to turn this memory segment into a shared memory segment with the System V shared memory C API, on Linux.
How can I do this?
I've tried stuff like
int shmid = shmget(IPC_PRIVATE, nbytes, IPC_CREAT | 0777);
void* attach = shmat(shmid, array, SHM_RND);
and like
int shmid = shmget(IPC_PRIVATE, nbytes, IPC_CREAT | 0777);
array = (int*)shmat(shmid, 0, 0);
but either shmat returns (void*)-1 or int* array gets all its values set to zero.
Here's a full program that shows the failures:
/*
gcc shm0.c -o shm0 && ./shm0
*/
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <sys/stat.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#define NELEMENTS 3
void array_show(int* array, size_t nelements){
puts("");
for(int i=0; i < nelements; ++i)
printf("%d\n", *(array + i));
}
int main(){
const size_t nbytes = NELEMENTS * sizeof(int);
int* array;
posix_memalign((void**)&array, 16*16*16, nbytes);
int shmid = shmget(IPC_PRIVATE, nbytes, IPC_CREAT | 0777);
// 1st try...
void* attach = shmat(shmid, array, SHM_RND);
if(attach == (void*)-1)
puts("shmat() failed!");
// 2nd try...
memset(array, 1, nbytes); // Fill array with dummy data!
array_show(array, NELEMENTS);
array = (int*)shmat(shmid, 0, 0);
array_show(array, NELEMENTS);
if(array[1] == 0)
puts("array was reset!");
// 3rd try
array = (int*)shmat(shmid, array, 0); // Now accessing, say, array[0] yields a segfault!
shmdt(array);
}
Why SYSV shared memory and not mmap()? Because I'm using the X11 MIT shared memory extension, and, according to the docs:
... to be able to use this extension, your system must provide the SYSV shared memory primitives. There is not an mmap-based version of this extension. To use shared memory on Sun systems, you must have built your kernel with SYSV shared memory enabled ...
(If someone knows a way around this, or how to do this using shm_open(), I'd be delighted to hear about it.)
As far as I understand, arrays are allocated contiguous blocks of memory. I wrote a program to test my understanding. It basically attaches a shared memory segment. Here is the code:
#include <stdio.h>
#include <sys/shm.h>
#include <sys/stat.h>
int main ()
{
int segment_id;
int* shared_memory;
struct shmid_ds shmbuffer;
int segment_size;
const int
shared_segment_size = 4096; //one page size
int *results;
int cpid=0;
int sum=0;
/* Allocate a shared memory segment. */
segment_id = shmget (IPC_PRIVATE, shared_segment_size,
IPC_CREAT | IPC_EXCL | S_IRUSR | S_IWUSR);
/* Attach the shared memory segment. Results now contain the starting address */
results =shmat (segment_id, 0, 0);
printf("%p\n",&results);
printf("%p\n",&results[1]);
printf("%p\n",&results[2]);
printf("%p\n",&results[3]);
return 0;
}
Here is the output:
0x7fffe4d62e40
0x7f6603987004
0x7f6603987008
0x7f660398700c
Can someone explain to me why the address of results[1] is:
0x7f6603987004
given that the address of results[0] is:
0x7fffe4d62e40
Shouldn't it be 0x7fffe4d62e44?
Am I missing something crucial?
I have created two programs a server.c and a client.c. I have a struct that holds an age. I have got the programs working together to read the shared memory and to change the shared memory, however this only works when using one variable in the struct. As soon as i have more than one variable in the struct i get a segmentation fault.
Server.c
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <stdlib.h>
typedef struct People
{
int age;
int isDone;
} Person;
int main()
{
Person aaron;
Person *p_aaron;
int id;
int key = 5432;
p_aaron = &aaron;
(*p_aaron).age = 19;
(*p_aaron).isDone = 0;
if ((id = shmget(key,sizeof(aaron), IPC_CREAT | 0666)) < 0)
{
perror("SHMGET");
exit(1);
}
if((p_aaron = shmat(id, NULL, 0)) == (Person *) -1)
{
perror("SHMAT");
exit(1);
}
(*p_aaron).age = 19;
printf("Shared Memory Age: %d\n", (*p_aaron).age);
*p_aaron = aaron;
while ((*p_aaron).age == 19)
{
sleep(1);
}
printf("Shared Memory Age Turned To: %d", (*p_aaron).age);
return 0;
}
Client.c
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#include <stdio.h>
#include <string.h>
typedef struct People
{
int age;
} Person;
int main()
{
Person aaron;
Person *p_aaron;
int id;
int key = 5432;
p_aaron = &aaron;
id = shmget(key,sizeof(aaron), IPC_CREAT | 0644);
p_aaron = shmat(id, NULL, 0);
printf("%d", (*p_aaron).age);
(*p_aaron).age = 21;
return 0;
}
Error message from Server.c
SHMGET: Invalid argument
RUN FINISHED; exit value 1; real time: 0ms; user: 0ms; system: 0ms
You don't show any code that deletes the shared memory segment.
If you look at the POSIX specification for shmget(), you will see that the EINVAL error you are reporting can be given if:
[EINVAL]
A shared memory segment is to be created and the value of size is less than the system-imposed minimum or greater than the system-imposed maximum.
[EINVAL]
No shared memory segment is to be created and a shared memory segment exists for key but the size of the segment associated with it is less than size.
I think you may be running into the second case; you're trying to create a bigger shared memory segment than the one that already exists.
Modify your code to clean up behind itself (shmdt(), shmctl()).
Use ipcrm to remove the existing shared memory segment.
Also, as I noted in a comment, you should make sure that the client and server programs agree on the size of the structure in shared memory. Anything else is a recipe for disaster. You should put the structure definition into a header, and both your programs should use that header, and both should be recompiled when you change the definition of the header.