Can I create a circular buffer on Linux? (current code segfaults) - c

Question

Inspired by this example for Windows. In short, they create a file handle (with CreateFileMapping) then create 2 different pointers to the same memory (MapViewOfFileEx or MapViewOfFile3)
So I tried to do the same thing with shm_open, ftruncate and mmap. I used mmap a few times in the past for memory and files but I never mixed it with shm_open or used shm_open.
My code fails on the second mmap with a segfault. I tried doing a syscall directly on both mmaps and it still segfaults :( How do I do this properly? The idea is I can do memcpy(p+len-10, src, 20) and have the first 10bytes of src be at the end of the memory and last 10 written to the start (hence circular)
#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <assert.h>
int main()
{
write(2, "Start\n", 6); //prints this
int len = 1024*1024*2;
int fd = shm_open("example", O_RDWR | O_CREAT, 0777);
assert(fd > 0); //ok
int r1 = ftruncate(fd, len);
assert(r1 == 0); //ok
char*p = (char*)mmap(0, len, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0);
assert((long long)p>0); //ok
//Segfaults on next line
char*p2 = (char*)mmap(p+len, len, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_FIXED, fd, 0); //segfaults
write(2, "Finish\n", 7); //doesn't print this
return 0;
}

Answer 1

Linux usually selects address space for mappings starting from a certain point and goes lower with each reservation. So your 2nd mmap call replaces one of previous file mappings (likely libc.so), which leads to SIGSEGV with SEGV_ACCERR - invalid access permissions. You are overwriting executable section of libc.so (that is being executed right now) with non-executable data.
Use strace to check what is going on inside:
$ strace ./a.out
...
openat(AT_FDCWD, "/dev/shm/example", O_RDWR|O_CREAT|O_NOFOLLOW|O_CLOEXEC, 0777) = 3
ftruncate(3, 2097152) = 0
mmap(NULL, 2097152, PROT_READ|PROT_WRITE, MAP_PRIVATE, 3, 0) = 0x7f134c1bf000
mmap(0x7f134c3bf000, 2097152, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0) = 0x7f134c3bf000
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_ACCERR, si_addr=0x7f134c4ccc37} ---
+++ killed by SIGSEGV +++
Compare addresses you are passing around with /proc/$pid/maps file and you will see what you are overwriting.
Your mistake was to assume MAP_FIXED can be used without reserving memory beforehand. To do this properly you need to:
Reserve memory by calling mmap with len * 2 size, PROT_NONE and MAP_ANONYMOUS | MAP_PRIVATE (and without file)
Use mmap with MAP_FIXED to overwrite portions of that mapping with the content you need
Additionally, you should prefer using memfd_create instead of shm_open on Linux to avoid shared memory files from staying around. Unlinking them with shm_unlink doesn't help if your program crashes. This also gives you a file that is private to your program instance.

Answer 2

You do not need to call mmap again to gen new pointer. (You even must not to do it.) Just increment it.
The pointer p2 will not point to the address just after the memory block allocated.
#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <assert.h>
int main()
{
write(2, "Start\n", 6); //prints this
int len = 1024*1024*2;
int fd = shm_open("example", O_RDWR | O_CREAT, 0777);
assert(fd > 0); //ok
int r1 = ftruncate(fd, len);
assert(r1 == 0); //ok
char*p = (char*)mmap(0, len, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0);
assert((long long)p>0); //ok
char*p2 = p+len;
write(2, "Finish\n", 7); //doesn't print this
return 0;
}