I am trying to ftruncate a shared memory object to a specific length. For example, I want to set its length to 1 byte using the following snippet:
#include <stdlib.h>
#include <stdio.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/mman.h>
int main() {
struct stat fd_stat;
int fd;
fd = shm_open("NAME", O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);
fstat(fd, &fd_stat);
printf("Size before: %lld\n", fd_stat.st_size);
ftruncate(fd, 1);
fstat(fd, &fd_stat);
printf("Size after: %lld\n", fd_stat.st_size);
}
Which in Ubuntu 20.04 prints:
Size before: 0
Size after: 1
That's the output I'd expect.
However, in macOS X Big Sur I get:
Size before: 0
Size after: 4096
As you see, it seems to be expanding the size to the size of a page.
The ftruncate Linux man page reads:
The truncate() and ftruncate() functions cause the regular file named by path or referenced by fd to be truncated to a size of precisely length bytes.
Nonetheless, the POSIX specification is not as specific (pun intended):
If fildes refers to a regular file, the ftruncate() function shall cause the size of the file to be truncated to length. [...] If the file previously was smaller than this size, ftruncate() shall increase the size of the file.
Does that mean that ftruncate always sets the length to exactly the specified number of bytes? If indeed it does, it would entail that macOS X Big Sur is not fully POSIX-compliant (even though it is certified to be so). If not, how can I guarantee that it truncates fd to the size I want?
In short, you cannot get a guarantee that your shared memory object will be of precisely the size you ask ftruncate to be. That's because, as #user3386109 said, "The portion of the POSIX spec that you quoted starts with "If fildes refers to a regular file"".
If you want to constrain yourself to an arbitrary length, you can always use an auxiliary variable to keep track of the size you assume it to be (even if the actual size might actually differ, which may not be that important after all). Your code would look like:
#include <stdlib.h>
#include <stdio.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/mman.h>
int main() {
struct stat fd_stat;
int fd;
off_t fd_size;
fd = shm_open("NAME", O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);
fstat(fd, &fd_stat);
printf("Size before: %lld\n", fd_stat.st_size);
fd_size = 1;
ftruncate(fd, fd_size);
fstat(fd, &fd_stat);
printf("Actual size: %lld\n", fd_stat.st_size);
printf("Perceived size: %lld\n", fd_size);
}
On top of that, if you want to share the size among different processes, you can turn fd_size into a mmaped shared memory object to keep track of the size you assume it to be across them all.
Related
I am trying to open a shared memory segment in my main process. My terminology may be incorrect in the question but this is what I am trying to achieve:
I collect information from 7 sensors and from that I evaluate the state. I made a structure senStruct. I want to share the 7 sensor and state information to other processes with use of shared memory. In code I am opening the senfile, mapping the senStruct into it. Now I simply want to store something into state and it gives me an error.
Here is the code:
#include <fcntl.h> /* Defines O_* constants */
#include <sys/stat.h> /* Defines mode constants */
#include <sys/mman.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
typedef struct senStruct {
int sensor[7];
int state;
}senStruct;
int main()
{
int fd;
size_t size = sizeof(senStruct);
senStruct *p;
fd = shm_open( "senfile" , O_CREAT | O_EXCL | O_RDWR, S_IRUSR | S_IWUSR );
ftruncate (fd, size);
printf("Before mapping...p points to %p\n",p);
p =(senStruct *) mmap (NULL , size, PROT_READ | PROT_WRITE,MAP_SHARED , fd, 0);
printf("After mapping...p points to %p\n",p);
p->state=1;
return 0;
}
Output is:
Before mapping...p points to (nil)
After mapping...p points to 0xffffffffffffffff
Segmentation fault (core dumped)
Address of p after p seems doubtful to me. Looking into gdb indicates segmentation fault at p->state = 1. Is my procedure incorrect or did I miss anything?
when trying to do shmat i get allocation error, telling me i cannot accsess the memory, it did not happen to me before and i really dont know what to do.
this is the error :
0xffffffffffffffff error: Cannot access memory at address 0xffffffffffffffff
and the wiered thing is that vecBoard is allocated in the process mapped area and only get crazy when shmat is triggered. thank you all!
#include <string.h>
#include <stdio.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdlib.h>
#include <sys/fcntl.h>
#include <errno.h>
#include <fcntl.h>
#include <time.h>
#include <signal.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#include <sys/sem.h>
#define SHM_SIZE 4096
#define FLAGS IPC_CREAT
#define COLUMNS 8
#define ROWS 8
key_t key;
int main()
{
char* vecBoard;
// Creating shared memory :
if ((key = ftok("ex31.c",'k')) == -1)
{
perror("ftok");
exit(1);
}
int shm_id;
shm_id=shmget(key,SHM_SIZE,FLAGS);
if(shm_id==-1)
{
printf("error creating shared memory\n");
exit(0);
}
printf("the shared memory segment ID is: %d\n",shm_id);
vecBoard = (char *)shmat(shm_id,0,0);
if((vecBoard = (char *)shmat(shm_id,0,0)) == (char*)-1)
{
printf("error in attaching to the shared memory\n");
exit(0);
}
}
The shown code calls shmat() twice, for the same memory segment.
That's obviously wrong.
Looking at errno would help diagnose the problem (i.e. call perror on failure instead of just printf). The OS gives you information when a system call fails; don't ignore it!
If you carefully read the man page for shmget, the third argument (flags) is supposed to contain, in its low 9 bits, the permissions desired for the shared memory segment. If you just pass IPC_CREAT here, those bits are cleared to 0, so you create a segment for which nobody has either read or write permissions. Thus shmat fails (with EACCES because you are (implicitly) asking to both read and write that segment, and you don't have permission to do either.
You probably want to change it to something like IPC_CREAT | 0600, if you want the current user to be able to access the segment.
Note that before testing this again, you'll probably have to remove the existing shared memory segment with the erroneous permissions. Use the ipcs and ipcrm tools for this.
I'm trying to limit the OS (Ubuntu Server 15.04) to a certain memory usage and reserve the rest but write a kernel module to read/write to the reserved memory. I figured out how to limit the usage/reserve memory using the kernel parameters "mem=4G memmap=4G#0 memmap=4G$4G" (4GB for OS and 4GB reserved, split at 4GB point) but I don't know how DMA to reserved memory works with kernel modules. I was thinking just create a proc file but I'm not sure if you can create one outside of the OS's allocated memory.
Any suggestions? Thanks!
Edit: This is for research so it doesn't need to be "nice"
Update:
Maybe I don't need to write a kernel module. I just found this and I'm going to give it a shot:
http://elinux.org/Memory_Management#Reserving_.28and_accessing.29_the_top_of_memory_on_startup
Update:
I tried the link above but I segfault whenever I try to write. Here's my code:
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/mann.h>
#define RESERVED_MEMORY_SIZE 0x100000000
int main() {
int fd;
char *reserved_memory;
fd = open("/dev/mem", O_RDWR | O_SYNC);
reserved_memory = (char *) mmap(0, RESERVED_MEMORY_SIZE, PROT_READ | PROT_WRITE, MAP_FILE | MAP_SHARED, fd, 4096);
reserved_memory[0] = 'a';
return 0;
}
dmesg shows:
a.out[1167]: segfault at ffffffffffffffff ip 00000000004005d7 sp 00007ffeffccbd80 error 7 in a.out[400000+1000]
For kicks I tried reserved_memory[1]:
a.out[1180]: segfault at 0 ip 00000000004005db sp 00007ffc388d77b0 error 6 in a.out[400000+1000]
I'll look into the format of those messages so I can figure out what it's telling me.
Update:
I found this question by somebody with the same issue as me however the only solution appears to be a kernel rebuild. I'm going to try to avoid this so maybe my best option is a custom kernel module again.
accessing mmaped /dev/mem?
Ok, so I think I solved it. Turns out I just didn't understand how mmap works and I guess the kernel has no restriction on writing/reading /dev/mem if it's in reserved memory. Below are two programs that will write to my reserved spot in memory and read from it.
Write "Hello World!":
#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>
#define RESERVED_MEMORY_OFFSET 0x100000000 /* Offset is 4GB */
int main() {
int fd;
char *reserved_memory;
char *buffer = "Hello World!";
fd = open("/dev/mem", O_RDWR | O_SYNC):
/* Returns a pointer to the 4GB point in /dev/mem - the start of my reserved memory. Only mapping 4096 bytes. */
reserved_memory = (char *) mmap(0, 4096, PROT_READ | PROT_WRITE, MAP_FILE | MAP_SHARED, fd, RESERVED_MEMORY_OFFSET);
if (reserved_memory == MAP_FAILED) {
printf("Failed to creating mapping.\n");
printf("ERRNO: %s\n", strerror(errno));
return -1;
}
sprintf(reserved_memory, "%s", buffer);
return 0;
}
Read from beginning of reserved memory:
#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>
#define RESERVED_MEMORY_OFFSET 0x100000000 /* Offset is 4GB */
int main() {
int fd;
char *reserved_memory;
char buffer[13];
fd = open("/dev/mem", O_RDWR | O_SYNC):
/* Returns a pointer to the 4GB point in /dev/mem - the start of my reserved memory. Only mapping 4096 bytes. */
reserved_memory = (char *) mmap(0, 4096, PROT_READ | PROT_WRITE, MAP_FILE | MAP_SHARED, fd, RESERVED_MEMORY_OFFSET);
if (reserved_memory == MAP_FAILED) {
printf("Failed to creating mapping.\n");
printf("ERRNO: %s\n", strerror(errno));
return -1;
}
snprintf(buffer, 13, "%s", reserved_memory);
printf("%s\n", buffer);
return 0;
}
Special thanks to #knm241!
Have a problem.
I have a file which contents look like number:error_description.
Now i need to put this file to shared memory (POSIX). If any contents are modified it should be saved to the base-file.
There is a need to search in the content in the shared memory (results will be sent to a client over a message queue).
How do I implement all this? First I thought I have to open (fopen("my_file", "r")) and then I have to create shared memory and mmap the file.
Can someone help me?
edit:
#include <stdio.h>
#include <stdlib.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#include <sys/mman.h>
#include <fcntl.h>
#include <unistd.h>
#include <semaphore.h>
/*
* \ /tmp/errors -> Error File
*/
#define MSGQ_HANDLER "/error_handler"
#define PATH_TO_FILE "/tmp/errors"
#define FILE_MODE (S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH)
int main(void) {
int fd = open(PATH_TO_FILE, O_RDWR);
struct stat file_stat;
fstat(fd, &file_stat);
printf("File size: %zd\n", file_stat.st_size);
char *byte_ptr = mmap(NULL, file_stat.st_size, PROT_READ | PROT_WRITE,
MAP_SHARED, fd, 0);
if(byte_ptr == MAP_FAILED){
perror("error:");
}
while(1){
printf("%s\n", byte_ptr);
if(byte_ptr)
exit(1);
}
return EXIT_SUCCESS;
}
So far it is what I have now.
Read a line works.
How do I change the content?
Don't use fopen and forget about shared memory (the sh* API I mean). mmap is all that's needed.
Open your file with open and the right options (read/write). Then use mmap with the option MAP_SHARED. All changes in the file will be reflected directly and visible to all processes that map the same file. On Linux and Solaris (on other systems I don't know, but it is not guaranteed by POSIX or any standard) you can even access the file concurrently with read/write. It is a bad idea though.
Concurrent memory accesses from different processes will, of course, need synchronisation (mutex, semaphores etc.).
Please tell me, why my simple application cannot mmap a small size of memory?
And, why such a specific boundary - 257UL?
// #define MAP_SIZE 256UL or below - fail
// #define MAP_SIZE 257UL - ok
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <errno.h>
#include <signal.h>
#include <fcntl.h>
#include <ctype.h>
#include <termios.h>
#include <sys/types.h>
#include <sys/mman.h>
#define FATAL do { fprintf(stderr, "Error at line %d, file %s (%d) [%s]\n", \
__LINE__, __FILE__, errno, strerror(errno)); exit(1); } while(0)
#define MAP_SIZE 4096UL
#define MAP_MASK (MAP_SIZE - 1)
int main(int argc, char **argv) {
int fd;
void *map_base, *virt_addr;
unsigned long read_result, writeval;
off_t target = strtoul("0x00002000", 0, 0);
if((fd = open("/dev/mem", O_RDWR | O_SYNC)) == -1) FATAL;
printf("/dev/mem opened.\n");
fflush(stdout);
map_base = mmap(0, MAP_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, target & ~MAP_MASK);
if(map_base == (void *) -1) FATAL;
printf("Memory mapped at address %p.\n", map_base);
fflush(stdout);
...
}
mmap works in multiples of the page size on your system. If you're doing this on i386/amd64 or actually most modern CPUs, this will be 4096.
In the man page of mmap on my system it says: "offset must be a multiple of the page size as returned by sysconf(_SC_PAGE_SIZE).". On some systems for historical reasons the length argument may be not a multiple of page size, but mmap will round up to a full page in that case anyway.
Probably you just don't have the rights to write to /dev/mem. This is probably not what you want, mapping all the low end physical memory into your address space.
Have a look into shm_open to open memory segments or MAP_ANONYMOUS to map anonymously.
Edit:
Do a man mem to know what the /dev/mem device node is about:
Byte addresses in mem are interpreted as physical memory
addresses.
References to nonexistent locations cause errors to be returned.
If you want to map to a device node to have a memory segment you should use /dev/zero, but nowadays the tools I describe above should be sufficient.
Then don't, really don't, run such a code with root privileges unless you really know what you are doing. Writing into the physical memory and thus overwriting kernel and userspace data and programs can only lead to catastrophes.