This question already has answers here:
How to change characters in a text file using C's mmap()?
(2 answers)
Closed 2 years ago.
I am trying to read and write a struct using mmap, however the changes I do after the mmap are not being persisted to the disk or are not being loaded correctly.
Here's my code sample, when run the first time the file is created and the prints show the correct data, on the second time when the file already exists the struct is empty, filled with 0's. So it looks like the changes were not written to the file but I am having trouble figuring out why.
#include <stddef.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <errno.h>
#include <unistd.h>
typedef struct {
int age;
char name[128];
} person;
int main(int argc, char *argv[argc]){
char filename [] = "data.person";
int fd = open(filename, O_RDWR | O_CREAT , S_IRWXU);
if (fd == -1) {
printf("Failed to create version vector file, error is '%s'", strerror(errno));
exit(1);
}
struct stat st;
fstat(fd, &st);
person *p;
if (st.st_size == 0) {
ftruncate(fd, sizeof(person));
p = (person *) mmap(0, sizeof(person), PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0);
strcpy(p[0].name, "Hello");
p[0].age = 10;
msync(p, sizeof(person), MS_SYNC);
}else{
p = (person *) mmap(0, st.st_size, PROT_READ, MAP_PRIVATE, fd, 0);
if( p == MAP_FAILED){
printf("mmap failed with error '%s'\n", strerror(errno));
exit(0);
}
}
printf("%s %d\n", p->name, p->age);
munmap(p, sizeof(person));
close(fd);
}
My OS is manjaro 20 and the gcc version is 10.1
Do not use MAP_PRIVATE because:
Create a private copy-on-write mapping. Updates to the
mapping are not visible to other processes mapping the same
file, and are not carried through to the underlying file. It
is unspecified whether changes made to the file after the
mmap() call are visible in the mapped region.
Related
I'm trying to mmap a file for reading and writing, and currently I am receiving errcode 19: ENODEV. I'm using a blank file with 2MB of space.
According to the man pages, this error occurs when the file is not compatible for mapping. If this is true, I don't understand why; It's just a regular empty file. Additionally, it was provided by my professor, so presumably, it should work fine. Is anything else going on?
Here is my code:
#include <stdio.h>
#include <sys/mman.h>
#include <fcntl.h>
#include <stdlib.h>
#include <errno.h>
int main(int argc, const char * argv[]) {
int fd = open("Drive2MB", O_RDWR, S_IRUSR | S_IWUSR | S_IXUSR);
if(fd < 0) {
printf("open failure\n");
exit(1);
}
char *drive = mmap(NULL, 1048576, PROT_WRITE, fd, MAP_SHARED, 0);
if(drive == MAP_FAILED) {
printf("map failure\n");
fprintf(stderr, "Value of errno: %d\n", errno);
exit(1);
}
//test
drive[513] = 'p';
printf("%c", drive[513]);
}
If it is indeed the file, how would I go about creating a file that is compatible with mmap().
Two mistakes:
char *drive = mmap(NULL, 1048576, PROT_WRITE, fd, MAP_SHARED, 0);
should be
char *drive = mmap(NULL, 1048576, PROT_WRITE, MAP_SHARED, fd, 0);
and you can't map an empty file but only a file that has the length already.
To populate Drive2MB do dd if=/dev/zero of=Drive2MB bs=1048576 count=1
MAP_SHARED happens to have the value of 1 so it tried to mmap stdout which is typically a terminal, which cannot be mapped.
I'm new in the area of memory mapped and I was wondering if there is any way I can read a text file using memory mapped to a string. I don't really know how to start to write the code.
The general idea with memory mapped I/O is that you tell the operating system (OS) what file you want, and it (after doing some amount of set-up work) tells you where that file now is in memory.
Once that contract is performed, you should be able to copy things to and from that memory in any way you wish (such as with memcpy), and it will magically handle the I/O for you.
Detail depends on which OS you're using since the ISO C standard doesn't specify this behaviour - it's therefore OS-specific.
For example, Windows uses a file mapping paradigm shown here, while Linux uses mmap to allow you to subject a file you've already opened to memory mapping (via its file descriptor).
By way of example, this Linux program, a little voluminous due mainly to its error checking and progress reports, memory maps the file.txt file and outputs its content:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <sys/mman.h>
#include <fcntl.h>
#include <unistd.h>
// Helper function to minimise error code in main.
static int clean(
int retVal, // value to return.
char *err, // error/NULL, allows optional %s for strerror(errno).
int fd, // fd/-1 to close.
void *filMem, // memory/NULL to unmap.
off_t sz, // size if unmapping.
void *locMem // memory/NULL to free.
) {
if (err) printf (err, strerror(errno));
if (locMem) free(locMem);
if (filMem) munmap(filMem, sz);
if (fd >= 0) close(fd);
return retVal;
}
int main(void) {
int fd = open("file.txt", O_RDONLY);
if (fd < 0) return clean(-1, "Can't open: %s\n", -1, NULL, 0, NULL);
printf("File opened okay, fd = %d.\n", fd);
off_t sz = lseek(fd, 0, SEEK_END);
if (sz == (off_t) -1) return clean(-1, "Can't seek: %s\n", fd, NULL, 0, NULL);
printf("File size is %ld.\n", sz);
void *fileArea = mmap(NULL, sz, PROT_READ, MAP_SHARED, fd, 0);
if (! fileArea) return clean(-1, "Can't map: %s\n", fd, NULL, 0, NULL);
printf("File mapped to address %p.\n", fileArea);
char *localArea = calloc(1, sz + 1);
if (! localArea) return clean(-1, "Can't allocate\n", fd, fileArea, sz, NULL);
memcpy(localArea, fileArea, sz);
printf("Data copied to %p, result is [\n%s]\n", localArea, localArea);
return clean(0, NULL, fd, fileArea, sz, localArea);
}
Running that on my local system, the results can be seen from the following transcript:
pax$ cat file.txt
...This is the input file content.
pax$ ./testprog
File opened okay, fd = 3.
File size is 35.
File mapped to address 0x7f868a93b000.
Data copied to 0x1756420, result is [
...This is the input file content.
]
I have a problem considering the usage of mmap. I am trying to map a pci device to a virtual address and read its content. In the future I am planning to write values to it as well.
The problem is that I (seemingly) successfully mapped the device to virtual memory space. However when I read the content of that virtual address all values are zero, despite the file not being empty.
Here is my code:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <errno.h>
#include "../include/types.h"
#include "../include/pci.h"
#define PRINT_ERROR \
do { \
fprintf(stderr, "Error at line %d, file %s (%d) [%s]\n", \
__LINE__, __FILE__, errno, strerror(errno)); exit(1);\
} while(0)
#define MAP_SIZE 4069
#define MAP_MASK (MAP_SIZE - 1)
int main(int argc, char *argv[])
{
int pci_dev;
int *mmap_base;
int *content;
char file[] = {"/sys/bus/pci/devices/0000:04:00.0/resource"};
int i;
printf("File to be read from: %s\n", file);
pci_dev = open(file, O_RDONLY);
if (pci_dev < 0)
{
PRINT_ERROR;
}
mmap_base = mmap(NULL, MAP_SIZE, PROT_READ, MAP_PRIVATE | MAP_ANON, pci_dev, 0);
if (mmap_base == (void *)-1 || mmap_base == NULL)
{
PRINT_ERROR;
}
printf("Mapped on address %p of size %d Byte\n", mmap_base, (int)MAP_SIZE);
content = (int *)mmap_base;
for(i = 0; i < 1024; i++)
{
printf("%x", content[i]);
}
return 0;
}
Here's the content of the first line from the file "/sys/bus/pci/devices/0000:04:00.0/resource" that I am trying to access:
0x00000000cd000000 0x00000000cd07ffff 0x0000000000040200
However the output I get is:
File to be read from: /sys/bus/pci/devices/0000:04:00.0/resource
Mapped on address 0xb7705000 of size 4096 Byte
0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000...
Am I doing something wrong? Every help is appreciated!
Actually you've got 2 mistakes:
Don't use MAP_ANON when you create a map for a real file on the file system, it's meant for IPC and requiring extra memory from OS e.g. while malloc().
When you remove the flag, the mmap() will likely return ENODEV, because linux sysfs doesn't support mmaping; So you have to use read()'s here.
I am currently using shmget and shmat to create a shared memory between two process. When the process die the shared memory is still alive and restarting the process mean we can start where we were before. But if the machine is turned off then turned on we are losing the data.
I would like to know if there was an option in shmget/shmat or another method to make shared memory between process in order to keep the data alive even in the case of a reboot.
Right now i am doing this kind of thing :
const char *ZoneFile = "/home/Zone.dat";'
key_t sharedKeyZone;
int sharedSpaceIdZone;
int descriptor = open(ZoneFile, O_CREAT | O_RDWR, S_IRWXU);
close(descriptor);
sharedKeyZone = ftok(ZoneFile, 1);
sharedSpaceIdZone = shmget(sharedKeyZone, 1 * sizeof(Zone_t), IPC_CREAT);
ZoneArray = (Zone_t *) shmat(sharedSpaceIdZone, NULL, 0);
Zone_t being a structure type, i can access every data in ZoneArray[0] from my 2 process without problem.
Right now the only solution i can think of would be to periodicaly write a ini file with the data to "save" the state of the system and when restart it read this file but this would be not flexible at all if the structure must evolve later.
EDIT : following the idea of #Wumpus Q. Wumbley i tried to use mmap with msync this way :
#include <stdio.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <fcntl.h>
#include <unistd.h>
#include <semaphore.h>
//add rt to eclipse librairies
typedef struct count {
int counter;
} count;
int main()
{
count *memory;
int fd = shm_open("MYmemory.txt", O_CREAT | O_RDWR, S_IRWXU);
if(fd == -1)
{
perror("shm_open");
return 1;
}
ftruncate(fd, sizeof(count));
memory = mmap(NULL, sizeof(count), PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
printf("before %d\n", memory->counter);
memory->counter = 5;
printf("after %d\n", memory->counter);
if(msync(memory, sizeof(count),MS_SYNC)<0)
{
printf("%s","msync ERROR.");
}
else { printf("%s","msync completed successfully.");}
return 0;
}
Same result as with shmget and shmat, after a reboot the data are 0. (the printf "before" show 0)
EDIT 2 :
This did it for me :
count *memory;
int fd = open(MYmemoryFile, O_CREAT | O_RDWR, S_IRWXU);
ftruncate(fd, sizeof(count));
memory = mmap(NULL, sizeof(count), PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
printf("before %d\n", memory->counter);
memory->counter = 5;
printf("after %d\n", memory->counter);
if(msync(memory, sizeof(count),MS_SYNC)<0)
{
printf("%s","msync ERROR.");
}
else { printf("%s","msync completed successfully.\n");}
Still not perfect since everywhere in my former code i was using my memory without pointer form (Zone[0].param is now Zone[0]->param for example) but this is a step forward, thanks to #Wumpus Q. Wumbley.
EDIT 3 : If someone search about it later, here is how i fixed it to work with structure :
#include <stdio.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <fcntl.h>
#include <unistd.h>
#include <semaphore.h>
//add rt to eclipse librairies
const char *MYmemoryFile = "/home/memorystruct.txt";
typedef struct count
{
int counter;
int test;
} match[5];
int main()
{
int fd = open(MYmemoryFile, O_CREAT | O_RDWR, S_IRWXU);
ftruncate(fd, sizeof(match));
struct count *memory = mmap(NULL, sizeof(match), PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
printf("before memory[0].counter %d\n", memory[0].counter);
printf("before memory[0].counter %d\n", memory[0].test);
printf("before memory[1].counter %d\n", memory[1].counter);
memory[0].counter = 5;
memory[0].test = 2;
memory[1].counter = 17;
printf("after memory[0].counter %d\n", memory[5].counter);
printf("after memory[1].counter %d\n", memory[1].counter);
if(msync(memory, sizeof(match),MS_SYNC)<0)
{
printf("%s","msync ERROR.");
}
else
{
printf("%s","msync completed successfully.\n");
}
return 0;
}
If you mmap a regular file with MAP_SHARED, you effectively have a shm segment that is also a permanent file. The copy on disk will lag behind the copy in memory, but you can call msync to flush it periodically.
So I'm using mmap to then write to another file. But the weird thing is, when my code hits mmap, what it does is clears the file. So I have a file that's populated with random characters (AB, HAA, JAK, etc...). What it's supposed to do is use mmap as read basically and then write that file to the new file. So that first if (argc == 3) is the normal read and write, the second if (argc ==4) is supposed to use mmap. Does anyone have any idea why on Earth this is happening?
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/io.h>
#include <sys/mman.h>
#include <sys/time.h>
#include <sys/resource.h>
int main(int argc, char const *argv[])
{
int nbyte = 512;
char buffer[nbyte];
unsigned char *f;
int bytesRead = 0;
int size;
int totalBuffer;
struct stat s;
const char * file_name = argv[1];
int fd = open (argv[1], O_RDONLY);
int i = 0;
char c;
int fileInput = open(argv[1], O_RDONLY);
int fileOutPut = open(argv[2], O_WRONLY | O_TRUNC | O_CREAT, S_IRUSR | S_IWUSR);
fstat(fileInput, &s);
size = s.st_size;
printf("%d\n", size);
if (argc == 3)
{
printf("size: %d\n", size);
printf("nbyte: %d\n", nbyte);
while (size - bytesRead >= nbyte)
{
read(fileInput, buffer, nbyte);
bytesRead += nbyte;
write(fileOutPut, buffer, nbyte);
}
read(fileInput, buffer, size - bytesRead);
write(fileOutPut, buffer, size - bytesRead);
}
else if (argc == 4)
{
int i = 0;
printf("4 arg\n");
f = (char *) mmap (0, size, PROT_READ, MAP_PRIVATE, fileInput, 0);
/* This is where it is being wipped */
}
close(fileInput);
close(fileOutPut);
int who = RUSAGE_SELF;
struct rusage usage;
int ret;
/* Get the status of the file and print some. Easy to do what "ls" does with fstat system call... */
int status = fstat (fd, & s);
printf("File Size: %d bytes\n",s.st_size);
printf("Number of Links: %d\n",s.st_nlink);
return 0;
}
EDIT: I wanted to mention that the first read and write works perfectly, it is only when you try to do it through the mmap.
If you mean it's clearing your destination file, then yes, that's exactly what your code will do.
It opens the destination with truncation and then, in your argc==4 section, you map the input file but do absolutely nothing to transfer the data to the output file.
You'll need a while loop of some description, similar to the one in the argc==3 case, but which writes the bytes in mapped memory to the fileOutput descriptor.