Suppose that you have a binary file. It consists of double's. Its size is enough small to be put in the memory. How to read all of these numbers by using mmap function?
I have tried to dereference output pointer. But it is only the first element of data. To use loop it's non-trivial how to control the quantity of the elements of the array.
int main(int argc, char* argv[]) { // we get filename as an argument from the command line
if (argc != 2)
return 1;
int fd = open(argv[1], O_RDWR, 0777);
size_t size = lseek(fd, 0, SEEK_END);
double m = 0;
int cnt = 0; // counter of doubles
void* mp = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0);
if (mp == MAP_FAILED)
return 1;
double* data = mp;
m += *data; // we want to count the sum of these doubles
++cnt;
int ump_res = munmap(mp, sizeof(double));
if (ump_res < sizeof(double))
return 1;
printf("%a\n", (m / (double)cnt)); // we output the average number of these doubles
close(fd);
return 0;
}
I expect that in stdout we will get the average of all doubles in the file, which name is give in argv[1] .
It is possible to cast void* to double* .Then you may iterate and handle elements:
void* mp = mmap(0, length, PROT_READ, MAP_PRIVATE, fd, 0);
if (mp == MAP_FAILED) {
close(fd);
return 1;
}
double* data = (double*)mp;
size_t cnt = length / sizeof(double);
for (size_t i = 0; i < cnt; ++i) {
m += data[i];
}
Hope you'll find it helpful.
Related
int fd = open(argv[1], O_RDONLY);
struct stat bf;
fstat(fd, &bf);
void *adr = mmap(NULL, bf.st_size, PROT_READ, MAP_PRIVATE, fd, 0);
for (size_t i = 0; i <= bf.st_size; i++)
{
printf("%c", adr[i]); // doesn't work invalid use of void expression
}
I'm trying to print the content a file, but I don't understand why I can't convert
to char*
printf("%c", (char*) adr[i]);
Trying to copy the contents of a file to another file by copying n bytes at a time in c. I believe the code below works for copying one byte at a time but am not sure how to make it work for n number of bytes, have tried making a character array of size n and changing the read/write functions to read(sourceFile , &c, n) and write(destFile , &c, n), but the buffer doesn't appear to work that way.
#include <fcntl.h>
#include <unistd.h>
#include <stdint.h>
#include <time.h>
void File_Copy(int sourceFile, int destFile, int n){
char c;
while(read(sourceFile , &c, 1) != 0){
write(destFile , &c, 1);
}
}
int main(){
int fd, fd_destination;
fd = open("source_file.txt", O_RDONLY); //opening files to be read/created and written to
fd_destination = open("destination_file.txt", O_RDWR | O_CREAT);
clock_t begin = clock(); //starting clock to time the copying function
File_Copy(fd, fd_destination, 100); //copy function
clock_t end = clock();
double time_spent = (double)(end - begin) / CLOCKS_PER_SEC; //timing display
return 0;
}
how to make it work for n number of bytes
Just read N number of bytes and copy that many bytes that you successfully read.
#define N 4096
void File_Copy(int sourceFile, int destFile, int n){
char c[N];
const size_t csize = sizeof(c)/sizeof(*c);
while (1) {
const ssize_t readed = read(sourceFile, c, csize);
if (readed <= 0) {
// nothing more to read
break;
}
// copy to destination that many bytes we read
const ssize_t written = write(destFile, c, readed);
if (written != readed) {
// we didn't transfer everything and destFile should be blocking
// handle error
abort();
}
}
}
You want to copy a buffer of size n at once:
void File_Copy(int sourceFile, int destFile, int n){
char c[n];
ssize_t st;
while((st = read(sourceFile , c, n)) > 0){
write(destFile , c, st);
}
}
Note, that not necessarily n bytes are always copied at once, it might be less. And you also have to check the return value of write() and handle the situation, when less bytes were written, as it fits your needs.
One example is a loop:
while (st > 0) {
int w = write(destFile, c, st);
if (w < 0) {
perror("write");
return;
}
st -= w;
}
Another issue: When you create the destination file here
fd_destination = open("destination_file.txt", O_RDWR | O_CREAT);
you do not specify the third mode parameter. This leads to a random mode, which might lead to this open() to fail the next time. So better add a valid mode, for example like this:
fd_destination = open("destination_file.txt", O_RDWR | O_CREAT, 0644);
This might have distorted your test results.
This is my version using lseek (no loop required):
It relies on read and write always processing the complete buffer and never a part of it (I don't know if this is guaranteed).
void File_Copy(int sourceFile, int destFile)
{
off_t s = lseek(sourceFile, 0, SEEK_END);
lseek(sourceFile, 0, SEEK_SET);
char* c = malloc(s);
if (read(sourceFile, c, s) == s)
write(destFile, c, s);
free(c);
}
The following code does not rely on this assumption and can also be used for file descriptors not supporting lseek.
void File_Copy(int sourceFile, int destFile, int n)
{
char* c = malloc(n);
while (1)
{
ssize_t readStatus = read(sourceFile, c, n);
if (readStatus == -1)
{
printf("error, read returned -1, errno: %d\n", errno);
return;
}
if (readStatus == 0)
break; // EOF
ssize_t bytesWritten = 0;
while (bytesWritten != readStatus)
{
ssize_t writeStatus = write(destFile, c + bytesWritten, readStatus - bytesWritten);
if (writeStatus == -1)
{
printf("error, write returned -1, errno is %d\n", errno);
return;
}
bytesWritten += writeStatus;
if (bytesWritten > readStatus) // should not be possible
{
printf("how did 'bytesWritten > readStatus' happen?");
return;
}
}
}
free(c);
}
On my system (PCIe SSD) I get best performance with a buffer between 1MB and 4MB (you can also use dd to find this size). Bigger buffers don't make sense. And you need big files (try 50GB) to see the effect.
Hello I am trying to back up a vector by mmap.
However, I have tried msync then munmap but it doesn't work. After I write to the (char *) then munmap the file, the file has no content. The mmap file is also created with flag MAP_SHARED. Would really appreciate it if anyone can help.
//update file descriptor
if ((fd = open(filename.c_str(), O_RDWR | S_IRWXU)) < 0) { //| O_CREAT
printf("ERROR opening file %s for writing", filename.c_str());
exit(1);
}
//lseek create a file large enough
off_t i = lseek(fd, frontier_size * URL_MAX_SIZE, SEEK_SET);
if (i != frontier_size * URL_MAX_SIZE) {
cout << "failed to seek";
}
//reposition and write 3 bytes to the file else will failed to read
char buff[3] = "ta";
ssize_t kk = lseek(fd, 0, SEEK_SET);
if (kk < 0) {
cout << "failed to reposition";
}
ssize_t temp_write = write(fd, (void *)& buff, 2);
if (temp_write < 0) {
cout << "failed to write";
cout << temp_write;
}
//reposition to begining
ssize_t k = lseek(fd, 0, SEEK_SET);
if (k < 0) {
cout << "failed to reposition";
}
char * map = (char *)mmap(0, frontier_size * URL_MAX_SIZE, PROT_WRITE, MAP_SHARED, fd, 0);
if (map == MAP_FAILED) {
printf("failed mmap");
exit(1);
}
mmap_frontier = map;
//write to frontier
for (int i = 0; i < frontier.size(); ++i) {
strcpy(mmap_frontier, frontier[i].c_str());
mmap_frontier += URL_MAX_SIZE;
}
mmap_frontier -= frontier.size() * URL_MAX_SIZE;
ssize_t k = lseek(fd, 0, SEEK_SET);
if (k < 0) {
cout << "failed to reposition";
}
int sync = msync((void *)0, frontier.size() * URL_MAX_SIZE, MS_ASYNC);
if (sync < 0 ) {
cout << "failed to sync";
}
int unmap = munmap((void *)0, frontier.size() * URL_MAX_SIZE);
if (unmap < 0) {
cout << "failed to unmap";
}
There are quite a few problems with your code, and with the question:
S_IRWXU is the 3rd argument to open(), not a flag for the 2nd parameter.
mmap() won't work correctly if the file is too small. You can use ftruncte() to set the file size correctly. You tried to seek past the total size of the mapping and write a couple of bytes ("ta"), but before doing that you issued the seek lseek(fd, 0, SEEK_SET) which means the file size was set to 3 rather than mapping_size+3.
You're not backing the vector with an mmapped file, the vector has nothing to do with it, the vector uses its own memory that isn't related in any way to this mapping (please edit your question...).
You called msync() with the address (void *)0, so the actual address which needs to be synced, map, is not being synced.
Likewise, you called munmap() with the address (void *)0, so the actual address which needs to be unmapped is not being unmapped.
You called msync() with MS_ASYNC, which means there's no guarantee that the sync happens before you read the file's contents.
Here's what's working for me (error handling omitted for brevity):
unsigned frontier_size = 2;
const unsigned URL_MAX_SIZE = 100;
int fd = open("data", O_RDWR);
loff_t size = frontier_size * URL_MAX_SIZE;
ftruncate(fd, size);
char *map = (char *)mmap(0, size, PROT_WRITE, MAP_SHARED, fd, 0);
strcpy(map, "hello there");
msync(map, size, MS_SYNC);
munmap(map, size);
close(fd);
I am trying to write multiple random numbers into the memory, but I am having an issue where the for loop does not initilize the number required, and I get an output of one random integer, and random number of spaces and new lines.
int random_range (unsigned const low, unsigned const high) {...}
/* Create and write to a shared file for communication with another process.
*
* argv[1] = file name
*
* Note: Error processing is omitted
*/
int main (int argc, char* const argv[]) {
int fd;
int* file_memory;
int numOfInt;
numOfInt = atoi(argv[2]); //Turning 2nd argument into an integer
/* seed the random number generator */
srand (time (NULL));
/* open or create a file to hold an unsigned integer */
fd = open (argv[1], O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);
/* write FILESIZE spaces */
for (int i=0; i<FILESIZE; i++) write (fd, " ", 1);
write (fd, "", 1); /* write a NULL at EOF */
file_memory = mmap (NULL, FILESIZE, PROT_WRITE, MAP_SHARED, fd, 0);
close (fd);
int* temp = file_memory;
/* write a random integer to memory-mapped area */
for(int x=0; x<numInt; ++x){
if(x!=(numInt-1))
sprintf((char*) file_memory++, "%d ", random_range (-100, 100));
else
sprintf((char*) file_memory++, "%d\n",random_range (-100, 100));
}
file_memory = temp;
/* release the memory */
munmap (file_memory, FILESIZE);
}
I am trying to implement the caesar cipher with mmap. I think the cipher works so fine but the problem is, the mmap. The idea of it is to update the file, if there was a change.
So far it isn't working. I can just read the memory mapped file and print it. But if I make any modificaiton, I get a segmentation fault. Unfortunately, I couldn't solve the problem myself. So, I would appreciate it, if you could help me with it.
Here is the code.
int main (int argc, char *argv[]) {
if(argc != 5)
fprintf(stdeer, "usage: ./cipher (encrypt|decrypt) <file name> (casar| vigenere) <key>\n");
// (encrypt / decrypt) can be found in argv[1]
// filename in argv[2]
// encryption method in argv[3]
// key in argv[4]
int fd = open(argv[2], O_RDWR, S_IWRITE | S_IREAD);
if (fd < 0)
hanle_error("open");
off_t len = lseek(fd, 0, SEEK_END);
if (len == (off_t)-1)
handle_error("lseek");
unsigned char* data = mmap(0, len, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); // Add PROT_WRITE
if (data == MAP_FAILED)
handle_error("mmap");
char c = *argv[4];
int key = 0;
if(strcmp(argv[3], "caesar") == 0) {
key = c - 48;
if(strcmp(argv[1], "decrpyt") == 0)
key = -key;
int num = 0;
for(int size_t i = 0; i < (size_t)len; i++) {
if(data[i] >= 97 && data[i] <= 122) {
num = data[i];
num +=key;
if(num > 'z') {
num -= 26;
data[i] = num + '0';
} else if (num < 'a') {
num += 26;
data[i] = num + '0';
} else {
data[i] = num + '0';
}
} else {
continue;
}
}
}
return 0;
}
A possible input can be anything e.g.
SsWd asdas
qwmkfd aw.
The algorithm above should just modify the lower case letters and leave the rest as it is.
I hope someone might be able to help me.
Also, I only implemented the caesar cipher.
EDIT: The seg fault is gone after I added PROT_WRITE. But know I get weird question marks for the modified lower case letters. Does anyone know why?
If you want to write to the file too, then
unsigned char* data = mmap(0, len, PROT_READ, MAP_SHARED, fd, 0);
should be
unsigned char* data = mmap(0, len, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
That may not be the only issue, but would explain the seg fault.
The reason is self explanatory!