mmap PCIe BAR and print content of addr is ffffffff - c

I tried to use mmap() in the linux user mode to map the resource2 file in sysfs to obtain the BAR of the pcie device. The code is shown below.
char *devname = "/sys/bus/pci/devices/0000:04:00.2/resource2";
res_fd = open(devname, O_RDWR);
if (res_fd < 0)
goto fail;
map_addr = mmap(NULL, pci->mem_resource[i].len,
PROT_READ | PROT_WRITE,
MAP_SHARED, res_fd, 0);
if (map_addr == MAP_FAILED)
goto fail;
printf ("%x\n", ((unsigned char*)map_addr)[0x100]);
Finally, the output of this code is ffffffff.
I am very sure that this is abnormal. I checked the datasheet of the device and its output should be a fixed constant, such as 0x37e3cf5.
Actually, I got ffffffff on ubuntu 18.04 kernel 4.19+, but the correct 0x37e3cf5 on ubuntu 18.04 kernel 5.4+.
I want to know what caused the difference of pcie's source file in sysfs. It has tortured me for a long time
Thank you!

I found the answer shortly after the question was posted, it may be caused by AMD’s Secure Memory Encryption function, so I turned off the sme function in the kernel parameters of gurb and everything was normal.

Related

Not able to write in /dev/mem

The issue that I am experimenting is not related with open() or mmap() function, which are executed properly. I have disabled CONFIG_STRICT_DEVMEM in the kernel so I can read from /dev/mem without problems. Actually, I can do the following:
const char *path = "/dev/mem"
int fd = open(path, O_RDWR); /* read and write flags*/
p = mmap(0, SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, BASE_ADDR); /* read and write flags*/
And the code does not fail. Nonetheless, I am using this code to write in the PCI address space. So, basically the BASE_ADDR is 0xc000000, and the size is 256 MiB (0x10000000, all the PCI address space).
Said that, when I try to write on these positions (with a specific offset, BDF format), nothing is written; again the code does not fail, it just does not write anything.
In case my code was wrong, I tried BusyBox, with the following parameters:
[horro# ~]$ sudo busybox devmem 0xc00b0a8c w 0xffffffff
[horro# ~]$ sudo busybox devmem 0xc00b0a8c
0x00000000
So, basically it is not writing anything.
There is a CONFIG_STRICT_DEVMEM kernel config option. My understanding is that it must be set at compile-time as CONFIG_STRICT_DEVMEM=n. This is a security reasons.

Mapping physical memory through /dev/mem fails despite MAP_SHARED, CONFIG_STRICT_DEVMEM not set and nopat

I am trying to access physical address through the /dev/mem without success. I can access the address space reserved for PCI devices but when I try to map my memory I get error. (I have mapped the virtual to physical through the pagemap interface).
I have added the nopat to the kernel command line and I am running my program as root.
virtual address: 0x7f925a266000
physical address: 0x1d3a66000
I have also tried aligning it to huge page boundaries without success.
Using the following code, mmap returns -1.
int *addr;
if ((fd = open("/dev/mem", O_RDWR|O_SYNC)) < 0 ) {
printf("Error opening file. \n");
close(fd);
return (-1);
}
addr = (int *)mmap(0, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE, fd, 0x1d3a66000);
//error message: Operation not permitted, no logs with dmesg
printf("addr: %p \n",addr);
printf("addr: %d \n",*addr); /* CRASH. */
Any ideas on how I can make it work, or if it's not possible through /dev/mem is there other way to map physical addresses? I am running Ubuntu with the latest kernel version.
Edit:
I have recompiled the kernel without the strict_devmem as #yano suggested and the problem is fixed.

mmap() fails while devmem2 succeeds (C/CPP) [Allwinner A20]

I am trying to access the hardware registers of an A20 SOM by mapping them to userspace. In this case, target is the PIO, listed at physical address 0x01C20800.
The official Olimex Debian7 (wheezy) image is being used. Kernel Linux a20-olimex 3.4.90+
I was able to verify the location by using the devmem2 tool and Allwinner's documentation on the said memory space (switched the pinmode and level with devmem).
The mmap call on the other hand
*map = mmap(
NULL,
BLOCK_SIZE, // = (4 * 1024)
PROT_READ | PROT_WRITE,
MAP_SHARED,
*mem_fd,
*addr_p
);
fails with mmap error: Invalid argument
Here's a more complete version of the code: http://pastebin.com/mfEuVdbJ
Don't worry about the pointers as the same code does work when accessing UART0 at 0x01C28000.
Although only UART0 (and UART4), which is used as serial console.
I've decompiled the script.bin (still in use despite DTB) without success, as UART 0, 7 and 8 are enabled there.
I am also logged in as user root
I would still guess something related to permissions but I'm pretty lost right now since devmem has no problem at all
> root#a20-olimex:~# devmem2 0x01c20800 w /dev/mem opened. Memory mapped
> at address 0xb6f85000.
While sourcejedi didn't certainly fix my issue, he gave me the right approach.
I took a look at the forementioned devmem tool's source to discover that the mmap call's address is masked
address & ~MAP_MASK to get the entire page, which is essentially the same operation as in my comment.
However, to get back to the right place after the mapping has been done, you have to add the mask back
final_address = mapped_address + (target_address & MAP_MASK);
This resulted in following code (based on OP's pastebin)
Where
MAP_MASK = (sysconf(_SC_PAGE_SIZE) - 1) in this case 4095
int map_peripheral(unsigned long *addr_p, int *mem_fd, void **map, volatile unsigned int **addr)
{
if (!(*addr_p)) {
printf("Called map_peripheral with uninitilized struct.\n");
return -1;
}
// Open /dev/mem
if ((*mem_fd = open("/dev/mem", O_RDWR | O_SYNC)) < 0) {
printf("Failed to open /dev/mem, try checking permissions.\n");
return -1;
}
*map = mmap(
NULL,
MAP_SIZE,
PROT_READ | PROT_WRITE,
MAP_SHARED,
*mem_fd, // file descriptor to physical memory virtual file '/dev/mem'
*addr_p & ~MAP_MASK // address in physical map to be exposed
/************* magic is here **************************************/
);
if (*map == MAP_FAILED) {
perror("mmap error");
return -1;
}
*addr = (volatile unsigned int *)(*map + (*addr_p & MAP_MASK));
/************* and here ******************************************/
return 0;
}
if you read the friendly manual
EINVAL Invalid argument (POSIX.1)
is the error code. (Not EPERM!). So we look it up for the specific function
EINVAL We don't like addr, length, or offset (e.g., they are too large,
or not aligned on a page boundary).
BLOCK_SIZE, // 1024 - ?
You want a multiple of sysconf(_SC_PAGE_SIZE). In practice it will be 4096. I won't bother with the fully general math to calculate it - you'll find examples if you need it.

mmap: Operation not permitted

I am trying to use mmap in user space to read the physical memory where 'mem_map' starts. It's an array that contains all the physical pages. This is a i386 machine running 3.0 kernel.
The code is like this:
....
//define page size
//
#define PAGE_SIZE 0x1000 //4096 bytes
#define PAGE_MASK (PAGE_SIZE - 1)
....
/* open /dev/mem file*/
if((fd = open("/dev/mem", O_RDWR | O_SYNC)) == -1) {
printf("/dev/mem could not be opened.\n");
perror("open");
exit(1);
} else {
printf("/dev/mem opened.\n");
}
/* Map one page */
printf(" mem_map is at physical addr: 0x%x\n", mem_map_phy_addr);
map_base = mmap(0, PAGE_SIZE, PROT_READ, MAP_SHARED, fd, (mem_map_phy_addr & ~PAGE_MASK)); //mem_map_phy_addr is at 0x356f2000
if(map_base == (void *) -1) {
printf("Memory map failed. err num = %d\n",errno);
perror("mmap"); //failed here
} else {
printf("Memory mapped at address %p.\n", map_base);
}
I ran this as a root. The output is:
/dev/mem opened.
mem_map is at physical addr: 0x356f2000
Memory map failed. err num = 1
mmap: Operation not permitted
To be sure, I googled the problem and added the following line to my /etc/sysctl.conf file:
vm.mmap_min_addr = 0
But this doesn't work either.
Anyone knows why mem_map operation like this is not permitted and how I can get around it?
Thanks.
It sounds like the kernel has been compiled with CONFIG_STRICT_DEVMEM enabled. This is a security feature to prevent user space access to (possibly sensitive) physical memory above 1MB (IIRC). You might be able to disable this with sysctl dev.mem.restricted.
I had a similar problem which occured when I was trying use flashrom on an APU2c4 Board with Arch Linux.
The sysctl option dev.mem.restricted wasn't available in my system and using a self compiled kernel was no option for me.
I worked around the problem by setting the iomem Kernelparameter to relaxed via Grub:
# /boot/grub/grub.cfg
linux /boot/vmlinuz-linux iomem=relaxed
Of course a reboot is nessesary for this solution.
Reference:
https://www.reddit.com/r/libreboot/comments/6wvyry/flashrom_failures_to_access/
https://www.flashrom.org/FAQ
https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt

mmap on /proc/pid/mem

Has anybody succeeded in mmap'ing a /proc/pid/mem file with Linux kernel 2.6? I am getting an ENODEV (No such device) error. My call looks like this:
char * map = mmap(NULL, PAGE_SIZE, PROT_READ, MAP_SHARED, mem_fd, offset);
And I have verified by looking at the /proc/pid/maps file while debugging that, when execution reaches this call, offset has the value of the top of the stack minus PAGE_SIZE. I have also verified with ptrace that mmap is setting errno to ENODEV.
See proc_mem_operations in /usr/src/linux/fs/proc/base.c: /proc/.../mem does not support mmap.

Resources