I am working on pulling data from Retroarch via JSON format using software called Gamehook.
I am trying to add support for the PSX with the Beetle PSX HW core.
The issue is that even with extensive searching through the core and retroarch's source code, I cannot find a proper memory map for it. I have tried adding the ranges from the two sources below, which gives me a "no memory map defined" error.
http://www.raphnet.net/electronique/psx_adaptor/Playstation.txt
0x8000_0000 0x801f_ffff Kernel and User Memory Mirror (2 Meg) Cached
0xa000_0000 0xa01f_ffff Kernel and User Memory Mirror (2 Meg) Uncached
0x0000_0000-0x0000_ffff Kernel (64K)
0x0001_0000 0x001f_ffff User Memory (1.9 Meg)
0x1f80_0000-0x1f80_03ff Scratch Pad (1024 bytes)
and
https://psx-spx.consoledev.net/memorymap/
KUSEG KSEG0 KSEG1
00000000h 80000000h A0000000h 2048K Main RAM (first 64K reserved for BIOS)
1F000000h 9F000000h BF000000h 8192K Expansion Region 1 (ROM/RAM)
1F800000h 9F800000h -- 1K Scratchpad (D-Cache used as Fast RAM)
1F801000h 9F801000h BF801000h 8K I/O Ports
1F802000h 9F802000h BF802000h 8K Expansion Region 2 (I/O Ports)
1FA00000h 9FA00000h BFA00000h 2048K Expansion Region 3 (SRAM BIOS region for DTL cards)
1FC00000h 9FC00000h BFC00000h 512K BIOS ROM (Kernel) (4096K max)
FFFE0000h (in KSEG2) 0.5K Internal CPU control registers (Cache Control)
Looking at consoleinfo.c in the Retroarch source code, it defines the memory regions for Playstation as.
/* ===== PlayStation ===== */
/* http://www.raphnet.net/electronique/psx_adaptor/Playstation.txt */
static const rc_memory_region_t _rc_memory_regions_playstation[] = {
{ 0x000000U, 0x00FFFFU, 0x000000U, RC_MEMORY_TYPE_SYSTEM_RAM, "Kernel RAM" },
{ 0x010000U, 0x1FFFFFU, 0x010000U, RC_MEMORY_TYPE_SYSTEM_RAM, "System RAM" }
};
static const rc_memory_regions_t rc_memory_regions_playstation = { _rc_memory_regions_playstation, 2 };
And the definition for rc_memory_region_t is
typedef struct rc_memory_region_t {
unsigned start_address; /* first address of block as queried by RetroAchievements */
unsigned end_address; /* last address of block as queried by RetroAchievements */
unsigned real_address; /* real address for first address of block */
char type; /* RC_MEMORY_TYPE_ for block */
const char* description; /* short description of block */
}
rc_memory_region_t;
I have asked in the Retroarch discord server and they have unfortunately given me conflicting information. Such as, I want to use the RetroAcheievements memory address or that I need to find out how Retroarch deals with memory addressing of the cores.
With that second portion, I have looked at the entire source code in depth and have only found the memory addressing information here.
So my question comes down to, does anyone know either how to find the mappable memory addresses from the cores in Retroarch or does anyone happen to know the mappable memory addresses from the cores in Retroarch?
Related
I use GNU for ARM and want to define some cell in RAM memory space as following:
#define FOO_LOCATION 0x20000000
#define foo (*((volatile uint32_t *) FOO_LOCATION ))
My question is - if such declaration will prohibit usage of the cell with FOO_LOCATION address in stack or heap? What address preffered to avoid memory fragmentation?
Update
I want to place some variable at a certain memory address and access it after watchdog reset. I guess that if i will declare it as usual
uint32_t foo;
it will have another physical location after reset. Also i read a post where said that most probably there is no such way to declare variable adddress. And i have idea to tell the GNU not to use some memory address. As for example special registers are not used by custom variables.
Update 2
In addition to previous definitions i added section to linker script
SECTIONS
{
. = 0x20000000
.fooSection :
{
*(.fooSection)
. = 0x04 /* size = 4 bytes */
}
/* other placements follow here... */
}
I am trying to brush up on C so I have been playing around with the linux kernel's system call table (on 3.13.0-32-generic). I found a resource online that searches for the system call table with the following function which I load into the kernel in an LKM:
static uint64_t **aquire_sys_call_table(void)
{
uint64_t offset = PAGE_OFFSET;
uint64_t **sct;
while (offset < ULLONG_MAX) {
sct = (uint64_t **)offset;
if (sct[__NR_close] == (uint64_t *) sys_close) {
printk("\nsys_call_table found at address: 0x%p\n", sys_call_table);
return sct;
}
offset += sizeof(void *);
}
return NULL;
}
The function works. I am able to use the address it returns to manipulate the system call table. What I don't understand is why the address returned by this function doesn't match the address in /boot/System.map-(KERNEL)
Here is what the function prints:
sys_call_table found at address: 0xffff880001801400
Here is what I get when I search system.map
$ sudo cat /boot/System.map-3.13.0-32-generic | grep sys_call_table
ffffffff81801400 R sys_call_table
ffffffff81809cc0 R ia32_sys_call_table
Why don't the two addresses match? Its my understanding that the module runs in the kernel's address space, so the address of the system call table should be the same.
The two virtual addresses have the same physical address.
From Documentation/x86/x86_64/mm.txt
<previous description obsolete, deleted>
Virtual memory map with 4 level page tables:
0000000000000000 - 00007fffffffffff (=47 bits) user space, different per mm
hole caused by [48:63] sign extension
ffff800000000000 - ffff87ffffffffff (=43 bits) guard hole, reserved for hypervisor
ffff880000000000 - ffffc7ffffffffff (=64 TB) direct mapping of all phys. memory
ffffc80000000000 - ffffc8ffffffffff (=40 bits) hole
ffffc90000000000 - ffffe8ffffffffff (=45 bits) vmalloc/ioremap space
ffffe90000000000 - ffffe9ffffffffff (=40 bits) hole
ffffea0000000000 - ffffeaffffffffff (=40 bits) virtual memory map (1TB)
... unused hole ...
ffffec0000000000 - fffffc0000000000 (=44 bits) kasan shadow memory (16TB)
... unused hole ...
ffffff0000000000 - ffffff7fffffffff (=39 bits) %esp fixup stacks
... unused hole ...
ffffffff80000000 - ffffffffa0000000 (=512 MB) kernel text mapping, from phys 0
ffffffffa0000000 - ffffffffff5fffff (=1525 MB) module mapping space
ffffffffff600000 - ffffffffffdfffff (=8 MB) vsyscalls
ffffffffffe00000 - ffffffffffffffff (=2 MB) unused hole
The direct mapping covers all memory in the system up to the highest
memory address (this means in some cases it can also include PCI memory
holes).
vmalloc space is lazily synchronized into the different PML4 pages of
the processes using the page fault handler, with init_level4_pgt as
reference.
Current X86-64 implementations only support 40 bits of address space,
but we support up to 46 bits. This expands into MBZ space in the page tables.
->trampoline_pgd:
We map EFI runtime services in the aforementioned PGD in the virtual
range of 64Gb (arbitrarily set, can be raised if needed)
0xffffffef00000000 - 0xffffffff00000000
-Andi Kleen, Jul 2004
we know the virtual address space ffff880000000000-ffffc7ffffffffff is direct mapping of all physical memory. When the kernel wants to access all physical memory, it uses direct mapping. It's also what you use for searching.
And the ffffffff80000000-ffffffffa0000000 is kernel text mapping. When the kernel code executed, rip register uses the kernel text mapping.
In arch/x86/include/asm/page_64.h, we can get the relation of virtual address and physical address.
static inline unsigned long __phys_addr_nodebug(unsigned long x)
{
unsigned long y = x - __START_KERNEL_map;
/* use the carry flag to determine if x was < __START_KERNEL_map */
x = y + ((x > y) ? phys_base : (__START_KERNEL_map - PAGE_OFFSET));
return x;
}
and
// arch/x86/include/asm/page_types.h
#define PAGE_OFFSET ((unsigned long)__PAGE_OFFSET)
// arch/x86/include/asm/page_64_types.h
#define __START_KERNEL_map _AC(0xffffffff80000000, UL)
#define __PAGE_OFFSET _AC(0xffff880000000000, UL)
As for the addresses mentioned in the question above:
what the function prints,
sys_call_table found at address: 0xffff880001801400
what system.map gives,
$ sudo cat /boot/System.map-3.13.0-32-generic | grep sys_call_table
ffffffff81801400 R sys_call_table
ffffffff81809cc0 R ia32_sys_call_table
both of them resolve to same physical address.
virt->phys conversion happens in such way that corresponding addresses in 'direct' mapping region and 'kernel text' mapping region resolve to same physical address.
Through the magic of virtual memory mapping, the address you use depends on where you are. The symbol table file System.map is to help attaching a gdb or crash utility to the running system. Inside the kernel, well, is inside the kernel.
You may also have a /proc/kallsym file for even more values :)
Only root can show the addresses in the /proc/kallsyms file! It is rarely disabled but you can enable it if it's disabled. But the addresses in the System.map and kallsyms file for the same sys_call are different.
If a person is using a kernel built by himself, then System.map is preferable but if you are using a pre-built kernel (like we mostly do), then kallsyms is the right place for you!
I was recently delving into the details of Linux's memory management as I want to implement something similar for my own toy kernel, so I was hoping if someone who's familiar with the details could help me understand one thing. Apparently the physical memory manager is a buddy algorithm, which is further specialised to return blocks of pages of a particular order (0 to 9, with 0 being just a single page). For each order the blocks are stored as a linked list. Say if a block of order 5 is requested but is not found on the list of order 5 blocks, the algorithm searches for a block in order 6, splits it into two, gives the requested half and moves the other half an order lower (as it is half in size).
What I don't get is how the kernel stores these structures, or how it allocates space for them. Since for order 0 pages you would need 1M entries (each is a 4KiB page), does it mean that the kernel allocates 1MiB * sizeof(struct page)? What about the blocks of order 1 and above? Does the kernel reuse allocated blocks by marking them as a higher order, and when it needs to split it in two just return the block and get one that is unused?
What I don't get is how the kernel stores these structures, or how it allocates space for them. Since for order 0 pages you would need 1M entries (each is a 4KiB page), does it mean that the kernel allocates 1MiB * sizeof(struct page)?
Initialization of zones is done by calling paging_init() (arch/x86/mm/init_32.c; some descriptions - https://www.kernel.org/doc/gorman/html/understand/understand005.html 2.3 Zone Initialisation and http://repo.hackerzvoice.net/depot_madchat/ebooks/Mem_virtuelle/linux-mm/vminit.html Initializing the Kernel Page Tables) from setup_arch() via (native_pagetable_init() and indirect call 1166 x86_init.paging.pagetable_init();):
690 /*
691 * paging_init() sets up the page tables - note that the first 8MB are
692 * already mapped by head.S.
...*/
697 void __init paging_init(void)
698 {
699 pagetable_init();
...
711 zone_sizes_init();
712 }
pagetable_init() creates kernel page tables in swapper_pg_dir array of 1024 pgd_ts.
zone_sizes_init() actually defines zones of physical memory and calls free_area_init_nodes() to initialize them with actual work done (for each NUMA node for_each_online_node(nid) {...}) in free_area_init_node() which calls three functions:
calculate_node_totalpages() prints page counts for every node in dmesg
alloc_node_mem_map() does actual job of allocating struct page for every physical page in this node; memory for them is allocated by bootmem allocator doc1 doc2 (you can see its debug with bootmem_debug=1 kernel boot option):
4936 size = (end - start) * sizeof(struct page);
4937 map = alloc_remap(pgdat->node_id, size);
if (!map) map = memblock_virt_alloc_node_nopanic(size, pgdat->node_id);
free_area_init_core() (with filling of bitmaps in struct zone). Functionality of free_area_init_core described for older kernels in http://repo.hackerzvoice.net/depot_madchat/ebooks/Mem_virtuelle/linux-mm/zonealloc.html#INITIALIZE as:
free_area_init_core() The memory map is built, and the freelists and buddy bitmaps initialized, in free_area_init_core().
Free lists of orders in each zone are initialized and orders are marked as having no any free page: free_area_init_core() -> init_currently_empty_zone() -> zone_init_free_lists:
4147 static void __meminit zone_init_free_lists(struct zone *zone)
4148 {
4149 unsigned int order, t;
4150 for_each_migratetype_order(order, t) {
4151 INIT_LIST_HEAD(&zone->free_area[order].free_list[t]);
4152 zone->free_area[order].nr_free = 0;
4153 }
4154 }
PS: There is init() in kernel, it is called start_kernel(), and LXR (Linux cross-reference) will help you to navigate between functions (I posted links to lxr.free-electrons.com, but there are several online LXRs):
501 asmlinkage __visible void __init start_kernel(void)
...
528 boot_cpu_init();
529 page_address_init();
530 pr_notice("%s", linux_banner);
531 setup_arch(&command_line);
Do you have idea how to initialize array of structs starting from specific address in memory (not virtual, physical DDR memory). I am working on implementation of TxRx on SoC (ARM-FPGA). Basically ARM (PS) and FPGA (PL) communicate to each other by using shared RAM memory. Currently I am working on transmitter side, so I need to constantly load packets that I get from MAC layer to memory, then my Tx reads data and sends it in air. To achieve this I want to implement circular FIFO buffer on (ARM) side, in way that I can store up to 6 packets into buffer and send them one by one, in same time loading other packets on places of already sent packages. Because I need to use specific memory addresses I am interested is it possible to initialize array of structure that will be stored on specific addresses in memory. For example I want that my array starts at adress 0x400000 and ends at address 0x400000 + MaximumNumberOfPackets x SizeOfPackets I know how to do it for one instantiate of structure for example like this:
buffer_t *tmp = (struct buffer_t *)234881024;
But how to do it for array of structures?
A pointer to a single struct (or int, float, or anything else) is inherently a pointer to an array of them. The pointer type provides the sizeof() value for an array entry, and thus allows pointer arithmetic to work.
Thus, given a struct buffer you can simply do
static struct buffer * const myFIFO = (struct buffer *) 0x40000
and then simply access myFIFO as an array
for (size_t i = 0; i < maxPackets; ++i)
{
buffer[i].someField = initialValue1;
buffer[i].someOtherField = 42;
}
This works just the way you expect.
What you can't do (using pure standard C) is declare an array at a particular address like this:
struct buffer myFIFO[23] # 0x400000;
However, your compiler may have extensions to allow it. Many embedded compilers do (after all, that's often how they declare memory-mapped device registers), but it will be different for every compiler vendor, and possibly for every chip because it is a vendor extension.
GCC does allow it for AVR processors via an attribute, for example
volatile int porta __attribute__((address (0x600)));
But it doesn't seem to support it for an ARM.
Generally #kdopen is right but for arm you should create an entry in MEMORY section linker script that shows to linker where is your memory:
MEMORY
{
...
ExternalDDR (w) : ORIGIN = 0x400000, LENGTH = 4M
}
And than, when you are declaring variable just use the
__attribute__((section("ExternalDDR")))
I found the way how to do it. So could I do it like this. I set this into linker script:
MEMORY {
ps7_ddr_0_S_AXI_BASEADDR : ORIGIN = 0x00100000, LENGTH = 0x1FF00000
ps7_ram_0_S_AXI_BASEADDR : ORIGIN = 0x00000000, LENGTH = 0x00030000
ps7_ram_1_S_AXI_BASEADDR : ORIGIN = 0xFFFF0000, LENGTH = 0x0000FE00
DAC_DMA (w) : ORIGIN = 0xE000000, LENGTH = 64K
}
.dacdma : {
__dacdma_start = .;
*(.data)
__dacdma_end = .;
} > DAC_DMA
And then I set this into code
static buffer_t __attribute__((section("DAC_DMA"))) buf_pool[6];
I want to find the physical address of a variable defined in a user-space process? Is there any way to do it using root privileges?
#include "stdio.h"
#include "unistd.h"
#include "inttypes.h"
uintptr_t vtop(uintptr_t vaddr) {
FILE *pagemap;
intptr_t paddr = 0;
int offset = (vaddr / sysconf(_SC_PAGESIZE)) * sizeof(uint64_t);
uint64_t e;
// https://www.kernel.org/doc/Documentation/vm/pagemap.txt
if ((pagemap = fopen("/proc/self/pagemap", "r"))) {
if (lseek(fileno(pagemap), offset, SEEK_SET) == offset) {
if (fread(&e, sizeof(uint64_t), 1, pagemap)) {
if (e & (1ULL << 63)) { // page present ?
paddr = e & ((1ULL << 54) - 1); // pfn mask
paddr = paddr * sysconf(_SC_PAGESIZE);
// add offset within page
paddr = paddr | (vaddr & (sysconf(_SC_PAGESIZE) - 1));
}
}
}
fclose(pagemap);
}
return paddr;
}
As partially answered before, normal programs should not need to worry about physical addresses as they run in a virtual address space with all its conveniences. Furthermore, not every virtual address has a physical address, the may belong to mapped files or swapped pages. However, sometimes it may be interesting to see this mapping, even in userland.
For this purpose, the Linux kernel exposes its mapping to userland through a set of files in the /proc. The documentation can be found here. Short summary:
/proc/$pid/maps provides a list of mappings of virtual addresses together with additional information, such as the corresponding file for mapped files.
/proc/$pid/pagemap provides more information about each mapped page, including the physical address if it exists.
This website provides a C program that dumps the mappings of all running processes using this interface and an explanation of what it does.
First, why would you want to do this? The purpose of modern VM systems is to remove the application programmer from the complexity of physocal memory layout. Gving them each their own uniform address space to make their life easyer.
If you did want to do this you would almost certanly need to use a kernel module. Get the virtual address of the variable in the normal way, use this to index into the processes page tables and read the value you find(the physical address of the frame). Then add the page offset to get the complete physical address. Note you wont be able to use this address while paging is enabled.
(If your lucky you may be able to get the frame address of a VM region from the /proc file system and thus wouldnt require to write a kernel module.)