I know there are two types of addresses. Virtual and Physical. Printing the address of an integer variable will print its virtual address. Is there a function that will facilitate to print the physical memory of that variable?
Does virtual memory mean a section on hard disk that is treated as RAM by the OS?
No, there is no such (portable) function. In modern operating systems implementing memory protection, user-space (as opposed to kernel-space, i.e. parts of the OS) cannot access physical addresses directly, that's simply not allowed. So there would be little point.
No, virtual memory does not need to involve hard disks, that's "swapping" or "paging". You can implement that once you have virtual memory, since it gives the OS a chance to intervene and manage which pages are kept in physical memory, thus making it possible to "page out" memory to other storage media.
For a very in-depth look at how the Linux kernel manages memory, this blog post is fantastic.
This is a complicated subject.
Physical memory addresses point to a real location in a hardware memory device, whether it be your system memory, graphics card memory, or network card buffer.
Virtual memory is the memory model presented to user-mode processes. Most devices on the system have some virtual memory address space mapped to them, which the processor can write to. When these physical memory addresses are given a virtual memory address, the OS recognises that read/write requests to those addresses need to be serviced by a particular device, and delegates that request off to it.
Related
when a program started the OS will create a virtual memory, which divided into stack, heap, data, text to run a process on it.I know that each segment is used for specification purpose such as text saves the binary code of program, data saves static and global variable. My question is why the OS need to create the virtual memory and divide it into the segments ? How about if OS just use the physical memory and the process run directly on the physical memory. I think maybe the answer is related with running many process at the same time, sharing memory between process but i am not sure. It is kind if you give me an example about the benefit of creating virtual memory and dividing it into the segments.
In an environment with memory protection via a memory mapping unit, all memory is virtual (mapped via the MMU). It's possible to simply map each virtual address linearly to physical addresses, while still using the protection capabilities of the MMU, but doing that makes no sense. There are many reasons to prefer that memory not be directly mappped, such as being able to share program instructions and shared library code between instances of the same program or different programs, being able to fork, etc.
If I have a Memory-mapped I/O device, and I want to write to a register for this device located at address 0x16D34, the 0x16D34 address is actually a virtual address, and the CPU will translate it to a physical address first, and then write the data to the physical address.
But what about Port-mapped I/O devices (for example: a serial port), so if I want to write to a register for a serial port located at address 0x3F8, is the 0x3F8 address a physical address or a virtual address?
Edit: I am on x86 architecture.
Port-mapped I/O on x86/x86-64 (most other modern architectures don't even support it) happens in an entirely separate address space. This address space is not subject to memory mapping, so there are no virtual port addresses, only physical ones. Special in and out instructions must be used to perform port I/O, simple memory access (e.g. with mov) can't access this separate address space. Access protection based on privilege level is possible; most modern OSes prevent user space processes from accessing I/O ports by default.
For details, you can for example check the chapter "INPUT/OUTPUT" of Intel's "Intel® 64 and IA-32 Architectures Developer's Manual: Vol. 1" (chapter 18 as of this writing).
Note that in the early days of x86, port addresses were hardwired in each device, including ISA add-in cards. If you were lucky, the card had a set of jumpers for selecting one of a limited set of possible port ranges for the device, in order to avoid range clashes between devices. Later, Plug & Play was introduced to make the selection dynamically during system boot. PCI further refined this, so that I/O BARs can pretty much be mapped anywhere within the 0x0000-0xffff address space by the operating system and/or firmware. Port-mapped I/O is now strongly discouraged when designing new hardware due to its many inherent limitations.
It seems your question would be the differences between memory-mapped I/O and Port-mapped IO. There normally two methods for processor to connect external devices, which are memory-mapped or port mapped I/O.
Memory-mapped I/O
Memory-mapped I/O uses the same address space to address both memory and I/O devices. So when an address is accessed by the CPU, it may refer to a portion of physical RAM, but it can also refer to memory of the I/O device (Based on Memory-mapped I/O on Wiki).
The value 0x16D34 in your first example would be virtual memory, and would be mapped to the physical memory. The I/O device would refer the same physical memory as well to allow the access from CPU.
Port mapped I/O
Port mapped I/O uses a separate, dedicated address space and is accessed via a dedicated set of microprocessor instructions. For 0x3F8 in your second example, it's the address of the own address specific to Memory and I/O devices. It's not the address shared between memory and I/O devices as we previously mentioned in Memory-mapped I/O. You might get more detailed in Memory-mapped IO vs Port-mapped IO
I'm new in Linux world, and would like to ask a question for this forum:
If I want to access physical address from user space, and I don't have any Kernel driver implementation for this specific hardware device - Can I do it?
I know this is not the "right" way to do so, just want to know if there is a way.
Thanks in advanced!
You cannot in general access physical addresses (from user-space program) on Linux.
However, you might perhaps want to mmap(2) the /dev/mem device (see mem(4) for details).
You'll still use virtual memory with virtual addresses, but you'll happen to see part of the physical RAM.
Read Advanced Linux Programming.
No it is not possible to access an arbitrary physical address from application without a system call. (That means you need an existing driver or you need to write a kernel space driver to do map required register for you)
Mmap also work on virtual address. So if your physical addresses maps to a logical address then only you can get a mapping of the wanted physical address and access it.
Depending on your hardware architecture you might not be able to directly access the address of a register because of the memory protection mechanics and because memory addresses "seen" by the kernel are different from the one seen from a user space process.
You will need some kernel space code that will map back and forth the register address to a memory address that makes sense for your user speca process. Try reading about memory mapping and mmap () related syscalls
I'm working on a Linux kernel project and i need to find a way to allocate Virtual Memory without allocating Physical Memory. For example if I use this :
char* buffer = my_virtual_mem_malloc(sizeof(char) * 512);
my_virtual_mem_malloc is a new SYSCALL implemented by my kernel module. All data written on this buffer is stocked on file or on other server by using socket (not on Physical Memory). So to complete this job, i need to request Virtual Memory and get access to the vm_area_struct structure to redefine vm_ops struct.
Do you have any ideas about this ?
thx
This is not architecturally possible. You can create vm areas that have a writeback routine that copies data somewhere, but at some level, you must allocate physical pages to be written to.
If you're okay with that, you can simply write a FUSE driver, mount it somewhere, and mmap a file from it. If you're not, then you'll have to just write(), because redirecting writes without allocating a physical page at all is not supported by the x86, at the very least.
There are a few approaches to this problem, but most of them require you to first write to an intermediate memory.
Network File System (NFS)
The easiest approach is simply to have the server open some sort of a shared file system such as NFS and using mmap() to map a remote file to a memory address. Then, writing to that address will actually write the OS's page cache, wich will eventually be written to the remote file when the page cache is full or after predefined system timeout.
Distributed Shared Memory (DSM)
An alternative approach is using DSM with a very small cache size.
In computer science, distributed shared memory (DSM) is a form of memory architecture where physically separated memories can be addressed as one logically shared address space.
[...] Software DSM systems can be implemented in an operating system, or as a programming library and can be thought of as extensions of the underlying virtual memory architecture. When implemented in the operating system, such systems are transparent to the developer; which means that the underlying distributed memory is completely hidden from the users.
It means that each virtual address is logically mapped to a virtual address on a remote machine and writing to it will do the following: (a) receive the page from the remote machine and gain exclusive access. (b) update the page data. (c) release the page and send it back to the remote machine when it reads it again.
On typical DSM implementation, (c) will only happen when the remote machine will read the data again, but you might start from existing DSM implementation and change the behavior so that the data is sent once the local machine page cache is full.
I/O MMU
[...] the IOMMU maps device-visible virtual addresses (also called device addresses or I/O addresses in this context) to physical addresses.
This basically means to write directly to the network device buffer, which is actually implementing an alternative driver for that device.
Such approach seems the most complicated and I don't see any benefit from that approach.
This approach is actually not using any intermediate memory but is definitely not recommended unless the system has a heavy realtime requirement.
I understand that each user process is given a virtual address space, and that can be dumped. But is there a way to dump the Physical Address Space? Suppose I have 32-bit system with 4GB memory, can i write a program to print each physical memory location.
I understand it violates memory protection etc. but if its possible how can convert this into a kernel process or lower level process to allow me access to the entire memory..?
I'd like to know how to write such code (if possible) on Windows/Linux platform( or kernel).. OR in case I've to use Assembly or something like that, how to shift to that privilege level.
In Linux, you can open and map the device file /dev/mem (if you have read permission to it). This corresponds to physical memory.
can i write a program to print each physical memory location.
I think no operating system gives the user access to physical memory location. So, you cann't. What ever, you are seeing are virtual addresses produced by the Operating System.
It is possible, on Windows, to access physical memory directly. Some of the things you can do:
Use the Device\PhysicalMemory object -- you can't access all physical memory, and user-mode access to it is restricted starting from Windows Server 2003 SP1.
Use Address Windowing Extensions -- you can control your own virtual-to-physical address mappings, so in a sense you are accessing physical memory directly, although still through page tables.
Write a kernel-mode driver -- there are kernel-mode APIs to access physical memory directly, to allocate physical memory pages, etc. One reason for that is DMA (Direct Memory Access).
None of these methods will give you easy, unrestricted access to any physical memory location.
If I may ask, what are you trying to accomplish?
I'm thinking you could probably do it with a kernel mode driver, but the result would be gibberish as what is in the user section of RAM at the time you grabbed it would be what the OS had paged in, it may be part of one application or a mish mash of a whole bunch. This previous SO question may also be helpful: How does a Windows Kernel mode Driver, access paged memory ?
Try this NTMIO - A WINDOWS COMMAND LINE TO ACCESS HARDWARE RESOURCES http://siliconkit.com/ocart/index.php?route=product/product&keyword=ntmio&category_id=0&product_id=285