I'm implementing RTEMS driver for Ethernet card by porting it from Linux. Much of the work is done, processor IO mode is working ok, as well as interrupt handling. Now I'm having problems implementing DMA.
Specifically, in Linux driver I use as a base, function dma_alloc_coherent() is used. This function will return two different addresses: one is address that driver code (host CPU) will see, and other one is the address that card will use to access same memory region via PCI during DMA.
I'm having problems finding appropriate replacement function. First I thought of using malloc() and then pci_pci2cpu to translate this address to the one one card can access, however, pci_pci2cpu returns 0xFFFFFFFF for IO and 0x0 for remaining two modes.
Second approach I considered is using dual ported memory manager, but I'm not finding useful examples of it's usage. For example, rtems_port_create() function requires pointers *internal_start and *external_start to be provided, but I'm not sure where are this pointers comming from?
I use Gaisler RTEMS version 4.11 and Sparc architecture (LEON3 cpu).
Best,
Ivan
Ok basically I got this figured out.
First, RTEMS has flat memory model, so address that malloc() returns is actual physical address in memory. That means I don't need dma_alloc_coherent() as malloc() is already doing same thing. For aligned memory, I used posix_memalign() which is also supported.
Second, I needed to see is there any address translation between the card and memory. This is not related to RTEMS, but rather to the system architecture, so after looking into GRLIB user manual and looking at RTEMS initialization code for grpci2 core, I found that there is no memory translation (it's set 1:1).
Bottom line is that if I allocate buffer with simple malloc and give that address to PCI card, it will be able to access (read/write) this buffer.
These were all assumptions that I started with, but in the end my problems were in faulty DMA chip. :)
I'm not sure I got the question right but anyway:
RTEMS does not implements an handler for the LEON family DMA.
To use the DMA you need to exploit the LEON structure you can find in leon.h header file.
That structure is linked to the memory addresses of the LEON3 processor.
Alternatively you can address the registers directly.
After that you need to go to http://www.gaisler.com/index.php/products/components/ut699
and download the functional manual of the UT699 (or search for the SoC you are using :) )
There you will find how to write the registries in the correct order to initiate a DMA transfer from/to PCI target.
Cheers
Related
I am re-implementing mmap in a device driver for DMA.
I saw this question: Linux Driver: mmap() kernel buffer to userspace without using nopage that has an answer using vm_insert_page() to map one page at a time; hence, for multiple pages, needed to execute in a loop. Is there another API that handles this?
Previously I used dma_alloc_coherent to allocate a chunk of memory for DMA and used remap_pfn_range to build a page table that associates process's virtual memory to physical memory.
Now I would like to allocate a much larger chunk of memory using __get_free_pages with order greater than 1. I am not sure how to build page table in that case. The reason is as follows:
I checked the book Linux Device Drivers and noticed the following:
Background:
When a user-space process calls mmap to map device memory into its address space, the system responds by creating a new VMA to represent that mapping. A driver that supports mmap (and, thus, that implements the mmap method) needs to help that process by completing the initialization of that VMA.
Problem with remap_pfn_range:
remap_pfn_range won’t allow you to remap conventional addresses, which include the ones you obtain by calling get_free_page. Instead, it maps in the zero page. Everything appears to work, with the exception that the process sees private, zero-filled pages rather than the remapped RAM that it was hoping for.
The corresponding implementation using get_free_pages with order 0, i.e. only 1 page in scullp device driver:
The mmap method is disabled for a scullp device if the allocation order is greater than zero, because nopage deals with single pages rather than clusters of pages. scullp simply does not know how to properly manage reference counts for pages that are part of higher-order allocations.
May I know if there is a way to create VMA for pages obtained using __get_free_pages with order greater than 1?
I checked Linux source code and noticed there are some drivers re-implementing struct dma_map_ops->alloc() and struct dma_map_ops->map_page(). May I know if this is the correct way to do it?
I think I got the answer to my question. Feel free to correct me if I am wrong.
I happened to see this patch: mm: Introduce new vm_map_pages() and vm_map_pages_zero() API while I was googling for vm_insert_page.
Previouly drivers have their own way of mapping range of kernel pages/memory into user vma and this was done by invoking vm_insert_page() within a loop.
As this pattern is common across different drivers, it can be generalized by creating new functions and use it across the drivers.
vm_map_pages() is the API which could be used to mapped kernel memory/pages in drivers which has considered vm_pgoff.
After reading it, I knew I found what I want.
That function also could be found in Linux Kernel Core API Documentation.
As for the difference between remap_pfn_range() and vm_insert_page() which requires a loop for a list of contiguous pages, I found this answer to this question extremely helpful, in which it includes a link to explanation by Linus.
As a side note, this patch mm: Introduce new vm_insert_range and vm_insert_range_buggy API indicates that the earlier version of vm_map_pages() was vm_insert_range(), but we should stick to vm_map_pages(), since under the hood vm_map_pages() calls vm_insert_range().
I am working in an Embedded Linux system which has the 2.6.35.3 kernel.
Within the device we require a 4MB+192kB contiguous DMA capable buffer for one of our data capture drivers. The driver uses SPI transfers to copy data into this buffer.
The user space application issues a mmap system call to map the buffer into user space and after that, it directly reads the available data.
The buffer is allocated using "alloc_bootmem_low_pages" call, because it is not possible to allocate more than 4 MB buffer using other methods, such as kmalloc.
However, due to a recent upgrade, we need to increase the buffer space to 22MB+192kB. As I've read, the Linux kernel has only 16MB of DMA capable memory. Therefore, theoretically this is not possible unless there is a way to tweak this setting.
If there is anyone who knows how to perform this, please let me know?
Is this a good idea or will this make the system unstable?
The ZONE_DMA 16MB limit is imposed by a hardware limitation of certain devices. Specifically, on the PC architecture in the olden days, ISA cards performing DMA needed buffers allocated in the first 16MB of the physical address space because the ISA interface had 24 physical address lines which were only capable of addressing the first 2^24=16MB of physical memory. Therefore, device drivers for these cards would allocate DMA buffers in the ZONE_DMA area to accommodate this hardware limitation.
Depending on your embedded system and device hardware, your device either is or isn't subject to this limitation. If it is subject to this limitation, there is no software fix you can apply to allow your device to address a 22MB block of memory, and if you modify the kernel to extend the DMA address space beyond 16MB, then of course the system will become unstable.
On the other hand, if your device is not subject to this limitation (which is the only way it could possibly write to a 22MB buffer), then there is no reason to allocate memory in ZONE_DMA. In this case, I think if you simply replace your alloc_bootmem_low_pages call with an alloc_bootmem_pages call, it should work fine to allocate your 22MB buffer. If the system becomes unstable, then it's probably because your device is subject to a hardware limitation, and you cannot use a 22MB buffer.
It looks like my first attempt at an answer was a little too generic. I think that for the specific i.MX287 architecture you mention in the comments, the DMA zone size is configurable through the CONFIG_DMA_ZONE_SIZE parameter which can be made as large as 32Megs. The relevant configuration option should be under "System Type -> Freescale i.MXS implementations -> DMA memory zone size".
On this architecture, it's seems safe to modify it, as it looks like it's not addressing a hardware limitation (the way it was on x86 architectures) but just determining how to lay out memory.
If you try setting it to 32Meg and testing both alloc_bootmem_pages and alloc_bootmem_low_pages in your own driver, perhaps one of those will work.
Otherwise, I think I'm out of ideas.
I am planning to put OS on my ARM board to get extra facility like memory management , File system etc..
For example if I want want to write a value to port x to blink led I can write (int*)0x00458=1 without OS . But if with OS if I write the same (int*)0x00458=1 .
The memory will be of user space virtual memory or real memory address .
Sorry I don't know how to express my question pardon me for that .
My goal is to put OS on my ARM board and write LED blink program with out using driver library.
It depends on the os, with linux you can use mmap to ask the operating system to map you a hole, with permissions, to punch through from the application layer to that physical address.
Windows there was the giveio thing for I/O, and likely a way to write a kernel driver for punching through with memory mapped I/O.
It is heavily operating system dependent so you have look at your operating system. You tagged linux so start with mmap, there should already be some examples of how to do this in stackoverflow answers. As well as elsewhere, only takes a few lines of code to get the pointer (note asking for a larger space like 0x10000000 bytes is more likely to get a pointer than 0x1000 bytes for example).
since you want to write 1 to memory this (int*)0x00458=1 is incorrect.
you must use
*(volatile int*)0x00458=1
to write in memory:
use volatile to prevent compiler from optimizing your code out.
this used to blibk a LED on port address 0x65 for example:
#include <stdint.h>
while (1)
{
//PORTG^=1;
*(volatile uint32_t*)0x65 ^= 1;
//delay
}
this instruction will compile to correct assembly code, no matter there is OS or not.
the only note for os is:
if your ARM CPU hase remapping for address you must find new address, again this instruction will be correct. for details about your os, you may read os documentation.
Hope this helps.
I was reading a paragraph from the "The Linux Kernel Module Programming Guide" and I have a couple of doubts related to the following paragraph.
The reason for copy_from_user or get_user is that Linux memory (on
Intel architecture, it may be different under some other processors)
is segmented. This means that a pointer, by itself, does not reference
a unique location in memory, only a location in a memory segment, and
you need to know which memory segment it is to be able to use it.
There is one memory segment for the kernel, and one for each of the
processes.
However it is my understanding that Linux uses paging instead of segmentation and that virtual addresses at and above 0xc0000000 have the kernel mapping in.
Do we use copy_from_user in order to accommodate older kernels?
Do the current linux kernels use segmentation in any way at all? If so how?
If (1) is not true, are there any other advantages to using copy_from_user?
Yeah. I don't like that explanation either. The details are essentially correct in a technical sense (see also Why does Linux on x86 use different segments for user processes and the kernel?) but as you say, linux typically maps the memory so that kernel code could access it directly, so I don't think it's a good explanation for why copy_from_user, etc. actually exist.
IMO, the primary reason for using copy_from_user / copy_to_user (and friends) is simply that there are a number of things to be checked (dangers to be guarded against), and it makes sense to put all of those checks in one place. You wouldn't want every place that needs to copy data in and out from user-space to have to re-implement all those checks. Especially when the details may vary from one architecture to the next.
For example, it's possible that a user-space page is actually not present when you need to copy to or from that memory and hence it's important that the call be made from a context that can accommodate a page fault (and hence being put to sleep).
Also, user-space data pointers need to be checked carefully to ensure that they actually point to user-space and that they point to data regions, and that the copy length doesn't wrap beyond the end of the valid regions, and so forth.
Finally, it's possible that user-space actually doesn't share the same page mappings with the kernel. There used to be a linux patch for 32-bit x86 that made the complete 4G of virtual address space available to user-space processes. In that case, kernel code could not make the assumption that a user-space pointer was directly accessible, and those functions might need to map individual user-space pages one at a time in order to access them. (See 4GB/4GB Kernel VM Split)
The main reason for I/O memory region is to read/write anything to that memory.
If the register address is given, we can use readx/writex (x stands for b/l/w).
Then why do we have to use the address returned by io_remap which is nothing but the same as the address of the particular register given in the data sheet?
ioremap is architecture specific function/macro. On some architectures it won't do anything and just basically return the address specified as an argument. It may do much more than that on other architectures, though. Take arm or x86 as an example - the ioremap will do a lot of checks before letting you using the memory region, for example.
What's more important than those checks, however, is that ioremap can setup a mapping of virtual addresses (from vmalloc area) to the requested physical ones and ensure that caching is disabled for addresses you are going to use. So in most cases pointer returned by ioremap will not be the same as just numeric address from the datasheet.
You want caching to be disabled because I/O registers are controlled by some external (from CPU point of view) devices. This means that processor can't know when its content changed, making cache content invalid.
The thing returned by request_mem_region is a struct resource *, you don't use it to access the I/O memory, and you don't have to do much of anything with it except check it for NULL. request_mem_region isn't part of the mapping you need to do to access the I/O, and your driver would actually (probably) work without it, but by calling it you make some information available in kernel data structures, and make sure that two drivers aren't trying to use overlapping memory ranges.