Hi folks, I'm currently trying to port a (currently for unreleased hardware and under nda) driver from x86 to arm and in doing so I've run into a problem with allocating memory using pci_alloc_consistent. Looking into it I can't allocate more than 1MB whereas I need to allocate at least 2MB of contiguous memory. Looking further into it, the conventional method to get around this at the moment seems to be to statically link the driver into the kernel and use alloc_bootmem or alloc_bootmem_low (mentioned in ldd3 and all over google). The problem I've run into however is that I get a warning at boot which is: WARNING: at mm/bootmem.c:672 alloc_arch_preferred_bootmem+0x34/0x64() Looking at that code in bootmem.c this is a warning which I think is there to tell me that SLAB is available and warn against trying to allocate at boot. However it reportedly allocates the memory but when the driver tries to use this memory later, then the system hangs. Is alloc_bootmem currently the correct way to get memory allocated at boot or is there a different call I should be using. Alternatively have I done something wrong by using alloc_bootmem as a drop-in replacement for pci_alloc_consistent? I've tried to grep the kernel sources for other drivers that use alloc_bootmem but there aren't many which appear in the listing and I find this strange for a technique which is referred to a lot. Thanks, Phil
I'm currently trying to port a (currently for unreleased hardware and under nda) driver from x86 to arm and in doing so I've run into a problem with allocating memory using pci_alloc_consistent. Looking into it I can't allocate more than 1MB whereas I need to allocate at least 2MB of contiguous memory. I guess pci_alloc_consistent calls dma_alloc_coherent, so why does it fail to allocate 2MB of memory? What is the error do you see?
Looking further into it, the conventional
method to get around this at the moment seems to be to statically link the driver into the kernel and use alloc_bootmem or alloc_bootmem_low (mentioned in ldd3 and all over google).
The problem I've run into however is that I get a warning at boot which is:
WARNING: at mm/bootmem.c:672 alloc_arch_preferred_bootmem+0x34/0x64()
You need to call alloc_bootmem at an early stage before slab is initialized.
Looking at that code in bootmem.c this is a warning which I think is there to tell me that SLAB is available and warn against trying to allocate at boot. However it reportedly allocates the memory but when the driver tries to use this memory later, then the system hangs.
Is the driver using the memory for DMA? Are you using the memory returned by alloc_bootmem directly? Remember DMA works on physical addresses not virtual. You might have to use dma_map_single/dma_unmap_single. Also does the h/w has any limitations of the physical memory range it can access? Like only first 64MB of RAM? regards -syed
sk.syed2 wrote:
I'm currently trying to port a (currently for unreleased hardware and under nda) driver from x86 to arm and in doing so I've run into a problem with allocating memory using pci_alloc_consistent. Looking into it I can't allocate more than 1MB whereas I need to allocate at least 2MB of contiguous memory. I guess pci_alloc_consistent calls dma_alloc_coherent, so why does it fail to allocate 2MB of memory? What is the error do you see?
Well there is no warning or panic, pci_alloc_consistent returns NULL if I request 2MB. If I change my request to 1MB then it succeeds, everything is allocated correctly and I can use the memory without issues, but of course the device requires 2MB (well 6 buffers all 2MB in size) so it's of no use. If I attempt to request a second 1MB then pci_alloc_consistent also returns NULL for that second allocation. I have looked at the code for dma_alloc_coherent etc and it seems to me that it's returning null because there isn't the space available to allocate the requested memory.
The problem I've run into however is that I get a warning at boot which is:
WARNING: at mm/bootmem.c:672 alloc_arch_preferred_bootmem+0x34/0x64()
You need to call alloc_bootmem at an early stage before slab is initialized.
I was thinking that this might be the problem, but in order to do that I'd have to put the alloc_bootmem in core kernel code wouldn't I? This doesn't seem very appealing.
Is the driver using the memory for DMA? Are you using the memory returned by alloc_bootmem directly?
Yes, it is used for DMA and I am using the memory returned by alloc_bootmem directly.
Remember DMA works on physical addresses not virtual. You might have to use dma_map_single/dma_unmap_single.
Yes, I was aware that dma uses physical addresses, isn't that what alloc_bootmem returns? I'm not aware of dma_map_single etc I will look at that soon.
Also does the h/w has any limitations of the physical memory range it can access? Like only first 64MB of RAM?
It's limited to 32bit addressing, but that should be fine for the 512mb of ram on the arm board which I have. Thanks, Phil
I have looked at the code for dma_alloc_coherent etc and it seems to me that it's returning null because there isn't the space available to allocate the requested memory. You might have to increase the consistent(or dma) memory size. In recent kernels its defined by CONSISTENT_DMA_SIZE (ach/arm/include/asm/memory.h). Which kernel version are you using? You might have to override this value to match your needs. Are there any other drivers that allocate using dma_alloc_coherent?
I was thinking that this might be the problem, but in order to do that I'd have to put the alloc_bootmem in core kernel code wouldn't I? This doesn't seem very appealing. Yes. Nobody wants to do that. Best option is to use dma_alloc_coherent. Alternatively you can use kmalloc in your driver itself and then use dma_map_single/dma_unmap_single.
Yes, I was aware that dma uses physical addresses, isn't that what alloc_bootmem returns? I'm not aware of dma_map_single etc I will look at that soon. No alloc_bootmem doesnt return physical addresses. It returns virtual address. That is the reason you need to use dma_map_single and dma_unmap_single. These apis return physical address(dma_addr_t) for the corresponding virtual address. Also they flush/invalidate the cache so there are no side effects. But dma_alloc_coherent gives you a memory whose cache lines are disabled, so you wouldnt have to worry about caching side effects.
-syed
sk.syed2 wrote:
I have looked at the code for dma_alloc_coherent etc and it seems to me that it's returning null because there isn't the space available to allocate the requested memory. You might have to increase the consistent(or dma) memory size. In recent kernels its defined by CONSISTENT_DMA_SIZE (ach/arm/include/asm/memory.h). Which kernel version are you using? You might have to override this value to match your needs. Are there any other drivers that allocate using dma_alloc_coherent?
Oh, awesome! This was it, I'm using kernel version 2.6.37 and changing CONSISTENT_DMA_SIZE for arm has allowed me to allocate enough memory using dma_alloc_coherent so I can use the original code which I had working on x86. Is there any reason why this isn't a kconfig option? Thanks for the help. Phil
Is there any reason why this isn't a kconfig option?
The memory/RAM related defaults are present in arch/arm/include/asm/memory.h. But you can override them by defining them in /arch/arm/mach-YOURMACHINE/include/mach/memory.h. I think the developers intended to keep it this way so machine specific defines go in machine specific directories, and it will not bloat the kernel config. -syed
participants (2)
-
Philip Downer -
sk.syed2