DMA, CMA, coherence and performance
Hi, I'm working on a device driver for a video device which continuously reads and writes image frames using DMA. The frames are fairly large, in the range of 2-8MB, and I would like the buffers for them to be contiguous because of my hardware. My understanding is that using the contiguous memory allocator is the current "right way" to get the buffers, and that CMA operates entirely behind the scenes when calls are made to dma_alloc_coherent(). However, it seems that for this system, a streaming DMA setup would be more appropriate. The buffer gets filled with data once, handed to the device, and then isn't touched again until it gets reused with new data. The resources I've read have hinted that streaming DMA has some performance benefits over coherent DMA, so this seems like the way to go. But I haven't seen any discussion of how to use CMA with streaming DMA (or whether such a thing is even necessary). Does the CMA also work behind get_free_pages, or other kernel memory allocation methods? Does it matter? The kernel newbies page on memory allocation (http://kernelnewbies.org/KernelMemoryAllocation) says that get_free_pages up to about 8MB are ok. Is that a generalization based on typical memory fragmentation, or a guarantee? Thanks, Steven
I think this discussion should help you: http://e2e.ti.com/support/embedded/linux/f/354/t/89419.aspx other failures: http://stackoverflow.com/questions/14625919/allocating-a-large-dma-buffer and some guideline here: https://www.kernel.org/doc/Documentation/DMA-API.txt https://lkml.org/lkml/2011/3/25/19 As I don't have any specific crashdump or error information, nothing I can comment further about your problem. It is quite difficult to make general comment. On Fri, Jan 3, 2014 at 7:20 AM, Steven Bell <botsnlinux@gmail.com> wrote:
Hi,
I'm working on a device driver for a video device which continuously reads and writes image frames using DMA. The frames are fairly large, in the range of 2-8MB, and I would like the buffers for them to be contiguous because of my hardware. My understanding is that using the contiguous memory allocator is the current "right way" to get the buffers, and that CMA operates entirely behind the scenes when calls are made to dma_alloc_coherent().
However, it seems that for this system, a streaming DMA setup would be more appropriate. The buffer gets filled with data once, handed to the device, and then isn't touched again until it gets reused with new data. The resources I've read have hinted that streaming DMA has some performance benefits over coherent DMA, so this seems like the way to go. But I haven't seen any discussion of how to use CMA with streaming DMA (or whether such a thing is even necessary).
Does the CMA also work behind get_free_pages, or other kernel memory allocation methods? Does it matter? The kernel newbies page on memory allocation (http://kernelnewbies.org/KernelMemoryAllocation) says that get_free_pages up to about 8MB are ok. Is that a generalization based on typical memory fragmentation, or a guarantee?
Thanks, Steven
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- Regards, Peter Teoh
On Sun, Jan 5, 2014 at 8:02 AM, Peter Teoh <htmldeveloper@gmail.com> wrote:
I think this discussion should help you:
http://e2e.ti.com/support/embedded/linux/f/354/t/89419.aspx
other failures:
http://stackoverflow.com/questions/14625919/allocating-a-large-dma-buffer
and some guideline here:
https://www.kernel.org/doc/Documentation/DMA-API.txt
https://lkml.org/lkml/2011/3/25/19
As I don't have any specific crashdump or error information, nothing I can comment further about your problem. It is quite difficult to make general comment.
On Fri, Jan 3, 2014 at 7:20 AM, Steven Bell <botsnlinux@gmail.com> wrote:
Hi,
I'm working on a device driver for a video device which continuously reads and writes image frames using DMA. The frames are fairly large, in the range of 2-8MB, and I would like the buffers for them to be contiguous because of my hardware. My understanding is that using the contiguous memory allocator is the current "right way" to get the buffers, and that CMA operates entirely behind the scenes when calls are made to dma_alloc_coherent().
However, it seems that for this system, a streaming DMA setup would be more appropriate. The buffer gets filled with data once, handed to the device, and then isn't touched again until it gets reused with new data. The resources I've read have hinted that streaming DMA has some performance benefits over coherent DMA, so this seems like the way to go. But I haven't seen any discussion of how to use CMA with streaming DMA (or whether such a thing is even necessary).
Does the CMA also work behind get_free_pages, or other kernel memory allocation methods? Does it matter? The kernel newbies page on memory allocation (http://kernelnewbies.org/KernelMemoryAllocation) says that get_free_pages up to about 8MB are ok. Is that a generalization based on typical memory fragmentation, or a guarantee?
Thanks, Steven
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- Regards, Peter Teoh
Thanks for the reply and the links. Unfortunately, they don't really answer my basic question - which is a general conceptual question, not a specific error or problem that I'm running into. I want to allocate a contiguous 8MB buffer and use it for streaming DMA. I don't need or want a coherent buffer, since I'm using streaming DMA, and so dma_alloc_coherent() seems like the wrong method to use. Is that true, or should I use it anyway? If not, what method do I use? The DMA-API and DMA-API-HOWTO documents don't seem to say anything about allocating memory for streaming DMA operations. Have I missed something? Thanks, Steven
For streaming DMA, you can use the function dma_map_single(). Alternatively, you can create your own DMA pool using dma_pool_create(), and allocate memory from that using dma_pool_alloc((). The later case will be more suitable for you, since you want a 8MB DMA buffer for your DMA. Hope that helps/ Thanks, Saumendra On Mon, Jan 6, 2014 at 11:48 AM, Steven Bell <botsnlinux@gmail.com> wrote:
On Sun, Jan 5, 2014 at 8:02 AM, Peter Teoh <htmldeveloper@gmail.com>wrote:
I think this discussion should help you:
http://e2e.ti.com/support/embedded/linux/f/354/t/89419.aspx
other failures:
http://stackoverflow.com/questions/14625919/allocating-a-large-dma-buffer
and some guideline here:
https://www.kernel.org/doc/Documentation/DMA-API.txt
https://lkml.org/lkml/2011/3/25/19
As I don't have any specific crashdump or error information, nothing I can comment further about your problem. It is quite difficult to make general comment.
On Fri, Jan 3, 2014 at 7:20 AM, Steven Bell <botsnlinux@gmail.com> wrote:
Hi,
I'm working on a device driver for a video device which continuously reads and writes image frames using DMA. The frames are fairly large, in the range of 2-8MB, and I would like the buffers for them to be contiguous because of my hardware. My understanding is that using the contiguous memory allocator is the current "right way" to get the buffers, and that CMA operates entirely behind the scenes when calls are made to dma_alloc_coherent().
However, it seems that for this system, a streaming DMA setup would be more appropriate. The buffer gets filled with data once, handed to the device, and then isn't touched again until it gets reused with new data. The resources I've read have hinted that streaming DMA has some performance benefits over coherent DMA, so this seems like the way to go. But I haven't seen any discussion of how to use CMA with streaming DMA (or whether such a thing is even necessary).
Does the CMA also work behind get_free_pages, or other kernel memory allocation methods? Does it matter? The kernel newbies page on memory allocation (http://kernelnewbies.org/KernelMemoryAllocation) says that get_free_pages up to about 8MB are ok. Is that a generalization based on typical memory fragmentation, or a guarantee?
Thanks, Steven
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- Regards, Peter Teoh
Thanks for the reply and the links. Unfortunately, they don't really answer my basic question - which is a general conceptual question, not a specific error or problem that I'm running into.
I want to allocate a contiguous 8MB buffer and use it for streaming DMA. I don't need or want a coherent buffer, since I'm using streaming DMA, and so dma_alloc_coherent() seems like the wrong method to use. Is that true, or should I use it anyway? If not, what method do I use?
The DMA-API and DMA-API-HOWTO documents don't seem to say anything about allocating memory for streaming DMA operations. Have I missed something?
Thanks, Steven
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
I want to allocate a contiguous 8MB buffer and use it for streaming DMA. I don't need or want a coherent buffer, since I'm using streaming DMA, and so dma_alloc_coherent() seems like the wrong method to use. Is that true, or should I use it anyway? If not, what method do I use?
The DMA-API and DMA-API-HOWTO documents don't seem to say anything about allocating memory for streaming DMA operations. Have I missed something?
On Mon, Jan 6, 2014 at 1:29 AM, saumendra dash <saumendradash@gmail.com> wrote:
For streaming DMA, you can use the function dma_map_single(). Alternatively, you can create your own DMA pool using dma_pool_create(), and allocate memory from that using dma_pool_alloc(().
The later case will be more suitable for you, since you want a 8MB DMA buffer for your DMA. Hope that helps/
I am using dma_map_single, but it doesn't allocate any memory. From what I understand, it takes a pointer to a buffer and performs the necessary cache flushes or invalidations to make sure that the memory is safe to DMA. Is there a correct way to allocate large buffers for streaming DMA? Right now, I'm allocating memory with get_free_pages, but that is prone to fail on large allocations, and only works up to 4MB on my system (i.e get_free_pages order 10).
From reading LDD3 and DMA-API.txt, DMA pools are for allocating "small dma-coherent memory regions". LDD3 specifically states that pools are useful where the driver needs regions smaller than 1 page. Since I want large, non-coherent buffers, I don't see how this helps.
Steven
There are two ways to do this 1. Allocate a physical memory at boot time itself by reserve_mem fn pointer and you could use this large buffer when you need dma using dma map single fn. 2. Patch the kernel with bigphysarea patch and allocate large buffer with that patch. And you can use that buffer for dma. I have used both methods it works for large physical buffer you need for dma. Thanks, Haresh Thanks, Haresh. On 7 Jan 2014 03:27, "Steven Bell" <botsnlinux@gmail.com> wrote:
I want to allocate a contiguous 8MB buffer and use it for streaming DMA. I don't need or want a coherent buffer, since I'm using streaming DMA, and so dma_alloc_coherent() seems like the wrong method to use. Is that true, or should I use it anyway? If not, what method do I use?
The DMA-API and DMA-API-HOWTO documents don't seem to say anything about allocating memory for streaming DMA operations. Have I missed something?
On Mon, Jan 6, 2014 at 1:29 AM, saumendra dash <saumendradash@gmail.com> wrote:
For streaming DMA, you can use the function dma_map_single(). Alternatively, you can create your own DMA pool using dma_pool_create(),
and allocate memory from that using dma_pool_alloc(().
The later case will be more suitable for you, since you want a 8MB DMA
buffer for your DMA.
Hope that helps/
I am using dma_map_single, but it doesn't allocate any memory. From what I understand, it takes a pointer to a buffer and performs the necessary cache flushes or invalidations to make sure that the memory is safe to DMA. Is there a correct way to allocate large buffers for streaming DMA? Right now, I'm allocating memory with get_free_pages, but that is prone to fail on large allocations, and only works up to 4MB on my system (i.e get_free_pages order 10).
From reading LDD3 and DMA-API.txt, DMA pools are for allocating "small dma-coherent memory regions". LDD3 specifically states that pools are useful where the driver needs regions smaller than 1 page. Since I want large, non-coherent buffers, I don't see how this helps.
Steven
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Hi, Ion memory, i think, is the solution to the scenario. This is boot time carved out memory which only you would have the access to. To have your own share, you can declare some in device tree files if you have access to them. You may want to read more about it here<http://lwn.net/Articles/480055/>. One more link <http://lwn.net/Articles/565469/>. In general you can search for ION memory and study the proper api's. Regards, Manty On Tue, Jan 7, 2014 at 10:45 AM, haresh langaraman <hareshel@gmail.com>wrote:
There are two ways to do this 1. Allocate a physical memory at boot time itself by reserve_mem fn pointer and you could use this large buffer when you need dma using dma map single fn.
2. Patch the kernel with bigphysarea patch and allocate large buffer with that patch. And you can use that buffer for dma.
I have used both methods it works for large physical buffer you need for dma.
Thanks, Haresh
Thanks, Haresh. On 7 Jan 2014 03:27, "Steven Bell" <botsnlinux@gmail.com> wrote:
I want to allocate a contiguous 8MB buffer and use it for streaming DMA. I don't need or want a coherent buffer, since I'm using streaming DMA, and so dma_alloc_coherent() seems like the wrong method to use. Is that true, or should I use it anyway? If not, what method do I use?
The DMA-API and DMA-API-HOWTO documents don't seem to say anything about allocating memory for streaming DMA operations. Have I missed something?
On Mon, Jan 6, 2014 at 1:29 AM, saumendra dash <saumendradash@gmail.com> wrote:
For streaming DMA, you can use the function dma_map_single(). Alternatively, you can create your own DMA pool using
dma_pool_create(), and allocate memory from that using dma_pool_alloc(().
The later case will be more suitable for you, since you want a 8MB DMA
buffer for your DMA.
Hope that helps/
I am using dma_map_single, but it doesn't allocate any memory. From what I understand, it takes a pointer to a buffer and performs the necessary cache flushes or invalidations to make sure that the memory is safe to DMA. Is there a correct way to allocate large buffers for streaming DMA? Right now, I'm allocating memory with get_free_pages, but that is prone to fail on large allocations, and only works up to 4MB on my system (i.e get_free_pages order 10).
From reading LDD3 and DMA-API.txt, DMA pools are for allocating "small dma-coherent memory regions". LDD3 specifically states that pools are useful where the driver needs regions smaller than 1 page. Since I want large, non-coherent buffers, I don't see how this helps.
Steven
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
participants (5)
-
haresh langaraman -
manty kuma -
Peter Teoh -
saumendra dash -
Steven Bell