the cost of vmalloc
Hello, I need to allocate fairly a large piece of memory, about 128KB or 256KB. I tried to use kmalloc, and it sometimes fails, but vmalloc in this case usually still work. But I'm not sure how costly vmalloc is. As far as I know, vmalloc needs to change the page table, and thus might need to invalidate TLB. It seems quite expensive. If it is, maybe I can allocate memory with kmalloc first, and then try to use vmalloc if kmalloc fails. Any suggestions? Thanks, Da
Hi :) On Sat, Sep 10, 2011 at 08:16, Zheng Da <zhengda1936@gmail.com> wrote:
Hello,
I need to allocate fairly a large piece of memory, about 128KB or 256KB.
128 K? Are you sure? That doesn't sound too big for me...but I might be wrong here..... if you say 4 megabyte, now that's big...
I tried to use kmalloc, and it sometimes fails, but vmalloc in this case usually still work.
easy to guess, vmalloc doesn't care if it's not physically contigous...only virtually contigous
But I'm not sure how costly vmalloc is. As far as I know, vmalloc needs to change the page table, and thus might need to invalidate TLB. It seems quite expensive. If it is, maybe I can allocate memory with kmalloc first, and then try to use vmalloc if kmalloc fails. Any suggestions?
invalidating page table, at some point yes. Also searching non physically contigous could be hard too. But again, this is your last resort to gain such a large memory area. just my 2 cents idea... -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
Hi! On 21:16 Fri 09 Sep , Zheng Da wrote:
Hello,
I need to allocate fairly a large piece of memory, about 128KB or 256KB. I tried to use kmalloc, and it sometimes fails, but vmalloc in this case usually still work. But I'm not sure how costly vmalloc is. As far as I know, vmalloc needs to change the page table, and thus might need to invalidate TLB. It seems quite expensive. If it is, maybe I can allocate memory with kmalloc first, and then try to use vmalloc if kmalloc fails. Any suggestions?
If it is at all possible, I would suggest avoiding to allocate more than PAGE_SIZE (usually 4KB) in one continuous chunk. You can use e.g. scatterlists (which are basically lists of pointers) if you need a bigger buffer. -Michi -- programing a layer 3+4 network protocol for mesh networks see http://michaelblizek.twilightparadox.com
On Sat, Sep 10, 2011 at 3:35 AM, Michael Blizek <michi1@michaelblizek.twilightparadox.com> wrote:
I need to allocate fairly a large piece of memory, about 128KB or 256KB. I tried to use kmalloc, and it sometimes fails, but vmalloc in this case usually still work. But I'm not sure how costly vmalloc is. As far as I know, vmalloc needs to change the page table, and thus might need to invalidate TLB. It seems quite expensive. If it is, maybe I can allocate memory with kmalloc first, and then try to use vmalloc if kmalloc fails. Any suggestions?
If it is at all possible, I would suggest avoiding to allocate more than PAGE_SIZE (usually 4KB) in one continuous chunk. You can use e.g. scatterlists (which are basically lists of pointers) if you need a bigger buffer. Actually, I have seen some problems when allocating large pieces of memory constantly. Occasionally, I see alloc_pages and kmalloc fails to allocate the memory I want. The whole point of allocating a large chunk of memory is to avoid extra memory copy because I need to run decompression algorithms on it. Or memory copy is cheaper? I'm also thinking of using slab allocator. The size of memory I need to allocate can change from 20KB to 128KB. Then I need quite a few slab allocators.
Thanks, Da
Hi! On 11:15 Sat 10 Sep , Zheng Da wrote:
On Sat, Sep 10, 2011 at 3:35 AM, Michael Blizek <michi1@michaelblizek.twilightparadox.com> wrote:
I need to allocate fairly a large piece of memory, about 128KB or 256KB. I tried to use kmalloc, and it sometimes fails, but vmalloc in this case usually still work. But I'm not sure how costly vmalloc is. As far as I know, vmalloc needs to change the page table, and thus might need to invalidate TLB. It seems quite expensive. If it is, maybe I can allocate memory with kmalloc first, and then try to use vmalloc if kmalloc fails. Any suggestions?
If it is at all possible, I would suggest avoiding to allocate more than PAGE_SIZE (usually 4KB) in one continuous chunk. You can use e.g. scatterlists (which are basically lists of pointers) if you need a bigger buffer. Actually, I have seen some problems when allocating large pieces of memory constantly. Occasionally, I see alloc_pages and kmalloc fails to allocate the memory I want.
This does not really suprise me. Once the memory fragments, allocating large continuous memory regions gets harder.
The whole point of allocating a large chunk of memory is to avoid extra memory copy because I need to run decompression algorithms on it.
In this case scatterlists solves 2 problems at once. First, you will not need to allocate large continuous memory regions. Second, you avoid wasting memory.
Or memory copy is cheaper?
This is hard to say. But by passing the scatterlist you should be able to avoid the copying.
I'm also thinking of using slab allocator. The size of memory I need to allocate can change from 20KB to 128KB. Then I need quite a few slab allocators.
The slab allocator is good if you have a large number of allocations which are the same size. To me it does not really sound like a good idea to use the slab allocator in your case. -Michi -- programing a layer 3+4 network protocol for mesh networks see http://michaelblizek.twilightparadox.com
Hi, On Sat, Sep 10, 2011 at 3:30 PM, Michael Blizek <michi1@michaelblizek.twilightparadox.com> wrote:
The whole point of allocating a large chunk of memory is to avoid extra memory copy because I need to run decompression algorithms on it.
In this case scatterlists solves 2 problems at once. First, you will not need to allocate large continuous memory regions. Second, you avoid wasting memory. The problem is that the decompression library works on contiguous memory, so I have to provide contiguous memory instead of scatterlists.
Thanks, Da
Hi! On 16:54 Sat 10 Sep , Zheng Da wrote:
Hi,
On Sat, Sep 10, 2011 at 3:30 PM, Michael Blizek <michi1@michaelblizek.twilightparadox.com> wrote:
The whole point of allocating a large chunk of memory is to avoid extra memory copy because I need to run decompression algorithms on it.
In this case scatterlists solves 2 problems at once. First, you will not need to allocate large continuous memory regions. Second, you avoid wasting memory. The problem is that the decompression library works on contiguous memory, so I have to provide contiguous memory instead of scatterlists.
Which decompression lib are you talking about? Even if it does not have explicit support for scatterlists, usually you should be able to call the decompress function multiple times. Otherwise, how would you (de)compress data which is larger than available memory? -Michi -- programing a layer 3+4 network protocol for mesh networks see http://michaelblizek.twilightparadox.com
Hi, On Sun, Sep 11, 2011 at 2:32 AM, Michael Blizek <michi1@michaelblizek.twilightparadox.com> wrote:
The whole point of allocating a large chunk of memory is to avoid extra memory copy because I need to run decompression algorithms on it.
In this case scatterlists solves 2 problems at once. First, you will not need to allocate large continuous memory regions. Second, you avoid wasting memory. The problem is that the decompression library works on contiguous memory, so I have to provide contiguous memory instead of scatterlists.
Which decompression lib are you talking about? Even if it does not have explicit support for scatterlists, usually you should be able to call the decompress function multiple times. Otherwise, how would you (de)compress data which is larger than available memory? Sorry for the late reply. I'm using LZO. The data is compressed in blocks of 128KB. I don't think I can split the compressed block and run LZO decompressor on the pieces multiple times. There is a lot of free memory, but the kernel can't find contiguous memory sometimes. vmalloc always succeeds when kmalloc fails.
Thanks, Da
Hi! On 21:50 Thu 15 Sep , Zheng Da wrote:
Hi,
On Sun, Sep 11, 2011 at 2:32 AM, Michael Blizek <michi1@michaelblizek.twilightparadox.com> wrote:
The whole point of allocating a large chunk of memory is to avoid extra memory copy because I need to run decompression algorithms on it.
In this case scatterlists solves 2 problems at once. First, you will not need to allocate large continuous memory regions. Second, you avoid wasting memory. The problem is that the decompression library works on contiguous memory, so I have to provide contiguous memory instead of scatterlists.
Which decompression lib are you talking about? Even if it does not have explicit support for scatterlists, usually you should be able to call the decompress function multiple times. Otherwise, how would you (de)compress data which is larger than available memory? Sorry for the late reply. I'm using LZO. The data is compressed in blocks of 128KB. I don't think I can split the compressed block and run LZO decompressor on the pieces multiple times. There is a lot of free memory, but the kernel can't find contiguous memory sometimes. vmalloc always succeeds when kmalloc fails.
Yes, it really does look as if lzo currently does not support scatterlists. The change looks fairly simple to me, but apparently there is no maintainer it :-( . -Michi -- programing a layer 3+4 network protocol for mesh networks see http://michaelblizek.twilightparadox.com
participants (3)
-
Michael Blizek -
Mulyadi Santosa -
Zheng Da