Re: ENOMEM failure on mmap call
Hi :) 2011/10/12 Ezequiel García <elezegarcia@yahoo.com.ar>:
Initially, you requested 8 MB. Knowing that you had 9 MB free RAM, I think you got ENOMEM since that would just leave 1 MB and quite likely that is under safe minimum free RAM allowed. I roughly recall that somewhere around 2-5% of RAM must be kept unallocated from user need to cope with urgent situation or something like that.
Okey, but I am not allocating since mmap won't consume any memory.
but you "commit"... and it is that "committed memory" that counts.... (in this situation IMO) Also, i forgot to mention that it is likely to that your mmap failed due to kernel inability to find such large continous virtual (let's stress the word virtual here) memory area... it could be due to fragmentation etc. So, IOW, physically kernel still able to satisfy your request, but unable to map it for you.
It was shown that, when you use 7.5 MB, that slight 0.5 MB made you didn't hit the mark...so you're safe now and kernel allow that.
But why does he allows 10 threads!!!
this is what I am not clear, do those 10 threads allocate 8 MB each? or 8 MB total? remember that threads share address space by default...
Sorry I couldn't point you to exact kernel source code line. lxr.linux.no is your best friend....
Thanks for the link and for the help. I guess I'll have to keep diggin :D
-- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
--- El jue 13-oct-11, Mulyadi Santosa <mulyadi.santosa@gmail.com> escribió:
this is what I am not clear, do those 10 threads allocate 8 MB each? or 8 MB total? remember that threads share address space by default...
I am sorry maybe I explained myself incorrectly, threads don't actually allocate anything; just mmap it's own stack space (8MB each). I looked at the link you provided me and I found this: http://lxr.linux.no/linux+v3.0.4/mm/mmap.c#L112 mmap.c, line 112: vm_enough_memory() It seems here is where the mm subsystem decides to fail with ENOMEM and uses commit parameters to decide... I found it most interesting. Ezequiel.
Hi :) 2011/10/13 Ezequiel García <elezegarcia@yahoo.com.ar>:
--- El jue 13-oct-11, Mulyadi Santosa <mulyadi.santosa@gmail.com> escribió:
this is what I am not clear, do those 10 threads allocate 8 MB each? or 8 MB total? remember that threads share address space by default...
I am sorry maybe I explained myself incorrectly, threads don't actually allocate anything; just mmap it's own stack space (8MB each).
I see, so 8 MB each.... and since you created 10 threads, I suppose that would commit 8*10=80 MB of RAM. CMIIW here.... (stack are individual, so they aren't shared between threads)
I looked at the link you provided me and I found this:
http://lxr.linux.no/linux+v3.0.4/mm/mmap.c#L112
mmap.c, line 112: vm_enough_memory()
It seems here is where the mm subsystem decides to fail with ENOMEM and uses commit parameters to decide... I found it most interesting.
I agree....as you can see there, there is some percentage of VM preserved for root...also for hugetlb etc. -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
--- El jue 13-oct-11, Mulyadi Santosa <mulyadi.santosa@gmail.com> escribió:
I am sorry maybe I explained myself incorrectly, threads don't actually allocate anything; just mmap it's own stack space (8MB each).
I see, so 8 MB each.... and since you created 10 threads, I suppose that would commit 8*10=80 MB of RAM. CMIIW here.... (stack are individual, so they aren't shared between threads)
Reading sources I think kernel let the process create several threads because as there is no real memory usage, the amount of free pages on each thread allocation is the same. I mean, when I start first thread I have enough free pages to allow the mmap(). When I start the second, third, etc. I always have the same amount of free pages, since threads don't alloc any pages they just 'prepare' the adresses. I'll try to put some printk inside vm_enough_memory() and try to re-run my tests. Do you think there is any other debugging aproach? By the way, I have already solved my problem (weeks ago) by reducing default thread stack size :) But I would love to understand what happened. It's a strange scenario for a linux kernel, since I have very low memory left and zero swap. Thanks a lot for the help!
hi again :) 2011/10/14 Ezequiel García <elezegarcia@yahoo.com.ar>:
Reading sources I think kernel let the process create several threads because as there is no real memory usage, the amount of free pages on each thread allocation is the same.
I agree, as long as Copy on Write hasn't kicked in.... sometimes it's the commited amount of memory that matters, sometimes how much memory you really allocate that matters. Around 2006, I wrote an article about Out of Memory killer, somehow related to this issue. Maybe you are interested to read it: http://linuxdevcenter.com/pub/a/linux/2006/11/30/linux-out-of-memory.html
I mean, when I start first thread I have enough free pages to allow the mmap(). When I start the second, third, etc. I always have the same amount of free pages, since threads don't alloc any pages they just 'prepare' the adresses. So, the stack area is shared among those threads?
I'll try to put some printk inside vm_enough_memory() and try to re-run my tests. Do you think there is any other debugging aproach?
IMO, that one way to go :) of course nowadays you have ftrace etc, but still, printk followed by kernel recompilation is still easiest IMHO :)
By the way, I have already solved my problem (weeks ago) by reducing default thread stack size :) But I would love to understand what happened.
Welcome to the OOM world :)
It's a strange scenario for a linux kernel, since I have very low memory left and zero swap.
I think it's not that strange, we just have to understand the behaviour. And perhaps your application must anticipate it by gradually decreasing such memory allocation whenever you hit such error.
Thanks a lot for the help! You welcome and sorry I don't lend too much help here...
-- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
participants (2)
-
Ezequiel García -
Mulyadi Santosa