How to detect slab memory leak
Hi everyone, When using slabtop to display the cache size growing, I see that kmalloc-1024 and kmalloc-256 are increasing about 200MB after 1 hour. Is there any way to detect which kernel components cause this slab increase? Thanks a lot, Brs, Naruto
On Sun, Dec 16, 2018 at 1:25 PM Naruto Nguyen <narutonguyen2018@gmail.com> wrote:
Hi everyone,
When using slabtop to display the cache size growing, I see that kmalloc-1024 and kmalloc-256 are increasing about 200MB after 1 hour. Is there any way to detect which kernel components cause this slab increase?
Thanks a lot, Brs, Naruto
Hello Naruto I think you can do it by instrumenting call to kmalloc(). These days, eBPF is the hot stuff to try for this matter. See if it fits yours too. regards, Mulyadi
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
You can try kernel config option "CONFIG_DEBUG_KMEMLEAK". https://www.kernel.org/doc/html/v4.10/dev-tools/kmemleak.html On Tue, Dec 18, 2018 at 1:14 PM Mulyadi Santosa <mulyadi.santosa@gmail.com> wrote:
On Sun, Dec 16, 2018 at 1:25 PM Naruto Nguyen <narutonguyen2018@gmail.com> wrote:
Hi everyone,
When using slabtop to display the cache size growing, I see that kmalloc-1024 and kmalloc-256 are increasing about 200MB after 1 hour. Is there any way to detect which kernel components cause this slab increase?
Thanks a lot, Brs, Naruto
Hello Naruto
I think you can do it by instrumenting call to kmalloc(). These days, eBPF is the hot stuff to try for this matter. See if it fits yours too.
regards,
Mulyadi
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- regards,
Mulyadi Santosa Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com _______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Hi Harsh and Mulyadi, Thanks a lot for your reply. Just another question, in the slabtop output, I see that we have kmalloc-64 and kmalloc-96 cache, when these caches increase their size, is it calculated in the "cache" of the "free" command or in "used". Is it reclaimable? and count in SReclaimable or SUnreclaim of cat /proc/meminfo? I know some cache like dentry or inode cache will be calculated in "cache" of free command output and SReclaimable of cat /proc/meminfo and can be freed by "echo 3 > /proc/sys/vm/drop_caches" but not sure if it the same situation for kmalloc-64 and kmalloc-96 cache. Thanks again, Brs, Bao On Tue, 18 Dec 2018 at 15:31, Harsh Jain <harshjain.prof@gmail.com> wrote:
You can try kernel config option "CONFIG_DEBUG_KMEMLEAK".
https://www.kernel.org/doc/html/v4.10/dev-tools/kmemleak.html
On Tue, Dec 18, 2018 at 1:14 PM Mulyadi Santosa <mulyadi.santosa@gmail.com> wrote:
On Sun, Dec 16, 2018 at 1:25 PM Naruto Nguyen <narutonguyen2018@gmail.com> wrote:
Hi everyone,
When using slabtop to display the cache size growing, I see that kmalloc-1024 and kmalloc-256 are increasing about 200MB after 1 hour. Is there any way to detect which kernel components cause this slab increase?
Thanks a lot, Brs, Naruto
Hello Naruto
I think you can do it by instrumenting call to kmalloc(). These days, eBPF is the hot stuff to try for this matter. See if it fits yours too.
regards,
Mulyadi
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- regards,
Mulyadi Santosa Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com _______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
On Wed, Dec 19, 2018 at 6:58 PM Naruto Nguyen <narutonguyen2018@gmail.com> wrote:
Hi Harsh and Mulyadi,
Thanks a lot for your reply.
Just another question, in the slabtop output, I see that we have kmalloc-64 and kmalloc-96 cache, when these caches increase their size, is it calculated in the "cache" of the "free" command or in "used".
If the cache is used for page cache related operations, then AFAIK it is calculated in cache. If not, say for mmap non file backed operation, then it is calculated in overall used.
Is it reclaimable?
AFAIK yes, as long as it is not locked after being allocated.
and count in SReclaimable or SUnreclaim of cat /proc/meminfo? I know some cache like dentry or inode cache will be calculated in "cache" of free command output and SReclaimable of cat /proc/meminfo and can be freed by "echo 3 > /proc/sys/vm/drop_caches" but not sure if it the same situation for kmalloc-64 and kmalloc-96 cache.
Thanks again, Brs, Bao
On Tue, 18 Dec 2018 at 15:31, Harsh Jain <harshjain.prof@gmail.com> wrote:
You can try kernel config option "CONFIG_DEBUG_KMEMLEAK".
https://www.kernel.org/doc/html/v4.10/dev-tools/kmemleak.html
On Tue, Dec 18, 2018 at 1:14 PM Mulyadi Santosa <mulyadi.santosa@gmail.com> wrote:
On Sun, Dec 16, 2018 at 1:25 PM Naruto Nguyen <
narutonguyen2018@gmail.com> wrote:
Hi everyone,
When using slabtop to display the cache size growing, I see that kmalloc-1024 and kmalloc-256 are increasing about 200MB after 1 hour. Is there any way to detect which kernel components cause this slab increase?
Thanks a lot, Brs, Naruto
Hello Naruto
I think you can do it by instrumenting call to kmalloc(). These days, eBPF is the hot stuff to try for this matter. See if it fits yours too.
regards,
Mulyadi
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- regards,
Mulyadi Santosa Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com _______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaig...> Virus-free. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaig...> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
participants (3)
-
Harsh Jain -
Mulyadi Santosa -
Naruto Nguyen