OOM killer hung the whole system

[Yubin Ruan]

2017-07-25 17:59 GMT+08:00 Rock Lee <rockdotlee at gmail.com>:
> Hi,
>
> I was using a program to test OOM killer, but OOM killer hung my board.  The
> test snippet is very simple, just malloc() without free().
>
>   6         int i=0;
>   7         char *ptrtest=NULL;
>   8          while (1)
>   9          {
>  10                    ptrtest = malloc(0x1000);
>  11                    if (ptrtest == NULL)
>  12                        printf("malloc failed.\n");
>  13          }
>  14          return 0;
>  15
>
> After I executed this program(a.out), there is no reaction of my shell. Only
> this messege was shown:
>
> [  180.138188] Out of memory: Kill process 2706 (qq) score 623 or sacrifice
> child
> [  180.164230] Killed process 2706 (qq) total-vm:98156kB, anon-rss:96400kB,
> file-rss:4kB
>
> I think it must be some problem with OOM killer, and tried a lot of way to
> get some infomation. And I got this:
>
> =======================================================
> Process: a.out, cpu: 0 pid: 1463 start: 0xd5849b00
> =====================================================
>     Task name: a.out pid: 1463 cpu: 0
>     state: 0x2 exit_state: 0x0 stack base: 0xd6190000
>     Stack:
>     [<c08394e4>] __schedule+0x2c8
>     [<c018df88>] squashfs_cache_get+0x108
>     [<c0191020>] squashfs_readpage_block+0x28
>     [<c018f648>] squashfs_readpage+0x624
>     [<c0090d4c>] __do_page_cache_readahead+0x228
>     [<c00882f4>] filemap_fault+0x1d4
>     [<c00a5f10>] __do_fault+0x34
>     [<c00a8ab0>] do_read_fault+0x19c
>     [<c00a9388>] handle_mm_fault+0x468
>     [<c00164b0>] do_page_fault+0x11c
>     [<c00084a4>] do_PrefetchAbort+0x34
>     [<c0011c1c>] ret_from_exception+0x0
>
> I am suprised, because a.out is in /tmp, there is no relation between a.out
> and squashfs(my rootfs). Could anyone give me a hint ? Is the problem of
> squashfs or OOM killer.

You say your rootfs is squashfs filesystem. Is /tmp using the same
device and same filesystem (i.e., squashfs) ?

Yubin