Estimating total memory usage in any Linux system

[Pintu Agarwal]

On Mon, 18 May 2020 at 23:27, Pintu Agarwal <pintu.ping at gmail.com> wrote:
>
> Hi,
>
> I was just trying to manually calculate the total memory available in
> a system to match up the MemTotal shown in /proc/meminfo
> I know "free -m" command will give the vary high level usage, but I
> wanted to breakdown the MemTotal numbers to see usage in each
> category.
>
> I am looking for a common formula that should match on almost all systems.
> So, I came up with below fields from meminfo to closely match the total.
> ~Total = {MemFree + Active + Inactive + Shmem + Mapped + Slab +
> KernelStack + PageTables + (init freed) + (Boot Diff) + (low
> watermarks)}
>
> But I am not sure if these fields are correct at least for arm32 system.
> In some boards I could able to match up these figures, but some not.
>
> So I have some doubts:
> * Is "Mapped" (NR_MAPPED_PAGES) already part of Active pages ?
> * Is Kernel allocation (alloc_pages, etc.) accounted somewhere ? Is it
> part of Anon pages ?
> * What about vmalloc / ioremap pages ? Do we also need to consider VmallocUsed ?
> ==> In my opinion vmalloc internally used kmalloc and page tables, so
> I guess it should be already part of Slab or PageTables ??
>
> Below are some data taken from a small embedded arm32 device with 512MB RAM:
>

Sorry, I forgot to mention the kernel version. Here it is 3.18 Kernel

> Kernel Boot Logs:
> Memory: 99244K/254976K available (10720K kernel code, 1336K rwdata,
> 4792K rodata, 400K init, 1602K bss, 135252K reserved, 20480K
> cma-reserved)
> Total = 99244 + 400 (init) + 20480 (cma) =  120124
> Boot Diff = 121204 - 120124 => 1080 kB
>
> MemTotal: 121204 kB
> MemFree: 2540 kB
> MemAvailable: 51924 kB
> Buffers: 0 kB
> Cached: 47492 kB
> SwapCached: 0 kB
> Active: 42604 kB
> Inactive: 24308 kB
> Active(anon): 19476 kB
> Inactive(anon): 92 kB
> Active(file): 23128 kB
> Inactive(file): 24216 kB
> Unevictable: 0 kB
> Mlocked: 0 kB
> SwapTotal: 0 kB
> SwapFree: 0 kB
> Dirty: 8 kB
> Writeback: 0 kB
> AnonPages: 19452 kB
> Mapped: 12644 kB
> Shmem: 148 kB
> Slab: 27560 kB
> SReclaimable: 7224 kB
> SUnreclaim: 20336 kB
> KernelStack: 3104 kB
> PageTables: 1896 kB
> NFS_Unstable: 0 kB
> Bounce: 0 kB
> WritebackTmp: 0 kB
> CommitLimit: 60600 kB
> Committed_AS: 1576740 kB
> VmallocTotal: 777216 kB
> VmallocUsed: 53696 kB
> VmallocChunk: 689148 kB
> --------------------
>
> These are my calculations from the above fields.
> ---------------------
> MemFree 2540
> Active 42604
> Inactive 24308
> Mapped 12644
> Slab 27560
> KernelStack 3104
> PageTables 1896
> Shmem 148
> init freed 400
> Diff from boot total 1080
> min_free_kbytes 1384
> user_reserve_kbytes 3537
> My Total ==> 121205  (add all the above)
>
> As you can see "My Total" closely matches with the MemTotal from /proc/meminfo
>
> But on some systems it does not match.
> So, I wanted to understand if there is something wrong in my
> calculation, or there is something wrong in the system itself (like
> leaks etc,)
>
> Note: I don't want to use any tools or utilize for these, but I wanted
> to manually understand it.
>
> I am also trying to explore every vm counter that populates these fields.
>
> But if you have any other opinion about it please let me know.
>
>
> Thanks,
> Pintu