Question about page count
Hi, I am currently analyzing the memory pages footprint in 2.6.18.8 kernel (1GB ram, 32-bit, Flat model, no NUMA), and got a question. Does page_count(page)==0 really means the page is free because the source code says so? If I just traverse mem_map for all memory descriptor, and count how many of them has page_count 0, then the number of free page will exceed the MemFree number reported from /proc/meminfo (actually from the zone list). After some investigation, I figure out that page->flags actually encode the zone information (DMA, NORMAL, HIGHMEM), and some of the page descriptors do not have correct zone number. But filtering them out doesn't help. In my case, HIGHMEM has 61 free pages reported by /proc/meminfo, but when I traverse the page descriptors array from (896MB - 1024MB), I count the pages with (1) flags stating its zone == ZONE_HIGHMEM (2) page_count(page)==0 My result is 186 but not 61. Could anyone help on explaining this? Thanks, Steve
On Mon, Jun 20, 2011 at 9:10 PM, Jui-Hao Chiang <windtracekimo@gmail.com> wrote:
Hi,
I am currently analyzing the memory pages footprint in 2.6.18.8 kernel (1GB ram, 32-bit, Flat model, no NUMA), and got a question. Does page_count(page)==0 really means the page is free because the source code says so?
If I just traverse mem_map for all memory descriptor, and count how many of them has page_count 0, then the number of free page will exceed the MemFree number reported from /proc/meminfo (actually from the zone list).
After some investigation, I figure out that page->flags actually encode the zone information (DMA, NORMAL, HIGHMEM), and some of the page descriptors do not have correct zone number. But filtering them out doesn't help.
In my case, HIGHMEM has 61 free pages reported by /proc/meminfo, but when I traverse the page descriptors array from (896MB - 1024MB), I count the pages with (1) flags stating its zone == ZONE_HIGHMEM (2) page_count(page)==0 My result is 186 but not 61.
The problem is that we can't see the details of your caculation. Sugguest you take a look at function nr_free_highpages in linux/mm/highmem.c.
Could anyone help on explaining this?
-- Shenzhen, CN twitter: @izhangxc
Hi... On 20/06/2011, Jui-Hao Chiang <windtracekimo@gmail.com> wrote:
Hi,
I am currently analyzing the memory pages footprint in 2.6.18.8 kernel (1GB ram, 32-bit, Flat model, no NUMA), and got a question. Does page_count(page)==0 really means the page is free because the source code says so?
looking at how page_count implemented, it checks whether the related page is gonna be reclaimed or a compound page (not sure what it is). If so, returns the first page of that "page". by comparing the definition of free page as "really free, not being used for any purpose" versus the above perception, I guess it's not surprising to see the difference -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
Hey Mulyadi, On Mon, Jun 20, 2011 at 10:15 PM, Mulyadi Santosa <mulyadi.santosa@gmail.com> wrote: ...snip...
looking at how page_count implemented, it checks whether the related page is gonna be reclaimed or a compound page (not sure what it is). If so, returns the first page of that "page".
On the ARM, you can setup the MMU to map 1Mb regions or 4Kb regions. A 1 Mb region only goes through the top level of the MMU, whereas a 4K page has to go through 2 levesl, so accessing pages through the 1 Mb mapping is faster. I believe (I'm not 100% sure) that the 256 x 4K pages that make up the 1Mb region are a compound page. -- Dave Hylands Shuswap, BC, Canada http://www.davehylands.com
Hi Dave :) On Tue, Jun 21, 2011 at 20:23, Dave Hylands <dhylands@gmail.com> wrote:
On the ARM, you can setup the MMU to map 1Mb regions or 4Kb regions. A 1 Mb region only goes through the top level of the MMU, whereas a 4K page has to go through 2 levesl, so accessing pages through the 1 Mb mapping is faster.
I believe (I'm not 100% sure) that the 256 x 4K pages that make up the 1Mb region are a compound page.
Oh, very valuable info...thanks for sharing that...yes, quite make sense to me. Anyway, I also found this info in Documentation/vm/pagemap.txt inside kernel tree. I paste here so everybody could take a quick peek: "The major consumers of compound pages are hugeTLB pages (Documentation/vm/hugetlbpage.txt), the SLUB etc. memory allocators and various device drivers. However in this interface, only huge/giga pages are made visible to end users." -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
Hi, Mulyadi: My kernel module actually does something similar to the following struct page *page = mem_map; for (i = 0; i < 262144; i++, page++) printk("[%d] flags %x, count %x, mapcount %x, private %x, mapping %x, " "index %x, lrunext %x, lruprev %x\n", i, page->flags, page->_count, page->_mapcount, page->private, page->mapping, page->index, page->lru.next, page->lru.prev); (1) no compound page has been found during this printing (2) I seemed to figure out how to compute the free page as meminfo does See the following 4 pages output [20] flags 80000, count 0, mapcount ffffffff, private 2, mapping 0, index 0, lrunext c069789c, lruprev c1001318 [21] flags 0, count 0, mapcount ffffffff, private 0, mapping 0, index 0, lrunext 100100, lruprev 200200 [22] flags 0, count 0, mapcount ffffffff, private 0, mapping 0, index 0, lrunext 100100, lruprev 200200 [23] flags 0, count 0, mapcount ffffffff, private 0, mapping 0, index 0, lrunext 100100, lruprev 200200 The 1st page (page frame 20) has PageBuddy(page)==1 from the flags, and its field private==2 means there are 2^2 contiguous free pages in the subsequent memory frames. So page 20,21, 22, 23 are actually a group (buddy pages) in the zone allocator. There seems no special flags in page 21, 22, 23, so the only way is to search for the PageBuddy and look at the private field to deduce them. With this rule to calculate the free pages, the result equals to MemFree reported from nr_free_pages() or /proc/meminfo (3) However there are some page frames that I can't not figure out the buddy information from the rules found in (2). For example, in the follow pages, I can't find any related buddy pages before them, but their count is still 0. [239339] flags c0000000, count 0, mapcount ffffffff, private 0, mapping 0, index 155, lrunext c17eeb78, lruprev c17912f8 [239340] flags c0000000, count 0, mapcount ffffffff, private 0, mapping 0, index ae, lrunext c178d8f8, lruprev c174e1b8 [239344] flags c0000000, count 0, mapcount ffffffff, private 0, mapping 0, index 93f6, lrunext c069a18c, lruprev c17f5738 That's the part I don't understand. Thanks, Jui-Hao On Tue, Jun 21, 2011 at 1:15 AM, Mulyadi Santosa <mulyadi.santosa@gmail.com> wrote:
Hi...
On 20/06/2011, Jui-Hao Chiang <windtracekimo@gmail.com> wrote:
Hi,
I am currently analyzing the memory pages footprint in 2.6.18.8 kernel (1GB ram, 32-bit, Flat model, no NUMA), and got a question. Does page_count(page)==0 really means the page is free because the source code says so?
looking at how page_count implemented, it checks whether the related page is gonna be reclaimed or a compound page (not sure what it is). If so, returns the first page of that "page".
by comparing the definition of free page as "really free, not being used for any purpose" versus the above perception, I guess it's not surprising to see the difference
-- regards,
Mulyadi Santosa Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
participants (4)
-
Dave Hylands -
Jui-Hao Chiang -
Mulyadi Santosa -
Wick