How to handle page allocation when memory exceeds a local node
Suppose that there are two nodes and each node has 16GiB memory size. When a process needs pages from the kernel, I understand that __alloc_pages_nodemask() will do allocation for the process. In the function, I noticed that if there is no available page on a zone (or a node) while it loops for_next_zone_zonelist_nodemask() (a macro in get_page_from_freelist()), the kernel would try to reclaim reclaimable page from the zone. This loops will continue until the kernel gets a free page. This means that the kernel would try to find a page from the remote node if there is no free page in the local node. My question is that when the kernel starts to allocate a page from the remote node due to no enough memory in the local node, does the kernel always check there is a reclaimable page from the local node? In other words, does the kernel always start to find a page from the very first entry (which would be the local node and the first zone) in the loop? In my perspective, if the kernel starts to allocate in the remote node, I think the scheduler should move the process to the remote node and it will allocate a page in the remote node at first in the loop (in the process view, the node would be the local now since it is moved). Would the scheduler do that? Regards, WK Choe
On Thu, 22 Aug 2019 17:41:22 +0900, Won-Kyo Choe said:
In my perspective, if the kernel starts to allocate in the remote node, I think the scheduler should move the process to the remote node and it will allocate a page in the remote node at first in the loop (in the process view, the node would be the local now since it is moved). Would the scheduler do that?
That's not the scheduler's job to do that. Plus... what do you do about the case where a process already has 12G of memory on one node, that node runs out and 1 4K page gets allocated on another node. Which is better, move the 12G, or every once in a while try to relocate that 1 4K page to a better node?
On Thu, Aug 22, 2019 at 06:00:01AM -0400, Valdis Klētnieks wrote:
On Thu, 22 Aug 2019 17:41:22 +0900, Won-Kyo Choe said:
In my perspective, if the kernel starts to allocate in the remote node, I think the scheduler should move the process to the remote node and it will allocate a page in the remote node at first in the loop (in the process view, the node would be the local now since it is moved). Would the scheduler do that?
That's not the scheduler's job to do that. Plus... what do you do about the case where a process already has 12G of memory on one node, that node runs out and 1 4K page gets allocated on another node. Which is better, move the 12G, or every once in a while try to relocate that 1 4K page to a better node?
I appreciate it for your answer. Firstly, I would say that the latter is better because of the small size but what do you exactly refer to a better node? Could you explain more specifically? I am not sure but I think that a better node means a node where the process can show better performance and this would be done by the scheduler in a certain way. Secondly, I think that you assume there should always be migration when the process has allocated pages in the remote node to try to exploit performance benefits on the local node. Is it a default policy of the kernel? My concern is that what if the process uses the whole memory capacity with two nodes and the process has one thread? In this case could I still get expectation that there is a myriad of migration or exchange between two nodes? Regards, WK Choe
participants (2)
-
Valdis Klētnieks -
Won-Kyo Choe