mmap how does kernel know a page is mmaped
Hi, Could anyone please explain, how mmap works underneath ? when kernel traveses pgd->pud->pmd->pte how does it know that a particular page is a mmaped page ? is there any special flag ? Thanks, Neophyte
On Thu, 25 Jul 2013 09:14:03 -0700, kernel neophyte said:
Could anyone please explain, how mmap works underneath ? when kernel traveses pgd->pud->pmd->pte how does it know that a particular page is a mmaped page ? is there any special flag ?
Why would the address mapping hardware even *care* that it's an mmap'ed page, once the mapping is set up? For that matter, why would most of the kernel code care? Only time an mmap'ed page is any different than any other process page is while the mmap is actually being set up, modified, or torn down. (And in fact, that's part of why getting the varions sync() calls to play nice with mmap() is so hard - because an mmap'ed file page is just a page. So noticing that a page got modified and knowing to do stuff like update the atime and mtime of the backing file is difficult...)
ok, I am sorry maybe I did not ask the question correctly, all I want to know is how mmap works underneath, given an address X how does kernel figure out its a mmaped page ? -Neo On Thu, Jul 25, 2013 at 10:04 AM, <Valdis.Kletnieks@vt.edu> wrote:
On Thu, 25 Jul 2013 09:14:03 -0700, kernel neophyte said:
Could anyone please explain, how mmap works underneath ? when kernel traveses pgd->pud->pmd->pte how does it know that a particular page is a mmaped page ? is there any special flag ?
Why would the address mapping hardware even *care* that it's an mmap'ed page, once the mapping is set up? For that matter, why would most of the kernel code care?
Only time an mmap'ed page is any different than any other process page is while the mmap is actually being set up, modified, or torn down.
(And in fact, that's part of why getting the varions sync() calls to play nice with mmap() is so hard - because an mmap'ed file page is just a page. So noticing that a page got modified and knowing to do stuff like update the atime and mtime of the backing file is difficult...)
On Thu, 25 Jul 2013 10:52:43 -0700, kernel neophyte said:
I am sorry maybe I did not ask the question correctly, all I want to know is how mmap works underneath, given an address X how does kernel figure out its a mmaped page ?
You missed the point. The kernel never checks if it's an mmap'ed page. It merely checks is this a mapped page, the same way it checks every single other memory reference whether it's a code segment, your stack, your heap... it doesn't care that address 0x3FB0CCF0 is mmap'ed or not. It just cares "does this throw a pagefault when I reference it?" - and then it gets handled just like any other page fault.
On Thu, Jul 25, 2013 at 1:52 PM, kernel neophyte <neophyte.hacker001@gmail.com> wrote:
ok,
I am sorry maybe I did not ask the question correctly, all I want to know is how mmap works underneath, given an address X how does kernel figure out its a mmaped page ?
-Neo
On Thu, Jul 25, 2013 at 10:04 AM, <Valdis.Kletnieks@vt.edu> wrote:
On Thu, 25 Jul 2013 09:14:03 -0700, kernel neophyte said:
Could anyone please explain, how mmap works underneath ? when kernel traveses pgd->pud->pmd->pte how does it know that a particular page is a mmaped page ? is there any special flag ?
Why would the address mapping hardware even *care* that it's an mmap'ed page, once the mapping is set up? For that matter, why would most of the kernel code care?
Only time an mmap'ed page is any different than any other process page is while the mmap is actually being set up, modified, or torn down.
(And in fact, that's part of why getting the varions sync() calls to play nice with mmap() is so hard - because an mmap'ed file page is just a page. So noticing that a page got modified and knowing to do stuff like update the atime and mtime of the backing file is difficult...)
Neo, First be aware this is bottom post only mailing list, as are most of the LKML lists, so if you are going to post on the public lists you will need to start bottom posting. As to your question, the more specific you are the better. There are 3 situations I can think of offhand, but I don't know which one you are interested in: 1) Userspace does a read/write to a data page that is not memory resident and the page is backed by a mmap'ed file. 2) Userspace does a read/write to a data page that is memory resident 3) The kernel decides a memory page is needed for another use and needs to be sync'ed to disk if any data updates are not to be lost. Valdis is saying for case 2) the kernel doesn't care. The kernel is not even in the loop. The userspace app has the memory pages available to it via the standard MMU mapping so the data accesses just work. The kernel in general only has to handle case 1) and case 3) Greg
On Thu, 25 Jul 2013 14:33:21 -0400, Greg Freemyer said:
There are 3 situations I can think of offhand, but I don't know which one you are interested in:
1) Userspace does a read/write to a data page that is not memory resident and the page is backed by a mmap'ed file.
2) Userspace does a read/write to a data page that is memory resident
3) The kernel decides a memory page is needed for another use and needs to be sync'ed to disk if any data updates are not to be lost.
And here, the kernel can cheat some more and re-use existing code. There's 2 parts to it: A) Writeback a page - which can be done the exact same way that any other page gets written back to disk. Only difference is writing to an offset in a file as opposed to an offset in a swap space. And if you consider the case of doing a 'swapon' on a regular disk file, there's no real difference between "write page 38F3E to block 3 of swap space on file /foo/bar/swap3" and "write page 38F3E to block 3 of file /foo/bar/swap3". Similarly, the retrieval of a non-resident page backed by an mmap file is pretty much the same as retrieving a non-resident page backed by swap space. B) Possibly invalidate the mapping, because the page frame is needed. At which point, future references to that page end up in case (1) above.
On Thu, Jul 25, 2013 at 11:33 AM, Greg Freemyer <greg.freemyer@gmail.com>wrote:
On Thu, Jul 25, 2013 at 1:52 PM, kernel neophyte <neophyte.hacker001@gmail.com> wrote:
ok,
I am sorry maybe I did not ask the question correctly, all I want to know is how mmap works underneath, given an address X how does kernel figure out its a mmaped page ?
-Neo
On Thu, Jul 25, 2013 at 10:04 AM, <Valdis.Kletnieks@vt.edu> wrote:
On Thu, 25 Jul 2013 09:14:03 -0700, kernel neophyte said:
Could anyone please explain, how mmap works underneath ? when kernel traveses pgd->pud->pmd->pte how does it know that a particular page
is a
mmaped page ? is there any special flag ?
Why would the address mapping hardware even *care* that it's an mmap'ed page, once the mapping is set up? For that matter, why would most of the kernel code care?
Only time an mmap'ed page is any different than any other process page is while the mmap is actually being set up, modified, or torn down.
(And in fact, that's part of why getting the varions sync() calls to play nice with mmap() is so hard - because an mmap'ed file page is just a page. So noticing that a page got modified and knowing to do stuff like update the atime and mtime of the backing file is difficult...)
Neo,
First be aware this is bottom post only mailing list, as are most of the LKML lists, so if you are going to post on the public lists you will need to start bottom posting.
As to your question, the more specific you are the better.
There are 3 situations I can think of offhand, but I don't know which one you are interested in:
1) Userspace does a read/write to a data page that is not memory resident and the page is backed by a mmap'ed file.
2) Userspace does a read/write to a data page that is memory resident
3) The kernel decides a memory page is needed for another use and needs to be sync'ed to disk if any data updates are not to be lost.
Valdis is saying for case 2) the kernel doesn't care. The kernel is not even in the loop. The userspace app has the memory pages available to it via the standard MMU mapping so the data accesses just work.
The kernel in general only has to handle case 1) and case 3)
I am sorry, its still not clear to me. All I am asking is I want to know and understand how mmap works, given an address *X*, how does the Linux kernel figure out that *X* is an mmaped page? Is there a special flag in the page table entry? Does the access generate a page fault ? If so, how does the handler find out it is an mmaped address? Sorry for the inconvenience caused, Thanks a lot for your time in replying. -Neo
Greg
On Thu, 25 Jul 2013 16:57:44 -0700, kernel neophyte said:
I am sorry, its still not clear to me. All I am asking is I want to know and understand how mmap works, given an address *X*, how does the Linux kernel figure out that *X* is an mmaped page?
Before you ask *how* it does it, first figure out if it does it *at all*. The reason you can't figure out how the swami is levitating is because they aren't actually levitating at all.
Is there a special flag in the page table entry?
No, because no flag is needed.
Does the access generate a page fault ?
Maybe, maybe not. If the page is resident in memory there's no page fault. And if it's not resident, it gets paged in from wherever the backing page happens to be.
If so, how does the handler find out it is an mmaped address?
The handler doesn't *care* if it's mmaped. All it has to know is (1) this page isn't in memory, (2) it needs to be in memory, and (3) so please schedule the I/O to read it from block NNNN of device XXYY just like any *other* page being read in because of a page fault. The *only* thing "magical" about an mmap'ed page is that the pointer to where to read/write it might (sometimes) point at someplace that's not a swap space. Though for some uses of mmap(), it *does* point at swap space (for instance, the anonymous pages created by mmap() as used from malloc() in glibc).
On Thu, Jul 25, 2013 at 5:18 PM, <Valdis.Kletnieks@vt.edu> wrote:
On Thu, 25 Jul 2013 16:57:44 -0700, kernel neophyte said:
I am sorry, its still not clear to me. All I am asking is I want to know and understand how mmap works, given an address *X*, how does the Linux kernel figure out that *X* is an mmaped page?
Before you ask *how* it does it, first figure out if it does it *at all*.
The reason you can't figure out how the swami is levitating is because they aren't actually levitating at all.
:-) Good one!
Is there a special flag in the page table entry?
No, because no flag is needed.
Does the access generate a page fault ?
Maybe, maybe not. If the page is resident in memory there's no page fault. And if it's not resident, it gets paged in from wherever the backing page happens to be.
If so,
how
does the handler find out it is an mmaped address?
The handler doesn't *care* if it's mmaped. All it has to know is (1) this page isn't in memory, (2) it needs to be in memory, and (3) so please schedule the I/O to read it from block NNNN of device XXYY just like any *other* page being read in because of a page fault.
Thanks, I get it now.
The *only* thing "magical" about an mmap'ed page is that the pointer to where to read/write it might (sometimes) point at someplace that's not a swap space.
Though for some uses of mmap(), it *does* point at swap space (for instance,
the anonymous pages created by mmap() as used from malloc() in glibc).
Thanks I get it now.
-Neo
participants (3)
-
Greg Freemyer -
kernel neophyte -
Valdis.Kletnieks＠vt.edu