What will happen if 2 processes map same physical page
Hi all, The question is it ipossiblle in Linux/MMU/TLB that 2 processes map to the same physical address? Will CPU or TLB discover that second process tries to reach occupied physical page? What if first process set page permission to read and second whats to write to this page ? Perhaps during context switch all page access permissions of first process is flashed out from MMU ? I confess I do not know what exactly happens in TLB and between Linux and hardware view of pages. I am particularly intereste in ARMv7, perhaps its architecture have some solutions to these question. Regards, Lev
On Wed, 20 Mar 2019 16:42:39 +0300, Lev Olshvang said:
The question is it ipossiblle in Linux/MMU/TLB that 2 processes map to the same physical address?
Totally possible. That's how mmap shared memory works, and why shared libraries are possible.
Will CPU or TLB discover that second process tries to reach occupied physical page?
Well, the hardware won't discover it as a "second" process, it only knows it's processing *this* memory access.
What if first process set page permission to read and second whats to write to this page ?
Perfectly OK - the two processes have separate page table mappings, with separate permission bits. So (for example) physical page 0x17F000 is mapped to virtual address 0x2034D000 with read-only permission n process 1's page tables, and to virtual address 0x98FF3000 with read-write permission in process 2's page tables. No problem. (And before you ask, yes it's possible for process 2 to running on one core doing a write to the page at the exact same time that process 1 is doing a read on another core. Depending on the hardware cache design, this may or may not get process 1 updated data. This is why locking and memory barriers are important. See Documentation/memory-barriers.txt for more details) "And then there's the Alpha" - a processor design that got much of its speed by being weird about this stuff. :)
Perhaps during context switch all page access permissions of first process is flashed out from MMU ?
Actually, the kernel just points the MMU at a new set of page table entries and lets the TLB reload as needed. In particular, on most architectures, the kernel tries really hard to ensure that all processes share at least part of their page table mappings so the kernel is always mapped at the same place, meaning that there's a better chance that on a syscall, the TLB already has hot entries for large parts of the kernel so no TLB reloads are needed.
Hi Vaaldis, Thanks for answer, I still wondering whether the kernel will allow write to a read-only page of shared library while it has mapped to several processes? Kernel knows that page's reference count >1, will it allow mmap/mprotect to change page protection ? Or will it allow direct right by physical address? I suppose that CPU should raise page fault when write is made to read only page, What is the sequence CPU raises page faul before write to page of after data is written Will CPU wait until kernel will consider what to do , whether agree and change PTE "writable " bit to 1 ? Or kernel may disagree and raise SEGFAULT? I checked in the handle_mm_fault() calls for arch_vma_access_permitted() which just returns true on most architectures which is very strange and contradicts my prediction of SEFFAULT. arch_vma_access_permitted() retutus true when is sees that access is made from foreign process? https://elixir.bootlin.com/linux/latest/ident/arch_vma_access_permitted I am totally confused. What do you think ? Regards, Lev 20.03.2019, 20:08, "Valdis Klētnieks" <valdis.kletnieks@vt.edu>:
On Wed, 20 Mar 2019 16:42:39 +0300, Lev Olshvang said:
The question is it ipossiblle in Linux/MMU/TLB that 2 processes map to the same physical address?
Totally possible. That's how mmap shared memory works, and why shared libraries are possible.
Will CPU or TLB discover that second process tries to reach occupied physical page?
Well, the hardware won't discover it as a "second" process, it only knows it's processing *this* memory access.
What if first process set page permission to read and second whats to write to this page ?
Perfectly OK - the two processes have separate page table mappings, with separate permission bits. So (for example) physical page 0x17F000 is mapped to virtual address 0x2034D000 with read-only permission n process 1's page tables, and to virtual address 0x98FF3000 with read-write permission in process 2's page tables. No problem.
(And before you ask, yes it's possible for process 2 to running on one core doing a write to the page at the exact same time that process 1 is doing a read on another core. Depending on the hardware cache design, this may or may not get process 1 updated data. This is why locking and memory barriers are important. See Documentation/memory-barriers.txt for more details)
"And then there's the Alpha" - a processor design that got much of its speed by being weird about this stuff. :)
Perhaps during context switch all page access permissions of first process is flashed out from MMU ?
Actually, the kernel just points the MMU at a new set of page table entries and lets the TLB reload as needed. In particular, on most architectures, the kernel tries really hard to ensure that all processes share at least part of their page table mappings so the kernel is always mapped at the same place, meaning that there's a better chance that on a syscall, the TLB already has hot entries for large parts of the kernel so no TLB reloads are needed.
On Thu, 21 Mar 2019 12:56:17 +0300 Lev Olshvang <levonshe@yandex.com> wrote:
Hi Vaaldis,
Thanks for answer, I still wondering whether the kernel will allow write to a read-only page of shared library while it has mapped to several processes? Kernel knows that page's reference count >1, will it allow mmap/mprotect to change page protection ? Or will it allow direct right by physical address? I suppose that CPU should raise page fault when write is made to read only page,
What is the sequence CPU raises page faul before write to page of after data is written Will CPU wait until kernel will consider what to do , whether agree and change PTE "writable " bit to 1 ? Or kernel may disagree and raise SEGFAULT?
Note that each process has its own PTE. So PTE in one process may say the page is writable and PTE in another process may say it's read-only.
I checked in the handle_mm_fault() calls for arch_vma_access_permitted() which just returns true on most architectures which is very strange and contradicts my prediction of SEFFAULT. arch_vma_access_permitted() retutus true when is sees that access is made from foreign process? https://elixir.bootlin.com/linux/latest/ident/arch_vma_access_permitted
I am totally confused.
What do you think ?
Regards, Lev
It looks like there are two separate questions in the email. 1) Will kernel allow the same physical page to be mapped as read-only in one process and as read-write in another process? 2) How page fault is generated? Answer for first is yes. Same physical page can be mapped with different permissions in two different processes. It means read-only process will ultimately (hopefully very soon) notice changes made by read-write process. Answer for second question is a bit complicated. However there is a trick to it. Once we know that, rest will become clear automaticaly. The trick (at least for x86 systems) is that permissions are maintained at two different levels: - VMA level - PTE level (or PUD level for larger page size but that is not relevant here) When a page in memory is accessed, permission on corresponding VMA is checked first. If the access is allowed by VMA then PTE permissions are checked. Otherwise segfault is generated. If permissions at PTE level don't match the access type then a page fault is generated. That's when page fault hander kicks in and tries to resolve the problem by faulting the page into RAM, copying the page in RAM (for copy-on-write) etc.
20.03.2019, 20:08, "Valdis Klētnieks" <valdis.kletnieks@vt.edu>:
On Wed, 20 Mar 2019 16:42:39 +0300, Lev Olshvang said:
The question is it ipossiblle in Linux/MMU/TLB that 2 processes map to the same physical address?
Totally possible. That's how mmap shared memory works, and why shared libraries are possible.
Will CPU or TLB discover that second process tries to reach occupied physical page?
Well, the hardware won't discover it as a "second" process, it only knows it's processing *this* memory access.
What if first process set page permission to read and second whats to write to this page ?
Perfectly OK - the two processes have separate page table mappings, with separate permission bits. So (for example) physical page 0x17F000 is mapped to virtual address 0x2034D000 with read-only permission n process 1's page tables, and to virtual address 0x98FF3000 with read-write permission in process 2's page tables. No problem.
(And before you ask, yes it's possible for process 2 to running on one core doing a write to the page at the exact same time that process 1 is doing a read on another core. Depending on the hardware cache design, this may or may not get process 1 updated data. This is why locking and memory barriers are important. See Documentation/memory-barriers.txt for more details)
"And then there's the Alpha" - a processor design that got much of its speed by being weird about this stuff. :)
Perhaps during context switch all page access permissions of first process is flashed out from MMU ?
Actually, the kernel just points the MMU at a new set of page table entries and lets the TLB reload as needed. In particular, on most architectures, the kernel tries really hard to ensure that all processes share at least part of their page table mappings so the kernel is always mapped at the same place, meaning that there's a better chance that on a syscall, the TLB already has hot entries for large parts of the kernel so no TLB reloads are needed.
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Hello Okash, Actually there were 3 question: Third question is : Is there any way to tell the kernel that PTE of same physical page should be equal in all processes? For example, shared lib mapped from different processes to same physical page must have same PTE, isn it? And the presence of SELinux feature SELINUX_CHECKREQPROT_VALUE indicates for me that kernel somehow knows the correct page protections. (although I do not see in code how it is done) But the question might be rephrased : IMHO Kernel should mandate same PTE flags no matter how many virtual mapping were made to the same physical page. What do you think? 21.03.2019, 13:45, "Okash Khawaja" <okash.khawaja@gmail.com>:
On Thu, 21 Mar 2019 12:56:17 +0300 Lev Olshvang <levonshe@yandex.com> wrote:
Hi Vaaldis,
Thanks for answer, I still wondering whether the kernel will allow write to a read-only page of shared library while it has mapped to several processes? Kernel knows that page's reference count >1, will it allow mmap/mprotect to change page protection ? Or will it allow direct right by physical address? I suppose that CPU should raise page fault when write is made to read only page,
What is the sequence CPU raises page faul before write to page of after data is written Will CPU wait until kernel will consider what to do , whether agree and change PTE "writable " bit to 1 ? Or kernel may disagree and raise SEGFAULT?
Note that each process has its own PTE. So PTE in one process may say the page is writable and PTE in another process may say it's read-only.
I checked in the handle_mm_fault() calls for arch_vma_access_permitted() which just returns true on most architectures which is very strange and contradicts my prediction of SEFFAULT. arch_vma_access_permitted() retutus true when is sees that access is made from foreign process? https://elixir.bootlin.com/linux/latest/ident/arch_vma_access_permitted
I am totally confused.
What do you think ?
Regards, Lev
It looks like there are two separate questions in the email.
1) Will kernel allow the same physical page to be mapped as read-only in one process and as read-write in another process?
2) How page fault is generated?
Answer for first is yes. Same physical page can be mapped with different permissions in two different processes. It means read-only process will ultimately (hopefully very soon) notice changes made by read-write process.
Answer for second question is a bit complicated. However there is a trick to it. Once we know that, rest will become clear automaticaly. The trick (at least for x86 systems) is that permissions are maintained at two different levels:
- VMA level - PTE level (or PUD level for larger page size but that is not relevant here)
When a page in memory is accessed, permission on corresponding VMA is checked first. If the access is allowed by VMA then PTE permissions are checked. Otherwise segfault is generated. If permissions at PTE level don't match the access type then a page fault is generated. That's when page fault hander kicks in and tries to resolve the problem by faulting the page into RAM, copying the page in RAM (for copy-on-write) etc.
20.03.2019, 20:08, "Valdis Klētnieks" <valdis.kletnieks@vt.edu>: > On Wed, 20 Mar 2019 16:42:39 +0300, Lev Olshvang said: >> The question is it ipossiblle in Linux/MMU/TLB that 2 processes >> map to the same physical address? > > Totally possible. That's how mmap shared memory works, and why > shared libraries are possible. > >> Will CPU or TLB discover that second process tries to reach >> occupied physical page? > > Well, the hardware won't discover it as a "second" process, it only > knows it's processing *this* memory access. > >> What if first process set page permission to read and second >> whats to write to this page ? > > Perfectly OK - the two processes have separate page table mappings, > with separate permission bits. So (for example) physical page > 0x17F000 is mapped to virtual address 0x2034D000 with read-only > permission n process 1's page tables, and to virtual address > 0x98FF3000 with read-write permission in process 2's page tables. > No problem. > > (And before you ask, yes it's possible for process 2 to running on > one core doing a write to the page at the exact same time that > process 1 is doing a read on another core. Depending on the > hardware cache design, this may or may not get process 1 updated > data. This is why locking and memory barriers are important. See > Documentation/memory-barriers.txt for more details) > > "And then there's the Alpha" - a processor design that got much of > its speed by being weird about this stuff. :) > >> Perhaps during context switch all page access permissions of >> first process is flashed out from MMU ? > > Actually, the kernel just points the MMU at a new set of page table > entries and lets the TLB reload as needed. In particular, on most > architectures, the kernel tries really hard to ensure that all > processes share at least part of their page table mappings so the > kernel is always mapped at the same place, meaning that there's a > better chance that on a syscall, the TLB already has hot entries > for large parts of the kernel so no TLB reloads are needed.
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Hi Lev, On Fri, 22 Mar 2019 12:15:49 +0300 Lev Olshvang <levonshe@yandex.com> wrote:
Hello Okash,
Actually there were 3 question:
Third question is : Is there any way to tell the kernel that PTE of same physical page should be equal in all processes? Not that I am aware of. You mention SELinux feature which might be a way but I don't know about it.
For example, shared lib mapped from different processes to same physical page must have same PTE, isn it?
This is policy vs. mechanism question. I think kernel - wherever possible - provides mechanism, leaving policy to be determined by user space. So in case of shared lib mapping, it's the dynamic linker which ensures that all mappings are similar across different processes.
And the presence of SELinux feature SELINUX_CHECKREQPROT_VALUE indicates for me that kernel somehow knows the correct page protections. (although I do not see in code how it is done)
But the question might be rephrased : IMHO Kernel should mandate same PTE flags no matter how many virtual mapping were made to the same physical page.
What do you think?
I'm not sure if that would be the best default option. One reason is that PTE flags aren't a property of physical page itself. Instead, PTE flags are a property of a process's view of the physical page. Physical page itself is a series of bytes. Meaning is given to those bytes by the process which accesses them. This is a bit more subjective opinion rather than an objective answer so I'm ot sure if this answers your query. Thanks, Okash
21.03.2019, 13:45, "Okash Khawaja" <okash.khawaja@gmail.com>:
On Thu, 21 Mar 2019 12:56:17 +0300 Lev Olshvang <levonshe@yandex.com> wrote:
Hi Vaaldis,
Thanks for answer, I still wondering whether the kernel will allow write to a read-only page of shared library while it has mapped to several processes? Kernel knows that page's reference count >1, will it allow mmap/mprotect to change page protection ? Or will it allow direct right by physical address? I suppose that CPU should raise page fault when write is made to read only page,
What is the sequence CPU raises page faul before write to page of after data is written Will CPU wait until kernel will consider what to do , whether agree and change PTE "writable " bit to 1 ? Or kernel may disagree and raise SEGFAULT?
Note that each process has its own PTE. So PTE in one process may say the page is writable and PTE in another process may say it's read-only.
I checked in the handle_mm_fault() calls for arch_vma_access_permitted() which just returns true on most architectures which is very strange and contradicts my prediction of SEFFAULT. arch_vma_access_permitted() retutus true when is sees that access is made from foreign process? https://elixir.bootlin.com/linux/latest/ident/arch_vma_access_permitted
I am totally confused.
What do you think ?
Regards, Lev
It looks like there are two separate questions in the email.
1) Will kernel allow the same physical page to be mapped as read-only in one process and as read-write in another process?
2) How page fault is generated?
Answer for first is yes. Same physical page can be mapped with different permissions in two different processes. It means read-only process will ultimately (hopefully very soon) notice changes made by read-write process.
Answer for second question is a bit complicated. However there is a trick to it. Once we know that, rest will become clear automaticaly. The trick (at least for x86 systems) is that permissions are maintained at two different levels:
- VMA level - PTE level (or PUD level for larger page size but that is not relevant here)
When a page in memory is accessed, permission on corresponding VMA is checked first. If the access is allowed by VMA then PTE permissions are checked. Otherwise segfault is generated. If permissions at PTE level don't match the access type then a page fault is generated. That's when page fault hander kicks in and tries to resolve the problem by faulting the page into RAM, copying the page in RAM (for copy-on-write) etc.
20.03.2019, 20:08, "Valdis Klētnieks" <valdis.kletnieks@vt.edu>: > On Wed, 20 Mar 2019 16:42:39 +0300, Lev Olshvang said: >> The question is it ipossiblle in Linux/MMU/TLB that 2 processes >> map to the same physical address? > > Totally possible. That's how mmap shared memory works, and why > shared libraries are possible. > >> Will CPU or TLB discover that second process tries to reach >> occupied physical page? > > Well, the hardware won't discover it as a "second" process, it only > knows it's processing *this* memory access. > >> What if first process set page permission to read and second >> whats to write to this page ? > > Perfectly OK - the two processes have separate page table mappings, > with separate permission bits. So (for example) physical page > 0x17F000 is mapped to virtual address 0x2034D000 with read-only > permission n process 1's page tables, and to virtual address > 0x98FF3000 with read-write permission in process 2's page tables. > No problem. > > (And before you ask, yes it's possible for process 2 to running on > one core doing a write to the page at the exact same time that > process 1 is doing a read on another core. Depending on the > hardware cache design, this may or may not get process 1 updated > data. This is why locking and memory barriers are important. See > Documentation/memory-barriers.txt for more details) > > "And then there's the Alpha" - a processor design that got much of > its speed by being weird about this stuff. :) > >> Perhaps during context switch all page access permissions of >> first process is flashed out from MMU ? > > Actually, the kernel just points the MMU at a new set of page table > entries and lets the TLB reload as needed. In particular, on most > architectures, the kernel tries really hard to ensure that all > processes share at least part of their page table mappings so the > kernel is always mapped at the same place, meaning that there's a > better chance that on a syscall, the TLB already has hot entries > for large parts of the kernel so no TLB reloads are needed.
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
On Fri, 22 Mar 2019 12:15:49 +0300, Lev Olshvang said:
But the question might be rephrased : IMHO Kernel should mandate same PTE flags no matter how many virtual mapping were made to the same physical page.
And exactly *why* should it be "mandated"? Certainly, for many classes of objects, such as shared libraries, it's a desirable feature (maybe - but see below). However, there's plenty of *other* use cases where the programmer may want to have one control process having read/write access to a memory segment, while a bunch of worker processes are merely reading the data. For instance, if you're serving out complicated computations to sub-processes that involve a lot of parameters and input data, the control process already *has* all this data (potentially megabytes of it) in memory. Using shared memory to transfer it to the worker process is a lot more efficient than having to stuff it all through a socket. And even for shared libraries, you may want one process to be able to write to the space while others are reading it, for live patching and similar functions. (Yes, there's a security trade-off there - and yes, there are sites that will accept the risk, and no, that sort of trade-off belongs in userspace, not in the kernel). The kernel does mechanism, not policy. So it's totally reasonable to have a defined way for userspace to say "this page can only be shared with these permissions" - that's mechanism. Having the kernel force a specific value without a good architectural reason is policy. (Sometimes the kernel does force things to work a specific way if it's required to guarantee system stability. That's why you can't use the write() system call on a directory even if you have write permissions - you can only use stuff like link() and open(). Permissions on shared memory pages don't involve that sort of kernel self-defense issue.
participants (3)
-
Lev Olshvang -
Okash Khawaja -
Valdis Klētnieks