PCI / PCIe Device Memory - Rationale for Choosing MMIO Over PMIO (and Visa-Versa)
Hi all, Over the last couple of months, I've been reading the hardware documentation and Linux device driver source code for a range of different PCI and PCIe devices. Those examined range from multi-function data acquisition cards through to avionics bus interface devices. In doing so, I have referenced numerous resources (including the Third Edition of LDD - what a great book - and the documentation available for the Linux PCI Bus Subsystem on kernel.org). One thing I'm still a little unclear on is why vendors might opt to map PCI / PCIe device memory into the system memory map as either Memory-Mapped I/O (MMIO) or Port-Mapped I/O (PMIO). That is, for what reasons would a device manufacturer choose to make use of one address space over the other for regions of a PCI / PCIe device's memory? Some of the general reasons are alluded to by the aforementioned resources (e.g., more instruction cycles are required to access data via PMIO, MMIO can be marked as prefetchable and handled as such, etc). Would anyone who's been engaged in the development of a PCI / PCIe device talk about their experience and what factors led to one address space being chosen over the other (for specific regions of a specific device)? Specific examples would really help me (and probably others) understand what factors are involved in this decision and how a suitable choice is made. Reading the driver source code, for specific devices, has been great for developing an initial understanding of the different approaches taken by device manufacturers, but source code and hardware documentation rarely provide any information on the rationale for a chosen implementation. Any specific technical accounts on this matter, etc, would be much appreciated. Jack
On Thu, Nov 05, 2020 at 02:27:58PM +0000, Jack Winch wrote:
Hi all,
Over the last couple of months, I've been reading the hardware documentation and Linux device driver source code for a range of different PCI and PCIe devices. Those examined range from multi-function data acquisition cards through to avionics bus interface devices. In doing so, I have referenced numerous resources (including the Third Edition of LDD - what a great book - and the documentation available for the Linux PCI Bus Subsystem on kernel.org).
One thing I'm still a little unclear on is why vendors might opt to map PCI / PCIe device memory into the system memory map as either Memory-Mapped I/O (MMIO) or Port-Mapped I/O (PMIO). That is, for what reasons would a device manufacturer choose to make use of one address space over the other for regions of a PCI / PCIe device's memory? Some of the general reasons are alluded to by the aforementioned resources (e.g., more instruction cycles are required to access data via PMIO, MMIO can be marked as prefetchable and handled as such, etc).
I think you should be talking to hardware people about this, as this is almost always due to hardware limitations/issues/design decisions. The PCI-SIG should have a bunch of resources about this, have you looked into that? good luck! greg k-h
Hey Greg,
I think you should be talking to hardware people about this, as this is almost always due to hardware limitations/issues/design decisions.
To be honest, I anticipated a response along these lines (after briefly speaking to another individual on this matter). They commented on how this decision was usually made and asserted by the hardware engineer(s).
The PCI-SIG should have a bunch of resources about this, have you looked into that?
I haven't as of yet, but I'll shift focus onto their resources next. Do any specific resources of theirs spring to mind offhand? No worries if not - I'm not afraid to spend time searching them out. Thanks for the quick response and your time. :) Jack
participants (2)
-
Greg KH -
Jack Winch