Remote I/O bus

[Greg KH]

On Fri, Oct 04, 2019 at 04:08:06PM +0200, Luca Ceresoli wrote:
> Hi Greg,
> 
> On 04/10/19 15:22, Greg KH wrote:
> > On Fri, Oct 04, 2019 at 01:04:56PM +0200, Luca Ceresoli wrote:
> >> Hi,
> >>
> >> on an embedded system I currently have a standard platform device:
> >>
> >> .-----.  data  .--------.
> >> | CPU |--------| DEVICE |
> >> '-----'   bus  '--------'
> >>
> >> The driver is a standard platform driver that uses ioread32() and
> >> iowrite32() to access registers.
> >>
> >> So far, so good.
> >>
> >> Now in a new design I have the same device in an FPGA, external to the
> >> SoC. The external FPGA is not reachable via an I/O bus, but via SPI (or
> >> I2C). A microprocessor in the FPGA acts as a bridge: as an SPI client it
> >> receives register read/write requests from the CPU, forwards them to the
> >> devices on the in-FPGA data bus as a master, then sends back the replies
> >> over SPI.
> >>
> >>                        SoC <- | -> FPGA
> >>
> >> .-----.  data   .---------.       .--------.  data   .--------.
> >> | CPU |---------| SPI CTL |-------| BRIDGE |---------| DEVICE |
> >> '-----'  bus A  '---------'  SPI  '--------'  bus B  '--------'
> >>
> >>
> >> What would be a proper way to model this in the Linux kernel?
> >>
> >> Of course I can hack the drivers to hijack them on SPI, but I'm trying
> >> to solve the problem in a better way. IMO "a proper way" implies that
> >> the platform driver does not need to be aware of the existence of the
> >> bridge.
> >>
> >> Note: in the real case there is more than one device to handle.
> >>
> >> At first sight I think this should be modeled with a "bridge" device that:
> >>
> >>  * is a SPI device
> >>  * implements a "platform bus" where regular platform devices can be
> >>    instantiated, similar to a "simple-bus"
> > 
> > Yes, make your own "bus", and have the SPI device be your "host
> > controller" in that it bridges the SPI bus to your "FPGA bus".
> > 
> > The driver model is set up for this, it should be not that complex to do
> > so.  If you have specific questions, just let me know.  "Clean" examples
> > of what to do is the greybus code as that's probably one of the newest
> > busses to be added to the kernel.
> > 
> >> In device tree terms:
> >>
> >> &amba { /* data bus A in picture */
> >>
> >>     spi0: spi at 42000000 {
> >>         reg = <0x42000000 0x1000>;
> >>         #address-cells = <1>;
> >>
> >>         io-over-spi-bridge at 1 { /* data bus B */
> >>             reg = <1>; /* slave select pin 1 */
> >>             compatible = "linux,io-over-spi-bridge";
> >>             #address-cells = <1>;
> >>             #size-cells = <1>;
> >>
> >>             mydevice at 4000 {
> >>                 /* 1 kB I/O space at 0x4000 on bus B */
> >>                 reg = <0x4000 0x1000>;
> >>             };
> >>         };
> >>     };
> >> };
> >>
> >> The io-over-spi driver is supposed to request allocation of a virtual
> >> memory area that:
> >>  1. is as large as the address space on bus B
> >>  2. is __iomem (non cached, etc)
> >>  3. is not mapped on the physical CPU address space (bus A)
> >>  4. page faults at every read/write access, triggering a callback
> >>     that starts an SPI transaction, waits for the result and returns
> > 
> > I don't think you can map memory to be "on an SPI bus", unless you have
> > support for that in your hardware controller itself.  Trying to map
> > memory in this way is odd, just treat the devices out off the bus as
> > "devices that need messages sent to them", and you should be fine.  It's
> > not memory-mapped iomemory, so don't think of it that way.
> 
> If I got you correctly, this means I cannot reuse the existing device
> drivers unmodified as I was hoping to.

You are switching from a "ioread/write" to "all data goes across an SPI
link".  No, you can't reuse the existing drivers, but you can modify
them to abstract out the "read/write data" functions to be transport
agnositic.

> They won't be 'struct platform_device' instances anymore, they will
> become 'struct mybus_device' instances. And as such they won't be
> allowed to call ioread32() / iowrite32(), but will have to call
> mybus_ioread32() and mybus_iowrite32(). Correct?

Yes.

But, if you do it right, the majority of your driver is the logic to
control the hardware, and interact with whatever other subsystem those
devices talk to.  Read/Write data and the bus the device talks to should
just be a tiny shim that you can split out into a separate module/file.

Do you have a pointer to your existing code anywhere?

thanks,

greg k-h