abort when using memory returned by io_remap_pfn_range

[Brian Magnuson]

On Fri, May 24, 2013 at 02:58:24PM -0400, Brian Magnuson wrote:
> Hi,
> 
> I'm attempting to export three regions of memory out to userspace via
> mmap and I've run into a total block.  Of the three regions two are
> reserved areas of DRAM and the last is in IO space and represents
> configuration registers.
> 
> The device is a Zynq-7000 Xilinx FPGA so it's a ARMv7 architecture.
> 
> All three calls to *remap_pfn_range succceed, non-NULL pointers are
> returned to user space, and the two DRAM regions work fine but any
> attempt to access the IO region from a user program fails with:
> 
> Unhandled fault: imprecise external abort (0xc06) at 0xb6fe9000
> 
> The code is below where ETA1_REGS_PHYS_OFFSET (the IO region) is
> 0x60E0_0000.
> 
> I can mmap /dev/mem at this address and poke at it without any issues.
> So... any help on what I'm doing wrong here?  I've spent the better part
> of two days on this and haven't a clue.
> 
> Incidentally, I'm using memmap on the kernel command line to reserve the
> DRAM areas.  I'm thinking maybe there's probably a better way to reserve
> large amounts of physically continguous memory?
> 
> Thanks,
> -Brian
> 
> 
> static int eta1_mmap(struct file *filp, struct vm_area_struct *vma) {
>    int ret, map;
> 
>    // mmap offset, size, physical memory address, IO
>    static uint32_t maps[3][4] = {
>       { ETA1_TX_BUFFER_VIRT_OFFSET, ETA1_TX_BUFFER_SIZE, ETA1_TX_BUFFER_PHYS_OFFSET, 1 }, //TX    64MB
>       { ETA1_RX_BUFFER_VIRT_OFFSET, ETA1_RX_BUFFER_SIZE, ETA1_RX_BUFFER_PHYS_OFFSET, 1 }, //RX    64MB
>       { ETA1_REGS_VIRT_OFFSET,      ETA1_REGS_SIZE,      ETA1_REGS_PHYS_OFFSET,      0 }  //Regs  16KB
>    };
> 
>    for (map = 0; map < 3; map++) {
>       if (vma->vm_pgoff == maps[map][0] >> PAGE_SHIFT) {
>          if (vma->vm_end - vma->vm_start != maps[map][1]) {
>             printk(KERN_ERR "Buffer %d size doesn't match\n", map);
>             return -EAGAIN;
>          }
>          if (maps[map][3]) { //DRAM
>             ret = remap_pfn_range(vma, vma->vm_start,
>                                  (unsigned long)(maps[map][2] >> PAGE_SHIFT),
>                                   vma->vm_end - vma->vm_start, vma->vm_page_prot);
>          } else { //IO memory - This just a define to remap_pfn_range on ARM but let's be nice
>             ret = io_remap_pfn_range(vma, vma->vm_start,
>                                  (unsigned long)(maps[map][2] >> PAGE_SHIFT),
>                                   vma->vm_end - vma->vm_start, vma->vm_page_prot);
>          }
>          return ret;
>       }
>    }
> 
>    printk(KERN_ERR "Didn't match base offset for any range\n");
>    return -EAGAIN;
> }
> 
> user:
> 
> void *mmap_wrap(int fd, size_t length, off_t offset) {
>    void *regs;
> 
>    regs = mmap(NULL, length, PROT_READ | PROT_WRITE, MAP_SHARED, fd, offset);
>    if (regs == MAP_FAILED) {
>       perror("mmap");
>       return NULL;
>    }
>    return regs;
> }
> 
> int main(void) {
>    int fd = open("/dev/eta1", O_RDWR | O_SYNC);
>    if (fd == -1) {
>       perror("Could not open device");
>       exit(-1);
>    }
> 
>    uint32_t *regs;
>    regs   = (uint32_t *)mmap_wrap(fd, ETA1_REGS_SIZE, ETA1_REGS_VIRT_OFFSET);
> 
>    regs[0] = 0x1; //BOOM
> }

Just a note that I managed to fix this myself.  Adding this before the
remap of the io region prevents any SIGBUS firings.

vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);

This isn't totally satisfing since it's not clear to me why disabling
caching prevents the SIGBUS.  It's certainly something I wanted to do
anyway since I want a bus cycle issued for every read/write, but that
should have just caused stale data to be around rather than crash.

This particular peripheral doesn't like anything other than 32b word
aligned accesses so maybe that has something to do with it?

-Brian