Thanks. I was reading the code of kernel 3.0. XFS starts to support concurrent direct IO since kernel 3.1.5. 
But concurrent direct IO write still doesn't work well in kernel 3.2. I wrote a test program that accesses a 4G file randomly (read and write), and I ran it with 8 threads and the machine has 8 cores. It turns out that only 1 core is running. I'm pretty sure xfs_rw_ilock is locked with XFS_IOLOCK_SHARED in xfs_file_dio_aio_write. 

lockstat shows me that there is a lot of wait time in ip->i_lock. It seems the lock is locked exclusively.
               &(&ip->i_lock)->mr_lock-W:         31568          36170           0.24       20048.25     7589157.99         130154        3146848           0.00         217.70     1238310.72
               &(&ip->i_lock)->mr_lock-R:         11251          11886           0.24       20043.01     2895595.18          46671         526309           0.00          63.80      264097.96
               -------------------------
                 &(&ip->i_lock)->mr_lock          36170          [<ffffffffa03be122>] xfs_ilock+0xb2/0x110 [xfs]
                 &(&ip->i_lock)->mr_lock          11886          [<ffffffffa03be15a>] xfs_ilock+0xea/0x110 [xfs]
               -------------------------
                 &(&ip->i_lock)->mr_lock          38555          [<ffffffffa03be122>] xfs_ilock+0xb2/0x110 [xfs]
                 &(&ip->i_lock)->mr_lock           9501          [<ffffffffa03be15a>] xfs_ilock+0xea/0x110 [xfs]

Then I used systemtap to instrument xfs_ilock and find there are at least 3 functions that lock ip->i_lock exclusively during write.
Is there any popular FS that supports concurrent direct IO well?

Thanks,
Da

On Sat, Jan 14, 2012 at 6:45 AM, Raghavendra D Prabhu <raghu.prabhu13@gmail.com> wrote:
Hi Zheng,


* On Fri, Jan 13, 2012 at 04:41:16PM -0500, Zheng Da <zhengda1936@gmail.com> wrote:
Hello,

I'm looking for a FS in Linux that supports concurrent direct IO write.
ext4 supports concurrent direct IO read if we mount it with dioread_nolock,
but doesn't support concurrent writes. XFS doesn't support concurrent
direct IO at all. It locks the inode exclusive if it's direct IO. I tried
btrfs, and it seems it doesn't support concurrent direct IO either though I
haven't looked into its code.
Is there a local FS that support concurrent direct IO write? It seems NFS
supports it (
http://kevinclosson.wordpress.com/2011/08/12/file-systems-for-a-database-choose-one-that-couples-direct-io-and-concurrent-io-whats-this-have-to-do-with-nfs-harken-back-5-2-years-to-find-out/),
but I'm looking for local FS.

Thanks,
Da

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

XFS locks inode exclusive only if it is an unaligned Direct IO, which is apparently done to prevent race conditions -- refer to this  http://oss.sgi.com/archives/xfs/2011-01/msg00157.html Also the behavior of Ext4 under dioread_nolock is supported by XFS by default and in a much better way. Also Ext4 is the only one which uses DIO_LOCKING while doing direct io.



Regards,
--
Raghavendra Prabhu
GPG Id : 0xD72BE977
Fingerprint: B93F EBCB 8E05 7039 CD3C A4B8 A616 DCA1 D72B E977
www: wnohang.net