Thanks. I was reading the code of kernel 3.0. XFS starts to support concurrent direct IO since kernel 3.1.5. <div>But concurrent direct IO write still doesn't work well in kernel 3.2. I wrote a test program that accesses a 4G file randomly (read and write), and I ran it with 8 threads and the machine has 8 cores. It turns out that only 1 core is running. I'm pretty sure xfs_rw_ilock is locked with XFS_IOLOCK_SHARED in xfs_file_dio_aio_write. </div>
<div><br></div><div>lockstat shows me that there is a lot of wait time in ip->i_lock. It seems the lock is locked exclusively.</div><div><div> &(&ip->i_lock)->mr_lock-W: 31568 36170 0.24 20048.25 7589157.99 130154 3146848 0.00 217.70 1238310.72</div>
<div> &(&ip->i_lock)->mr_lock-R: 11251 11886 0.24 20043.01 2895595.18 46671 526309 0.00 63.80 264097.96</div><div>
-------------------------</div>
<div> &(&ip->i_lock)->mr_lock 36170 [<ffffffffa03be122>] xfs_ilock+0xb2/0x110 [xfs]</div><div> &(&ip->i_lock)->mr_lock 11886 [<ffffffffa03be15a>] xfs_ilock+0xea/0x110 [xfs]</div>
<div> -------------------------</div><div> &(&ip->i_lock)->mr_lock 38555 [<ffffffffa03be122>] xfs_ilock+0xb2/0x110 [xfs]</div><div> &(&ip->i_lock)->mr_lock 9501 [<ffffffffa03be15a>] xfs_ilock+0xea/0x110 [xfs]</div>
<div><br></div><div>Then I used systemtap to instrument xfs_ilock and find there are at least 3 functions that lock ip->i_lock exclusively during write.</div><div>Is there any popular FS that supports concurrent direct IO well?</div>
<div><br></div><div>Thanks,<br>Da</div><br><div class="gmail_quote">On Sat, Jan 14, 2012 at 6:45 AM, Raghavendra D Prabhu <span dir="ltr"><<a href="mailto:raghu.prabhu13@gmail.com" target="_blank">raghu.prabhu13@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Zheng,<div><div><br>
<br>
* On Fri, Jan 13, 2012 at 04:41:16PM -0500, Zheng Da <<a href="mailto:zhengda1936@gmail.com" target="_blank">zhengda1936@gmail.com</a>> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hello,<br>
<br>
I'm looking for a FS in Linux that supports concurrent direct IO write.<br>
ext4 supports concurrent direct IO read if we mount it with dioread_nolock,<br>
but doesn't support concurrent writes. XFS doesn't support concurrent<br>
direct IO at all. It locks the inode exclusive if it's direct IO. I tried<br>
btrfs, and it seems it doesn't support concurrent direct IO either though I<br>
haven't looked into its code.<br>
Is there a local FS that support concurrent direct IO write? It seems NFS<br>
supports it (<br>
<a href="http://kevinclosson.wordpress.com/2011/08/12/file-systems-for-a-database-choose-one-that-couples-direct-io-and-concurrent-io-whats-this-have-to-do-with-nfs-harken-back-5-2-years-to-find-out/" target="_blank">http://kevinclosson.wordpress.<u></u>com/2011/08/12/file-systems-<u></u>for-a-database-choose-one-<u></u>that-couples-direct-io-and-<u></u>concurrent-io-whats-this-have-<u></u>to-do-with-nfs-harken-back-5-<u></u>2-years-to-find-out/</a>),<br>
but I'm looking for local FS.<br>
<br>
Thanks,<br>
Da<br>
</blockquote>
<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
______________________________<u></u>_________________<br>
Kernelnewbies mailing list<br>
<a href="mailto:Kernelnewbies@kernelnewbies.org" target="_blank">Kernelnewbies@kernelnewbies.<u></u>org</a><br>
<a href="http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies" target="_blank">http://lists.kernelnewbies.<u></u>org/mailman/listinfo/<u></u>kernelnewbies</a><br>
</blockquote>
<br>
XFS locks inode exclusive only if it is an unaligned Direct IO, which is apparently done to prevent race conditions -- refer to this <a href="http://oss.sgi.com/archives/xfs/2011-01/msg00157.html" target="_blank">http://oss.sgi.com/archives/<u></u>xfs/2011-01/msg00157.html</a> Also the behavior of Ext4 under dioread_nolock is supported by XFS by default and in a much better way. Also Ext4 is the only one which uses DIO_LOCKING while doing direct io.<br>
<br>
<br>
<br>
Regards,<span><font color="#888888"><br>
-- <br>
Raghavendra Prabhu<br>
GPG Id : 0xD72BE977<br>
Fingerprint: B93F EBCB 8E05 7039 CD3C A4B8 A616 DCA1 D72B E977<br>
www: <a href="http://wnohang.net" target="_blank">wnohang.net</a><br>
</font></span></blockquote></div><br></div>