OOT: sharing about my research about why stat and ls show difference used block count upon a file
Hi all... As the subject says, here is my blog post that discuss that issue: http://the-hydra.blogspot.com/2011/07/stat-or-ls-which-one-showing-correct.h... Few kind people already share their opinions on Linkedin. Here is the summary so far: http://the-hydra.blogspot.com/2011/07/feedback-regarding-my-stat-or-ls-post.... Have opinions? Please share and let's discuss it. And thanks in advance for your time reading my blog. PS: I thank Manish Katiyar for his help during this research. Thanks man! -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
Hello. Nice post. I had never noticed that. And I am able to reproduce the ls-stat behavior on a debian box with ext4 fs and no SELinux, or any other ACL's whatsoever. In any case, from here, On Jul 2, 2011, at 8:00 AM, Mulyadi Santosa wrote:
Few kind people already share their opinions on Linkedin. Here is the summary so far: http://the-hydra.blogspot.com/2011/07/feedback-regarding-my-stat-or-ls-post....
You state:
But my friend pointed that stat was accouting extra blocks that might (I say "might" because my friend is not so sure) contain metadata such as SELinux and ACL.
I am no ext3 expert, but I cannot wrap my head around the FS accounting for *metadata* blocks when stat is invoked. And I don't believe the ACLs would be kept within data blocks either, so this makes little sense to me. In any case, after looking at ext3, I now believe the 'getattr' method in 'struct inode_operations' is left for the VFS to handle with its 'generic_fillattr()' method, and it pretty much copies everything relevant from the inode to the 'struct kstat'. Thing is, does the inode 'i_blocks' field keep track of both metadata and data blocks, or only data blocks? (I think only the latter makes any sense, but hey, that's just me). --- João Eduardo Luís gpg key: 477C26E5 from pool.keyserver.eu
Hi Joao! :) 2011/7/2 João Eduardo Luís <jecluis@gmail.com>:
Hello.
Nice post. I had never noticed that. And I am able to reproduce the ls-stat behavior on a debian box with ext4 fs and no SELinux, or any other ACL's whatsoever.
OK, so far I can conclude it's not 100% reproducible on every case...and it looks it is indeed due to enabled ACL and/or SELinux...hmmmmmm
You state:
But my friend pointed that stat was accouting extra blocks that might (I say "might" because my friend is not so sure) contain metadata such as SELinux and ACL.
I am no ext3 expert, but I cannot wrap my head around the FS accounting for *metadata* blocks when stat is invoked. And I don't believe the ACLs would be kept within data blocks either, so this makes little sense to me.
I am also still digging a bit further about that. Spare time is my only problem now plus my health. So, I am open for suggestion for other people here. But still, thanks for sharing your opinions. It means a lot for me (who is no expert in fs :D )
In any case, after looking at ext3, I now believe the 'getattr' method in 'struct inode_operations' is left for the VFS to handle with its 'generic_fillattr()' method, and it pretty much copies everything relevant from the inode to the 'struct kstat'. Thing is, does the inode 'i_blocks' field keep track of both metadata and data blocks, or only data blocks? (I think only the latter makes any sense, but hey, that's just me).
Sssshhhh, I felt this will lead into another accounting journey, just like when I wrote about /proc/meminfo...great :D Wonder what tools could help me here.... cscope? watchpoints of gdb..... ufffff -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
2011/7/2 Mulyadi Santosa <mulyadi.santosa@gmail.com>:
Hi Joao! :)
2011/7/2 João Eduardo Luís <jecluis@gmail.com>:
Hello.
Nice post. I had never noticed that. And I am able to reproduce the ls-stat behavior on a debian box with ext4 fs and no SELinux, or any other ACL's whatsoever.
OK, so far I can conclude it's not 100% reproducible on every case...and it looks it is indeed due to enabled ACL and/or SELinux...hmmmmmm
You state:
But my friend pointed that stat was accouting extra blocks that might (I say "might" because my friend is not so sure) contain metadata such as SELinux and ACL.
Hi, After reading a bit of ext3 code, I'm now fairly sure the blocks allocated for extended attributes are counted as i_blocks (which stat returns) for an inode. Extented attributes for an inode can either reside in the inode or can at max span 1 filesystem block{size.}. The extra allocated block number for the inode is stored in i_file_acl field of the inode. As an example, kernel has various code paths which given a symlink tries to determine whether the symlink is a fast or a slow one. "fast" means the path is less than 64 bytes long and resides in inode itself while slow means the path exists in a block and the block needs to be loaded to resolve the symlink. So for fast symlinks i_blocks should be 0 for an inode. Now have a look at the code which determines whether the symlink is fast or slow. /* * Test whether an inode is a fast symlink. */ static int ext3_inode_is_fast_symlink(struct inode *inode) { int ea_blocks = EXT3_I(inode)->i_file_acl ? (inode->i_sb->s_blocksize >> 9) : 0; return (S_ISLNK(inode->i_mode) && inode->i_blocks - ea_blocks == 0); } Further reading revealed that we do allocate a new block for inode when we set xattrs. static int ext3_xattr_block_set() { .......... ext3_new_block(); ..... } So, I think if you have extended attributes which reside in an external block, it will be counted in i_blocks. -- Thanks - Manish
Dear Manish... On Sun, Jul 3, 2011 at 03:13, Manish Katiyar <mkatiyar@gmail.com> wrote:
After reading a bit of ext3 code, I'm now fairly sure the blocks allocated for extended attributes are counted as i_blocks (which stat returns) for an inode. Extented attributes for an inode can either reside in the inode or can at max span 1 filesystem block{size.}. The extra allocated block number for the inode is stored in i_file_acl field of the inode.
Thank you very much for your help! It was really confusing, but slowly I gather the pieces now. I got feedback from Linkedin group too, I'll share about that later. Again, thanks a lot -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
participants (3)
-
João Eduardo Luís -
Manish Katiyar -
Mulyadi Santosa