After reading through the code in inode.c today , I am curious about the comment and the following code I will paste below. I am curious if this corner case is hit often enough for me to write a patch to improve the speed of this corner case. Furthermore , compress_file_range is the function name, in case you can't guess by the pasted code. Regards Nick 411 /* 412 * we don't want to send crud past the end of i_size through 413 * compression, that's just a waste of CPU time. So, if the 414 * end of the file is before the start of our current 415 * requested range of bytes, we bail out to the uncompressed 416 * cleanup code that can deal with all of this. 417 * 418 * It isn't really the fastest way to fix things, but this is a 419 * very uncommon corner. 420 */ 421 if (actual_end <= start) 422 goto cleanup_and_bail_uncompressed;
If it's a corner case, it won't be hit often enough right? And if it was hit often enough, it wouldn't be corner case!? :) These 2 are mutually exclusive! On Tue, Aug 26, 2014 at 3:47 PM, Nick <xerofoify@gmail.com> wrote:
After reading through the code in inode.c today , I am curious about the comment and the following code I will paste below. I am curious if this corner case is hit often enough for me to write a patch to improve the speed of this corner case. Furthermore , compress_file_range is the function name, in case you can't guess by the pasted code. Regards Nick 411 /* 412 * we don't want to send crud past the end of i_size through 413 * compression, that's just a waste of CPU time. So, if the 414 * end of the file is before the start of our current 415 * requested range of bytes, we bail out to the uncompressed 416 * cleanup code that can deal with all of this. 417 * 418 * It isn't really the fastest way to fix things, but this is a 419 * very uncommon corner. 420 */ 421 if (actual_end <= start) 422 goto cleanup_and_bail_uncompressed;
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
On 08/26/2014 06:58 PM, Mandeep Sandhu wrote:
If it's a corner case, it won't be hit often enough right? And if it was hit often enough, it wouldn't be corner case!? :)
These 2 are mutually exclusive!
On Tue, Aug 26, 2014 at 3:47 PM, Nick <xerofoify@gmail.com> wrote:
After reading through the code in inode.c today , I am curious about the comment and the following code I will paste below. I am curious if this corner case is hit often enough for me to write a patch to improve the speed of this corner case. Furthermore , compress_file_range is the function name, in case you can't guess by the pasted code. Regards Nick 411 /* 412 * we don't want to send crud past the end of i_size through 413 * compression, that's just a waste of CPU time. So, if the 414 * end of the file is before the start of our current 415 * requested range of bytes, we bail out to the uncompressed 416 * cleanup code that can deal with all of this. 417 * 418 * It isn't really the fastest way to fix things, but this is a 419 * very uncommon corner. 420 */ 421 if (actual_end <= start) 422 goto cleanup_and_bail_uncompressed;
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies I get that my question is if this corner case is hit, enough for me to write a patch to optimize it. In addition the comment states it isn't but want to known for standard compression workloads in btrfs if it's hit enough for me to work on this and how much speed degradation are me we doing my not writing it better. Nick
Then setup a test system which will reproduce hitting this corner case and instrument your code to see is there's any gain at all. On Tue, Aug 26, 2014 at 4:14 PM, Nick <xerofoify@gmail.com> wrote:
On 08/26/2014 06:58 PM, Mandeep Sandhu wrote:
If it's a corner case, it won't be hit often enough right? And if it was hit often enough, it wouldn't be corner case!? :)
These 2 are mutually exclusive!
On Tue, Aug 26, 2014 at 3:47 PM, Nick <xerofoify@gmail.com> wrote:
After reading through the code in inode.c today , I am curious about the comment and the following code I will paste below. I am curious if this corner case is hit often enough for me to write a patch to improve the speed of this corner case. Furthermore , compress_file_range is the function name, in case you can't guess by the pasted code. Regards Nick 411 /* 412 * we don't want to send crud past the end of i_size through 413 * compression, that's just a waste of CPU time. So, if the 414 * end of the file is before the start of our current 415 * requested range of bytes, we bail out to the uncompressed 416 * cleanup code that can deal with all of this. 417 * 418 * It isn't really the fastest way to fix things, but this is a 419 * very uncommon corner. 420 */ 421 if (actual_end <= start) 422 goto cleanup_and_bail_uncompressed;
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies I get that my question is if this corner case is hit, enough for me to write a patch to optimize it. In addition the comment states it isn't but want to known for standard compression workloads in btrfs if it's hit enough for me to work on this and how much speed degradation are me we doing my not writing it better. Nick
On Tue, 26 Aug 2014, Nick wrote:
On 08/26/2014 06:58 PM, Mandeep Sandhu wrote:
If it's a corner case, it won't be hit often enough right? And if it was hit often enough, it wouldn't be corner case!? :)
These 2 are mutually exclusive!
On Tue, Aug 26, 2014 at 3:47 PM, Nick <xerofoify@gmail.com> wrote:
After reading through the code in inode.c today , I am curious about the comment and the following code I will paste below. I am curious if this corner case is hit often enough for me to write a patch to improve the speed of this corner case. Furthermore , compress_file_range is the function name, in case you can't guess by the pasted code. Regards Nick 411 /* 412 * we don't want to send crud past the end of i_size through 413 * compression, that's just a waste of CPU time. So, if the 414 * end of the file is before the start of our current 415 * requested range of bytes, we bail out to the uncompressed 416 * cleanup code that can deal with all of this. 417 * 418 * It isn't really the fastest way to fix things, but this is a 419 * very uncommon corner. 420 */ 421 if (actual_end <= start) 422 goto cleanup_and_bail_uncompressed;
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies I get that my question is if this corner case is hit, enough for me to write a patch to optimize it. In addition the comment states it isn't but want to known for standard compression workloads in btrfs if it's hit enough for me to work on this and how much speed degradation are me we doing my not writing it better. Nick
Here's how I would go about it: 1. Understand when the case is met (in theory). 2. Try to trigger it on a real system multiple times. 3. Try to explore systematically under what circumstances the case is met and rank them by plausibility (if the notion of plausibility makes any sense in a real world scenario -- I don't know). 4. Estimate cost vs. benefit. I don't know if this is a good way but notice how you can do all this on yourself which I think is a plus for everyone. And if you decide in step 4 to write a patch: 5. Use your results from step 3 to create an environment that benefits from your patch (notice how 4 guarantees that there exists such a system with reasonable connection to real needs). Note the numbers. 6. Test your patch on as many regular configurations as possible. Note the numbers. If it degrades performance on any of those, abort. 7. Do *NOT* send the patch out. Regards, Tobi -- "There's an old saying: Don't change anything... ever!" -- Mr. Monk
On 08/26/2014 08:05 PM, Tobias Boege wrote:
On Tue, 26 Aug 2014, Nick wrote:
On 08/26/2014 06:58 PM, Mandeep Sandhu wrote:
If it's a corner case, it won't be hit often enough right? And if it was hit often enough, it wouldn't be corner case!? :)
These 2 are mutually exclusive!
On Tue, Aug 26, 2014 at 3:47 PM, Nick <xerofoify@gmail.com> wrote:
After reading through the code in inode.c today , I am curious about the comment and the following code I will paste below. I am curious if this corner case is hit often enough for me to write a patch to improve the speed of this corner case. Furthermore , compress_file_range is the function name, in case you can't guess by the pasted code. Regards Nick 411 /* 412 * we don't want to send crud past the end of i_size through 413 * compression, that's just a waste of CPU time. So, if the 414 * end of the file is before the start of our current 415 * requested range of bytes, we bail out to the uncompressed 416 * cleanup code that can deal with all of this. 417 * 418 * It isn't really the fastest way to fix things, but this is a 419 * very uncommon corner. 420 */ 421 if (actual_end <= start) 422 goto cleanup_and_bail_uncompressed;
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies I get that my question is if this corner case is hit, enough for me to write a patch to optimize it. In addition the comment states it isn't but want to known for standard compression workloads in btrfs if it's hit enough for me to work on this and how much speed degradation are me we doing my not writing it better. Nick
Here's how I would go about it:
1. Understand when the case is met (in theory). 2. Try to trigger it on a real system multiple times. 3. Try to explore systematically under what circumstances the case is met and rank them by plausibility (if the notion of plausibility makes any sense in a real world scenario -- I don't know). 4. Estimate cost vs. benefit.
I don't know if this is a good way but notice how you can do all this on yourself which I think is a plus for everyone. And if you decide in step 4 to write a patch:
5. Use your results from step 3 to create an environment that benefits from your patch (notice how 4 guarantees that there exists such a system with reasonable connection to real needs). Note the numbers. 6. Test your patch on as many regular configurations as possible. Note the numbers. If it degrades performance on any of those, abort. 7. Do *NOT* send the patch out.
Regards, Tobi
Thanks Tobi,
From reading the code off the bat, seems to not need to be written as this case is rarely meet for large files or files that are huge and take a lot of time to write. Was more curious about how to test things like this if I need to :). Nick
On Tue, 26 Aug 2014 20:13:10 -0400, Nick said:
From reading the code off the bat, seems to not need to be written as this case is rarely meet for large files or files that are huge and take a lot of time to write.
Nope. Same exact logic for small files. The code is doing "Don't bother trying to decompress if we're off the end of the file entirely". Doesn't matter if the file is 43K in size or 43G.
On Tue, 26 Aug 2014, Nick wrote:
On 08/26/2014 08:05 PM, Tobias Boege wrote:
On Tue, 26 Aug 2014, Nick wrote:
On 08/26/2014 06:58 PM, Mandeep Sandhu wrote:
If it's a corner case, it won't be hit often enough right? And if it was hit often enough, it wouldn't be corner case!? :)
These 2 are mutually exclusive!
On Tue, Aug 26, 2014 at 3:47 PM, Nick <xerofoify@gmail.com> wrote:
After reading through the code in inode.c today , I am curious about the comment and the following code I will paste below. I am curious if this corner case is hit often enough for me to write a patch to improve the speed of this corner case. Furthermore , compress_file_range is the function name, in case you can't guess by the pasted code. Regards Nick 411 /* 412 * we don't want to send crud past the end of i_size through 413 * compression, that's just a waste of CPU time. So, if the 414 * end of the file is before the start of our current 415 * requested range of bytes, we bail out to the uncompressed 416 * cleanup code that can deal with all of this. 417 * 418 * It isn't really the fastest way to fix things, but this is a 419 * very uncommon corner. 420 */ 421 if (actual_end <= start) 422 goto cleanup_and_bail_uncompressed;
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies I get that my question is if this corner case is hit, enough for me to write a patch to optimize it. In addition the comment states it isn't but want to known for standard compression workloads in btrfs if it's hit enough for me to work on this and how much speed degradation are me we doing my not writing it better. Nick
Here's how I would go about it:
1. Understand when the case is met (in theory). 2. Try to trigger it on a real system multiple times. 3. Try to explore systematically under what circumstances the case is met and rank them by plausibility (if the notion of plausibility makes any sense in a real world scenario -- I don't know). 4. Estimate cost vs. benefit.
I don't know if this is a good way but notice how you can do all this on yourself which I think is a plus for everyone. And if you decide in step 4 to write a patch:
5. Use your results from step 3 to create an environment that benefits from your patch (notice how 4 guarantees that there exists such a system with reasonable connection to real needs). Note the numbers. 6. Test your patch on as many regular configurations as possible. Note the numbers. If it degrades performance on any of those, abort. 7. Do *NOT* send the patch out.
Regards, Tobi
Thanks Tobi,
From reading the code off the bat, seems to not need to be written as this case is rarely meet for large files or files that are huge and take a lot of time to write.
Thanks for letting me compose my mail before you take a closer look at your matter and decide it's not worth it.
Was more curious about how to test things like this if I need to :).
Then you need to phrase that -- in the *first* mail of a thread. There is no need to hide your real questions behind different ones. I think the steps 1 and 2 above can still be used to answer your question. Basically, to test something you set up and run your system and then you do things. What your "system" is constituted of and what "things" are depends upon the subject and 1 and 2 might help you clear that up. Regards, Tobi -- "There's an old saying: Don't change anything... ever!" -- Mr. Monk
On 08/26/2014 08:37 PM, Tobias Boege wrote:
On Tue, 26 Aug 2014, Nick wrote:
On 08/26/2014 08:05 PM, Tobias Boege wrote:
On Tue, 26 Aug 2014, Nick wrote:
On 08/26/2014 06:58 PM, Mandeep Sandhu wrote:
If it's a corner case, it won't be hit often enough right? And if it was hit often enough, it wouldn't be corner case!? :)
These 2 are mutually exclusive!
On Tue, Aug 26, 2014 at 3:47 PM, Nick <xerofoify@gmail.com> wrote:
After reading through the code in inode.c today , I am curious about the comment and the following code I will paste below. I am curious if this corner case is hit often enough for me to write a patch to improve the speed of this corner case. Furthermore , compress_file_range is the function name, in case you can't guess by the pasted code. Regards Nick 411 /* 412 * we don't want to send crud past the end of i_size through 413 * compression, that's just a waste of CPU time. So, if the 414 * end of the file is before the start of our current 415 * requested range of bytes, we bail out to the uncompressed 416 * cleanup code that can deal with all of this. 417 * 418 * It isn't really the fastest way to fix things, but this is a 419 * very uncommon corner. 420 */ 421 if (actual_end <= start) 422 goto cleanup_and_bail_uncompressed;
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies I get that my question is if this corner case is hit, enough for me to write a patch to optimize it. In addition the comment states it isn't but want to known for standard compression workloads in btrfs if it's hit enough for me to work on this and how much speed degradation are me we doing my not writing it better. Nick
Here's how I would go about it:
1. Understand when the case is met (in theory). 2. Try to trigger it on a real system multiple times. 3. Try to explore systematically under what circumstances the case is met and rank them by plausibility (if the notion of plausibility makes any sense in a real world scenario -- I don't know). 4. Estimate cost vs. benefit.
I don't know if this is a good way but notice how you can do all this on yourself which I think is a plus for everyone. And if you decide in step 4 to write a patch:
5. Use your results from step 3 to create an environment that benefits from your patch (notice how 4 guarantees that there exists such a system with reasonable connection to real needs). Note the numbers. 6. Test your patch on as many regular configurations as possible. Note the numbers. If it degrades performance on any of those, abort. 7. Do *NOT* send the patch out.
Regards, Tobi
Thanks Tobi,
From reading the code off the bat, seems to not need to be written as this case is rarely meet for large files or files that are huge and take a lot of time to write.
Thanks for letting me compose my mail before you take a closer look at your matter and decide it's not worth it.
Was more curious about how to test things like this if I need to :).
Then you need to phrase that -- in the *first* mail of a thread. There is no need to hide your real questions behind different ones.
I think the steps 1 and 2 above can still be used to answer your question. Basically, to test something you set up and run your system and then you do things. What your "system" is constituted of and what "things" are depends upon the subject and 1 and 2 might help you clear that up.
Regards, Tobi
Sorry , thought it was a stupid question. Will note for next time. Regards Nick
On August 26, 2014 8:13:10 PM EDT, Nick <xerofoify@gmail.com> wrote:
On 08/26/2014 08:05 PM, Tobias Boege wrote:
On Tue, 26 Aug 2014, Nick wrote:
On 08/26/2014 06:58 PM, Mandeep Sandhu wrote:
If it's a corner case, it won't be hit often enough right? And if it was hit often enough, it wouldn't be corner case!? :)
These 2 are mutually exclusive!
On Tue, Aug 26, 2014 at 3:47 PM, Nick <xerofoify@gmail.com> wrote:
After reading through the code in inode.c today , I am curious about the comment and the following code I will paste below. I am curious if this corner case is hit often enough for me to write a patch to improve the speed of this corner case. Furthermore , compress_file_range is the function name, in case you can't guess by the pasted code. Regards Nick 411 /* 412 * we don't want to send crud past the end of i_size through 413 * compression, that's just a waste of CPU time. So, if the 414 * end of the file is before the start of our current 415 * requested range of bytes, we bail out to the uncompressed 416 * cleanup code that can deal with all of this. 417 * 418 * It isn't really the fastest way to fix things, but this is a 419 * very uncommon corner. 420 */ 421 if (actual_end <= start) 422 goto cleanup_and_bail_uncompressed;
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies I get that my question is if this corner case is hit, enough for me to write a patch to optimize it. In addition the comment states it isn't but want to known for standard compression workloads in btrfs if it's hit enough for me to work on this and how much speed degradation are me we doing my not writing it better. Nick
Here's how I would go about it:
1. Understand when the case is met (in theory). 2. Try to trigger it on a real system multiple times. 3. Try to explore systematically under what circumstances the case is met and rank them by plausibility (if the notion of plausibility makes any sense in a real world scenario -- I don't know). 4. Estimate cost vs. benefit.
I don't know if this is a good way but notice how you can do all this on yourself which I think is a plus for everyone. And if you decide in step 4 to write a patch:
5. Use your results from step 3 to create an environment that benefits from your patch (notice how 4 guarantees that there exists such a system with reasonable connection to real needs). Note the numbers. 6. Test your patch on as many regular configurations as possible. Note the numbers. If it degrades performance on any of those, abort. 7. Do *NOT* send the patch out.
Regards, Tobi
Thanks Tobi,
From reading the code off the bat, seems to not need to be written as this case is rarely meet for large files or files that are huge and take a lot of time to write. Was more curious about how to test things like this if I need to :). Nick
If you want to do anything more than code beautification for btrfs, you absolutely have to learn how to write xfstest scripts. Dispute the name, the Linux kernel now uses xfstests to exercise all kernel file systems. I don't follow btrfs, but xfs as an obvious example is pretty strict about only accepting meaningful patches if they are accompanied by a user space xfstest tool to exercise it. Surprisingly that seems to be less true of new functionality. Bug fix patches have an almost mandatory request for an accompanying xfstest. The other side of that is if you read thru the kernel bugzilla and find a filesystem bug you want to fix, a good first step is to focus on writing a xfstest script that will show the bug and fail appropriately. A decade plus ago when I first got interested in using xfs for a production system, the freeze feature would randomly fail for me. I was able to update the xfstest that exercised that code and get a reproducer. I didn't do the actual kernel fix, but I still think of that as my first significant contribution to the kernel. Greg -- Sent from my Android phone with K-9 Mail. Please excuse my brevity.
On Tue, 26 Aug 2014 21:49:51 -0400, Greg Freemyer said:
A decade plus ago when I first got interested in using xfs for a production system, the freeze feature would randomly fail for me. I was able to update the xfstest that exercised that code and get a reproducer. I didn't do the actual kernel fix, but I still think of that as my first significant contribution to the kernel.
Never underestimate the value of that. Clued users who submit good bug reports and who can replicate/test patches or provide good test cases are worth their weight in gold.
On 14-08-27 12:08 AM, Valdis.Kletnieks@vt.edu wrote:
On Tue, 26 Aug 2014 21:49:51 -0400, Greg Freemyer said:
A decade plus ago when I first got interested in using xfs for a production system, the freeze feature would randomly fail for me. I was able to update the xfstest that exercised that code and get a reproducer. I didn't do the actual kernel fix, but I still think of that as my first significant contribution to the kernel.
Never underestimate the value of that. Clued users who submit good bug reports and who can replicate/test patches or provide good test cases are worth their weight in gold.
Good to known after testing file systems and what tools are the standard in the community. Regards Nick
Nick, Have a look here please: http://www.brendangregg.com/linuxperf.html Aruna On Wed, Aug 27, 2014 at 12:47 AM, nick <xerofoify@gmail.com> wrote:
On 14-08-27 12:08 AM, Valdis.Kletnieks@vt.edu wrote:
On Tue, 26 Aug 2014 21:49:51 -0400, Greg Freemyer said:
A decade plus ago when I first got interested in using xfs for a production system, the freeze feature would randomly fail for me. I was able to update the xfstest that exercised that code and get a reproducer. I didn't do the actual kernel fix, but I still think of that as my first significant contribution to the kernel.
Never underestimate the value of that. Clued users who submit good bug reports and who can replicate/test patches or provide good test cases are worth their weight in gold.
Good to known after testing file systems and what tools are the standard in the community. Regards Nick
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
On 14-08-27 01:09 AM, Aruna Hewapathirane wrote:
Nick,
Have a look here please: http://www.brendangregg.com/linuxperf.html
Aruna
On Wed, Aug 27, 2014 at 12:47 AM, nick <xerofoify@gmail.com> wrote:
On 14-08-27 12:08 AM, Valdis.Kletnieks@vt.edu wrote:
On Tue, 26 Aug 2014 21:49:51 -0400, Greg Freemyer said:
A decade plus ago when I first got interested in using xfs for a production system, the freeze feature would randomly fail for me. I was able to update the xfstest that exercised that code and get a reproducer. I didn't do the actual kernel fix, but I still think of that as my first significant contribution to the kernel.
Never underestimate the value of that. Clued users who submit good bug reports and who can replicate/test patches or provide good test cases are worth their weight in gold.
Good to known after testing file systems and what tools are the standard in the community. Regards Nick
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
Thanks for the web page I will start reading it later. Cheers Nick
On Tue, 26 Aug 2014 18:47:41 -0400, Nick said:
After reading through the code in inode.c today , I am curious about the comment and the following code I will paste below. I am curious if this corner case is hit often enough for me to write a patch to improve the speed of this corner case.
421 if (actual_end <= start) 422 goto cleanup_and_bail_uncompressed; Wow, that's a heck of a small optimization target. How to approach it? Step 3: Become proficient enough to instrument the code and actually measure it. Because upstream will never accept a patch optimizing a "corner case" unless you have actual numbers that show an actual improvement under production workloads as opposed to microbenchmarks. Step 2: Read the code, and ask yourself how the hell you intend to make the code any faster than the relatively cheap if/goto that's currently in place. Note that the combination of superscalar and speculative execution on modern processors and good optimizing compilers means that it's possible that the 2 lines of code will execute in effectively *zero* cycles. Hard to make it faster than that. :) Step 1: Note that the zero-cycles branch is in a loop that includes a call to btrfs_compress_pages(), which is going to take *at least* 4096 cycles to process a 4K page (simply because you have to look at each byte in the buffer at least once). So the percentage savings is pretty low here, because it's swamped by calls to compress and memset and other stuff that chews a lot more cycles. Step 0: Ask yourself how often programs either *ignore* an EOF condition on a file they are reading, and keep trying to read past the end of the file, or do an explicit lseek()/read() beyond the end of the file, and whether it's worth optimizing that case at the expense of making the code more complex and probalby slower in the non-corner-case case. The tl;dr: You're barking up the wrong tree here.
participants (7)
-
Aruna Hewapathirane -
Greg Freemyer -
Mandeep Sandhu -
Nick -
nick
-
Tobias Boege -
Valdis.Kletnieks＠vt.edu