On 11/5/2018 11:53 AM, valdis.kletnieks@vt.edu wrote:
On Mon, 05 Nov 2018 11:28:49 -0500, "Austin S. Hemmelgarn" said:
Also, it's probably worth noting that BTRFS doesn't need to decompress the entire file to read or write blocks in the middle, it splits the file into 128k blocks and compresses each of those independent of the others, so it can just decompress the 128k block that holds the actual block that's needed.
Presumably it does something sane with block allocation for the now-compressed 128K that's presumably much smaller. Also, that limits the damage from writing to the middle of a compression unit....
That *does* however increase the memory requirement - you can OOM or deadlock if your read/write from the swap needs an additional 128K for the compression buffer at an inconvenient time...
Indeed, and I can't really comment on how it might behave under those circumstances (the systems I did the testing on never saw memory pressure quite _that_ bad, and I had them set up to swap things out pretty early and really aggressively).