Re: Creating compressed backing_store as swapfile
On 11/5/2018 10:58 AM, Adam Borowski wrote:
On Mon, Nov 05, 2018 at 08:31:46PM +0530, Pintu Agarwal wrote:
Hi,
I have one requirement: I wanted to have a swapfile (64MB to 256MB) on my system. But I wanted the data to be compressed and stored on the disk in my swapfile. [Similar to zram, but compressed data should be moved to disk, instead of RAM].
Note: I wanted to optimize RAM space, so performance is not important right now for our requirement.
So, what are the options available, to perform this in 4.x kernel version. My Kernel: 4.9.x Board: any - (arm64 mostly).
As I know, following are the choices: 1) ZRAM: But it compresses and store data in RAM itself 2) frontswap + zswap : Didn't explore much on this, not sure if this is helpful for our case. 3) Manually creating swapfile: but how to compress it ? 4) Any other options ?
Loop device on any filesystem that can compress (such as btrfs)? The performance would suck, though -- besides the indirection of loop, btrfs compresses in blocks of 128KB while swap wants 4KB writes. Other similar option is qemu-nbd -- it can use compressed disk images and expose them to a (local) nbd client.
Swap on any type of a networked storage device (NBD, iSCSI, ATAoE, etc) served from the local system is _really_ risky. The moment the local server process for the storage device gets forced out to swap, you deadlock. Performance isn't _too_ bad for the BTRFS case though (I've actually tested this before), just make sure you disable direct I/O mode on the loop device, otherwise you run the risk of data corruption.
On Mon, 05 Nov 2018 11:07:12 -0500, "Austin S. Hemmelgarn" said:
Performance isn't _too_ bad for the BTRFS case though (I've actually tested this before), just make sure you disable direct I/O mode on the loop device, otherwise you run the risk of data corruption.
Did you test that for random-access. or just sequential read/write? (Also, see the note in my other mail regarding doing a random-access write to the middle of the file...)
On 11/5/2018 11:14 AM, valdis.kletnieks@vt.edu wrote:
On Mon, 05 Nov 2018 11:07:12 -0500, "Austin S. Hemmelgarn" said:
Performance isn't _too_ bad for the BTRFS case though (I've actually tested this before), just make sure you disable direct I/O mode on the loop device, otherwise you run the risk of data corruption.
Did you test that for random-access. or just sequential read/write? (Also, see the note in my other mail regarding doing a random-access write to the middle of the file...)
Actual swap usage. About 16 months ago, I had been running a couple of Intel NUC5PPYH boxes (Pentium N3700 CPU's, 4GB of DDR3-1333 RAM) for some network prototyping. On both, I had swap set up to use a file on BTRFS via a loop device, and I made a point to test both with LZ4 inline compression and without any compression, and saw negligible performance differences (less than 1% in most cases). It was, of course, significantly worse than running on ext4, but on a system that's so resource constrained that both storage and memory are at a premium to this degree, the performance hit is probably going to be worth it. Also, it's probably worth noting that BTRFS doesn't need to decompress the entire file to read or write blocks in the middle, it splits the file into 128k blocks and compresses each of those independent of the others, so it can just decompress the 128k block that holds the actual block that's needed.
On Mon, 05 Nov 2018 11:28:49 -0500, "Austin S. Hemmelgarn" said:
Also, it's probably worth noting that BTRFS doesn't need to decompress the entire file to read or write blocks in the middle, it splits the file into 128k blocks and compresses each of those independent of the others, so it can just decompress the 128k block that holds the actual block that's needed.
Presumably it does something sane with block allocation for the now-compressed 128K that's presumably much smaller. Also, that limits the damage from writing to the middle of a compression unit.... That *does* however increase the memory requirement - you can OOM or deadlock if your read/write from the swap needs an additional 128K for the compression buffer at an inconvenient time...
On 11/5/2018 11:53 AM, valdis.kletnieks@vt.edu wrote:
On Mon, 05 Nov 2018 11:28:49 -0500, "Austin S. Hemmelgarn" said:
Also, it's probably worth noting that BTRFS doesn't need to decompress the entire file to read or write blocks in the middle, it splits the file into 128k blocks and compresses each of those independent of the others, so it can just decompress the 128k block that holds the actual block that's needed.
Presumably it does something sane with block allocation for the now-compressed 128K that's presumably much smaller. Also, that limits the damage from writing to the middle of a compression unit....
That *does* however increase the memory requirement - you can OOM or deadlock if your read/write from the swap needs an additional 128K for the compression buffer at an inconvenient time...
Indeed, and I can't really comment on how it might behave under those circumstances (the systems I did the testing on never saw memory pressure quite _that_ bad, and I had them set up to swap things out pretty early and really aggressively).
On Mon, Nov 5, 2018 at 9:37 PM Austin S. Hemmelgarn <ahferroin7@gmail.com> wrote:
On 11/5/2018 10:58 AM, Adam Borowski wrote:
On Mon, Nov 05, 2018 at 08:31:46PM +0530, Pintu Agarwal wrote:
Hi,
I have one requirement: I wanted to have a swapfile (64MB to 256MB) on my system. But I wanted the data to be compressed and stored on the disk in my swapfile. [Similar to zram, but compressed data should be moved to disk, instead of RAM].
Note: I wanted to optimize RAM space, so performance is not important right now for our requirement.
So, what are the options available, to perform this in 4.x kernel version. My Kernel: 4.9.x Board: any - (arm64 mostly).
As I know, following are the choices: 1) ZRAM: But it compresses and store data in RAM itself 2) frontswap + zswap : Didn't explore much on this, not sure if this is helpful for our case. 3) Manually creating swapfile: but how to compress it ? 4) Any other options ?
Loop device on any filesystem that can compress (such as btrfs)? The performance would suck, though -- besides the indirection of loop, btrfs compresses in blocks of 128KB while swap wants 4KB writes. Other similar option is qemu-nbd -- it can use compressed disk images and expose them to a (local) nbd client.
Swap on any type of a networked storage device (NBD, iSCSI, ATAoE, etc) served from the local system is _really_ risky. The moment the local server process for the storage device gets forced out to swap, you deadlock.
Performance isn't _too_ bad for the BTRFS case though (I've actually tested this before), just make sure you disable direct I/O mode on the loop device, otherwise you run the risk of data corruption.
Sorry, btrfs is not an option for us. We want something more lighter weight as our requirement is just < 200 MBs.
participants (3)
-
Austin S. Hemmelgarn -
Pintu Agarwal -
valdis.kletnieks＠vt.edu