Btrfs Questions for Personal Data Archive

nick xerofoify at gmail.com
Mon Apr 10 23:28:31 EDT 2017


Greetings All,
I am assuming this is coming from me it's just going to get  no reply. However if someone
who works on btrfs is willing to reply to this I have a few questions. Firstly I have a 
lot of personal storage for archiving various things probably around ~37TB filled currently 
in external drives and estimate the archive to grow to around ~300 to 400 TB. All of
these numbers are unformatted for people's information. I have been using btrfs but
need to ask a few questions for how to create a central archive best:

1.Does the compression algorithms used as according to my knowledge they don't do
it well enough to enable it for non text files? What are the performance disadvantages
from a IO perspective(I can Google this but I want numbers from someone who works
in a data center)? This is primary due to the central archive needing to be used
for many different systems at a time. I assume around 3-4 on average maybe more
including during a rebuild. Some of it is also hit bit rate encodes I have done
for bluray which are around 10-22mbps per second and may need to be watched during
a raid rebuild. GPU processing will be client side so don't worry about that.

2. Same as above but for software raid versus hardware raid in btrfs.I am stressing
rebuild time here on a raid 5/6. Probably going to use raid 60 as that gives me 
a mirror plus two drives failure for redundancy. 

3. I have found very little information on this but does the metadata being on a SSD
actually help with performance? Very little information exist off the wiki for this.
I literally goggled for a few hours and found very little even in terms of papers.

Further more the archive is going to be on external hard drives for a while, as hard
drives are not big enough. If people want more information or have advice on how
to scale this out that would be very helpful. Generally I am looking for information
related to btrfs.

If anyone replies huge thanks,
Nick
P.S. If anyone tells me to send my questions to the btrfs list I did some stupid things and
was banned from vger so I am just sending it here for now.



More information about the Kernelnewbies mailing list