Split RAID: Proposal for archival RAID using incremental batch checksum
I'd a appreciate any help/pointers in implementing the proposal below including the right path to get this into the kernel itself. ---------------------------------- I'm outlining below a proposal for a RAID device mapper virtual block device for the kernel which adds "split raid" functionality on an incremental batch basis for a home media server/archived content which is rarely accessed. Given a set of N+X block devices (of the same size but smallest common size wins) the SplitRAID device mapper device generates virtual devices which are passthrough for N devices and write a Batched/Delayed checksum into the X devices so as to allow offline recovery of block on the N devices in case of a single disk failure. Advantages over conventional RAID: - Disks can be spun down reducing wear and tear over MD RAID Levels (such as 1, 10, 5,6) in the case of rarely accessed archival content - Prevent catastrophic data loss for multiple device failure since each block device is independent and hence unlike MD RAID will only lose data incrementally. - Performance degradation for writes can be achieved by keeping the checksum update asynchronous and delaying the fsync to the checksum block device. In the event of improper shutdown the checksum may not have all the updated data but will be mostly up to date which is often acceptable for home media server requirements. A flag can be set in case the checksum block device was shutdown properly indicating that a full checksum rebuild is not required. Existing solutions considered: - SnapRAID (http://snapraid.sourceforge.net/) which is a snapshot based scheme (Its advantages are that its in user space and has cross platform support but has the huge disadvantage of every checksum being done from scratch slowing the system, causing immense wear and tear on every snapshot and also losing any information updates upto the snapshot point etc) I'd like to get opinions on the pros and cons of this proposal from more experienced people on the list to redirect suitably on the following questions: - Maybe this can already be done using the block devices available in the kernel? - If not, Device mapper the right API to use? (I think so) - What would be the best block devices code to look at to implement? Regards, Anshuman Aggarwal
On November 21, 2014 5:15:43 AM EST, Anshuman Aggarwal <anshuman.aggarwal@gmail.com> wrote:
I'd a appreciate any help/pointers in implementing the proposal below including the right path to get this into the kernel itself. ---------------------------------- I'm outlining below a proposal for a RAID device mapper virtual block device for the kernel which adds "split raid" functionality on an incremental batch basis for a home media server/archived content which is rarely accessed.
Given a set of N+X block devices (of the same size but smallest common size wins)
the SplitRAID device mapper device generates virtual devices which are passthrough for N devices and write a Batched/Delayed checksum into the X devices so as to allow offline recovery of block on the N devices in case of a single disk failure.
Advantages over conventional RAID:
- Disks can be spun down reducing wear and tear over MD RAID Levels (such as 1, 10, 5,6) in the case of rarely accessed archival content
- Prevent catastrophic data loss for multiple device failure since each block device is independent and hence unlike MD RAID will only lose data incrementally.
- Performance degradation for writes can be achieved by keeping the checksum update asynchronous and delaying the fsync to the checksum block device.
In the event of improper shutdown the checksum may not have all the updated data but will be mostly up to date which is often acceptable for home media server requirements. A flag can be set in case the checksum block device was shutdown properly indicating that a full checksum rebuild is not required.
Existing solutions considered:
- SnapRAID (http://snapraid.sourceforge.net/) which is a snapshot based scheme (Its advantages are that its in user space and has cross platform support but has the huge disadvantage of every checksum being done from scratch slowing the system, causing immense wear and tear on every snapshot and also losing any information updates upto the snapshot point etc)
I'd like to get opinions on the pros and cons of this proposal from more experienced people on the list to redirect suitably on the following questions:
- Maybe this can already be done using the block devices available in the kernel?
- If not, Device mapper the right API to use? (I think so)
- What would be the best block devices code to look at to implement?
Regards,
Anshuman Aggarwal
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
I think I understand the proposal. You say N pass-through drives. I assume concatenated? If the N drives were instead in a Raid-0 stripe set and your X drives was just a single parity drive, then you would have described Raid-4. There are use cases for raid 4 and you have described a good one (rarely used data where random w/o performance is not key). I don't know if mdraid supports raid-4 or not. If not I suggest adding raid-4 support is something else you might want to look at. Anyway, at a minimum add raid-4 to the existing solutions considered section. Greg -- Sent from my Android phone with K-9 Mail. Please excuse my brevity.
N pass through but with their own filesystems. Concatenation is via some kind of union fs solution not at the block level. Data is not supposed to be striped (this is critical so as to prevent all drives to be required to be accessed for consecutive data) Idea is that each drive can work independently and the last drive stores parity to save data in case of failure of any one drive. Any suggestions from anyone on where to start with such a driver..it seems like a block driver for the parity drive but which depends on intercepting the writes to other drives. On 21 November 2014 17:11, Greg Freemyer <greg.freemyer@gmail.com> wrote:
On November 21, 2014 5:15:43 AM EST, Anshuman Aggarwal <anshuman.aggarwal@gmail.com> wrote:
I'd a appreciate any help/pointers in implementing the proposal below including the right path to get this into the kernel itself. ---------------------------------- I'm outlining below a proposal for a RAID device mapper virtual block device for the kernel which adds "split raid" functionality on an incremental batch basis for a home media server/archived content which is rarely accessed.
Given a set of N+X block devices (of the same size but smallest common size wins)
the SplitRAID device mapper device generates virtual devices which are passthrough for N devices and write a Batched/Delayed checksum into the X devices so as to allow offline recovery of block on the N devices in case of a single disk failure.
Advantages over conventional RAID:
- Disks can be spun down reducing wear and tear over MD RAID Levels (such as 1, 10, 5,6) in the case of rarely accessed archival content
- Prevent catastrophic data loss for multiple device failure since each block device is independent and hence unlike MD RAID will only lose data incrementally.
- Performance degradation for writes can be achieved by keeping the checksum update asynchronous and delaying the fsync to the checksum block device.
In the event of improper shutdown the checksum may not have all the updated data but will be mostly up to date which is often acceptable for home media server requirements. A flag can be set in case the checksum block device was shutdown properly indicating that a full checksum rebuild is not required.
Existing solutions considered:
- SnapRAID (http://snapraid.sourceforge.net/) which is a snapshot based scheme (Its advantages are that its in user space and has cross platform support but has the huge disadvantage of every checksum being done from scratch slowing the system, causing immense wear and tear on every snapshot and also losing any information updates upto the snapshot point etc)
I'd like to get opinions on the pros and cons of this proposal from more experienced people on the list to redirect suitably on the following questions:
- Maybe this can already be done using the block devices available in the kernel?
- If not, Device mapper the right API to use? (I think so)
- What would be the best block devices code to look at to implement?
Regards,
Anshuman Aggarwal
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
I think I understand the proposal.
You say N pass-through drives. I assume concatenated?
If the N drives were instead in a Raid-0 stripe set and your X drives was just a single parity drive, then you would have described Raid-4.
There are use cases for raid 4 and you have described a good one (rarely used data where random w/o performance is not key).
I don't know if mdraid supports raid-4 or not. If not I suggest adding raid-4 support is something else you might want to look at.
Anyway, at a minimum add raid-4 to the existing solutions considered section.
Greg
-- Sent from my Android phone with K-9 Mail. Please excuse my brevity.
Top posting is strongly discouraged on all kernel related mailing lists including this one. I've moved your reply to the bottom and then replied after that. In future I will ignore replies that are top posted.
On 21 November 2014 17:11, Greg Freemyer <greg.freemyer@gmail.com> wrote:
On November 21, 2014 5:15:43 AM EST, Anshuman Aggarwal
<anshuman.aggarwal@gmail.com> wrote:
I'd a appreciate any help/pointers in implementing the proposal below including the right path to get this into the kernel itself. ---------------------------------- I'm outlining below a proposal for a RAID device mapper virtual block device for the kernel which adds "split raid" functionality on an incremental batch basis for a home media server/archived content which is rarely accessed.
Given a set of N+X block devices (of the same size but smallest common size wins)
the SplitRAID device mapper device generates virtual devices which are passthrough for N devices and write a Batched/Delayed checksum into the X devices so as to allow offline recovery of block on the N devices in case of a single disk failure.
Advantages over conventional RAID:
- Disks can be spun down reducing wear and tear over MD RAID Levels (such as 1, 10, 5,6) in the case of rarely accessed archival content
- Prevent catastrophic data loss for multiple device failure since each block device is independent and hence unlike MD RAID will only lose data incrementally.
- Performance degradation for writes can be achieved by keeping the checksum update asynchronous and delaying the fsync to the checksum block device.
In the event of improper shutdown the checksum may not have all the updated data but will be mostly up to date which is often acceptable for home media server requirements. A flag can be set in case the checksum block device was shutdown properly indicating that a full checksum rebuild is not required.
Existing solutions considered:
- SnapRAID (http://snapraid.sourceforge.net/) which is a snapshot based scheme (Its advantages are that its in user space and has cross platform support but has the huge disadvantage of every checksum being done from scratch slowing the system, causing immense wear and tear on every snapshot and also losing any information updates upto the snapshot point etc)
I'd like to get opinions on the pros and cons of this proposal from more experienced people on the list to redirect suitably on the following questions:
- Maybe this can already be done using the block devices available in the kernel?
- If not, Device mapper the right API to use? (I think so)
- What would be the best block devices code to look at to implement?
Regards,
Anshuman Aggarwal
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
I think I understand the proposal.
You say N pass-through drives. I assume concatenated?
If the N drives were instead in a Raid-0 stripe set and your X drives was just a single parity drive, then you would have described Raid-4.
There are use cases for raid 4 and you have described a good one (rarely used data where random w/o performance is not key).
I don't know if mdraid supports raid-4 or not. If not I suggest adding raid-4 support is something else you might want to look at.
Anyway, at a minimum add raid-4 to the existing solutions considered section.
Greg On November 21, 2014 1:48:57 PM EST, Anshuman Aggarwal <anshuman.aggarwal@gmail.com> wrote: N pass through but with their own filesystems. Concatenation is via some kind of union fs solution not at the block level. Data is not supposed to be striped (this is critical so as to prevent all drives to be required to be accessed for consecutive data)
I'm ignorant of how unionfs works, so I can offer no feedback about it. I see no real issue doing it with a block level solution with device mapper (dm) as the implementation. I'm going to ignore implementation for the rest of this email and discuss the goal. Can you detail what you see a single page write to D1 doing? You talked about batching / delaying the checksum writes, but I didn't understand how that made things more efficient, nor the reason for the delay. I assume you know raid 4 and 5 work like this: Read D1old Read Pold Pnew=(Pold^D1old)^D1new Write Pnew Write D1new Ie. 2 page reads and 2 page writes to update a single page. The 2 reads and the 2 writes take place in parallel, so if the disks are otherwise idle, then the time involved is one disk seek and 2 disk rotations. Let's say 25 msecs for the seek and 12 msecs per rotation. That is 49 msecs total. I think that is about right for a low performance rotating drive, but I didn't pull out any specs to double check my memory. While that is a lot of i/o overhead (4x), it is how raid 4 and 5 work and I assume your split raid would have to do something similar. With a normal non raided disk a single block write requires a seek and a rotation, so 37 msecs, thus very little clock time overhead for raid 4 or 5 for small random i/o block writes. Is that also true of your split raid? The delayed checksum writes confuse me. --- Where I'm concerned about your solution for performance is with a full stride write. Let's look at how a 4 disk raid 4 would write a full stride: Pnew = D1new ^ D2new ^ D3new Write D1 Write D2 Write D3 Write P So only 4 writes to write 3 data blocks. Even better all take place in parallel so you can accomplish 3x the data writes to disk that a single non-raided disk can. Thus for streaming writes, raid 4 or 5 see a performance boost over a single drive. I see nothing similar in your split raid. The same is true of streaming reads, raid 4 and 5 get performance gains from reading from the drives in parallel. I don't see any ability for that same gain in your split raid. In the real world raid 4 is rarely used because having a static parity drive offers no advantage I know of over having the parity handled as raid 5 does it. === Thus if your split raid was in kernel and I was setting up a streaming media server the choice would be between raid 5 and your split raid. Raid 5 I believe would have superior performance, but split raid would have a less catastrophic failure mode if 2 drives failed at once. Do I have right? Greg
Idea is that each drive can work independently and the last drive stores parity to save data in case of failure of any one drive.
Any suggestions from anyone on where to start with such a driver..it seems like a block driver for the parity drive but which depends on intercepting the writes to other drives.
-- Sent from my Android phone with K-9 Mail. Please excuse my brevity.
On 22 November 2014 at 18:47, Greg Freemyer <greg.freemyer@gmail.com> wrote:
Top posting is strongly discouraged on all kernel related mailing lists including this one. I've moved your reply to the bottom and then replied after that. In future I will ignore replies that are top posted.
On 21 November 2014 17:11, Greg Freemyer <greg.freemyer@gmail.com> wrote:
On November 21, 2014 5:15:43 AM EST, Anshuman Aggarwal
<anshuman.aggarwal@gmail.com> wrote:
I'd a appreciate any help/pointers in implementing the proposal below including the right path to get this into the kernel itself. ---------------------------------- I'm outlining below a proposal for a RAID device mapper virtual block device for the kernel which adds "split raid" functionality on an incremental batch basis for a home media server/archived content which is rarely accessed.
Given a set of N+X block devices (of the same size but smallest common size wins)
the SplitRAID device mapper device generates virtual devices which are passthrough for N devices and write a Batched/Delayed checksum into the X devices so as to allow offline recovery of block on the N devices in case of a single disk failure.
Advantages over conventional RAID:
- Disks can be spun down reducing wear and tear over MD RAID Levels (such as 1, 10, 5,6) in the case of rarely accessed archival content
- Prevent catastrophic data loss for multiple device failure since each block device is independent and hence unlike MD RAID will only lose data incrementally.
- Performance degradation for writes can be achieved by keeping the checksum update asynchronous and delaying the fsync to the checksum block device.
In the event of improper shutdown the checksum may not have all the updated data but will be mostly up to date which is often acceptable for home media server requirements. A flag can be set in case the checksum block device was shutdown properly indicating that a full checksum rebuild is not required.
Existing solutions considered:
- SnapRAID (http://snapraid.sourceforge.net/) which is a snapshot based scheme (Its advantages are that its in user space and has cross platform support but has the huge disadvantage of every checksum being done from scratch slowing the system, causing immense wear and tear on every snapshot and also losing any information updates upto the snapshot point etc)
I'd like to get opinions on the pros and cons of this proposal from more experienced people on the list to redirect suitably on the following questions:
- Maybe this can already be done using the block devices available in the kernel?
- If not, Device mapper the right API to use? (I think so)
- What would be the best block devices code to look at to implement?
Regards,
Anshuman Aggarwal
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
I think I understand the proposal.
You say N pass-through drives. I assume concatenated?
If the N drives were instead in a Raid-0 stripe set and your X drives was just a single parity drive, then you would have described Raid-4.
There are use cases for raid 4 and you have described a good one (rarely used data where random w/o performance is not key).
I don't know if mdraid supports raid-4 or not. If not I suggest adding raid-4 support is something else you might want to look at.
Anyway, at a minimum add raid-4 to the existing solutions considered section.
Greg On November 21, 2014 1:48:57 PM EST, Anshuman Aggarwal <anshuman.aggarwal@gmail.com> wrote: N pass through but with their own filesystems. Concatenation is via some kind of union fs solution not at the block level. Data is not supposed to be striped (this is critical so as to prevent all drives to be required to be accessed for consecutive data)
I'm ignorant of how unionfs works, so I can offer no feedback about it.
I see no real issue doing it with a block level solution with device mapper (dm) as the implementation. I'm going to ignore implementation for the rest of this email and discuss the goal.
Can you detail what you see a single page write to D1 doing?
You talked about batching / delaying the checksum writes, but I didn't understand how that made things more efficient, nor the reason for the delay.
I assume you know raid 4 and 5 work like this:
Read D1old Read Pold Pnew=(Pold^D1old)^D1new Write Pnew Write D1new
Ie. 2 page reads and 2 page writes to update a single page.
The 2 reads and the 2 writes take place in parallel, so if the disks are otherwise idle, then the time involved is one disk seek and 2 disk rotations. Let's say 25 msecs for the seek and 12 msecs per rotation. That is 49 msecs total. I think that is about right for a low performance rotating drive, but I didn't pull out any specs to double check my memory.
While that is a lot of i/o overhead (4x), it is how raid 4 and 5 work and I assume your split raid would have to do something similar. With a normal non raided disk a single block write requires a seek and a rotation, so 37 msecs, thus very little clock time overhead for raid 4 or 5 for small random i/o block writes.
Is that also true of your split raid? The delayed checksum writes confuse me. ---
Where I'm concerned about your solution for performance is with a full stride write. Let's look at how a 4 disk raid 4 would write a full stride:
Pnew = D1new ^ D2new ^ D3new Write D1 Write D2 Write D3 Write P
So only 4 writes to write 3 data blocks. Even better all take place in parallel so you can accomplish 3x the data writes to disk that a single non-raided disk can.
Thus for streaming writes, raid 4 or 5 see a performance boost over a single drive.
I see nothing similar in your split raid.
The same is true of streaming reads, raid 4 and 5 get performance gains from reading from the drives in parallel. I don't see any ability for that same gain in your split raid.
In the real world raid 4 is rarely used because having a static parity drive offers no advantage I know of over having the parity handled as raid 5 does it.
=== Thus if your split raid was in kernel and I was setting up a streaming media server the choice would be between raid 5 and your split raid. Raid 5 I believe would have superior performance, but split raid would have a less catastrophic failure mode if 2 drives failed at once.
Do I have right?
Greg
Idea is that each drive can work independently and the last drive stores parity to save data in case of failure of any one drive.
Any suggestions from anyone on where to start with such a driver..it seems like a block driver for the parity drive but which depends on intercepting the writes to other drives.
-- Sent from my Android phone with K-9 Mail. Please excuse my brevity.
You have the motivation and goal quite opposite from what is intended. In a home media server, the RAID6 mdadm setup that I currently have keeps all the disks spinning and running for writes which could be done just to the last disk while the others are in sleep mode (head parked etc) Its not about performance at all. Its about longevity of the HDDs. The entire proposal is focused entirely on extending the life of the drives. By not using stripes, we restrict writes to happen to just 1 drive and the XOR output to the parity drive which then explains the delayed and batched checksum (resulting in fewer writes to the parity drive). The intention is that if a drive fails then maybe we lose 1 or 2 movies but the rest is restorable from parity. Also another advantage over RAID5 or RAID6 is that in the event of multiple drive failure we only lose the content on the failed drive not the whole cluster/RAID. Did I clarify better this time around?
On Sat, Nov 22, 2014 at 8:22 AM, Anshuman Aggarwal <anshuman.aggarwal@gmail.com> wrote:
By not using stripes, we restrict writes to happen to just 1 drive and the XOR output to the parity drive which then explains the delayed and batched checksum (resulting in fewer writes to the parity drive). The intention is that if a drive fails then maybe we lose 1 or 2 movies but the rest is restorable from parity.
Also another advantage over RAID5 or RAID6 is that in the event of multiple drive failure we only lose the content on the failed drive not the whole cluster/RAID.
Did I clarify better this time around?
I still don't understand the delayed checksum/parity. With classic raid 4, writing 1 GB of data to just D1 would require 1 GB of data first be read from D1 and 1 GB read from P then 1 GB written to both D1 and P. 4 GB worth of I/O total. With your proposal, if you stream 1 GB of data to a file on D1: - Does the old/previous data on D1 have to be read? - How much data goes to the parity drive? - Does the old data on the parity drive have to be read? - Why does delaying it reduce that volume compared to Raid 4? - In the event drive 1 fails, can its content be re-created from the other drives? Greg -- Greg Freemyer
On 22 November 2014 at 19:33, Greg Freemyer <greg.freemyer@gmail.com> wrote:
On Sat, Nov 22, 2014 at 8:22 AM, Anshuman Aggarwal <anshuman.aggarwal@gmail.com> wrote:
By not using stripes, we restrict writes to happen to just 1 drive and the XOR output to the parity drive which then explains the delayed and batched checksum (resulting in fewer writes to the parity drive). The intention is that if a drive fails then maybe we lose 1 or 2 movies but the rest is restorable from parity.
Also another advantage over RAID5 or RAID6 is that in the event of multiple drive failure we only lose the content on the failed drive not the whole cluster/RAID.
Did I clarify better this time around?
I still don't understand the delayed checksum/parity.
With classic raid 4, writing 1 GB of data to just D1 would require 1 GB of data first be read from D1 and 1 GB read from P then 1 GB written to both D1 and P. 4 GB worth of I/O total.
With your proposal, if you stream 1 GB of data to a file on D1:
- Does the old/previous data on D1 have to be read?
- How much data goes to the parity drive?
- Does the old data on the parity drive have to be read?
- Why does delaying it reduce that volume compared to Raid 4?
- In the event drive 1 fails, can its content be re-created from the other drives?
Greg -- Greg Freemyer
Two things: Delayed writes basically to allow the parity drive to spin down if the parity writing is only 1 block instead of spinning up the drive for every write (obviously the data drive has to be spun up). Delays will be both time and size constrained. For a large write such as a 1 GB of data to file it would trigger a configurable maximum delaying limit which would then dump to parity drive immediately preventing memory overuse. This again ties in to the fact that the content is not 'critical' so if parity was not dumped when a drive fails, worst case you only lose the latest file. Delayed writes may be done via bcache or a similar implementation which caches the writes in memory and need not be part of the split raid driver at all.
On November 22, 2014 9:43:23 AM EST, Anshuman Aggarwal <anshuman.aggarwal@gmail.com> wrote:
On 22 November 2014 at 19:33, Greg Freemyer <greg.freemyer@gmail.com> wrote:
On Sat, Nov 22, 2014 at 8:22 AM, Anshuman Aggarwal <anshuman.aggarwal@gmail.com> wrote:
By not using stripes, we restrict writes to happen to just 1 drive and the XOR output to the parity drive which then explains the delayed and batched checksum (resulting in fewer writes to the parity drive). The intention is that if a drive fails then maybe we lose 1 or 2 movies but the rest is restorable from parity.
Also another advantage over RAID5 or RAID6 is that in the event of multiple drive failure we only lose the content on the failed drive not the whole cluster/RAID.
Did I clarify better this time around?
I still don't understand the delayed checksum/parity.
With classic raid 4, writing 1 GB of data to just D1 would require 1 GB of data first be read from D1 and 1 GB read from P then 1 GB written to both D1 and P. 4 GB worth of I/O total.
With your proposal, if you stream 1 GB of data to a file on D1:
- Does the old/previous data on D1 have to be read?
- How much data goes to the parity drive?
- Does the old data on the parity drive have to be read?
- Why does delaying it reduce that volume compared to Raid 4?
- In the event drive 1 fails, can its content be re-created from the other drives?
Greg -- Greg Freemyer
Two things: Delayed writes basically to allow the parity drive to spin down if the parity writing is only 1 block instead of spinning up the drive for every write (obviously the data drive has to be spun up). Delays will be both time and size constrained. For a large write such as a 1 GB of data to file it would trigger a configurable maximum delaying limit which would then dump to parity drive immediately preventing memory overuse.
This again ties in to the fact that the content is not 'critical' so if parity was not dumped when a drive fails, worst case you only lose the latest file.
Delayed writes may be done via bcache or a similar implementation which caches the writes in memory and need not be part of the split raid driver at all.
That provided little clarity. File systems like xfs queue (delay) significant amounts of actual data before writing it to disk. The same is true of journal data. If all you are doing is caching the parity up until their is enough to bother with, then a filesystem designed for streamed data already does the for the data drive, thus you don't need to do anything new for the parity drive, just run it in sync with the data drive. At this point I interpret your proposal to be: Implement a Raid 4 like setup, but instead if stripping the date data drives, concatenate them. That is something I haven't seen done, but I can see why you would want it. Implementing via unionfs I don't understand, but as a new device mapper mechanism it seems very logical. Obviously, I'm not a device mapper maintainer, so I'm not saying it would be accepted, but if I'm right you can now have a discussion of just a few sentences which explain your goal. Greg -- Sent from my Android phone with K-9 Mail. Please excuse my brevity.
On Sat, Nov 22, 2014 at 8:24 PM, Greg Freemyer <greg.freemyer@gmail.com> wrote:
On November 22, 2014 9:43:23 AM EST, Anshuman Aggarwal < anshuman.aggarwal@gmail.com> wrote:
On 22 November 2014 at 19:33, Greg Freemyer <greg.freemyer@gmail.com> wrote:
On Sat, Nov 22, 2014 at 8:22 AM, Anshuman Aggarwal <anshuman.aggarwal@gmail.com> wrote:
By not using stripes, we restrict writes to happen to just 1 drive and the XOR output to the parity drive which then explains the delayed and batched checksum (resulting in fewer writes to the parity drive). The intention is that if a drive fails then maybe we lose 1 or 2 movies but the rest is restorable from parity.
Also another advantage over RAID5 or RAID6 is that in the event of multiple drive failure we only lose the content on the failed drive not the whole cluster/RAID.
Did I clarify better this time around?
I still don't understand the delayed checksum/parity.
With classic raid 4, writing 1 GB of data to just D1 would require 1 GB of data first be read from D1 and 1 GB read from P then 1 GB written to both D1 and P. 4 GB worth of I/O total.
With your proposal, if you stream 1 GB of data to a file on D1:
- Does the old/previous data on D1 have to be read?
- How much data goes to the parity drive?
- Does the old data on the parity drive have to be read?
- Why does delaying it reduce that volume compared to Raid 4?
- In the event drive 1 fails, can its content be re-created from the other drives?
Greg -- Greg Freemyer
Two things: Delayed writes basically to allow the parity drive to spin down if the parity writing is only 1 block instead of spinning up the drive for every write (obviously the data drive has to be spun up). Delays will be both time and size constrained. For a large write such as a 1 GB of data to file it would trigger a configurable maximum delaying limit which would then dump to parity drive immediately preventing memory overuse.
This again ties in to the fact that the content is not 'critical' so if parity was not dumped when a drive fails, worst case you only lose the latest file.
Delayed writes may be done via bcache or a similar implementation which caches the writes in memory and need not be part of the split raid driver at all.
That provided little clarity.
File systems like xfs queue (delay) significant amounts of actual data before writing it to disk. The same is true of journal data. If all you are doing is caching the parity up until their is enough to bother with, then a filesystem designed for streamed data already does the for the data drive, thus you don't need to do anything new for the parity drive, just run it in sync with the data drive.
At this point I interpret your proposal to be:
Implement a Raid 4 like setup, but instead if stripping the date data drives, concatenate them.
That is something I haven't seen done, but I can see why you would want it. Implementing via unionfs I don't understand, but as a new device mapper mechanism it seems very logical.
Obviously, I'm not a device mapper maintainer, so I'm not saying it would be accepted, but if I'm right you can now have a discussion of just a few sentences which explain your goal.
RAID4 support does not exist in the mainline. Anshuman, you might want to reach out to Neil Brown who is the maintainer for dmraid. IIUC, your requirement can be well implemented by writing a new device mapper target. That will make it modular and will help you make improvements to it easily.
Greg -- Sent from my Android phone with K-9 Mail. Please excuse my brevity.
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- Regards, Sandeep. “To learn is to change. Education is a process that changes the learner.”
Sandeep, This isn't exactly RAID4 (only thing in common is a single parity disk but the data is not striped at all). I did bring it up on the linux-raid mailing list and have had a short conversation with Neil. He wasn't too excited about device mapper but didn't indicate why or why not. I would like to have this as a layer for each block device on top of the original block devices (intercepting write requests to the block devices and updating the parity disk). Is device mapper the write interface? What are the others? Also if I don't store the metadata on the block device itself (to allow the block device to be unaware of the RAID4 on top...how would the kernel be informed of which devices together form the Split RAID. Appreciate the help. Thanks, Anshuman On 24 November 2014 at 11:06, SandeepKsinha <sandeepksinha@gmail.com> wrote:
On Sat, Nov 22, 2014 at 8:24 PM, Greg Freemyer <greg.freemyer@gmail.com> wrote:
On November 22, 2014 9:43:23 AM EST, Anshuman Aggarwal <anshuman.aggarwal@gmail.com> wrote:
On 22 November 2014 at 19:33, Greg Freemyer <greg.freemyer@gmail.com> wrote:
On Sat, Nov 22, 2014 at 8:22 AM, Anshuman Aggarwal <anshuman.aggarwal@gmail.com> wrote:
By not using stripes, we restrict writes to happen to just 1 drive and the XOR output to the parity drive which then explains the delayed and batched checksum (resulting in fewer writes to the parity drive). The intention is that if a drive fails then maybe we lose 1 or 2 movies but the rest is restorable from parity.
Also another advantage over RAID5 or RAID6 is that in the event of multiple drive failure we only lose the content on the failed drive not the whole cluster/RAID.
Did I clarify better this time around?
I still don't understand the delayed checksum/parity.
With classic raid 4, writing 1 GB of data to just D1 would require 1 GB of data first be read from D1 and 1 GB read from P then 1 GB written to both D1 and P. 4 GB worth of I/O total.
With your proposal, if you stream 1 GB of data to a file on D1:
- Does the old/previous data on D1 have to be read?
- How much data goes to the parity drive?
- Does the old data on the parity drive have to be read?
- Why does delaying it reduce that volume compared to Raid 4?
- In the event drive 1 fails, can its content be re-created from the other drives?
Greg -- Greg Freemyer
Two things: Delayed writes basically to allow the parity drive to spin down if the parity writing is only 1 block instead of spinning up the drive for every write (obviously the data drive has to be spun up). Delays will be both time and size constrained. For a large write such as a 1 GB of data to file it would trigger a configurable maximum delaying limit which would then dump to parity drive immediately preventing memory overuse.
This again ties in to the fact that the content is not 'critical' so if parity was not dumped when a drive fails, worst case you only lose the latest file.
Delayed writes may be done via bcache or a similar implementation which caches the writes in memory and need not be part of the split raid driver at all.
That provided little clarity.
File systems like xfs queue (delay) significant amounts of actual data before writing it to disk. The same is true of journal data. If all you are doing is caching the parity up until their is enough to bother with, then a filesystem designed for streamed data already does the for the data drive, thus you don't need to do anything new for the parity drive, just run it in sync with the data drive.
At this point I interpret your proposal to be:
Implement a Raid 4 like setup, but instead if stripping the date data drives, concatenate them.
That is something I haven't seen done, but I can see why you would want it. Implementing via unionfs I don't understand, but as a new device mapper mechanism it seems very logical.
Obviously, I'm not a device mapper maintainer, so I'm not saying it would be accepted, but if I'm right you can now have a discussion of just a few sentences which explain your goal.
RAID4 support does not exist in the mainline. Anshuman, you might want to reach out to Neil Brown who is the maintainer for dmraid. IIUC, your requirement can be well implemented by writing a new device mapper target. That will make it modular and will help you make improvements to it easily.
Greg -- Sent from my Android phone with K-9 Mail. Please excuse my brevity.
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- Regards, Sandeep.
“To learn is to change. Education is a process that changes the learner.”
On November 24, 2014 1:48:48 AM EST, Anshuman Aggarwal <anshuman.aggarwal@gmail.com> wrote:
Sandeep, This isn't exactly RAID4 (only thing in common is a single parity disk but the data is not striped at all). I did bring it up on the linux-raid mailing list and have had a short conversation with Neil. He wasn't too excited about device mapper but didn't indicate why or why not.
If it was early in your proposal it may simply be he didn't understand it. The delayed writes to the parity disk you described would have been tough for device mapper to manage. It doesn't typically maintain its own longer term buffers, so that would have been something that might have given him concern. The only reason you provided was reduced wear and tear for the parity drive. Reduced wear and tear in this case is a red herring. The kernel already buffers writes to the data disk, so no need to separately buffer parity writes.
I would like to have this as a layer for each block device on top of the original block devices (intercepting write requests to the block devices and updating the parity disk). Is device mapper the write interface?
I think yes, but dm and md are actually separate. I think of dm as a subset of md, but if you are going to really do this you will need to learn the details better than I know them: https://www.kernel.org/doc/Documentation/device-mapper/dm-raid.txt You will need to add code to both the dm and md kernel code. I assume you know that both mdraid (mdadm) and lvm userspace tools are used to manage device mapper, so you would have to add user space support to mdraid/lvm as well.
What are the others?
Well btrfs as an example incorporates a lot of raid capability into the filesystem. Thus btrfs is a monolithic driver that has consumed much of the dm/md layer. I can't speak to why they are doing that, but I find it troubling. Having monolithic aspects to the kernel has always been something the Linux kernel avoided.
Also if I don't store the metadata on the block device itself (to allow the block device to be unaware of the RAID4 on top...how would the kernel be informed of which devices together form the Split RAID.
I don't understand the question. I haven't thought through the process, but with mdraid/lvm you would identify the physical drives as under dm control. (mdadm for md, pvcreate for dm). Then configure the split raid setup. Have you gone through the process of creating a raid5 with mdadm. If not at least read a howto about it. https://raid.wiki.kernel.org/index.php/RAID_setup I assume you would have mdadm form your multi-disk split raid volume composed of all the physical disks, then use lvm commands to define the block range on the the first drive as a lv (logical volume). Same for the other data drives. Then use mkfs to put a filesystem on each lv. The filesystem has no knowledge there is a split raid below it. It simply reads/writes to the overall, device mapper is layered below it and triggers the required i/o calls. Ie. For a read, it is a straight passthrough. For a write, the old data and old parity have to be read in, modified, written out. Device mapper does this now for raid 4/5/6, so most of the code is in place.
Appreciate the help.
Thanks, Anshuman
I just realized I replied to a top post. Seriously, don't do that on kernel lists if you want to be taken seriously. It immediately identifies you as unfamiliar with the kernel mailing list netiquette. Greg -- Sent from my Android phone with K-9 Mail. Please excuse my brevity.
On 24 November 2014 at 18:49, Greg Freemyer <greg.freemyer@gmail.com> wrote:
On November 24, 2014 1:48:48 AM EST, Anshuman Aggarwal <anshuman.aggarwal@gmail.com> wrote:
Sandeep, This isn't exactly RAID4 (only thing in common is a single parity disk but the data is not striped at all). I did bring it up on the linux-raid mailing list and have had a short conversation with Neil. He wasn't too excited about device mapper but didn't indicate why or why not.
If it was early in your proposal it may simply be he didn't understand it.
The delayed writes to the parity disk you described would have been tough for device mapper to manage. It doesn't typically maintain its own longer term buffers, so that would have been something that might have given him concern. The only reason you provided was reduced wear and tear for the parity drive.
Reduced wear and tear in this case is a red herring. The kernel already buffers writes to the data disk, so no need to separately buffer parity writes.
Fair enough, the delay in buffering for the parity writes is an independent issue which can be deferred easily.
I would like to have this as a layer for each block device on top of the original block devices (intercepting write requests to the block devices and updating the parity disk). Is device mapper the write interface?
I think yes, but dm and md are actually separate. I think of dm as a subset of md, but if you are going to really do this you will need to learn the details better than I know them:
https://www.kernel.org/doc/Documentation/device-mapper/dm-raid.txt
You will need to add code to both the dm and md kernel code.
I assume you know that both mdraid (mdadm) and lvm userspace tools are used to manage device mapper, so you would have to add user space support to mdraid/lvm as well.
What are the others?
Well btrfs as an example incorporates a lot of raid capability into the filesystem. Thus btrfs is a monolithic driver that has consumed much of the dm/md layer. I can't speak to why they are doing that, but I find it troubling. Having monolithic aspects to the kernel has always been something the Linux kernel avoided.
Also if I don't store the metadata on the block device itself (to allow the block device to be unaware of the RAID4 on top...how would the kernel be informed of which devices together form the Split RAID.
I don't understand the question.
mdadm typically has a metadata superblock stored on the block device which identifies the block device as part of the RAID and typically prevents it from directly recognized by file system code . I was wondering if Split RAID block devices can be made to be unaware to the RAID scheme on top and be fully mountable and usable without the raid drivers (of course invalidating the parity if any of them are written to). This allows a parity disk to be added to existing block devices without having to setup the superblock on the underlying devices. Hope that is clear now?
I haven't thought through the process, but with mdraid/lvm you would identify the physical drives as under dm control. (mdadm for md, pvcreate for dm). Then configure the split raid setup.
Have you gone through the process of creating a raid5 with mdadm. If not at least read a howto about it.
Actually, I have maintained a RAID5, RAID6 6 disk cluster with mdadm for more than a few years and handled multiple failures. I am reasonably familiar with md reconstruction too. It is the performance oriented but disk intensive nature of mdadm that I would like to vary on for a home media server.
I assume you would have mdadm form your multi-disk split raid volume composed of all the physical disks, then use lvm commands to define the block range on the the first drive as a lv (logical volume). Same for the other data drives.
Then use mkfs to put a filesystem on each lv.
Maybe it can also be done via md raid creating a partitionable array where each partition corresponds to an underlying block device without any striping.
The filesystem has no knowledge there is a split raid below it. It simply reads/writes to the overall, device mapper is layered below it and triggers the required i/o calls.
Ie. For a read, it is a straight passthrough. For a write, the old data and old parity have to be read in, modified, written out. Device mapper does this now for raid 4/5/6, so most of the code is in place.
Exactly. Reads are passthrough, writes lead to the parity write being triggered. Only remaining concern for me is that the md super block will require block device to be initialized using mdadm. That can be acceptable I suppose, but an ideal solution would be able to use existing block devices (which would be untouched)...put passthrough block device on top of them and manage the parity updation on the parity block device. The information about which block devices comprise the array can be stored in a config file etc and does not need a superblock as badly as a raid setup.
Appreciate the help.
Thanks, Anshuman
I just realized I replied to a top post.
Seriously, don't do that on kernel lists if you want to be taken seriously. It immediately identifies you as unfamiliar with the kernel mailing list netiquette.
Greg -- Sent from my Android phone with K-9 Mail. Please excuse my brevity.
Sorry. Just getting used to the kernel mailing list and most tools put the default reply on the top. Thanks for replying and reminding me. Anshuman
On Mon, 24 Nov 2014 22:58:08 +0530, Anshuman Aggarwal said:
prevents it from directly recognized by file system code . I was wondering if Split RAID block devices can be made to be unaware to the RAID scheme on top and be fully mountable and usable without the raid drivers (of course invalidating the parity if any of them are written
Well, there's two basic cases: 1) You have one device and you're adding a parity device - which is basically just creating a raid-1 mirror when you get down to it. 2) You have some collection of devices in a stripe/concat/whatever, and are adding a parity device. This only works if the existing stripe/concat is already functional *without* the parity device (which implies that said stripe or concat has to be an already-supported structure)
On November 24, 2014 12:28:08 PM EST, Anshuman Aggarwal <anshuman.aggarwal@gmail.com> wrote:
On 24 November 2014 at 18:49, Greg Freemyer <greg.freemyer@gmail.com> wrote:
On November 24, 2014 1:48:48 AM EST, Anshuman Aggarwal
<anshuman.aggarwal@gmail.com> wrote:
Sandeep, This isn't exactly RAID4 (only thing in common is a single parity disk but the data is not striped at all). I did bring it up on the linux-raid mailing list and have had a short conversation with Neil. He wasn't too excited about device mapper but didn't indicate why or why not.
If it was early in your proposal it may simply be he didn't understand it.
The delayed writes to the parity disk you described would have been tough for device mapper to manage. It doesn't typically maintain its own longer term buffers, so that would have been something that might have given him concern. The only reason you provided was reduced wear and tear for the parity drive.
Reduced wear and tear in this case is a red herring. The kernel already buffers writes to the data disk, so no need to separately buffer parity writes.
Fair enough, the delay in buffering for the parity writes is an independent issue which can be deferred easily.
I would like to have this as a layer for each block device on top of the original block devices (intercepting write requests to the block devices and updating the parity disk). Is device mapper the write interface?
I think yes, but dm and md are actually separate. I think of dm as a
subset of md, but if you are going to really do this you will need to learn the details better than I know them:
https://www.kernel.org/doc/Documentation/device-mapper/dm-raid.txt
You will need to add code to both the dm and md kernel code.
I assume you know that both mdraid (mdadm) and lvm userspace tools
are used to manage device mapper, so you would have to add user space support to mdraid/lvm as well.
What are the others?
Well btrfs as an example incorporates a lot of raid capability into
the filesystem. Thus btrfs is a monolithic driver that has consumed much of the dm/md layer. I can't speak to why they are doing that, but I find it troubling. Having monolithic aspects to the kernel has always been something the Linux kernel avoided.
Also if I don't store the metadata on the block device itself (to allow the block device to be unaware of the RAID4 on top...how would the kernel be informed of which devices together form the Split RAID.
I don't understand the question.
mdadm typically has a metadata superblock stored on the block device which identifies the block device as part of the RAID and typically prevents it from directly recognized by file system code . I was wondering if Split RAID block devices can be made to be unaware to the RAID scheme on top and be fully mountable and usable without the raid drivers (of course invalidating the parity if any of them are written to). This allows a parity disk to be added to existing block devices without having to setup the superblock on the underlying devices.
Hope that is clear now?
Thank you, I knew about the superblock, but didn't realize that was what you were talking about. Does this address your desire? https://raid.wiki.kernel.org/index.php/RAID_superblock_formats#mdadm_v3.0_--... Fyi: I'm ignorant of any real details and I have not used the above new feature, but it seems to be what you asking for.
I haven't thought through the process, but with mdraid/lvm you would
identify the physical drives as under dm control. (mdadm for md, pvcreate for dm). Then configure the split raid setup.
Have you gone through the process of creating a raid5 with mdadm. If
not at least read a howto about it.
Actually, I have maintained a RAID5, RAID6 6 disk cluster with mdadm for more than a few years and handled multiple failures. I am reasonably familiar with md reconstruction too. It is the performance oriented but disk intensive nature of mdadm that I would like to vary on for a home media server.
I assume you would have mdadm form your multi-disk split raid volume
composed of all the physical disks, then use lvm commands to define the block range on the the first drive as a lv (logical volume). Same for the other data drives.
Then use mkfs to put a filesystem on each lv.
Maybe it can also be done via md raid creating a partitionable array where each partition corresponds to an underlying block device without any striping.
I think I agree.
The filesystem has no knowledge there is a split raid below it. It
simply reads/writes to the overall, device mapper is layered below it and triggers the required i/o calls.
Ie. For a read, it is a straight passthrough. For a write, the old
data and old parity have to be read in, modified, written out. Device mapper does this now for raid 4/5/6, so most of the code is in place.
Exactly. Reads are passthrough, writes lead to the parity write being triggered. Only remaining concern for me is that the md super block will require block device to be initialized using mdadm. That can be acceptable I suppose, but an ideal solution would be able to use existing block devices (which would be untouched)...put passthrough block device on top of them and manage the parity updation on the parity block device. The information about which block devices comprise the array can be stored in a config file etc and does not need a superblock as badly as a raid setup.
Hopefully the new user space feature does just that. Greg -- Sent from my Android phone with K-9 Mail. Please excuse my brevity.
On 25 November 2014 at 10:26, Greg Freemyer <greg.freemyer@gmail.com> wrote:
On November 24, 2014 12:28:08 PM EST, Anshuman Aggarwal <anshuman.aggarwal@gmail.com> wrote:
On 24 November 2014 at 18:49, Greg Freemyer <greg.freemyer@gmail.com> wrote:
On November 24, 2014 1:48:48 AM EST, Anshuman Aggarwal
<anshuman.aggarwal@gmail.com> wrote:
Sandeep, This isn't exactly RAID4 (only thing in common is a single parity disk but the data is not striped at all). I did bring it up on the linux-raid mailing list and have had a short conversation with Neil. He wasn't too excited about device mapper but didn't indicate why or why not.
If it was early in your proposal it may simply be he didn't understand it.
The delayed writes to the parity disk you described would have been tough for device mapper to manage. It doesn't typically maintain its own longer term buffers, so that would have been something that might have given him concern. The only reason you provided was reduced wear and tear for the parity drive.
Reduced wear and tear in this case is a red herring. The kernel already buffers writes to the data disk, so no need to separately buffer parity writes.
Fair enough, the delay in buffering for the parity writes is an independent issue which can be deferred easily.
I would like to have this as a layer for each block device on top of the original block devices (intercepting write requests to the block devices and updating the parity disk). Is device mapper the write interface?
I think yes, but dm and md are actually separate. I think of dm as a
subset of md, but if you are going to really do this you will need to learn the details better than I know them:
https://www.kernel.org/doc/Documentation/device-mapper/dm-raid.txt
You will need to add code to both the dm and md kernel code.
I assume you know that both mdraid (mdadm) and lvm userspace tools
are used to manage device mapper, so you would have to add user space support to mdraid/lvm as well.
What are the others?
Well btrfs as an example incorporates a lot of raid capability into
the filesystem. Thus btrfs is a monolithic driver that has consumed much of the dm/md layer. I can't speak to why they are doing that, but I find it troubling. Having monolithic aspects to the kernel has always been something the Linux kernel avoided.
Also if I don't store the metadata on the block device itself (to allow the block device to be unaware of the RAID4 on top...how would the kernel be informed of which devices together form the Split RAID.
I don't understand the question.
mdadm typically has a metadata superblock stored on the block device which identifies the block device as part of the RAID and typically prevents it from directly recognized by file system code . I was wondering if Split RAID block devices can be made to be unaware to the RAID scheme on top and be fully mountable and usable without the raid drivers (of course invalidating the parity if any of them are written to). This allows a parity disk to be added to existing block devices without having to setup the superblock on the underlying devices.
Hope that is clear now?
Thank you, I knew about the superblock, but didn't realize that was what you were talking about.
Does this address your desire?
https://raid.wiki.kernel.org/index.php/RAID_superblock_formats#mdadm_v3.0_--...
Fyi: I'm ignorant of any real details and I have not used the above new feature, but it seems to be what you asking for.
It doesn't seem to because it appears that the unified container would still need to be the created before putting any data on the device. Ideally, the split raid can be added as an after thought by just adding a parity disk (block device) to an existing set of disks (block devices)
I haven't thought through the process, but with mdraid/lvm you would
identify the physical drives as under dm control. (mdadm for md, pvcreate for dm). Then configure the split raid setup.
Have you gone through the process of creating a raid5 with mdadm. If
not at least read a howto about it.
Actually, I have maintained a RAID5, RAID6 6 disk cluster with mdadm for more than a few years and handled multiple failures. I am reasonably familiar with md reconstruction too. It is the performance oriented but disk intensive nature of mdadm that I would like to vary on for a home media server.
I assume you would have mdadm form your multi-disk split raid volume
composed of all the physical disks, then use lvm commands to define the block range on the the first drive as a lv (logical volume). Same for the other data drives.
Then use mkfs to put a filesystem on each lv.
Maybe it can also be done via md raid creating a partitionable array where each partition corresponds to an underlying block device without any striping.
I think I agree.
The filesystem has no knowledge there is a split raid below it. It
simply reads/writes to the overall, device mapper is layered below it and triggers the required i/o calls.
Ie. For a read, it is a straight passthrough. For a write, the old
data and old parity have to be read in, modified, written out. Device mapper does this now for raid 4/5/6, so most of the code is in place.
Exactly. Reads are passthrough, writes lead to the parity write being triggered. Only remaining concern for me is that the md super block will require block device to be initialized using mdadm. That can be acceptable I suppose, but an ideal solution would be able to use existing block devices (which would be untouched)...put passthrough block device on top of them and manage the parity updation on the parity block device. The information about which block devices comprise the array can be stored in a config file etc and does not need a superblock as badly as a raid setup.
Hopefully the new user space feature does just that.
Greg
Although the user space feature doesn't seem to, Neil has suggested a way to try out using RAID-4 in a manner so as to create a split raid like array. Will post on this mailing list if it succeeds.
-- Sent from my Android phone with K-9 Mail. Please excuse my brevity.
On Thu, Nov 27, 2014 at 12:50 PM, Anshuman Aggarwal <anshuman.aggarwal@gmail.com> wrote:
On 25 November 2014 at 10:26, Greg Freemyer <greg.freemyer@gmail.com> wrote:
On November 24, 2014 12:28:08 PM EST, Anshuman Aggarwal <anshuman.aggarwal@gmail.com> wrote:
On 24 November 2014 at 18:49, Greg Freemyer <greg.freemyer@gmail.com> wrote:
<snip>
Also if I don't store the metadata on the block device itself (to allow the block device to be unaware of the RAID4 on top...how would the kernel be informed of which devices together form the Split RAID.
I don't understand the question.
mdadm typically has a metadata superblock stored on the block device which identifies the block device as part of the RAID and typically prevents it from directly recognized by file system code . I was wondering if Split RAID block devices can be made to be unaware to the RAID scheme on top and be fully mountable and usable without the raid drivers (of course invalidating the parity if any of them are written to). This allows a parity disk to be added to existing block devices without having to setup the superblock on the underlying devices.
Hope that is clear now?
Thank you, I knew about the superblock, but didn't realize that was what you were talking about.
Does this address your desire?
https://raid.wiki.kernel.org/index.php/RAID_superblock_formats#mdadm_v3.0_--...
Fyi: I'm ignorant of any real details and I have not used the above new feature, but it seems to be what you asking for.
It doesn't seem to because it appears that the unified container would still need to be the created before putting any data on the device. Ideally, the split raid can be added as an after thought by just adding a parity disk (block device) to an existing set of disks (block devices)
So what precisely does "creating a container" really do? ie. have you run strace on "mdadm --create --verbose /dev/md/imsm /dev/sd[b-g] --raid-devices 4 --metadata=imsm"? I'm assuming for your use case /etc/ could hold a metadata file thast defined a container and then a second metadata file that defined the splitRAID setup.
The filesystem has no knowledge there is a split raid below it. It
simply reads/writes to the overall, device mapper is layered below it and triggers the required i/o calls.
Ie. For a read, it is a straight passthrough. For a write, the old
data and old parity have to be read in, modified, written out. Device mapper does this now for raid 4/5/6, so most of the code is in place.
Exactly. Reads are passthrough, writes lead to the parity write being triggered. Only remaining concern for me is that the md super block will require block device to be initialized using mdadm. That can be acceptable I suppose, but an ideal solution would be able to use existing block devices (which would be untouched)...put passthrough block device on top of them and manage the parity updation on the parity block device. The information about which block devices comprise the array can be stored in a config file etc and does not need a superblock as badly as a raid setup.
Hopefully the new user space feature does just that.
Greg
Although the user space feature doesn't seem to, Neil has suggested a way to try out using RAID-4 in a manner so as to create a split raid like array. Will post on this mailing list if it succeeds.
I've used hardware raid setup with raid-1 that did what you want. If needed, you could pull out a drive and connected straight to another computer and everything just worked (except mirroring). Since you're working with Neil you have the expert on the case, but don't forget most drives have unused space between sector 1 and the start of the first partition. ie. Traditionally sectors 1-62 were unused/blank. Newer systems start the first partition at sector 2048, so sectors 1-2047 are blank. I don't recall off-hand which sectors a GPT setup uses, but I assume you can find an area that is rarely used. Greg
-- Sent from my Android phone with K-9 Mail. Please excuse my brevity.
participants (4)
-
Anshuman Aggarwal -
Greg Freemyer -
SandeepKsinha -
Valdis.Kletnieks＠vt.edu