Author Topic: bitrot prior to parity update?  (Read 14856 times)

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: bitrot prior to parity update?
« Reply #15 on: January 03, 2014, 01:25:56 pm »
The concern is valid, but the resolution is not to validate prior to update. Just schedule a regular Validate operation and catch bit rots then.

Validating at each Update will be wasteful. Additionally, aborting an Update because of bit rot is not the best course of action. If your array was to sustain a disk failure, an out of sync array will have more severe consequences than a sync'ed array with a bit rot.

Offline Shadowsoul

  • Newbie
  • *
  • Posts: 49
  • Karma: +0/-0
    • View Profile
Re: bitrot prior to parity update?
« Reply #16 on: January 03, 2014, 02:06:25 pm »
Why would not running a validate of the files that are part of an update a solution? Wasteful...well, yes, but so is a full validate if we have to run it every day to all but remove the risk of an undetected bit-rot sneaking into the parity.

As stated, if a silent error occurs between a file being part of an update and the validation of the array we have data corruption that can never ever be repaired barring us running yet another separate system to handle these cases...

While a disk failure in an unsynced array may have more severe consequences than the bit-rot that in itself is a separate concern..
If we get warned that we have bit-rot as soon as the update command runs we can repair that file and then perform the update.



Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: bitrot prior to parity update?
« Reply #17 on: January 03, 2014, 02:46:07 pm »
It is wasteful because:
1. It doubles the Update operation duration.
2. It is short sided. As previously stated, having your array in sync is more important that not having a bit rot into the parity.
3. Chances are, you won't be to resolve the bit rot even if you abort the Update because the data that would have otherwise been used to restore the bit rot would have likely changed on the other drives (likely failure beyond the tolerance level).

Wishing to use parity to restore from bit rot is something that is realistically feasible in real-time parity only as you can synchronize the changes. The same is not possible in Snapshot RAID.

In Snapshot RAID, bit rot detection is purely informational so that you can either restore is the array is in sync for the affected area, restore from backup, re-download, etc.

So, using the Validate operation to detect bit rot after the array is in sync is most appropriate than aborting an Update operation.

Offline Shadowsoul

  • Newbie
  • *
  • Posts: 49
  • Karma: +0/-0
    • View Profile
Re: bitrot prior to parity update?
« Reply #18 on: January 05, 2014, 01:25:12 pm »
Not in a 100% agreement with your points, but I do appreciate you taking the time to respond.


At least now we all know exactly how it works so there is no confusion. :)

Offline sonofwatt

  • Full Member
  • ***
  • Posts: 103
  • Karma: +1/-0
    • View Profile
Re: bitrot prior to parity update?
« Reply #19 on: January 20, 2014, 12:05:37 am »
3. Chances are, you won't be to resolve the bit rot even if you abort the Update because the data that would have otherwise been used to restore the bit rot would have likely changed on the other drives (likely failure beyond the tolerance level).

I'm curious why you say that. My understanding of bitrot is that it's highly dependant on physical characteristics of a particular drive.  If that is the case, and the bitrot isn't on the PPU, a restore should fix the issue since the parity data is still accurate.  If the bitrot is on the PPU an update should fix things by updating the parity info.  Assuming it was merely an issue of the hard drive's bit polarities becoming to weak to read, and not an issue with the physical media (harddrive surface).
Q6600, 8GB RAM, 1PPU, 1URU, 12x1TB DRUs, Server 2012, ReFS, RAID-F