Author Topic: Difference between update/verify/validate  (Read 16114 times)

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: Difference between update/verify/validate
« Reply #15 on: October 03, 2011, 11:31:00 am »
I guess I don't understand this one.  I am running WHS v1.  My Server downloads from usenet using Sabnzbd from RSS feeds, so shares change from time to time.  I was fooling around with the scheduler and initially set up a Cron job to do an update twice a day.  However, I began thinking this through:  What if I had a hard drive corrupt or die while the server is unattended (I am out of town).   Then having a "scheduled" update would wipe out all the data I would want to recover if I was there!   Seems to me you always want to run validate and depending on the outcome (if error, check logs) before you execute update.  Is this right?  If so, I am going to have to learn how to write a script so update won't execute unless the array passes the validate check ok.

No.
This has been covered a number of times already.
If a drive fails (or really a DRU path fails), so will the update operation.
Essentially, the update task does its own validation to determine if it is safe to update first before proceeding.

Offline jasonwc

  • Full Member
  • ***
  • Posts: 197
  • Karma: +5/-0
    • View Profile
Re: Difference between update/verify/validate
« Reply #16 on: October 03, 2011, 01:02:00 pm »
If one of your disks fails, the update process will fail and you will be instructed to restore the failed DRU. So, even if one of your disks fails, and you schedule updates to run automatically, the update will not cause you to lose data.

When you run an update, it first checks for new and modified files (eseentially, a quick-validate operation). You don't need to run a Validate/Verify operation before running an Update. In fact, the purpose of the Validate/Verify operations is to ensure the integrity of your data; not to check for new/modified/renamed files. That's done by the quick-validate operation, or as part of an Update operation.

However, you should run an update before a Verify/Validate operation to ensure the array is synced BEFORE checking the integrity of the data.

See http://wiki.flexraid.com/2011/08/28/maintaining-your-snapshot-raid/.

EDIT: Sorry, Brahim got to your question while I was writing up my response.
« Last Edit: October 03, 2011, 01:04:01 pm by jasonwc »
Norco RPC-4224 4U Rackmount Case (24 Hotswap Bays) | Asus P8B Workstation Motherboard (four PCI-E 16x slots @ 8x/8x/4x/4x and dual 82574L Intel server NICs) | Core i3-2100 | Supermicro AOC-SASLP-MV8 SAS Controller | Corsair TX650v2 PSU | Currently 16 TB array, 2 TB parity | WHS 2011

Offline 580guy

  • Newbie
  • *
  • Posts: 49
  • Karma: +0/-0
    • View Profile
Re: Difference between update/verify/validate
« Reply #17 on: October 03, 2011, 02:08:20 pm »

See http://wiki.flexraid.com/2011/08/28/maintaining-your-snapshot-raid/.

Thanks for the clarification.  I did read that wiki, but I guess it confused me.  The first line under Maintaining you array:

Anytime your data changes significantly, you’ll want to update your array. In the “Command Execution Center” click on “Validate”.

Sort of threw me into thinking that you had to run validate first before update.  Reading it over a bunch of times finally made it sink in that if update says ONLY 100% complete and nothing else, THEN you have to check the logs to determine if there's a problem or not.

Along these same lines I would like to ask perhaps another stupid question:

Most drive failures I have experienced usually are corruption problems.  The drive is still powered on, spinning, and present in Disk Management, yet many files are unreadable due to platter or read head problems, etc.  I can see where if a DRU is no longer present (drive totally failed, motor dies, etc) and update would tell you to restore the DRU.  Would it also say this in the former situation where the DRU is present, but corrupted? 

Again, apologize is this is a dumb question.

Thanks in advance.

Lenovo TS-440 13x2TB 1XSSD | Sans Digital TR5M-B eSATA 5 x 2TB | MediaSonic HF2-SU2S2 eSATA 4 x 2TB | WHS 2011 |Stablebit Drivepool |Stablebit Scanner| FlexRAID 2.0 Snapshot RAID|

Offline jasonwc

  • Full Member
  • ***
  • Posts: 197
  • Karma: +5/-0
    • View Profile
Re: Difference between update/verify/validate
« Reply #18 on: October 03, 2011, 02:42:48 pm »
My understanding (and it could be wrong) is that before undertaking an update process FlexRAID scans your disks for new files and modified files. It does this by accessing the file system metadata such as last modified date. Since silent data corruption wouldn't modify the metadata, but only the file data itself, the update process wouldn't update the stored parity for such files.

When you run a Validate or Verify task, FlexRAID scans the entire file and if it finds data corruption, you will be informed of the error and have the opportunity to run a restore.

Again, this is only my understanding of FlexRAID's operation. I could be wrong.

Also, I can't seem to find that line you quoted in the wiki article. Are you sure it's not from a different article?
« Last Edit: October 03, 2011, 02:44:44 pm by jasonwc »
Norco RPC-4224 4U Rackmount Case (24 Hotswap Bays) | Asus P8B Workstation Motherboard (four PCI-E 16x slots @ 8x/8x/4x/4x and dual 82574L Intel server NICs) | Core i3-2100 | Supermicro AOC-SASLP-MV8 SAS Controller | Corsair TX650v2 PSU | Currently 16 TB array, 2 TB parity | WHS 2011

Offline 580guy

  • Newbie
  • *
  • Posts: 49
  • Karma: +0/-0
    • View Profile
Re: Difference between update/verify/validate
« Reply #19 on: October 03, 2011, 05:27:12 pm »

Also, I can't seem to find that line you quoted in the wiki article. Are you sure it's not from a different article?

It's on this page:

http://wiki.flexraid.com/2011/03/26/beginners-guide-to-flexraid-2-0/

First line under Maintaining Your Array.
Lenovo TS-440 13x2TB 1XSSD | Sans Digital TR5M-B eSATA 5 x 2TB | MediaSonic HF2-SU2S2 eSATA 4 x 2TB | WHS 2011 |Stablebit Drivepool |Stablebit Scanner| FlexRAID 2.0 Snapshot RAID|

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: Difference between update/verify/validate
« Reply #20 on: October 03, 2011, 06:09:41 pm »
@jasonwc
You got it right. Data rot is only an issue when restoring, and the restore process will tell you of it (albeit too late).
That's why users should run Validate or Verify from time to time to detect those types of errors before a drive failure.

@580guy
Yeah, I agree, that line is confusing and mostly a typo, but we will have to wait for the author to clarify. :)

Offline xliv

  • Global Moderator
  • Full Member
  • *****
  • Posts: 208
  • Karma: +7/-1
    • View Profile
Re: Difference between update/verify/validate
« Reply #21 on: October 03, 2011, 06:37:55 pm »
Yeah, that guide is a bit dated but I did not want to remove it completely. I guess with the others that are online now, we could remove it. But I guess some part of its content still make sense, that's why I've kept it online - It's offline now, and I'll try to find some time to check whether I can take parts of it in a new article.

@580guy: jasonwc referred to another article: http://wiki.flexraid.com/2011/08/28/maintaining-your-snapshot-raid/
« Last Edit: October 03, 2011, 06:41:16 pm by xliv »
Server HW: ASUS P8B-E/4L + PIKE 2008 SAS | Xeon E3-1230 | 8GB DDR3 ECC 1333 [Kingston - KVR1333D3E9SK2/8G] | Intel 320 - 80 Go (OS)
Server OS: Win7 64b EN
FlexRAID 2.0 latest build, Cruise Control, 7 DRUs (6x3TB,1x2TB), 1 PPU (2x2TB)