Author Topic: Transparent RAID (tRAID) Version 1.1 Preview release  (Read 6627 times)

Offline adridolf

  • Jr. Member
  • **
  • Posts: 93
  • Karma: +0/-0
    • View Profile
Re: Transparent RAID (tRAID) Version 1.1 Preview release
« Reply #45 on: February 12, 2017, 02:37:46 pm »
Would it be possible to just reinstall the 1.0 versions and see whether files come back?

Offline adridolf

  • Jr. Member
  • **
  • Posts: 93
  • Karma: +0/-0
    • View Profile
Re: Transparent RAID (tRAID) Version 1.1 Preview release
« Reply #46 on: February 12, 2017, 02:43:08 pm »
Just mounted the source drive: Files are still not shown, but used space still includes them. So FlexRaid is not hiding them, but must have made them invisible somehow in the first place.
« Last Edit: February 12, 2017, 03:19:45 pm by adridolf »

Offline adridolf

  • Jr. Member
  • **
  • Posts: 93
  • Karma: +0/-0
    • View Profile
Re: Transparent RAID (tRAID) Version 1.1 Preview release
« Reply #47 on: February 12, 2017, 02:46:26 pm »
Despite that, during every second/third mount of the tRaid drive the mount fails and gives my the appearance of an MBR-formatted drive in Disk Management (see screenshot). When I stop/start the array again, the drive is just mounted fine.

Offline adridolf

  • Jr. Member
  • **
  • Posts: 93
  • Karma: +0/-0
    • View Profile
Re: Transparent RAID (tRAID) Version 1.1 Preview release
« Reply #48 on: February 12, 2017, 03:03:52 pm »
Any ideas how to recover the data in this case? I am not very keen on spending 50-100 Euros for a ReFS recovery software.

Quite strange BTW:
Only this ReclaiMe software display the missing files (looks like it does a complete scan of the drive in the background).
RStudio is not even able to correctly show me the existing files!!
« Last Edit: February 12, 2017, 04:15:28 pm by adridolf »

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,483
  • Karma: +201/-16
    • View Profile
Re: Transparent RAID (tRAID) Version 1.1 Preview release
« Reply #49 on: February 12, 2017, 11:13:44 pm »
Any ideas how to recover the data in this case? I am not very keen on spending 50-100 Euros for a ReFS recovery software.

Quite strange BTW:
Only this ReclaiMe software display the missing files (looks like it does a complete scan of the drive in the background).
RStudio is not even able to correctly show me the existing files!!
I truly have no idea on this one. Corrupted file system maybe?
Did you check the Windows event logs for possible file system error notifications?

Offline adridolf

  • Jr. Member
  • **
  • Posts: 93
  • Karma: +0/-0
    • View Profile
Re: Transparent RAID (tRAID) Version 1.1 Preview release
« Reply #50 on: February 13, 2017, 12:30:15 am »
Should look in the event log more often:

Got four Errors 133 (two times each):
The file system detected a checksum error and was not able to correct it. The name of the file or folder is "\\?\Volume{ID}\<unable to determine file name>".
The file system detected a checksum error and was not able to correct it. The name of the file or folder is "\\?\Volume{Other ID}\FilmeHD\_InputHD"

So, I've checked and I have exactly the same issue for another folder on another disk (!). While for one disk I would have blamed ReFS, in this case I doubt it's a coincidence.

So it narrows down to two question:
1. How/why did the file system got corrupted?
2. How can one deal with a corrupt ReFS file system other than reformatting?

Both folder were not accessed by me at the specified time and have definitely not been written to.
« Last Edit: February 13, 2017, 12:34:57 am by adridolf »

Offline adridolf

  • Jr. Member
  • **
  • Posts: 93
  • Karma: +0/-0
    • View Profile
Re: Transparent RAID (tRAID) Version 1.1 Preview release
« Reply #51 on: February 13, 2017, 04:46:02 am »
Despite what caused my problems yesterday, I consider leaving ReFS and going back to NTFS for my array (while I still can). Therefore, I would like your advice on two aspects:

1. If you do have one, I would be interested in your opinion on NTFS vs ReFS (1.2)

2. From a technical perspective, I would use the following procedure to migrate my whole array to NTFS with uninterrupted parity protection (do you confirm that?):
   a) disable pool and mount tRaid drives
   b) move data off one of the tRaid disks to external drives
   c) reformat the empty drive to NTFS (SWO/TCQ off?)
   d) move data from the second (refs) drive to the first, now empty, drive (NTFS)
   e) reformat second drive, move 3->2, reformat 3rd, move 4->3, reformat 4th
   f) move the data from the external drive to the, now empty, 4th drive

If I do that, the parity should protect everything on the pool, while the only data at risk are the files on the external drives? Without the array, mixed mode of NTFS/ReFS should not be a problem, will it?
« Last Edit: February 13, 2017, 04:48:07 am by adridolf »

Offline JasonX

  • Newbie
  • *
  • Posts: 8
  • Karma: +0/-0
    • View Profile
Re: Transparent RAID (tRAID) Version 1.1 Preview release
« Reply #52 on: February 13, 2017, 05:08:49 am »
Which log file?

This is answered by adridolf already: FlexRAID.nzfx.log

SMART will auto disable if querying the drive for it fails more than 10 times.

Of course, this makes sense, since the drives go into spin-down after some time. What does not makes sense: The second option to "skip SMART monitoring" if the drives are in Spin-Down. As of my understanding: The combination of both parameters make sure the SMART monitoring is enabled but will not spin-up any drive, so the monitoring is done once the drive is online again.
Does your answere mean, the option is somehow useless? I think the monitoring should be still keep enabled in this combination!?

But I've checked this: After drives fall into spin-down mode, the option is gone after some time.

Is it possible to implement a better/smarter handling of this? Even by a third option, maybe "ignore communication errors" or something!?

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,483
  • Karma: +201/-16
    • View Profile
Re: Transparent RAID (tRAID) Version 1.1 Preview release
« Reply #53 on: February 13, 2017, 11:15:02 am »
...
1. How/why did the file system got corrupted?
...

Do you have caching enabled on the pool? There is a limitation that the tRAID disks should NOT be be accessed (they should not have a drive letter) while the pool is running with caching enabled.

Despite what caused my problems yesterday, I consider leaving ReFS and going back to NTFS for my array (while I still can). Therefore, I would like your advice on two aspects:

1. If you do have one, I would be interested in your opinion on NTFS vs ReFS (1.2)

2. From a technical perspective, I would use the following procedure to migrate my whole array to NTFS with uninterrupted parity protection (do you confirm that?):
   a) disable pool and mount tRaid drives
   b) move data off one of the tRaid disks to external drives
   c) reformat the empty drive to NTFS (SWO/TCQ off?)
   d) move data from the second (refs) drive to the first, now empty, drive (NTFS)
   e) reformat second drive, move 3->2, reformat 3rd, move 4->3, reformat 4th
   f) move the data from the external drive to the, now empty, 4th drive

If I do that, the parity should protect everything on the pool, while the only data at risk are the files on the external drives? Without the array, mixed mode of NTFS/ReFS should not be a problem, will it?
1. I don't use ReFS. File systems from my experience take a very long time to fully mature.

2. You could do as you outlined. Things would go much faster if you were to delete the config, regain control of your disks, and do the migration offline.


Offline adridolf

  • Jr. Member
  • **
  • Posts: 93
  • Karma: +0/-0
    • View Profile
Re: Transparent RAID (tRAID) Version 1.1 Preview release
« Reply #54 on: February 13, 2017, 11:43:29 am »
Do you have caching enabled on the pool? There is a limitation that the tRAID disks should NOT be be accessed (they should not have a drive letter) while the pool is running with caching enabled.

What caching do you mean precisely?
If have OS caching disabled and SWO/TCQ enabled. Storage pool caching has always been disabled for this particular reason.
However, during the time when the file system failed only the pool was mounted and none of the individual drives.

During my own research today I've found several people referring to problems with ReFS and RAID systems in general, e.g. the last post on https://social.technet.microsoft.com/Forums/windowsserver/en-US/171a1808-157e-4ef9-b0dd-8a507ff6fcef/refs-corruption-when-filled-to-capacity?forum=winserver8gen

However, everything I found was very vague.

I am done with ReFS and will switch to NTFS now.

(Since I don't think this is particularly related to the preview, you may move the ReFS failure posts to a different topic if you like, since a cannot do that.)
« Last Edit: February 13, 2017, 11:46:16 am by adridolf »

Offline adridolf

  • Jr. Member
  • **
  • Posts: 93
  • Karma: +0/-0
    • View Profile
Re: Transparent RAID (tRAID) Version 1.1 Preview release
« Reply #55 on: February 13, 2017, 11:45:16 am »
2. You could do as you outlined. Things would go much faster if you were to delete the config, regain control of your disks, and do the migration offline.

Thanks for the response.

I agree. However, in this case, I would not have the possibility to recover any data in case of a drive failure during migration until the parity is recomputed, would I?
« Last Edit: February 13, 2017, 11:46:57 am by adridolf »

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,483
  • Karma: +201/-16
    • View Profile
Re: Transparent RAID (tRAID) Version 1.1 Preview release
« Reply #56 on: February 13, 2017, 07:06:28 pm »
Thanks for the response.

I agree. However, in this case, I would not have the possibility to recover any data in case of a drive failure during migration until the parity is recomputed, would I?
I worry more about you formatting the wrong disk(s) and other user errors more so than a disk failing in that window. The shorter the window the better - and, doing it offline will go faster.

Parity won't help you much if you format the wrong disk or corrupt a disk. By doing it outside of the RAID, you are also not stressing the parity disk.

Offline adridolf

  • Jr. Member
  • **
  • Posts: 93
  • Karma: +0/-0
    • View Profile
Re: Transparent RAID (tRAID) Version 1.1 Preview release
« Reply #57 on: February 14, 2017, 05:12:04 am »
There is no way to control that on small screens. Readability improve with screen size.

I was referring to the white text and the white border of the pie sectors: I would consider it more readable with black text and without the white borders, like for the pool chart.

That's a good thing. ;)
You want the least amount of Write Update.

A Write Update is one where the same data is being rewritten due to parity stripe alignment.
http://wiki.flexraid.com/2013/06/27/understanding-the-raid-monitoring-plugin/

I read the article before and understood it differently then. But if the screenshot does not look odd to you, it's okay.
« Last Edit: February 14, 2017, 05:17:24 am by adridolf »

Offline adridolf

  • Jr. Member
  • **
  • Posts: 93
  • Karma: +0/-0
    • View Profile
Re: Transparent RAID (tRAID) Version 1.1 Preview release
« Reply #58 on: February 14, 2017, 05:17:07 am »
When reformatting from ReFS to NTFS, I had to switch from PERFORMANCE to ENERGY_EFFICIENCY to be able to format the disk (in addition to switching off TCQ and SWO).

This has not been necessary earlier, when I formatted new disks during Expansions. Thus, this could be either specific to NTFS (earlier formats were ReFS) or to the preview.

Offline adridolf

  • Jr. Member
  • **
  • Posts: 93
  • Karma: +0/-0
    • View Profile
Re: Transparent RAID (tRAID) Version 1.1 Preview release
« Reply #59 on: February 16, 2017, 10:35:20 am »
Current state on the moving files problem:

Scenario 1: Accessing pool via network share and moving folder to local drive (drag and drop)
-> Folder structure is preserved during move (=successful)
-> moved files do NOT show up in the URB (which I would call reasonable behavior)
= desired behavior

Scenario 2: Accessing pool locally (via Remote Desktop) and moving folder to non-FlexRaid drive (drag and drop)
-> Target folder is empty
-> moved folder and subfiles DO show up in the URB
= earlier behavior

Scenario 3: Accessing pool locally as in Scenario 2 but moving files with PowerShell's MoveItem
-> Target folder is created and complete
-> the URB only contains the moved folder without subfiles/-folders
EDIT:
-> the URB contains all subfiles and -folders as individual entries, indicating that they are moved and deleted sequentially by the command

In any case, although the current behavior is somewhat erratic, it is much preferable to the previous situation, as network share access is the most common usage scenario for me and no data is lost in any configuration.

Exactly the same behavior after having changed to NTFS (this issue is not ReFS-specific)