Author Topic: Phasing out RT RAID in favor of Transparent RAID  (Read 8783 times)

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,541
  • Karma: +204/-16
    • View Profile
Phasing out RT RAID in favor of Transparent RAID
« on: March 31, 2013, 05:04:33 pm »
Phasing out (dropping) RT RAID in favor of Transparent RAID
Although the next release of FlexRAID will still include Real-Time RAID-F (RT RAID), I am contemplating eventually dropping it.

The reasons for this are as follow:
1. First and most of all, RT RAID simply cannot compete against Transparent RAID (TRAID).
RT RAID was a great effort on my part; however, TRAID is a much better and robust technology.
I am not dropping RT RAID outright yet since most of you have not tested TRAID yet to agree that it is better and that RT RAID should be dropped. Hopefully, this will be clearly decided in the coming months after we've all had a good feel for TRAID.

2. I want to align the product offering and create less confusion.
I can see the typical user new to both RT RAID and TRAID struggling to understand which one to go with if both are offered.
I think it would be preferable and less confusing to have RAID-F focus only on Snapshot RAID and TRAID focus on Real-Time RAID.

For those not familiar with TRAID, read here: http://www.openegg.org/2013/02/21/flexraids-raid-f-vs-nzfs/

Offline DrBlaze

  • Sr. Member
  • ****
  • Posts: 281
  • Karma: +14/-0
    • View Profile
Re: Phasing out RT RAID in favor of Transparent RAID
« Reply #1 on: March 31, 2013, 08:42:58 pm »
Hmmm interesting....

Sounds like Traid will be easy to maintain, pretty much acting the same as a hardware raid does (obviously without the striping).

I'm concerned about loosing all the flexibility I enjoy now.  Since this new product doesn't operate above the filesystem like now, it looks like we might need to recalculate parity more often like when we want to upgrade drives.

Oh well, I guess we'll see.  Does this mean than RT will not now ever get the option for more than 1 PPU? For me, that's its only real drawback, I've learned to deal with its other small foibles.

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,541
  • Karma: +204/-16
    • View Profile
Re: Phasing out RT RAID in favor of Transparent RAID
« Reply #2 on: March 31, 2013, 11:10:06 pm »
Nothing is decided till TRAID is out and all opinions are in.

If there is enough interest in RAID-F RT RAID, then its development will continue.
The new storage pooling engine will bring some needed improvement to RT RAID and dual Parity support has already been implemented.
So, the platform is viable.

Thinking about eclipsing RT RAID is purely in light of TRAID as already stated.
TRAID is borrowed from the NZFS RAID stack. As such, it has amazing flexibility built in when it comes to expanding/contracting the array.

I think when both releases are out in a few weeks that things will get clearer.
« Last Edit: April 01, 2013, 09:19:26 am by Brahim »

Offline b-earl

  • Hero Member
  • *****
  • Posts: 651
  • Karma: +13/-1
    • View Profile
Re: Phasing out RT RAID in favor of Transparent RAID
« Reply #3 on: April 01, 2013, 05:47:40 am »
Brahim I think it looks like the right step to do. From the comparison table on the link site I already a few days ago though I would change from RT to Traid. So now I am anxiously await it to test it and see for myself. Thanks for your good products.
Server HW: Chenbro RM41416 case | Supermicro X10SLM-F + LSI SAS 9305-16i | Xeon E3-1231 v3 | 16 GB DDR3 ECC Ram
Server OS:   Windows Server 2016 (UEFI) on 250 GB Samsung 850 Evo ssd
Transparent RAID 1.1.0 2017.02.11
Backupserver: Supermicro X9SCM-F UEFI + LSI SAS9211-8i IT FW
Server OS: Win 2016

Offline NLS

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1,018
  • Karma: +29/-4
  • Look ma, no hands!
    • View Profile
    • iLogic
Re: Phasing out RT RAID in favor of Transparent RAID
« Reply #4 on: April 03, 2013, 09:09:15 am »
Vote up. (as it seems to do what I actually needed from RT RAID)

---
NLS
Production system: SBS2011 fully patched, intel Core2 Quad, 8GB, 12 disks (1 system IDE, 1 backup IDE, 10 for array and parity most SATA3), parity is 3TB, largest data disk is 3TB, millions of smaller files, common browser Chrome latest.

Offline pete248

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-1
    • View Profile
Re: Phasing out RT RAID in favor of Transparent RAID
« Reply #5 on: April 07, 2013, 05:12:14 am »
tRaid looks rather good, just a few questions about it though:

How close is this next release? And I assume moving to tRaid will require a full rebuild of the array from RT raid?

Will the upgrade to tRaid require a new licence? I only bought a flexRAID licence a few weeks ago and it would be a shame to have to pay again so soon if I wanted to upgrade to tRaid.

How is datarot managed under the new tRaid system? Is this covered by the self-healing feature discussed in other posts?

The only definite downside I can see is upgrading to larger drives in the array requires re computation of parity? Other than that it looks awesome!

Offline Mondo

  • Jr. Member
  • **
  • Posts: 53
  • Karma: +0/-0
    • View Profile
Re: Phasing out RT RAID in favor of Transparent RAID
« Reply #6 on: April 08, 2013, 01:57:05 am »
Well - currently I only use RAID-F in snapshot mode so it doesn't really concern me. This is because I have a large amount of data that doesn't change, so RT RAID esp given it is experimental seemed like overkill.

I guess what you are saying is that you have limited resources and you see those resources better diverted to T-RAID rather than further development of the RT_RAID side of FlexRAID (RAID-F)..

It does seem a pity as if you could make a truly robust version of RAID-F with the exception of the auto-rebuild feature it would surpass the proposed T-RAID in nearly every way... and it would be a truly unique product. As you say yourself the arguments for RAID-F are compellng. 

One thing I don't see there is storage pool implementation with T-RAID - I take it that is a given. I think that is the one feature everyone loves about FlexRAID.
Norco RPC-4224 4U Rackmount Case (24 Hotswap Bays) | Supermicro X9SCM-F Motherboard (four PCI-E 8x slots @ 8x/8x/4x/4x)|4Gi RAM  | Core i3-2100 |2 x Supermicro AOC-SASLP-MV8 SAS Controller | Corsair TX650v2 PSU | Currently 12 TB array, 2 TB parity | WHS 2011

Offline NLS

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1,018
  • Karma: +29/-4
  • Look ma, no hands!
    • View Profile
    • iLogic
Re: Phasing out RT RAID in favor of Transparent RAID
« Reply #7 on: April 08, 2013, 03:19:49 am »
Man you have to see this TRAID. Brahim is making a way more professional product. When this gets debugged, everybody will say "Why wasn't like that before?", "Why didn't anybody make this all these years?", "When is MSFT buying this?".

---
NLS
Production system: SBS2011 fully patched, intel Core2 Quad, 8GB, 12 disks (1 system IDE, 1 backup IDE, 10 for array and parity most SATA3), parity is 3TB, largest data disk is 3TB, millions of smaller files, common browser Chrome latest.

Offline pete248

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-1
    • View Profile
Re: Phasing out RT RAID in favor of Transparent RAID
« Reply #8 on: April 08, 2013, 12:41:16 pm »
Man you have to see this TRAID. Brahim is making a way more professional product. When this gets debugged, everybody will say "Why wasn't like that before?", "Why didn't anybody make this all these years?", "When is MSFT buying this?".

From the screenshot at the end of this page (I know this is NZFS, but still): http://openegg.org/2013/02/12/a-first-look-at-nzfs-and-replacing-unraid-with-nzfss-transparent-raid-traid/, it does look incredible. I'm in snapshot at the moment but ideally want RT RAID (I found Flexraid's real time raid to be quite unstable on my system), so I can't wait to try it, even if it is a beta.

Offline Cavemanz

  • Newbie
  • *
  • Posts: 40
  • Karma: +0/-0
    • View Profile
Re: Phasing out RT RAID in favor of Transparent RAID
« Reply #9 on: April 11, 2013, 06:34:47 am »
Man you have to see this TRAID. Brahim is making a way more professional product. When this gets debugged, everybody will say "Why wasn't like that before?", "Why didn't anybody make this all these years?", "When is MSFT buying this?".

Careful what you wish for ;P
I was at NIC 2013 in Oslo, Norway in january where Jeffrey Snover held the keynote.
To those that dont know, he's the inventor of PowerShell, and is currently Lead Architect in the Windows Server division at MSFT.

I spoke to him directly asking if he had heard about FlexRAID, which he hadnt. I suggested to him that he would check it out, and pointed out a few things I wasnt very satisfied with in Storage Spaces, and where FlexRAID did things better.

Long way from making him aware of FlexRAID, to MS buying it tho ;P

Offline NLS

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1,018
  • Karma: +29/-4
  • Look ma, no hands!
    • View Profile
    • iLogic
Re: Phasing out RT RAID in favor of Transparent RAID
« Reply #10 on: April 11, 2013, 07:33:32 am »
I know some people in MSFT too.
You never know.

---
NLS
Production system: SBS2011 fully patched, intel Core2 Quad, 8GB, 12 disks (1 system IDE, 1 backup IDE, 10 for array and parity most SATA3), parity is 3TB, largest data disk is 3TB, millions of smaller files, common browser Chrome latest.

Offline pete248

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-1
    • View Profile
Re: Phasing out RT RAID in favor of Transparent RAID
« Reply #11 on: April 15, 2013, 01:30:25 pm »
Had another look at this and seen that tRAID also supports expanding to multiple PPUs and live data reconstruction?! I completely missed that the first time, I can't wait to try this out now! Those are two awesome features. I assume this means you would assign a drive to be a "spare" and then as soon as one drive fails it starts rebuilding the failed drive onto the spare?

Offline thewierd

  • Newbie
  • *
  • Posts: 27
  • Karma: +0/-0
    • View Profile
Re: Phasing out RT RAID in favor of Transparent RAID
« Reply #12 on: July 07, 2013, 03:46:09 pm »
Is it possible to get more info on the progress?
My RT needs to be Reconciled and Force Sync Verify very often(read way to often), so any change is welcome from my part! :D
« Last Edit: July 07, 2013, 04:20:54 pm by thewierd »

Offline tphelps19

  • Jr. Member
  • **
  • Posts: 51
  • Karma: +0/-0
    • View Profile
Re: Phasing out RT RAID in favor of Transparent RAID
« Reply #13 on: November 20, 2013, 12:06:51 pm »
I don't mean this to be disrespectful or harsh but I purchased FlexRaid a year ago and Real-Time Raid was buggy as all get out.  25% of the time the FlexRaid service wouldn't auto start (and this was on a fresh install of Windows 7 Ultimate), writing large chunks of data seemed to crash the array, large size arrays are not handled very well at all (I was doing a 20 drive RAID 5 array with 2 parity drives--I know that's a lot for RAID 5 but still should in theory work).  Reading data seemed to be ok but writing data was terrible, it would crap out and just hang after writing about 1 or 2 TB or so and a lot of times I'd have to log into the GUI manager and relaunch the array manually.  I personally still don't understand why anyone would want snapshot Raid, I don't want to have access to my data "sometimes" or get a "copy" of my data that is old.

I really have high hopes for FlexRaid so my question is have any of these bugs / problems been addressed in the last year?  I have very valuable data and I'm nervous to create an array because I don't have a way of backing any of this up due to the size (50TB of data).  Any help is appreciated.

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,541
  • Karma: +204/-16
    • View Profile
Re: Phasing out RT RAID in favor of Transparent RAID
« Reply #14 on: November 20, 2013, 12:32:09 pm »
@tphelps19
RT RAID does not address all problems. It is actually very stable under the right scenario.
The only thing keeping it under experimental stage is that it does not deal well with highly fragmented data, which I suspect is the issue in your case.

FlexRAID is a platform with different features each having different pros and cons.
This is why it is best to test them all for your particular setup before committing to any of the features.

Overall, I plan to redesign RT RAID as I think Real-Time RAID over File System has a lot of benefits to tap into.
The problems with RT RAID are architectural issues and can only be addressed so. Till then, only deploy RT RAID in scenarios where the file system will not sustain high fragmentation. This includes only using programs that pre-allocate the space for files before copying them.