Author Topic: Real-Time RAID performance  (Read 5130 times)

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 6,673
  • Karma: +145/-13
    • View Profile
Real-Time RAID performance
« on: October 19, 2011, 09:51:24 pm »
This thread is almost useless without real numbers, but I am very happy with FlexRAID's real-time RAID performance.

Read speeds match that of the drive being read from and write speeds are 50% or better (as much as 70%  :o) of the target drive (remember that parity data is being captured real-time).
So with FlexRAID, no need for crappy things like a cache drive, smooth streaming, or other half-baked workarounds other implementations have to fall back to.

Nothing in the market currently comes close to this performance including expensive solutions like Drobos.
So, giving up on hardware RAID finally does not have to suck.  8)


Offline NLS

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1,024
  • Karma: +29/-4
  • Look ma, no hands!
    • View Profile
    • iLogic
Re: Real-Time RAID performance
« Reply #1 on: October 20, 2011, 06:13:17 am »
You think we can actually start using real-time for production?
Or you still label it "α"? (as in α,β,γ)

---
NLS
Production system: SBS2011 fully patched, intel Core2 Quad, 8GB, 12 disks (1 system IDE, 1 backup IDE, 10 for array and parity most SATA3), parity is 3TB, largest data disk is 3TB, millions of smaller files, common browser Chrome latest.

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 6,673
  • Karma: +145/-13
    • View Profile
Re: Real-Time RAID performance
« Reply #2 on: October 20, 2011, 08:03:23 am »
I am upgrading the Real-Time RAID functionality to Beta from Alpha.
It is also functionally complete. It supports adding drives with or without pre-existing data, removing drives with or without data, recovery (of course), verification, and can fix itself of certain errors if ever.

Only further testing by the community will say if it is production ready or not. ;)

Offline webs0r

  • Jr. Member
  • **
  • Posts: 99
  • Karma: +2/-1
  • Hello, world.
    • View Profile
Re: Real-Time RAID performance
« Reply #3 on: October 20, 2011, 04:52:03 pm »
I'm curious to give it a shot after your next big release - assuming the view is rock solid.

Do I have to re-create all the parity data in moving to real-time from snapshot? Imagine if you could just toggle a switch  ;D
FlexRAID expert/snapshot RAID/Storage Pool mode
Windows Server 2008 R2, 23 TB pool, Array1: 3TB redundancy, Array 2: 4TB redundancy, 11 drives total

Offline ausraid

  • Newbie
  • *
  • Posts: 6
  • Karma: +0/-0
    • View Profile
Re: Real-Time RAID performance
« Reply #4 on: October 20, 2011, 08:53:27 pm »
im moving to a new flexraid bilt (from freenas) and want to use real time raid.

What are the current limitations or know bugs? is there any configarations we should avoid?
« Last Edit: October 20, 2011, 08:59:02 pm by ausraid »

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 6,673
  • Karma: +145/-13
    • View Profile
Re: Real-Time RAID performance
« Reply #5 on: October 20, 2011, 10:27:01 pm »
I'm curious to give it a shot after your next big release - assuming the view is rock solid.

Do I have to re-create all the parity data in moving to real-time from snapshot? Imagine if you could just toggle a switch  ;D

You will have to recreate it for now.
Implementing a migration path between Snapshot RAID and Real-Time RAID would have to include an automatic conversion from Expert mode to Cruise Control mode and a new wizard to collect needed re-configuration information. Basically, more work for me than I want to take on right now. So, that's a nice to have I am pushing for later.

im moving to a new flexraid bilt (from freenas) and want to use real time raid.

What are the current limitations or know bugs? is there any configarations we should avoid?
Yes. For Real-Time RAID, avoid the Expert mode and use the Cruise Control mode.
I am labeling the Real-Time RAID in Expert mode experimental, and I am only supporting it in Cruise Control mode.
You will also be limited to the T1+ engine for now.


Offline NLS

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1,024
  • Karma: +29/-4
  • Look ma, no hands!
    • View Profile
    • iLogic
Re: Real-Time RAID performance
« Reply #6 on: October 21, 2011, 06:00:03 am »
Fair enough (on all).

(loose) Timetable for the next build?

---
NLS
Production system: SBS2011 fully patched, intel Core2 Quad, 8GB, 12 disks (1 system IDE, 1 backup IDE, 10 for array and parity most SATA3), parity is 3TB, largest data disk is 3TB, millions of smaller files, common browser Chrome latest.

Offline webs0r

  • Jr. Member
  • **
  • Posts: 99
  • Karma: +2/-1
  • Hello, world.
    • View Profile
Re: Real-Time RAID performance
« Reply #7 on: November 30, 2011, 11:47:55 pm »
Hm I haven't had a chance to do some proper performance tests, but I'm finding that the RT performance is about 33% of what it was before just looking at my SAB history RAR extract timings.

I am plenty happy with this actually. => Given that before I had to do a separate parity update that would likely take the other 66% of the time. + This way I get immediate protection.

Still, tonight I'll go home and run some controlled (as much as I can) tests.

I'll copy from a 7200 Samsung 1TB onto a 2TB WD Green pool drive, with the parity drive as a 3TB WD Green and see what numbers come up.
FlexRAID expert/snapshot RAID/Storage Pool mode
Windows Server 2008 R2, 23 TB pool, Array1: 3TB redundancy, Array 2: 4TB redundancy, 11 drives total

Offline webs0r

  • Jr. Member
  • **
  • Posts: 99
  • Karma: +2/-1
  • Hello, world.
    • View Profile
Re: Real-Time RAID performance
« Reply #8 on: December 01, 2011, 05:20:52 am »
OK some interesting observations (from Windows Server 2008 R2):

Copy from my Samsung -> WD 2TB goes at about 60 Meg/s, basically at full speed while Windows caches the writes in RAM.
After noticing the significant caching I thought to test with a 25 Gb file instead of a 1 Gb one.

Unfortunately I lost my pool so unable to test until I can bring it back up.

I suspect perhaps the SAB unrar process bypasses the caching or something else is holding back the unpack to the pool speed, I'm guessing with the 25Gb test I will find that it will drop to ~40% of previous in my setup based on the SAB experience.

FlexRAID expert/snapshot RAID/Storage Pool mode
Windows Server 2008 R2, 23 TB pool, Array1: 3TB redundancy, Array 2: 4TB redundancy, 11 drives total

Offline whelp

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
    • View Profile
Re: Real-Time RAID performance
« Reply #9 on: December 03, 2011, 01:30:00 am »
Where I can read how real time raid works? Does it writes parity at the same time as data comes, or it updates automatically every N seconds? I tried to write a file to RT raid, and didnt noticed any slow-downs. 80MB/s on my "green" HDDs, looks good enough, but when I started µtorrent, it took about 30 seconds just to show the list of torrents (90 torrents atm), so probably linear read/write speed is okay, but accessing file list or directory stucture has some big lags.

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 6,673
  • Karma: +145/-13
    • View Profile
Re: Real-Time RAID performance
« Reply #10 on: December 03, 2011, 10:15:41 am »
Where I can read how real time raid works? Does it writes parity at the same time as data comes, or it updates automatically every N seconds? I tried to write a file to RT raid, and didnt noticed any slow-downs. 80MB/s on my "green" HDDs, looks good enough, but when I started µtorrent, it took about 30 seconds just to show the list of torrents (90 torrents atm), so probably linear read/write speed is okay, but accessing file list or directory stucture has some big lags.

It writes the parity as its writes the data.
Indeed, RT RAID reads near speed and can write at near speed (90%+) and 50% or better on average.
In regard to utorrent, yeah, users have reported this strange behavior with the slow scan.
If you can search Mantis and add your logs to the bug report there (if none, please open one).

Offline whelp

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
    • View Profile
Re: Real-Time RAID performance
« Reply #11 on: December 04, 2011, 05:45:09 am »
Sorry, I switched back to snapshot already, I will try to test RT raid more when I get some spare time.