Author Topic: SnapRAID dumps all content to one disk  (Read 7347 times)

Offline Valkyria

  • Newbie
  • *
  • Posts: 15
  • Karma: +0/-0
    • View Profile
SnapRAID dumps all content to one disk
« on: November 26, 2012, 05:27:31 am »
I don't know if i am gravely mistaken, or im not using the program correctly, but i followed the installation guide for snapRAID step-by-step.
http://wiki.flexraid.com/2011/08/29/setting-up-your-snapshot-raid/

I created my array with 4 DRUs and 1 PPU, with 3tb drives.
So i wanted to test how it worked. As expected, i detect all my 5 drives in windows, as i only got the RAID-verison of the program, and not the storage-pooling.
I tried to transfer a folder of ~40gb of data from an old HDD to one of the drives, then Update my RAID from the FlexRAID control panel. Here is where the problem evolves. It dumps all the files from the drive over to the designated PPU drive, occupying the exact amout of data in .flxr files. Should'nt a RAID of this type scatter these files over all the harddrives in the array?
If i copy another file of 40gb to another drive, and update the raid, will the PPU drive still take all the data, and then be filled up ridiculously fast, leaving me with one drive filled with 3tb and 4 drives with 750gb?
I hope i am using the program incorrectly.
As you can probably interpret, i am looking for a RAID5 solution.

Offline webs0r

  • Full Member
  • ***
  • Posts: 107
  • Karma: +4/-1
  • Hello, world.
    • View Profile
Re: SnapRAID dumps all content to one disk
« Reply #1 on: November 26, 2012, 05:42:34 am »
You misinterpret the FlexRAID's features. This is by design. It does not "stripe" parity data across all your drives like a hardware RAID solution. This is a key difference to typical RAID operation.
FlexRAID expert/snapshot RAID/Storage Pool mode
Windows Server 2008 R2, 23 TB pool, Array1: 3TB redundancy, Array 2: 4TB redundancy, 11 drives total

Offline Valkyria

  • Newbie
  • *
  • Posts: 15
  • Karma: +0/-0
    • View Profile
Re: SnapRAID dumps all content to one disk
« Reply #2 on: November 26, 2012, 06:02:47 am »
So it is not possible to stripe save the data with flexraid?
I was Looking to save alot of money with this solution over expensive hardwareraid solutions.
Is there any way i can reconfigure my raid to make the data striped, or do i have to look into hardwareraid controllers? Do you know any other cheap/free software raid which can do raid5/6 and still be able to add drives to the array at a later time?

Offline DrBlaze

  • Sr. Member
  • ****
  • Posts: 281
  • Karma: +14/-0
    • View Profile
Re: SnapRAID dumps all content to one disk
« Reply #3 on: November 26, 2012, 10:40:01 am »
No.  Why do you want to stripe your data?

The only good reason would be for the increased read speeds, and if that's the case you'll want hardware raid.

Most people here use Flexraid because it DOESN'T stripe data, so the data is much safer.  With traditional striped raid if you lose more than 1 drive in raid5 or 2 in raid6 then ALL Your data is gone (this can happen even if you have a tiny read error during recovery).  With Flexraid all the rest of your drives would still be fine and readable in any computer.

Offline Valkyria

  • Newbie
  • *
  • Posts: 15
  • Karma: +0/-0
    • View Profile
Re: SnapRAID dumps all content to one disk
« Reply #4 on: November 26, 2012, 12:36:12 pm »
Then what is it im not understanding?
I want to be able to lose a random drive in my array, and all the data should be intact. How is this achieved unless you spread the content on different drives?
I currently have ~7tb of data i dont want to lose, and i have five 3tb drives to put them on. I dont want to designate drives that should be able to fail when all the content is just as important as any.
I went for the flexRAID solution because someone told me it would make big quanteties of data just as safe as hardwareraid, only difference was in performance, which isn't a big deal.
From my point of view this is just RAID1'ing several drives onto designated drives. This carries the safety of RAID1, but also the huge amount of lost space on the harddrives.

Can i not use my five 3tb drives to back up to one another, and then lose a random drive, and rebuild the data from the other drives, giving me an effective 4*3tb = 12tb of data which should be recoverable?

Offline b-earl

  • Hero Member
  • *****
  • Posts: 660
  • Karma: +13/-1
    • View Profile
Sv: SnapRAID dumps all content to one disk
« Reply #5 on: November 26, 2012, 02:44:31 pm »
Hi Valkyria,
If you use flexraid then you will have one 3 tb drive for the parity and the other four 3tb drives are used for storage. Do you will have 12 tb of storage space. If the parity drive fails then the parity is made when the stud is reinitialized. If one of the DRU drives fail, the data us reconstructed from the parity drive.
Hope I could explain it.
Server HW: Chenbro RM41416 case | Supermicro X11SCZ-F + LSI SAS 9305-16i | i3-8300 | 16 GB DDR4 ECC Ram
Server OS:   Windows Server 2016 (UEFI) on  250 GB Samsung 970 Evo M.2 ssd
tRAID 1.1.0 2017.02.11
Backupserver: Supermicro X10SLM-F UEFI + LSI SAS9211-8i IT FW
Server OS: Win 2016 + tRAID

Offline webs0r

  • Full Member
  • ***
  • Posts: 107
  • Karma: +4/-1
  • Hello, world.
    • View Profile
Re: SnapRAID dumps all content to one disk
« Reply #6 on: November 26, 2012, 02:50:23 pm »
You don't need to stripe the data to achieve redundancy, that's completely irrelevant from a redundancy point of view.

So with typical RAID 5, let's say you have a 10 drive array. You get 1 drive's worth of "redundancy". All data and parity is striped across all your drives in you array.
So parts of imaginary file "Movie1.mkv" will be present on all your drives.

Let's say Drive 1 fails. You are OK because if you need "Movie1.mkv" the RAID controller will "regenerate it" from the parity data. Let's say Drive 2 fails before you have replaced Drive 1. You lose everything. All 9 drives x 3TB, gone.

Take the same situation with FlexRAID. You have a designated PPU drive containing parity only and 9 DRUs (your data drives).
If you lose 2 data drives before you manage to reconstruct your 1 drive of failed data, you *only* lose those 2 drives. So you lose 6 TB.

In either case hopefully you had a spare drive handy and you immediately recover the array before the 2nd drive fails avoiding the situation altogether :)

There are other benefits you can think of:
- no RAID controller necessary, means you can read these drives at any time, anywhere (they are portable)
- you don't have to have all drives spinning, all the time, to access your data (although somewhat negated over time as you'll find you have big folders split across all drives)
- save cost of RAID hardware
-...
FlexRAID expert/snapshot RAID/Storage Pool mode
Windows Server 2008 R2, 23 TB pool, Array1: 3TB redundancy, Array 2: 4TB redundancy, 11 drives total

Offline Valkyria

  • Newbie
  • *
  • Posts: 15
  • Karma: +0/-0
    • View Profile
Re: SnapRAID dumps all content to one disk
« Reply #7 on: November 26, 2012, 03:59:49 pm »
I think you both explained things nicely :)
But before i continue experimenting with moving files to my array, perhaps you can shed some light on the issue as to how the one PPU (Parity drive?) can fit info from all the other drives?
Like i said, i moved 40gb to a DRU, and the same amount of data was transfered to my PPU.
Should i transfer, 2.5tb, will my PPU only have 0.5tb of space remaining? How then will it fit data from my other 3 drives? :(

Offline DrBlaze

  • Sr. Member
  • ****
  • Posts: 281
  • Karma: +14/-0
    • View Profile
Re: SnapRAID dumps all content to one disk
« Reply #8 on: November 26, 2012, 10:01:18 pm »
Your PPU will grow to be as big as your largest DRU, and no bigger.   You can have as many DRUs as you want, your PPU will not fill up.

As to how Raid systems calculate many HDD's worth of data onto only 1 parity HDD... well it is kindof complicated and works much the same for many types of raid.  Here is an article explaining how parity is calculated and how lost drives can be recovered using it (just ignore the part about striping, as it does not apply to Flexraid)

http://www.pcguide.com/ref/hdd/perf/raid/concepts/genParity-c.html

Offline Valkyria

  • Newbie
  • *
  • Posts: 15
  • Karma: +0/-0
    • View Profile
Re: SnapRAID dumps all content to one disk
« Reply #9 on: November 27, 2012, 05:48:52 am »
It sounds like it works just like i want it to then :P
However, my mind cannot comprehend how 1 drive can store data from ∞ drives, by just storing the amount of data which is on the largest DRU o.0
Where are the backup data from the other drives stored in that case?
If i split my 7tb of data on my 4 DRUs, that should fill up 1.75tb on each drive, including the PPU.
Now i am able to lose a random DRU and rebuild the data from the PPU?
Should i split the data, 1.75tb to two drives, and 1.5tb to one, and 2tb to the remaining. That should leave me with a 2tb PPU, and all the drives recoverable, correct?

Offline webs0r

  • Full Member
  • ***
  • Posts: 107
  • Karma: +4/-1
  • Hello, world.
    • View Profile
Re: SnapRAID dumps all content to one disk
« Reply #10 on: November 27, 2012, 02:42:09 pm »
DrBlaze's link explains it.
I don't know if I can explain it any better, but its the same as hardware RAID

Take the equation
D1 x D2 x D3 = P1

If D2 fails, you can reconstruct by
D1 x ? x D3 = P1
D2 = P1 / D1 / D3

P1 is a mathematical combination of data of the data on D1, D2 and D3. It has to be as large as the largest DRU, because in this case, say D2 was the largest DRU
D1 = 0
D2 = has data
D3 = 0
P1 reflects D2 only and 0, 0 as inputs to the parity calc.

Aside from some funky spanning some people use, it doesn't matter how you split the data, you will always get 1 unit of DRU coverage with 1 PPU. (The default Flex engine).
In this scenario it would suggest you should put all your data on one drive if you could, cos you get 1 drive redundancy and if you get simultaneous 2 drive failure, you won't lose any data. (effectively mirroring in this case)
FlexRAID expert/snapshot RAID/Storage Pool mode
Windows Server 2008 R2, 23 TB pool, Array1: 3TB redundancy, Array 2: 4TB redundancy, 11 drives total

Offline Valkyria

  • Newbie
  • *
  • Posts: 15
  • Karma: +0/-0
    • View Profile
Re: SnapRAID dumps all content to one disk
« Reply #11 on: November 28, 2012, 12:47:21 pm »
I understand now :D
Should i concider buying the storeagepooling software as well, how will my computer detect my total diskspace with 1 PPU?
5x3tb = 15tb     or
4x3tb = 12tb ?
Is it possible to combine flexraid with other pooling software?