Author Topic: Storage Pool: re-group folder (minimize its split)  (Read 2143 times)

Offline xliv

  • Global Moderator
  • Full Member
  • *****
  • Posts: 208
  • Karma: +7/-1
    • View Profile
Storage Pool: re-group folder (minimize its split)
« on: September 26, 2011, 02:59:27 am »
After some time, folders are more and more split, even in auto-folder-priority.

It would be nice to have the ability of selecting 1 folder, and asking FlexRAID to  "re-group" it.

I guess 2 possibilities:
- easy one: FlexRAID will move all parts of folder to a drive having enough space
- harder one: FlexRAID will move other things to have enough space (but that could cause more split on another folder)

See http://bug.flexraid.com/view.php?id=108
Server HW: ASUS P8B-E/4L + PIKE 2008 SAS | Xeon E3-1230 | 8GB DDR3 ECC 1333 [Kingston - KVR1333D3E9SK2/8G] | Intel 320 - 80 Go (OS)
Server OS: Win7 64b EN
FlexRAID 2.0 latest build, Cruise Control, 7 DRUs (6x3TB,1x2TB), 1 PPU (2x2TB)

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: Storage Pool: re-group folder (minimize its split)
« Reply #1 on: September 26, 2011, 09:16:45 pm »
As posted on Mantis, I made changes that should eliminate this issue.
So, let see how the next release behaves.

Offline webs0r

  • Full Member
  • ***
  • Posts: 107
  • Karma: +4/-1
  • Hello, world.
    • View Profile
Re: Storage Pool: re-group folder (minimize its split)
« Reply #2 on: September 29, 2011, 03:18:28 am »
This is a pretty tough one.

The typical usage pattern is:
Drive 1      2     3      4        5
        Full   Full  Full   Part    Empty

So the easy case can't do anything much.

Only the hard case will be of value.

But then for that to work you kind of need spare / empty drives and a pretty crazy algorithm to basically to a full re-balance to minimise splits.

I have a folder that has 6 TB in it... another one with 4 TB...

Good luck!  ;D
FlexRAID expert/snapshot RAID/Storage Pool mode
Windows Server 2008 R2, 23 TB pool, Array1: 3TB redundancy, Array 2: 4TB redundancy, 11 drives total

Offline xliv

  • Global Moderator
  • Full Member
  • *****
  • Posts: 208
  • Karma: +7/-1
    • View Profile
Re: Storage Pool: re-group folder (minimize its split)
« Reply #3 on: September 29, 2011, 03:40:45 am »
The typical usage pattern is:
Drive 1      2     3      4        5
        Full   Full  Full   Part    Empty

So the easy case can't do anything much.
No, I guess the easy case will work in that pattern: re-grouping folders from the 4 first drives to the 5th one. But that will "break" little by little the fact that first drives are full and next are empty.

But if the folder you're trying to re-group is larger than a single drive, there won't be anything possible to do to re-group it...
Server HW: ASUS P8B-E/4L + PIKE 2008 SAS | Xeon E3-1230 | 8GB DDR3 ECC 1333 [Kingston - KVR1333D3E9SK2/8G] | Intel 320 - 80 Go (OS)
Server OS: Win7 64b EN
FlexRAID 2.0 latest build, Cruise Control, 7 DRUs (6x3TB,1x2TB), 1 PPU (2x2TB)

Offline Mondo

  • Jr. Member
  • **
  • Posts: 53
  • Karma: +0/-0
    • View Profile
Re: Storage Pool: re-group folder (minimize its split)
« Reply #4 on: October 24, 2011, 10:25:07 pm »
I think there is an easier way - don't split folders below a certain directory level. Make the split heirarchal rather than size based.

IE you have a CLIENT folder then beneath that a PROJECT folder - you may not care that the same client gets split across different drives but you definitely want individual projects only on one drive. So if you structure was :\CLIENTS\CLIENT_NAME\PROJECT_NAME the driectory split level is 3. Or 4 depending on how you set up the logic (whether ROOT is included as a level).

You can see how this could also work for MOVIES \ GENRE \ MOVIE_TITLE (you might want keep all movies of the same GENRE together or you might just want to ensure that the one movie stays together so tha playback is only ever off one drive...

The downside of this method is you need some kind of warning system if that "don't split" level runs out of space... wouldn't happen with known quantities like a DVD or BD - just set your free space to a minimum for that format, but for projects like mine which can take up 500 GiB or more it is a little more problematic. The word "manual" control comes to mind :D.

Norco RPC-4224 4U Rackmount Case (24 Hotswap Bays) | Supermicro X9SCM-F Motherboard (four PCI-E 8x slots @ 8x/8x/4x/4x)|4Gi RAM  | Core i3-2100 |2 x Supermicro AOC-SASLP-MV8 SAS Controller | Corsair TX650v2 PSU | Currently 12 TB array, 2 TB parity | WHS 2011