Author Topic: FlexRAID Snapshot - My Experiences  (Read 929 times)

Offline TheCodeMan

  • Newbie
  • *
  • Posts: 28
  • Karma: +0/-0
    • View Profile
FlexRAID Snapshot - My Experiences
« on: October 04, 2017, 08:48:15 am »
I've been using FlexRAID for years. While it's not a perfect solution, it does have its merits. This is not really a review but rather what I see as the benefits and negatives of using FlexRAID to protect data. Hopefully this info may help others decide if FlexRAID may be right for their application.

First and foremost, FlexRAID does exactly what it claims to...sort of. It does protect data. It will restore data from parity. Does it work flawlessly every time - not really.

The Good
FlexRAID protects your data. It will monitor your drives. It provides many different notification options. The interface is quite easy to use, though things aren't always where you expect them to be.

The Bad
FlexRAID does everything above but rarely by itself. While the S.M.A.R.T. monitoring works well, it does not work with some controllers. This is really one of the biggest shortcomings in my opinion. If you can't depend on FlexRAID to monitor some of your drives, then you have to find an alternative. I've tried many but the two I've had the most luck with are HD Sentinel and StableBit Scanner, the latter pretty much works well with all my controllers. FYI, I have five servers running FlexRAID, protecting a little over 100TB of data.

Want an example of how things can really go wrong? Setting scheduled updates, validates and verifies, while not monitoring your drives properly.  If this happens, your bound to lose data. If a drive goes bad, corrupt data will get written to the parity drives if it's not caught and replaced before updates. This has happened to me several times. Nothing is more painful than losing 3-4 TB of data due to pure negligence and stupidity. This is not FlexRAID's fault, it is doing exactly what you told it to do. The issue with FlexRAID is the lack of ability to cancel these scheduled tasks when a faulty drive is detected. If there is a method to do this, I am not aware of it. So watch your drives and check them before performing any updates.

In short, pair FlexRAID with a good drive scanner than monitors all your drives, perform updates manually after a drive scan, and you should be good. If your lucky enough that FlexRAID's built-in S.M.A.R.T. monitoring detects all your drives, that may be all you need.

The Ugly
And finally, we get to the few things that have really left a bad taste in my mouth when it comes to FlexRAID. The licensing process and model is a little more difficult than I would expect it to be. And what many people may not realize when purchasing FlexRAID - you are not buying a copy of software that you can use indefinitely, even on a single licensed host. The licensing is tied to the hardware, drives, processor, memory, etc. No one really know the algorithm used, short of the developers. From what I understand the license can fail even with a bulk change of hard drives. Keep this in mind if you have six 1TB drives and you plan to swap them out with 4TB drives. On a side note, this exact thing has happened to me and the FlexRAID team was kind enough to reset my license as a one off so I could continue to use the software.

The level of support is questionable. Keep in mind that you are purchasing a product without support, unless you want to pay a premium. While this forum is available, to my knowledge there are no dedicated support personnel that monitor and respond to questions. They may go unanswered, for days at a time. Some questions go unanswered indefinitely. Here is where FlexRAID in my opinion, really dropped the ball. Documentation is lackluster and it's spread out. You can glean a lot by reading through the forums but again, don't set high expectations that your post will receive an immediate answer.

As it is, I can't really view FlexRAID as a turnkey product. If you don't mind digging in a little, spending some time in the forums, reading about other people's successes and failures, you might be able to tune FlexRAID to your liking. So to wrap things up, I'm not really a major proponent for FlexRAID, nor am I against it. It fills a niche where other products fall short. Quite simply, the ability to apply parity RAID over existing data, remove a drive from the RAID, and mount on another PC to retrieve data - very few if any other products offer this.

Full disclosure: I will likely be purchasing my 6th copy of FlexRAID soon as I plan to build out another media server soon. Experimenting with UnRAID, FreeNAS and SS all don't seem to measure up for me.

Offline Bazinga

  • Jr. Member
  • **
  • Posts: 64
  • Karma: +2/-1
    • View Profile
Re: FlexRAID Snapshot - My Experiences
« Reply #1 on: October 04, 2017, 10:32:45 am »
Thanks for sharing. I've been using Snapshot for quite a few years as well and, fortunately, didn't get into situations with SMART that you are describing. What I'm interested in - is your experience with other solutions. What I really like in FlexRaid is the fact that I just bring any drive I want and it does party checks on data, so I don't need to create actual raids / format drives into some proprietary mode / etc. But I've been feeling uneasy with the level of FlexRaid support, that's for sure.

So, my question is - what does bring you back to Flex from other solutions?

Offline TheCodeMan

  • Newbie
  • *
  • Posts: 28
  • Karma: +0/-0
    • View Profile
Re: FlexRAID Snapshot - My Experiences
« Reply #2 on: October 04, 2017, 11:19:42 am »
My primary reason for going with FlexRAID over all others is the familiarity of the Windows interface. While I can and do occasionally use Linux I wanted the familiar interface, both for me and my family members.

Storage pooling was an absolute requirement.

The thought of booting and running from a USB thumb drive scares me for some reason.

I have access to a Windows-based backup and recovery system, so that helped sway my decision.

Again, having the ability to pull a drive from an existing parity RAID and read it on any PC was a big sway factor.

Proprietary formatting absolutely scares the hell out of me when we are talking 20+ TB.

While I have lost a drive or two worth of data due to my own negligence, with a proprietary system it could have been much worse.

Really, the two things that I'm most disappointed in when it comes to FlexRAID are the licensing and support models. Other than that, it is a good fit for me.

One of the strongest contenders for me has been Microsoft Storage Spaces, but the requirement of starting fresh with the drives is a big negative. This is where FlexRAID really shines, attach with existing data and go.

In reading through many posts from the forums, many people seem to be affected by updates running against failed or failing drives. I firmly believe that if you manually check your drives beforehand, these problems will go away. Of course, most people aren't going to want to hear that. Everyone wants automation, set it and forget it, etc. We are talking about terabytes of data here.

"That kind of automation doesn't come easy, or cheap."

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: FlexRAID Snapshot - My Experiences
« Reply #3 on: October 05, 2017, 01:40:54 pm »
@TheCodeMan
I think the most critical thing is using Snapshot RAID for its right purpose or Real-Time RAID otherwise.

I see RAID-F being deployed in scenarios where it simply isn't the right solution over tRAID. The allure of the flexibility of RAID-F can be a trap.
If you struggle with the maintenance tasks or they become too cumbersome, then look into tRAID.
Snapshot RAID is designed for data that is static or semi static. The only time parity is compromised is when data is edited or deleted without the recycle bin feature.
When used properly, the worst case for Snapshot RAID are newly added files not being restored since they were not included in the parity.

If you are editing content too frequently or deleting files without the recycle bin feature, then you are taking a departure from the product's primary intended usage.

I highly recommend that Snapshot RAID users setup a staging disk where content lands first before being warehoused in a Snapshot array.

Now, Snapshot RAID absolutely need some improvements as you have hinted. The most critical one being the ability to auto can the Update task on SMART reports. This was not done initially because being able to get SMART is not guaranteed for all disks. Still, it should/will be an option. Please add a feature request to bug.flexraid.com so we can track it.

There have also been many other enhancements, which are pending release. So, those fixes should improve things. No ETA on the release, but should be soon.

Offline TheCodeMan

  • Newbie
  • *
  • Posts: 28
  • Karma: +0/-0
    • View Profile
Re: FlexRAID Snapshot - My Experiences
« Reply #4 on: October 05, 2017, 03:28:55 pm »
I have my setup across all five servers as you are suggesting with a staging disk. The data is rarely manipulated on the RAID. The problem I have had in the past on multiple occasions is not catching a failed disk before multiple updates, verifies and validates are run. I know it has been mentioned in other threads that the update will cancel if a missing disk is detected. Does this also hold true for validates and verifies?

In just the last week, I had a 2 TB drive go missing, actually a complete failure of the drive. I was able to successfully restore using FlexRAID, and the bulk of the data remained intact. The restore showed successful, however, 75% of the movies files were corrupt. This leads me to believe that an update, validate or verify sent the corrupt data to the parity disks. The restore basically just restored the corrupt data back to a new disk.

Since I so infrequently change the data, I have just disabled all my scheduled update, validate and verify tasks. I monitor my drives very closely now and when I need to add new data, I verify all my disks are showing good before starting the update manually. This may not be the preferred way of doing it but I have lost data multiple times so there is some disconnect somewhere.

I know FlexRAID restores work because I tested it extensively before purchasing.


Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: FlexRAID Snapshot - My Experiences
« Reply #5 on: October 05, 2017, 03:50:50 pm »
.. I know it has been mentioned in other threads that the update will cancel if a missing disk is detected. Does this also hold true for validates and verifies?
...
Validate is designed to validate what it can and report on what if finds missing. So, by design, it does not abort on disk failure.
Verify/Verify Sync will abort if a disk is missing.
Note that some files can be flagged as restored corrupted when they are actually fine. So, all files flagged as restored corrupted should be manually checked before determining whether they are indeed corrupted.

Quote
...
In just the last week, I had a 2 TB drive go missing, actually a complete failure of the drive. I was able to successfully restore using FlexRAID, and the bulk of the data remained intact. The restore showed successful, however, 75% of the movies files were corrupt. This leads me to believe that an update, validate or verify sent the corrupt data to the parity disks.
...
Only the Update and Verify/Sync tasks can alter the parity.
One thing to check is that you don't have a Virus Scanner or the likes intruding with the FlexRAID service process. There have been cases where they corrupt the in-memory data. So, do configure your AV to ignore the FlexRAID service process.

One more tip, do set your exclusions patterns when applicable. For those storing media content, most cataloging tools create metadata files and album/movie cover images, which they override every time they do an update. It is best to exclude those files from Snapshot RAID.
« Last Edit: October 05, 2017, 04:01:11 pm by Brahim »

Offline Bazinga

  • Jr. Member
  • **
  • Posts: 64
  • Karma: +2/-1
    • View Profile
Re: FlexRAID Snapshot - My Experiences
« Reply #6 on: October 06, 2017, 07:57:55 pm »
In my case it's really mostly static data. Large media resources with some rarely changed backups that consist of smaller files. I've been contemplating to look into other solutions, but I simply don't want to move into a RAID like setup (had that years ago, had a card die on me - barely managed to recover my data) - with 30+ TB it'll be a pain to move to a RAID-like setup.

However, even though I've been using the product for many years, I was totally oblivious to SMART support. Not sure how I managed to miss that. Looking at that now I see that my drives do not show up under SMART UI at all, so, I guess, the integration isn't working. I've got my drives connected via a Supermicro SAS2 card and I wasn't able to find anything telling me that it isn't supported.

All these years I've been running in an auto update mode (which is really what it should be), so switching to manual for SMART checks and Update process will be painful - I do tend to forget things every now and then. Besides, the chance of a drive failing during Update process isn't really that much lower in this case.

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile

Offline Bazinga

  • Jr. Member
  • **
  • Posts: 64
  • Karma: +2/-1
    • View Profile
Re: FlexRAID Snapshot - My Experiences
« Reply #8 on: October 07, 2017, 12:43:46 pm »
Brahim, thanks for the pointer. That wiki page links to a page that's not there anymore. I think it should link to this now: https://www.smartmontools.org/browser/trunk/smartmontools/smartctl.8.in

Took me a while to understand what that is all about, however, I'm still not really sure. I need to somehow match the results of "--scan" to drive letter, and so far I'm not sure what's the best way to do that.

When running "smartctl --scan" I get this:

C:\Program Files\smartmontools\bin>smartctl --scan
/dev/sda -d ata # /dev/sda, ATA device
/dev/sdb -d ata # /dev/sdb, ATA device
/dev/sdc -d ata # /dev/sdc, ATA device
/dev/sdd -d ata # /dev/sdd, ATA device
/dev/sde -d ata # /dev/sde, ATA device
/dev/sdf -d ata # /dev/sdf, ATA device
/dev/sdg -d sat # /dev/sdg [SAT], ATA device
/dev/sdh -d sat # /dev/sdh [SAT], ATA device
/dev/sdi -d sat # /dev/sdi [SAT], ATA device
/dev/sdj -d sat # /dev/sdj [SAT], ATA device
/dev/sdk -d sat # /dev/sdk [SAT], ATA device
/dev/sdl -d sat # /dev/sdl [SAT], ATA device
/dev/sdm -d sat # /dev/sdm [SAT], ATA device
/dev/sdn -d sat # /dev/sdn [SAT], ATA device
/dev/csmi1,0 -d ata # /dev/csmi1,0, ATA device
/dev/csmi1,1 -d ata # /dev/csmi1,1, ATA device
/dev/csmi1,2 -d ata # /dev/csmi1,2, ATA device
/dev/csmi1,3 -d ata # /dev/csmi1,3, ATA device
/dev/csmi1,4 -d ata # /dev/csmi1,4, ATA device

I believe that [SAT] devices are the ones behind my SAS2 card.

Also, my "System Control Panel" tabs only have any information on Overview - i can pick a drive and get a pie char with a temp, but nothing else.

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: FlexRAID Snapshot - My Experiences
« Reply #9 on: October 07, 2017, 09:13:56 pm »
As per that wiki page, worry only about the type mapping. FlexRAID will typically handle the path mapping.