Author Topic: Coming at a crossroads - The future of FlexRAID  (Read 59612 times)

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Coming at a crossroads - The future of FlexRAID
« on: February 29, 2012, 08:01:30 am »
Preamble
Many of you know that I am feverishly working on NZFS.
To start with, NZFS will be a commercial product.
NZFS has the potential of reaching the enterprise market as well as serving the home user market.
That's not to say that NZFS is better than RAID-F (http://www.openegg.org/2012/02/22/flexraid-vs-nzfs/).
NZFS is not a replacement for RAID-F. In fact, the products can be complementary (but more on that later).

So, what is happening of RAID-F (FlexRAID as currently known)?

The history of FlexRAID
I started FlexRAID as a simple tool for my own use. I then decided to share it with a few users on AVSforum since I found it very useful.
There was never any ambition for me to spend more than a weekend or two on it. But then, it turned into a technical exercise.
At the time, I was running Openegg.org as a site for my many projects I have in incubation, wanting to turn them into open source projects.

My day job is that of a technical architect, and I love outside of the box thinking and pushing the limits of technology. Nonetheless, as an architect, I spend a lot of time evaluating theoretical principles. My many projects have been a way for me to bridge the gap between theory and practical implementation. Due to lack of time, I then decided to hibernate the projects and focus on them one at a time.
I expected FlexRAID to be a very short project (a few months top), which is why I decided to focus on it first. Of course, I was wrong on that one. ;D

FlexRAID might look like a very simple tool, but it is a multi-tier architectural gem. I won't go into further details on the architectural aspects that have kept me interested into the project, but for those that have been wondering what my motivations have been for the time put into the project, there you have it. :)

To open source it or to commercialize it
So, it has been fun and my technical thirst has been quenched in regard to FlexRAID's RAID-F. So, what's next?

In my original plan, I wanted to open source FlexRAID. However, concerns over people just stealing my code to go start their own projects and not contributing to FlexRAID has left me reluctant to take that route.
I have contributed to a number of open source projects and know far too well the pitfalls of the open source mantra.
Most successful open source projects are those with some real vested interest and financial backing. People tend to work best when there is some incentive.

What's next for me is NZFS from both a technical aspect and that it could make me some money.

After some serious thoughts, I think I am also going to commercialize RAID-F, and here is why:
  • Support: one of the reasons products tend to fail is from lack of support. Writing and giving away a product for free is one thing. Supporting it for free is quite another thing. My time is scarce and precious. If I am going to spend some of it supporting a product, then I need some financial incentive. In fact, this is how most open source projects are funded. The product is free, but support is not.
  • FlexRAID is written by more than just me: People tend to think that FlexRAID is the work of just one person (me). The truth is, FlexRAID is made possible thanks to the hard work of many individuals through various organizations. If RAID-F goes commercial, there are a number of licenses for technologies integrated into FlexRAID I will have to buy. This is a good thing since all these guys need to be paid for their hard work.

Let's face it, selling RAID-F will never pay for the time I have put into it (unless you guys are willing to pay me $1k a pop... I'll take it ;D).
So, the money raised will essentially go into paying for hefty (and I mean hefty) software licenses.

I know that there are plenty of users that are more than willing to pay for FlexRAID as long as they know it will be well supported.
Heck, many users like it so much that they generously donated to the project. Some even went out of their way and donated both money and their precious time.
FlexRAID pays for itself in hard drive cost alone. When you add energy cost, time not having to deal with finicky hardware RAID, the sheer flexibility to protect adhoc data, it is easy to see that the value is there.

I need to step up and give a quality well supported product to the community. I see this as a stepping stone for what's to come with NZFS.
For those wanting to use FlexRAID for free because they cannot afford it, we will work something out. I understand that what's cheap on one end of the world is not necessarily cheap on the other end. ;)

RAID-F as a commercial product:
Real-Time RAID as implemented in RAID-F should be considered experimental and not ready for production use.
Until it is stable, users will be able to download and use FlexRAID in RAID-F RT mode without needing a license.
I have full intent to make it stable, but for now what's being sold with support is Snapshot RAID + Storage Pooling.

Planned pricing for home users: (business users, please contact me)
RAID-F Storage Pooling only: $49.95 => $29.95 (sale)
RAID-F RAID: $69.95 => $39.95 (sale)
RAID-F RAID + Pooling: $99.95 => $49.95 (sale)
NZFS: TBD

What happens to donations?
Everyone that donated to the project prior to this post will get a free license of FlexRAID's RAID-F regardless of the donation amount.
Those that donated more than $50 or precious time will also get a free license of FlexRAID's NZFS when it is out. :)

Now, let the comments rip.  :P
« Last Edit: June 04, 2012, 10:28:25 am by Brahim »

Offline abs0lut.zer0

  • Newbie
  • *
  • Posts: 29
  • Karma: +0/-0
    • View Profile
Re: Coming at a crossroads - The future of FlexRAID
« Reply #1 on: February 29, 2012, 09:19:14 am »

For those wanting to use FlexRAID for free because they cannot afford it, we will work something out. I understand that what's cheap on one end of the world is not necessarily cheap on the other end. ;)

good to know.. ;D

so there will be no free model to try to see if the technology is suitable for the user's use ?


Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: Coming at a crossroads - The future of FlexRAID
« Reply #2 on: February 29, 2012, 09:55:28 am »
good to know.. ;D

so there will be no free model to try to see if the technology is suitable for the user's use ?

Sure, there will be a free license with a 3 drives max limit.
The RT RAID functionality will have no limit while still in beta.

Offline dasmarx

  • Newbie
  • *
  • Posts: 30
  • Karma: +0/-0
    • View Profile
Re: Coming at a crossroads - The future of FlexRAID
« Reply #3 on: February 29, 2012, 10:02:32 am »
This sounds awesome.

So this means, you will put more time into the development of flexraid and nzfs.

Also, you should make a preorder of nzfs in some bundle with RAID-F RAID + Pooling.  I am happy with the stuff right now but would use nzfs the day it releases (pretty sure about that).

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: Coming at a crossroads - The future of FlexRAID
« Reply #4 on: February 29, 2012, 10:30:50 am »
This sounds awesome.

So this means, you will put more time into the development of flexraid and nzfs.

Also, you should make a preorder of nzfs in some bundle with RAID-F RAID + Pooling.  I am happy with the stuff right now but would use nzfs the day it releases (pretty sure about that).

Indeed.
This does mean that my commitment to RAID-F and NZFS will increase.
The key thing here is that I won't be able to use the excuse of this being free software anymore.  :P

Great idea about pre-ordering NZFS, but I am trying (very very hard) to not leak any more info on it till the grand reveal.
I have a lot of eyes on me... people ready to copy everything I do. So, I am building a big enough of a gap.

Offline sekkey

  • Newbie
  • *
  • Posts: 7
  • Karma: +0/-0
    • View Profile
Re: Coming at a crossroads - The future of FlexRAID
« Reply #5 on: February 29, 2012, 11:19:47 am »
This is a great idea. Do you have plans to implement a support ticket system or will support be in the forums only with a private owners section or something similar?

Offline sbose

  • Newbie
  • *
  • Posts: 27
  • Karma: +1/-0
    • View Profile
Re: Coming at a crossroads - The future of FlexRAID
« Reply #6 on: February 29, 2012, 12:00:08 pm »
so the question is, when can we expect a final version of flexraid (snapshot raid) and Storage pool available for download for purchase ?, also do we get a discount on future NZFS product once it gets released ?
« Last Edit: February 29, 2012, 12:02:45 pm by sbose »

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: Coming at a crossroads - The future of FlexRAID
« Reply #7 on: February 29, 2012, 01:01:59 pm »
so the question is, when can we expect a final version of flexraid (snapshot raid) and Storage pool available for download for purchase ?, also do we get a discount on future NZFS product once it gets released ?

I am going to be doing work on RAID-F starting this coming weekend (clean ups and bug fixes).
I am thinking the final version should be out within the next 4 weeks.
My hope is two have about two releases in between to validate the bug fixes.

So, release schedule:
- Friday, March 9th
- Friday, March 16th
- Final release on March 23rd or 30th (at the latest)

Yeah, users that order RAID-F will get a discount on NZFS which will be stack-able with any other discount NZFS might have.
How much discount exactly will be determined once I price NZFS.

Offline pko

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
    • View Profile
Re: Coming at a crossroads - The future of FlexRAID
« Reply #8 on: March 01, 2012, 06:40:13 am »
Hello, Brahim.

I have mixed feelings about this, on one hand, I think it is just fair to have decent compensation for your effort, in another  I find the product way too immature (at least on linux) to be a for-sale solution, at least right now. Also, I think the 3 drive limitation prohibits real tests, In my opinion you should be capable of trying the product in a *real* configuration (I doubt anybody will use it for just 3 drives!), perhaps a time-limited or throughput-limited configuration would be better.

Also I find the cost a little steep, specially for storage pooling as there are a number of free alternatives (I was thinking of using alternatives even when considered S.P. free, because I am afraid of its stability) What advantages has SP compared to the other, free or otherwise, solutions?

Please understand I am not bashing your decision of going for-pay, as I said I think people like you which put a real effort to create a good and easy-to-use product on hand to normal users deserve a compensation, only that I find the chosen scheme to be inappropriate, specially given the state of the product.

I tried it and found it far from release-ready, at least it was a couple months ago and reading the forum I think it still is. I am speaking of far from release-ready in the sense of my confidence of committing to use it daily (S.P.) without problems/instability and confidence that in a worst case scenario I will be capable of retrieving my data (R-F) and not found then that some weird problem like the drive structure, a hardware peculiarity or some special characters in my files' names has prevented having usable redundancy data.

That were my feelings about it *before*, when FR was a free product and was waiting for a better version to give it a try in my real configuration (even then, in a secondary server, not my main one).

When there is a product that has been tried and worked without mayor glitches in a decent range of hardware/software configurations for at least a few months, *then* it can be said that it is production-ready and people can be in confidence charged for it, in my opinion. Else, if there are paying customers who find glitches in basic features (and in FR that glitches may mean not being capable of using their data or, worst, having lost it forever), that will create very bad blood.

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: Coming at a crossroads - The future of FlexRAID
« Reply #9 on: March 01, 2012, 10:22:33 am »
1. Like every software out there, things work for some and not for others.
If the basis was for the software to work for all before going retail, all software vendors including Microsoft would be out of business.
Almost none of the commercial software out there started out bug free.
Snapshot RAID has been proven and is mature as people have successfully been using it for years now.
The pooling feature has a few lingering bugs which will be fixed soon, but it is usable and works well overall.

2. Reiterating on number 1 and contrary to your belief, FlexRAID is working very stable for plenty of users on Linux (and of course on Windows).
I can't control the environment where FlexRAID is installed, and that has plenty to do with how it is working for different users.
So, if it works for 90% of the users, the remaining 10% are the minority and should make and wait for bug fixes or feel free to look for alternatives.

3. Relative to storage prices, RAID-F is down right cheap.
I am not selling the product to people on a string budget or to those that want everything for free. I am selling it to people that can afford it and are willing to pay. ;)
As posted above, I am more than willing to work with people that cannot afford it due to macro-economic reasons (countries where the income level is low).

4. If RAID-F is to get any better, it will need paying customers as an incentive.
If the product sucked entirely, it really would not matter whether it was free or not.
If people use it and donate to the project despite alternatives and currently outstanding bugs, it is because it has indisputable value.

Ultimately, I am defining my audience and establishing a new level of expectation. :)


Offline sbose

  • Newbie
  • *
  • Posts: 27
  • Karma: +1/-0
    • View Profile
Re: Coming at a crossroads - The future of FlexRAID
« Reply #10 on: March 01, 2012, 10:35:20 am »
Very Well said Brahim  :)

I will be happy to pay for this great Product.

PS : Just make the storage pool rock solid as the snapshot raid :-), looks like some bugs are still floating around.

Offline abs0lut.zer0

  • Newbie
  • *
  • Posts: 29
  • Karma: +0/-0
    • View Profile
Re: Coming at a crossroads - The future of FlexRAID
« Reply #11 on: March 01, 2012, 10:54:43 am »
Ultimately, I am defining my audience and establishing a new level of expectation. :)

this is a very intriguing sentence....  ::)

Offline pko

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
    • View Profile
Re: Coming at a crossroads - The future of FlexRAID
« Reply #12 on: March 01, 2012, 11:03:19 am »
Forgive me for this long post, that may also appear to be arrogance in my part, but in my opinion many good programs that start life as free fail in the long term because they do not choose a model that works, and many times the errors are blatant... perhaps the following considerations are useful to you:

First, be clear about you REAL objective:

You want that everyone that is willing to pay for your product actually pays for it (these are the paying customers, PC)

Many people wastes too much effort in something that is NOT your goal:

Your goal IS NOT to prevent people that is not willing to pay for your software to actually using it

There are people that, no matter how much it costs or how useful it is for them, will not pay for it. Let's call them "free riders" (FR) and some will for their own reasons perform irrational efforts to use your software without paying. Do not waste too much resources and time with them, if your software is good there will be many of them, just assume that as an unfair fact of life and move on.

To maximize the number of PC you must account what one person needs to become a PC:

1) Must know your software: that exists and that fulfills reliably some desired task
2) Can actually use it: its easy to install and use, fits his/her software and hardware profile, etc.
3) Can pay for it
4) Is willing to pay for it

See that FRs will expand your user base and so will improve on points 1) and 2), attracting more potential PCs, so FRs in the right amount are actually a very good thing. Too many of them, on the other hand, probably show that there is something that you are not doing well: you are giving away too much for free, or you are asking too much and people uses pirated versions, or paying is difficult, or your attitude or behavior impels people to not desire to pay you, etc. etc.

Point 3) is mostly your decision: if you charge too much or it is difficult to pay, potential PC will be just FR or not users at all (that last is the worst), if the price is too low you will profit less...

Point 1) is mostly out of your control except by spending money in advertising

Point 2) I'm assuming is your main goal as a programmer, to have your software to be the best possible for the wider user base as possible, so probably there is nothing to do there that you are not already doing now.

So the main point you can control is 4), if you give everything for free and allow anyone to use your software, only a few people will donate and many potential PC will become just FR. If you completely close your software, only PCs will use it, except for people pirating your software and becoming FR (by the way, that is not really bad, most of the pirates would not pay anyway so those are not real sales lost, but they will present your software to other people, potential PCs that may become actual PCs out of willing to do the right thing or just because they have no access to the pirated version)

But in my opinion, the sweet point is one that makes the software useful enough to maximize its distribution (so, reaching as many potential PCs as possible) and, at the same time, imposes some limits that compel to pay to get rid of them without actually preventing to use the software altogether.

In this particular case, I think that the best option is to make the software fully usable so people commits to use it, but in the long run, when they have grown accustomed to using it, will pay just to get the full functionality unencumbered. Some ideas on how to achieve that:

1) Give full functionality forever, but limit throughput
2) Give full functionality at full speed, but force to make some actions regularly, for example, do not allow using redundancy data that is older than 96h hours, thus forcing to fully rebuild parity at least thrice a week. If they have a hardware failure and parity is too old, they must then pay for a license to rebuild their data... but make that fact plain and clear BEFORE allowing installation, of course, you do not want to appear a cheater and alienate people.
3) Give full functionality forever at full throughput, but insert some forced interactive stops to impede automation. At the same time, that interaction will remind users that they are behaving like cheap scoundrels. This is what actually the program "Total Commander" does!

These are just a few ideas, but basically, show people that you are behaving fairly and giving them something they need and enjoy and many people will come to terms to paying for what they actually use.

Offline sbose

  • Newbie
  • *
  • Posts: 27
  • Karma: +1/-0
    • View Profile
Re: Coming at a crossroads - The future of FlexRAID
« Reply #13 on: March 01, 2012, 11:29:24 am »
some very good points, I would take apple's app pricing approach, make it cheap enough that new users are willing to try it without thinking too much into it. That will give prospective buyers more incentive to implement flexraid, just my 2 cents. I agree with Brahim about paying for the software (its really unique and a very smart piece of programming deserving the cost), and its mature enough for prime time provided the outstanding bugs are fixed which brahim already indicated that it will be fixed my march end.

The only thing that I would recommend is to lower the initial cost to make it more attractive for future customers.

don't get me wrong i will buy it even for the full price listed, because i have been using it for months now and I trust at least the snapshot raid mode with my data. But for someone new the idea of playing around with their live data while paying for the software at listed price would make them think long and hard

Just my 2 Cents

Offline pko

  • Newbie
  • *
  • Posts: 12
  • Karma: +0/-0
    • View Profile
Re: Coming at a crossroads - The future of FlexRAID
« Reply #14 on: March 01, 2012, 11:59:38 am »
1. Like every software out there, things work for some and not for others.
If the basis was for the software to work for all before going retail, all software vendors including Microsoft would be out of business.

Well, yes, but there are things that are basics and things that are let's call it secondary... and in free software, people are much more permissive of glitches.

Snapshot RAID has been proven and is mature as people have successfully been using it for years now.
The pooling feature has a few lingering bugs which will be fixed soon, but it is usable and works well overall.

It is true that most of what I find most problematic I think it pertains to pooling, these are from the forums from the last months:

1) I can't understand how could have taken years to detect that you must use UUID and not hardware paths. I have installed linux in like 20 computers, and detected the problem of changing hardware addresses in at least 2 of them (and many others may behave the same, but since they had just one drive, the problem did not presents itself). I've been using UUID for a few years now; I used some ad-hoc homemade solutions in the past. That is not something that happens rarely to weird hardware, but a fairly common thing.

2) People are reporting instabilities when copying large files or make some medium to heavy use of storage pools, that happened at least until the beta 4 or so

3) There were also reports of problems with files named with special characters (special in english, but absolutely common in other languages).

Those problems I think they are pretty basic, but that have been discovered very recently. They may be many others lurking there! I'm afraid that the harware and user base where flecraid has been tested is too reduced!

3. Relative to storage prices, RAID-F is down right cheap.

Of course, that is a matter of opinion :-D I'm not trying to bargain here, but I think the pricing is more or less on par with other solutions like unraid (at least the sale version, $69.95 compared to $69 for UR "plus"). And of course there are other options for free like ZFS or common RAID that they are not really comparable, and there is also the option of not using RAID at all, but simply copy the files to another place :D in pre-flood prices, $100 is like 2.5TB.

4. If RAID-F is to get any better, it will need paying customers as an incentive.

1500 users @20 is better than 200 users @100. And these things are not linear, if it reaches certain popularity levels, the numbers explode.

If the product sucked entirely, it really would not matter whether it was free or not.

Of course, I think the product is great, although there are certain edges that need to be tamed.

Ultimately, I am defining my audience and establishing a new level of expectation. :)

Of course again, please, do not interpret my words as empty criticism, I'm trying to be constructive here and you as creator have the last word and I deeply respect that, whatever it is.