Author Topic: Starting the array with multiple failed disks  (Read 1861 times)

Offline SirMaster

  • Jr. Member
  • **
  • Posts: 78
  • Karma: +4/-0
    • View Profile
Starting the array with multiple failed disks
« on: July 12, 2013, 02:21:18 pm »
I know I've been posting other issues I've had, but I'm making a new topic again to keep them separated, here is the latest one I can't seem to overcome.

I still use a somewhat simple 3 DRU 2 PPU array.  To keep things quick and easy each disk is just a 1GB disk in VMWare.

My array works like normal so that's all fine as a starting block.  When I'm testing the live rebuilding when there are failed disks it seems to work when there is only 1 failed disk.

However, If I set 2 of the 3 DRUs to failed at the same time and try to start the array, what happens is the still working DRU mounts as passthrough just fine, but the other 2 disks that are marked as failed are created as un-initialized disks and thus do not get passthroughed and the pool doesnt start.  If I try to force the pool to start again, then it usually does, but it only has the unfailed DRU in it, (the pool is smaller, only 1 disk when it should be 3).

Is tRAID supposed to be able to perform live reconstruction on 2 failed DRUs at once if you have 2 PPUs?

I have yet to get into the swapping out disks to restore from parity part yet.

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: Starting the array with multiple failed disks
« Reply #1 on: July 13, 2013, 07:15:16 am »
Looking into this.

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: Starting the array with multiple failed disks
« Reply #2 on: July 21, 2013, 02:10:07 pm »
Okay, this issue is now resolved for the next build (RC3).

Thanks for the report.

Offline SirMaster

  • Jr. Member
  • **
  • Posts: 78
  • Karma: +4/-0
    • View Profile
Re: Starting the array with multiple failed disks
« Reply #3 on: July 21, 2013, 05:10:52 pm »
Awesome   :D

Keep up the great work Brahim!

Offline SirMaster

  • Jr. Member
  • **
  • Posts: 78
  • Karma: +4/-0
    • View Profile
Re: Starting the array with multiple failed disks
« Reply #4 on: July 22, 2013, 09:15:30 am »
Well, I tried out RC3 last night, but I am still having this problem.

Actually, with RC3 I can't get any live reconstruction to work, not even with just 1 failed DRU.  It always keeps creating the DRUs I have set to failed as uninitialized disks when I start the array.

I'll keep trying and post some logs when I can.  I must be doing something wrong.
« Last Edit: July 22, 2013, 03:28:00 pm by SirMaster »

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: Starting the array with multiple failed disks
« Reply #5 on: July 22, 2013, 04:27:54 pm »
@SirMaster
No, you were correct.
In RC3, I only fixed the issue for when the disks dropped out while the array is online.
The case where the array was offline with multiple dropped disks was not being properly handled.

I have that fixed now and will release RC3r2 later tonight.
Thanks for the report.

Offline SirMaster

  • Jr. Member
  • **
  • Posts: 78
  • Karma: +4/-0
    • View Profile
Re: Starting the array with multiple failed disks
« Reply #6 on: July 23, 2013, 01:12:20 am »
Awesome.

I can report that live reconstruction is working for me now with 1 and 2 failed DRUs at once!


I did find that there may be some weird issue happening when there is 1 failed DRU and 1 failed PPU at the same time (in 2 PPU array) when you start the array.

It did eventually work it looks like, but I don't think it worked the first time.  Maybe I just didn't wait long enough.  When I do this, the UI thinks everythign is done, yet the system has yet to create the NZFS passthrough disks and the pool.

This doesn't happen when only failing DRUs.  Disks and the pool create themselves immediately when it's just DRUs that are failed.

Perhaps you can try it yourself to see what happens with a better possible understanding.

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: Starting the array with multiple failed disks
« Reply #7 on: July 23, 2013, 06:20:55 am »
Yeah, I was able to replicate the odd behavior with 1 failed DRU and 1 failed PPU.
It works, but one of the disks (a non-failed one in my case) took a while to initialize.

You just have to wait long enough to see the disk finally initialize.
This also delayed the pool initialization in my case. So, when the pool finally initialized, all was okay.