Author Topic: Flexraid keeps going readonly mode  (Read 2554 times)

Offline MasterCATZ

  • Jr. Member
  • **
  • Posts: 66
  • Karma: +0/-0
    • View Profile
Flexraid keeps going readonly mode
« on: May 11, 2013, 04:18:20 pm »
Code: [Select]
LABEL=FlexRaidPPU1 /mnt/FlexRaidPPU1 btrfs defaults 0 0
LABEL=FlexRaidDRU1 /mnt/FlexRaidDRU1 btrfs defaults 0 0
LABEL=FlexRaidDRU2 /mnt/FlexRaidDRU2 btrfs defaults 0 0
LABEL=FlexRaidDRU3 /mnt/FlexRaidDRU3 btrfs defaults 0 0
LABEL=FlexRaidDRU4 /mnt/FlexRaidDRU4 btrfs defaults 0 0
LABEL=FlexRaidDRU5 /mnt/FlexRaidDRU5 btrfs defaults 0 0
LABEL=FlexRaidDRU6 /mnt/FlexRaidDRU6 btrfs defaults 0 0
LABEL=FlexRaidDRU7 /mnt/FlexRaidDRU7 btrfs defaults 0 0
LABEL=FlexRaidDRU8 /mnt/FlexRaidDRU8 btrfs defaults 0 0

is how the drives are mounted for flexraid on startup

Code: [Select]
root@aio:~/FlexRAID-Managed-Pool# ls
class1_0
root@aio:~/FlexRAID-Managed-Pool# cd class1_0/
root@aio:~/FlexRAID-Managed-Pool/class1_0# ls
{004f3a72-1b8d-4855-9f24-4838f5355bcd}  {48a5abe3-8cb4-4dde-8254-a874a3284b52}
{023eb29c-eca9-4081-abe1-6a28d969c9a0}  {6f4d93c3-81cc-4416-8a77-bf59d6bfee9b}
{06556123-bddc-41bb-b291-97470eaccda7}  {814a2ae7-0da9-4c69-8774-78d14b17160a}
{25d44a05-bdbe-49b3-8554-edd16f5caf65}  {a45d4ebd-fe9a-4a4f-8c5c-fe221944b653}
{46e726e7-13d5-432b-801e-ad280b4e962b}  {efe0c4bd-1a11-4341-aa1c-5377b6ebf89b}
root@aio:~/FlexRAID-Managed-Pool/class1_0#

is what flexraid creates

I then have flexraid mount into the /mnt folder

Code: [Select]
FlexRaidDRU1  FlexRaidDRU4  FlexRaidDRU7  FlexRAID_POOL
FlexRaidDRU2  FlexRaidDRU5  FlexRaidDRU8  FlexRaidPPU1
FlexRaidDRU3  FlexRaidDRU6  FlexRaidDRU9
root@aio:/mnt#

all those mounts are accessible


if I copy data directly to a HDD I seem to not have problems , but if I copy data into
FlexRAID_POOL it will randomly lock off the HDD mount of drive that has data going to and lock out FlexRAID_POOL

needing reboot to go back into write mode

using Ubuntu and btrfs

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: Flexraid keeps going readonly mode
« Reply #1 on: May 11, 2013, 07:35:41 pm »
Are you using RT RAID or Snapshot RAID?
Also, look into the sys logs for possible clues.

Offline MasterCATZ

  • Jr. Member
  • **
  • Posts: 66
  • Karma: +0/-0
    • View Profile
Re: Flexraid keeps going readonly mode
« Reply #2 on: May 11, 2013, 09:20:41 pm »
Snapshot RAID as that’s the only one with out RAID restrictions :S


still yet to make a snapshot either

I have tried allowing/disallowing root/other users in the advanced settings as well does not effect it

flexraid is started as root all mounts are root with full RW all users  , but when files are copied I see the mount permissions drop from root to user level and then get read-only access

re:look into the sys logs for possible clues

can you elaborate a bit more on that my syslog and kernel log files are 16 gig ...

compressing them now so I can remove reboot and see if I can get to them before they are that large again

« Last Edit: May 11, 2013, 09:39:56 pm by MasterCATZ »

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile

Offline MasterCATZ

  • Jr. Member
  • **
  • Posts: 66
  • Karma: +0/-0
    • View Profile
Re: Flexraid keeps going readonly mode
« Reply #4 on: May 11, 2013, 10:59:53 pm »
log files before going more verbose


pretty much rebooted .. started a file copy to flexraid pool .. did not take long for it to make write protected

Offline MasterCATZ

  • Jr. Member
  • **
  • Posts: 66
  • Karma: +0/-0
    • View Profile
Re: Flexraid keeps going readonly mode
« Reply #5 on: May 11, 2013, 11:25:29 pm »
verbose logs

Offline MasterCATZ

  • Jr. Member
  • **
  • Posts: 66
  • Karma: +0/-0
    • View Profile
Re: Flexraid keeps going readonly mode
« Reply #6 on: May 11, 2013, 11:26:39 pm »
your more then welcome to pass the blame onto using btrfs as well

if you think that is the problem I will switch to another fs you recommend that will work with ubuntu ...

I decided to do a snapshot .. it also would not work until I restarted

just clicked update now and waiting to see if it completes
*edit* already failed .. I might switch to another fs and try again


attached flexraid logs made during snapshot process

Code: [Select]
May 12 13:38:36 aio kernel: [   11.566906] btrfs: relocating block group 51568967680 flags 1
May 12 13:38:36 aio kernel: [   11.581565] btrfs: unlinked 2 orphans
May 12 13:38:36 aio kernel: [   11.581569] btrfs: truncated 1 orphans
May 12 13:38:36 aio kernel: [   11.590344] device label FlexRaidPPU1 devid 1 transid 35 /dev/sdd1
May 12 13:38:36 aio kernel: [   11.590911] btrfs: disk space caching is enabled
May 12 13:38:36 aio kernel: [   11.621407] device label FlexRaidDRU1 devid 1 transid 591 /dev/sdc1
May 12 13:38:36 aio kernel: [   11.621841] btrfs: disk space caching is enabled
May 12 13:38:36 aio kernel: [   11.649542] btrfs: found 1 extents
May 12 13:38:36 aio kernel: [   11.694710] btrfs: found 1 extents
May 12 13:38:36 aio kernel: [   11.713121] btrfs: relocating block group 50495225856 flags 1
May 12 13:38:36 aio kernel: [   11.734504] btrfs: relocating block group 49421484032 flags 1
May 12 13:38:36 aio kernel: [   11.752446] btrfs: relocating block group 48347742208 flags 1
May 12 13:38:36 aio kernel: [   11.769634] btrfs: found 1 extents
May 12 13:38:36 aio kernel: [   11.810709] btrfs: found 1 extents
May 12 13:38:36 aio kernel: [   11.827891] btrfs: relocating block group 47274000384 flags 1
May 12 13:38:36 aio kernel: [   11.853667] btrfs: found 3 extents
May 12 13:38:36 aio kernel: [   11.927923] btrfs: found 3 extents
May 12 13:38:36 aio kernel: [   11.954307] btrfs: relocating block group 46200258560 flags 1
May 12 13:38:36 aio kernel: [   11.979357] btrfs: relocating block group 45126516736 flags 1
May 12 13:38:36 aio kernel: [   12.289952] btrfs: found 359 extents
May 12 13:38:36 aio kernel: [   12.318657] btrfs: unlinked 1 orphans
May 12 13:38:36 aio kernel: [   12.340825] device label FlexRaidDRU2 devid 1 transid 24 /dev/sdf1
May 12 13:38:36 aio kernel: [   12.341495] btrfs: disk space caching is enabled
May 12 13:38:36 aio kernel: [   12.360352] device label FlexRaidDRU3 devid 1 transid 22 /dev/sdj1
May 12 13:38:36 aio kernel: [   12.360742] btrfs: disk space caching is enabled
May 12 13:38:36 aio kernel: [   12.384250] device label FlexRaidDRU4 devid 1 transid 24 /dev/sde1
May 12 13:38:36 aio kernel: [   12.384758] btrfs: disk space caching is enabled
May 12 13:38:36 aio kernel: [   12.414515] device label FlexRaidDRU5 devid 1 transid 441 /dev/sdh1
May 12 13:38:36 aio kernel: [   12.414910] btrfs: disk space caching is enabled
May 12 13:38:36 aio kernel: [   12.663448] device label FlexRaidDRU6 devid 1 transid 24 /dev/sdl1
May 12 13:38:36 aio kernel: [   12.664294] btrfs: disk space caching is enabled
May 12 13:38:36 aio kernel: [   12.700792] device label FlexRaidDRU7 devid 1 transid 24 /dev/sdg1
May 12 13:38:36 aio kernel: [   12.701414] btrfs: disk space caching is enabled
May 12 13:38:36 aio kernel: [   12.738831] device label FlexRaidDRU8 devid 1 transid 22 /dev/sdi1
May 12 13:38:36 aio kernel: [   12.739488] btrfs: disk space caching is enabled
May 12 13:38:36 aio kernel: [   12.762052] device label FlexRaidDRU9 devid 1 transid 115 /dev/sdk1
May 12 13:38:36 aio kernel: [   12.762976] btrfs: disk space caching is enabled
May 12 13:38:36 aio kernel: [   12.997687] btrfs: found 359 extents
May 12 13:38:36 aio kernel: [   13.015398] btrfs: relocating block group 44052774912 flags 1
May 12 13:38:36 aio kernel: [   13.029155] btrfs: relocating block group 42979033088 flags 1
May 12 13:38:36 aio kernel: [   13.043323] btrfs: found 4 extents
May 12 13:38:36 aio kernel: [   13.093911] btrfs: found 4 extents
May 12 13:38:36 aio kernel: [   13.176240] btrfs: relocating block group 41905291264 flags 1
and a snip  of a 500 meg sys log
Code: [Select]
May 12 13:50:04 aio kernel: [  700.735464] ata5.00: exception Emask 0x0 SAct 0x7bfff80f SErr 0x0 action 0x6 frozen
May 12 13:50:04 aio kernel: [  700.735472] ata5.00: failed command: WRITE FPDMA QUEUED
May 12 13:50:04 aio kernel: [  700.735477] ata5.00: cmd 61/00:00:e8:1d:80/04:00:5d:00:00/40 tag 0 ncq 524288 out
May 12 13:50:04 aio kernel: [  700.735477]          res 40/00:00:09:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
May 12 13:50:04 aio kernel: [  700.735480] ata5.00: status: { DRDY }
May 12 13:50:04 aio kernel: [  700.735482] ata5.00: failed command: WRITE FPDMA QUEUED
May 12 13:50:04 aio kernel: [  700.735486] ata5.00: cmd 61/00:08:e8:21:80/04:00:5d:00:00/40 tag 1 ncq 524288 out
May 12 13:50:04 aio kernel: [  700.735486]          res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
May 12 13:50:04 aio kernel: [  700.735488] ata5.00: status: { DRDY }
May 12 13:50:04 aio kernel: [  700.735490] ata5.00: failed command: WRITE FPDMA QUEUED
May 12 13:50:04 aio kernel: [  700.735493] ata5.00: cmd 61/10:10:e8:25:80/00:00:5d:00:00/40 tag 2 ncq 8192 out
May 12 13:50:04 aio kernel: [  700.735493]          res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
May 12 13:50:04 aio kernel: [  700.735495] ata5.00: status: { DRDY }
May 12 13:50:04 aio kernel: [  700.735497] ata5.00: failed command: WRITE FPDMA QUEUED
May 12 13:50:04 aio kernel: [  700.735500] ata5.00: cmd 61/00:18:20:12:61/02:00:05:00:00/40 tag 3 ncq 262144 out
May 12 13:50:04 aio kernel: [  700.735500]          res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
May 12 13:50:04 aio kernel: [  700.735502] ata5.00: status: { DRDY }

Code: [Select]
May 12 13:51:45 aio kernel: [  801.841524] Write(16): 8a 00 00 00 00 00 00 08 44 b0 00 00 00 08 00 00
May 12 13:51:45 aio kernel: [  801.841548] end_request: I/O error, dev sdc, sector 541872
May 12 13:51:45 aio kernel: [  801.841555] btrfs: bdev /dev/sdc1 errs: wr 1, rd 0, flush 0, corrupt 0, gen 0
May 12 13:51:45 aio kernel: [  801.841581] sd 5:0:0:0: [sdc] Unhandled error code
May 12 13:51:45 aio kernel: [  801.841586] sd 5:0:0:0: [sdc] 
May 12 13:51:45 aio kernel: [  801.841589] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 12 13:51:45 aio kernel: [  801.841593] sd 5:0:0:0: [sdc] CDB:
May 12 13:51:45 aio kernel: [  801.841596] Write(16): 8a 00 00 00 00 00 00 08 2c a0 00 00 00 08 00 00
May 12 13:51:45 aio kernel: [  801.841617] end_request: I/O error, dev sdc, sector 535712
May 12 13:51:45 aio kernel: [  801.841623] btrfs: bdev /dev/sdc1 errs: wr 2, rd 0, flush 0, corrupt 0, gen 0
May 12 13:51:45 aio kernel: [  801.841637] sd 5:0:0:0: [sdc] Unhandled error code
May 12 13:51:45 aio kernel: [  801.841642] sd 5:0:0:0: [sdc] 
May 12 13:51:45 aio kernel: [  801.841646] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 12 13:51:45 aio kernel: [  801.841651] sd 5:0:0:0: [sdc] CDB:
May 12 13:51:45 aio kernel: [  801.841653] Write(16): 8a 00 00 00 00 00 00 08 15 d8 00 00 00 10 00 00
May 12 13:51:45 aio kernel: [  801.841676] end_request: I/O error, dev sdc, sector 529880
May 12 13:51:45 aio kernel: [  801.841681] btrfs: bdev /dev/sdc1 errs: wr 3, rd 0, flush 0, corrupt 0, gen 0
May 12 13:51:45 aio kernel: [  801.841695] sd 5:0:0:0: [sdc] Unhandled error code
May 12 13:51:45 aio kernel: [  801.841700] sd 5:0:0:0: [sdc] 
May 12 13:51:45 aio kernel: [  801.841704] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
May 12 13:51:45 aio kernel: [  801.841708] sd 5:0:0:0: [sdc] CDB:
May 12 13:51:45 aio kernel: [  801.841710] Write(16): 8a 00 00 00 00 00 00 08 7e b8 00 00 00 08 00 00
May 12 13:51:45 aio kernel: [  801.841732] end_request: I/O error, dev sdc, sector 556728
May 12 13:51:45 aio kernel: [  801.841737] btrfs: bdev /dev/sdc1 errs: wr 4, rd 0, flush 0, corrupt 0, gen 0
May 12 13:51:45 aio kernel: [  801.841754] sd 5:0:0:0: [sdc] Unhandled error code
May 12 13:51:45 aio kernel: [  801.841759] sd 5:0:0:0: [sdc] 
May 12 13:51:45 aio kernel: [  801.841762] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK

« Last Edit: May 11, 2013, 11:55:42 pm by MasterCATZ »

Offline MasterCATZ

  • Jr. Member
  • **
  • Posts: 66
  • Karma: +0/-0
    • View Profile
Re: Flexraid keeps going readonly mode
« Reply #7 on: May 12, 2013, 02:55:16 am »
well switched to ext4 its extremely slow
( copies hangs copies hangs again maybe 30mbs average when it was doing 100mbs average before )

but unsure if thats because I have not enabled Direct I/O: this time or not , just waiting to see if its going to fail this time

*edit* ok HANG is the good word its been 1 hr and its still on the same file I last saw it at
if I start a new copy it does a few then also stops


*edit* ok now it seems I have a really bad batch 3TB WD of disks

out of 16x their was 1x DOA no spin up
1x that causes the sata bus to reset randomly ( smart status was set as failed as NEW )
1x that randomly dropped out that no longer gets picked up even tho it spins up fine
1x thats slowly been failing ( the first drive that flexraid writes to ) data is now coming back as corrupted partition tables and causing slow boot up time just about to do extended smart test to see if I can get it to fail for RMA ..
« Last Edit: May 12, 2013, 04:17:29 am by MasterCATZ »