Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.


Messages - lrissman

Pages: [1] 2 3 ... 8
1
Snapshot RAID / Re: Very old spanned DRU failed, replace with bigger??
« on: January 15, 2013, 08:20:00 pm »
About the spanned DRU,  If you do as you suggest, and fill the 3TB you will get errors about not enough space for parity.

Personally, I would add the 3TB as a new DRU, and reduce the spanned dru to a single 1.5TB disk.   Unfortuantly I do not use CruiseControl, so maybe somebody else on the forum can help you with the step-by-step to swap a spanned dru for a non-spanned one.

Worst case, do a validate on your array (ensure you dont have any bad sectors), delete the config and recreate as above, and redo parity.

Btw.  Disk bought at around the same time from the same mfgr, etc typically will have a high likelyhood of failing together.   I recently suffered 8x1TB disks out of 13 that all failed inside 6 months.

2
Storage Pooling / Re: Flexraid volume not using available space
« on: January 15, 2013, 07:07:33 am »
check out the Merge Modes section in the wiki: http://wiki.flexraid.com/about/flexraid-storage-pooling/.

I dont use cruise control, so I couldnt tell you exactly where it is on yours setup but it should be somewhere in an advanced settings tab or such.

3
Thanks Brahim,

Due to a lack of response on this one, I decided to reinstall the whole system so that I can get my array back online -- not a great solution, but the only one that i had available -- this is in progress.   I would like to know there is a fix so I am not hit again mysteriously as such I would appreciate if you let me know if you found the issue.

Remember that it was working perfectly for some time before suddenly core dumping, so it might be hard to replicate.      I can confirm that I had not made any software changes on the server at the time of the crash -- I was using the pool via SMB.

4
Buying Recommendations / Re: New Home Server build
« on: January 14, 2013, 08:51:03 am »
I'll third Tonym's statement.    Too many things can go wrong with external disks.. (such as a bumped cable or a dropped disk).   

Buy a case that fits your needs

For CPUs, doesnt matter so much.   Any modern CPU can handle the requirements of basic raid/fileserving.   If you plan to do more with the box than fileserving, then buy the appropriate CPU (I run a i7 2nd generation because I do lots of transcoding).

for a motherboard, I like newegg's interface: http://www.newegg.ca/Product/ProductList.aspx?Submit=ENE&N=100007626%20600054095&IsNodeId=1&bop=And&ShowDeactivatedMark=False&Order=PRICE&PageSize=20#     

On the left side choose: " more options"  and then specify what you want.  (eg, 10 SATA 3 ports, or 4 pcie 16x slots).   Then see the prices and listing and go from there.    Asus, asrock, gigabyte, supermicro, msi.    I have had good luck with all. (and some bad). 

Personally, I am pretty cheap so I bought a cheap board with a balanced number of sata slots for about 70 bucks, an i7 for it (expensive) and then a few 4 port pcie SiI cards from ebay and things tick along well enough.      I also have a norco hot-swap case with a fully SATA backplane (so I dont have to deal with SAS and breakout cables).   

5
RAID-F on Windows / Re: menu acting up in preferences and settings
« on: January 14, 2013, 05:23:27 am »
if you are remote you can use the local browser to hit flexraid on the system.   I used to do it frequently on my linux setup and I managed flexraid from my remote windows box.  http://<serverip>:8080

6
RAID-F on Windows / Re: HELP!! Where has my data gone!
« on: January 14, 2013, 05:21:51 am »
in the system log you will see errors from "disk"  that would point to disk errors (if you have them).

another thought,  Did you happen to move the disk to a different controller?

Llarge disks (I think starting at 2.1TB but for sure 3TB and beyond) require a bit of a different system to manage.  I am sure that when you first got your 4TB disks you discovered that you could not use a standard partition table and had to go with a GPT.   

Well, where I am going with the above is many controllers flat-out dont support 64bit lba (48 bit is what the norm is prior to these large disks).   Some controllers support it and that is all good.    where the fun comes in is that some controllers start to do strange translation, such as using 4k sector sizes that way there are enough addresses in 48bit lba to address the 4TB.   

If you take one of these large disks and move it between controllers you could easily end up with a readable partition table showing you the partition, but unable to read the data thus the partition is RAW. 

As to recovery tools, it is not as complete as others, but it is free and does a good job when it can do the job:  Pandora Recovery

7
Real-Time RAID / Re: How to go from Snapshot to RT Raid config?
« on: January 13, 2013, 04:21:53 pm »
I dont know that it is possible.     You may find the only option is to delete your config, and create a new config in RT-Raid mode.    You can also try to preserve the parity on the ppu  -- if it is used the same it might actually work.   

Worst case I see is that you will have to recalculate all your parity. 

I would recommend that you do a full verify to ensure you dont have any bad sectors prior to destroying your snapshot raid.

8
Real-Time RAID / Re: How to shutdown safely?
« on: January 13, 2013, 03:31:50 pm »
My apologies,  I was using the "unread posts since last visit" and missed that you posted in Real-time raid.

honestly I am not sure if flexraid will finish an update; however, realtime raid will update parity in realtime, so if you have finished writing 100G of data, parity should be done very very shortly after as it is writing in realtime.   

Thus you will probably find it is quite difficult to shutdown windows in the middle of a write of parity. 

I'll look to those with more experience in RT-raid to confirm though.

9
RAID-F on Windows / Re: HELP!! Where has my data gone!
« on: January 13, 2013, 03:29:30 pm »
a) Google is your friend, if you have a technical question, ask google.   A quick search on "windows 8 event viewer" produced this nice how-to:  http://windows.microsoft.com/en-CA/windows-vista/Open-Event-Viewer
2) Flexraid will not update parity if you lost a DRU.  It will error out.

Yes, you can try that plan.   You might find most of the data on DRU3 is recoverable using tools.   If you are certain that you have a disk issue, then i would skip checking the event log and move right to the vendor's disk test tools to certify you have a disk problem and not something simple like a cable problem.

If you cant recover, then your best option might be to use flexraid to recover DRU3 to an alternate disk and see what you get.   It wont wipe your existing DRU3 if you specify an alternate path.

10
Real-Time RAID / Re: How to shutdown safely?
« on: January 13, 2013, 01:37:18 pm »
A) Flexraid runs as a service.   Windows stops services before shutting down, so flexraid will have been stopped
B) Flexraid will not update parity unless told to do so (not even on a shutdown)
C) with Expression scripts you can write your own pre and post scripts to do stuff around updates.   shutdown could be achieved with a simple .cmd executed after the update with "Shutdown /p"   or reboot "Shutdown /R /t 0"

11
RAID-F on Windows / Re: HELP!! Where has my data gone!
« on: January 13, 2013, 01:35:24 pm »
check your windows event viewer.  Sounds like you are getting read errors.   

12
Storage Pooling / Re: Flexraid volume not using available space
« on: January 13, 2013, 01:34:21 pm »
Here is what I would do

1) Verify that the permissions on the new disks compared to the working ones
2) try restarting the flexraid service (i sometimes found that a restart was needed)
3) once I needed a reboot of the machine
4) look at your balancing priority.  I use by space to balance the disks evenly.

13
You could be right -- robocopy and rsync are just my go-to tools for data migration so I had used it a great deal with FlexRaid before coming across those posts..   Worked for me, so I simply didnt worry.   :)

My cataloging tools work of the last modified date.   I also need to preserve permissions, as such I dont use normal copy.   However, there are a boatload of various tools that can do those two things other than robocopy.   

14
I have used robocopy quite a bit with lots of success and no issues; however, ymmv.

I have also used Microsoft's Synctoy successfully for scheduled replication.   

15
RAID-F on Linux / My Ubuntu 12.10 setup
« on: January 13, 2013, 07:07:10 am »
Hello all,  I took some notes when I setup my ubuntu 12.10 flexraid server and thought I would share them.   I should warn that I never got to finish it as my flexraid install seems to have become corrupted and is yet unresolved (See other posts by me to see where that is)

Ubuntu Install
==========
- Install Ubuntu 12.10 server as per normal
  - File and Print Server and Virtual Host if you want to use KVM (I use VirtualBox)
- After install, from the command line run:  sudo apt-get install gnome-desktop
reboot
- setup remote desktop sharing (Search google for some guides)
   - This allows you to use any VNC client remotely to access the gui/desktop
- Setup autologin for ubuntu 12.10 (http://askubuntu.com/questions/210823/how-do-iset-automatic-login-in-ubuntu-12-10)
- sudo apt-get install smartmontools python-glade2 system-config-samba sysstat
- mkdir /disk-mounts
- mkdir /disk-mounts/dru1
- mkdir /disk-mounts/dru2
- mkdir /disk-mounts/dru3
- mkdir /disk-mounts/dru
  • -- as many as you like

- mkdir /disk-mounts/ppu1
- mkdir /disk-mounts/ppu
  • -- as many as you like


Now, here is where you need to do your own thing.   For each disk you have, either format or use existing (note, do not use NTFS in linux as it is very slow -- it works, but is slow).
- If you are using existing, tune2fs -L dru1 /dev/sd[x1]        (this will apply a label of dru1 to the device specified (note you need to repace [x1] with your disk letter and partition number.
- If you are formatting new,   mkfs.ext4 -L dru1 /dev/sd[x1]     (same as above but formats disk so it is blank)

- Setup /etc/fstab is next
- sudo vi /etc/fstab  (or your favourite editor)
- Add the following lines and modify as needed
#FlexRaid disks
LABEL=ppu1 /disk-mounts/ppu1 ext4 defaults,nofail 0 2
LABEL=dru1 /disk-mounts/dru1 ext4 defaults,nofail 0 2
LABEL=dru2 /disk-mounts/dru2 ext4 defaults,nofail 0 2
LABEL=dru3 /disk-mounts/dru3 ext4 defaults,nofail 0 2

next mount the disks (or just reboot)
sudo mount /disk-mounts/ppu1
sudo mount /disk-mounts/dru1
sudo mount /disk-mounts/dru2
sudo mount /disk-mounts/dru3

Flexraid Setup
==========
I used these guides to install flexraid:
- install flexraid as per http://www.havetheknowhow.com/Configure-the-server/Install-FlexRAID-v2.x.html

In essence I created a folder /usr/lib/flexraid and gave it permissions as per the guide.   Then from my user (not root) I ran the normal linux install and pointed it to there.  To automatically start flexraid, I used the guide in the flexraid wiki -- essnetially creating a basic script in /usr/lib/flexraid that runs flexraid and using startup manager in ubuntu.  (From dash search for Startup and you will find it).

At this point you can configure flexraid remotely by using http://serverip:8080  -- I would recommend you add a userid/password for security.   Also note that the shares manager doesnt work in my linux expert mode..   ensure you setup your mount point as samba will require it.    So next we setup samba.

Note:  Flexraid does not always unmount the pool from the FileSystem when you stop the pool -- not sure why.   If it doesnt unmount and you get errors about files in the flexraid pool mount point, run sudo umount -l <flexraid mount point>.  I also add this to my stop script.

My pool start and stop scripts are below with virtualBox setup.    I also never made it to scheduling my verify and update cycles; however, you will want to create similar before-update, after-update, before-verify, and after-verify, to pause various processes and programs, then use expression language to put it all together.   
 Create Expression scripts and schedule the Before scripts for update/verify
    - before-update that runs your script and runs update
    - update that runs the update or verify and then runs after-update
     - after-update that runs your after-update script and exits

Samba Setup
=========
sudo vi /etc/samba/smb.conf
- For windows 8 access, you need to enable SMB2 support in samba:  http://wiki.samba.org/index.php/Samba_3.6_Features_added/changed
   - Add max protocol = SMB2  in the [global] section
   - Add your shares:

[storage]
        path = /data/storage
        public = no
        guest ok = no
        read only = yes
        create mask = 0664
        directory mask = 0775
        browseable = yes
        valid users = movies andreia lrissman
        write list = andreia lrissman
        read list = movies


- Note, the valid users are those that have access (you need to create the samba users:  http://www.cyberciti.biz/faq/adding-a-user-to-a-samba-smb-share/)
- Write list are the users that have write access to the share
- Read list are the users that have read access to the share.
- Share name is in the [ ] at the top

Installing VirtualBox
==============
http://www.itworld.com/software/304613/install-virtualbox-ubuntu-1210

#download the ubuntu 12.10 .deb of virtualbox
#apt-get install dkms
#dep -i <downloaded vbox .deb>

create your vm..
setup the init script in your /etc/init.d dir for each of your VMS but dont have it autostart
http://www.glump.net/howto/virtualbox_as_a_service

-- Read: http://wiki.flexraid.com/2012/03/04/executing-custom-scripts-before-andor-after-the-storage-pool-starts-or-stops/
bring vm online with the pool (and offline)

-- I put theses scripts into my flexraid install dir:
user@server:/flexraid$ cat pool-after-start.sh
#!/bin/bash
/etc/init.d/vbox-Win7VM start
/etc/init.d/crashplan start

user@server:/flexraid$ cat pool-before-stop.sh
#!/bin/bash
/etc/init.d/vbox-Win7VM stop
/etc/init.d/crashplan stop

user@server:/flexraid$ cat pool-after-stop.sh
umount -l /data/storage

Crashplan install (if you want)
============
Install Crashplan normally
remove /etc/rc2.d/S99crashplan
add it to your start and stop to the pool as above

Monitoring I/O Performance
===================
http://www.upubuntu.com/2012/06/list-of-best-system-monitoring.html
iostat, vmstat, mpstat

iostat -x -m 2
Look for %util

Troubleshooting Read errors
====================
watch /var/log/syslog and if you see ata read errors you have issues:

http://unix.stackexchange.com/questions/13960/how-to-map-atax-0-identifiers-in-kern-log-error-messages-to-actual-dev-sdy-devi


Enjoy.



Pages: [1] 2 3 ... 8