Author Topic: Fail to copy a large file  (Read 2254 times)

Offline Gasman3055

  • Newbie
  • *
  • Posts: 37
  • Karma: +0/-0
    • View Profile
Fail to copy a large file
« on: July 06, 2013, 12:18:55 pm »
I want to know if the following would be expected to fail. If the answer is no, then I will upload logs, details, etc. This concerns copying files larger than the Reserve but that should fit well on the drives without splitting the files across disks if FlexRaid and Windows Explorer can handle it.

Setup: Realtime Raid Pool with 3 each 2 TB drives all freshly formatted. Reserve set to 200 GB.

First: Copy using Windows Explorer (Win 7): 1.5 TB of data, one of which is a 994 GB file. All goes well. Not sure of the exact time to copy but less than a day for both files combined.

Then: Copy a single 880 GB file using Windows Explorer. I specifically chose this as smaller than the 994 GB but large enough that if FlexRaid did not anticipate that it would not fit on the first disk, it might fail. Two weeks later, about ¼ the way through and progress bar has not noticeably moved since the first day.

Abort the copy in Windows Explorer. Attempt to stop pool. Much later that day, still “Processing . . .”


I suspect that FlexRaid does can lock up if it starts to copy a large file that is larger than the Reserve and for which there is insufficient room on the disk it starts writing to (even if the other disk is empty). I want somebody to confirm or deny this conjecture. If this failure is not expected, then I will upload the logs.






Offline NLS

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1,018
  • Karma: +29/-4
  • Look ma, no hands!
    • View Profile
    • iLogic
Re: Fail to copy a large file
« Reply #1 on: July 06, 2013, 03:14:42 pm »
The reserve should be larger than the larger file you plan to copy.

Your needs seem pretty unique. Curious, what kind of files are 994GB or 880GB in single file?
Hard disk images? Truecrypt files?


A trick would be to make the reserve huge. Make ONLY the large file copies. Then set it to smaller ammount.
Other solution would be to delete the array, access the native drives, put the files where you want them (I suspect you will not moving 994GB single files too much...) and then build array.



« Last Edit: July 06, 2013, 03:17:47 pm by NLS »
---
NLS
Production system: SBS2011 fully patched, intel Core2 Quad, 8GB, 12 disks (1 system IDE, 1 backup IDE, 10 for array and parity most SATA3), parity is 3TB, largest data disk is 3TB, millions of smaller files, common browser Chrome latest.

Offline DrBlaze

  • Sr. Member
  • ****
  • Posts: 281
  • Karma: +14/-0
    • View Profile
Re: Fail to copy a large file
« Reply #2 on: July 06, 2013, 10:25:20 pm »
You could even mount the target drives, copy the insanely large files to their respective drives, then unmount them and run reconcile.

Obviously not "by the book", but sometimes you have to go against the rules with RT.

Offline Gasman3055

  • Newbie
  • *
  • Posts: 37
  • Karma: +0/-0
    • View Profile
Re: Fail to copy a large file
« Reply #3 on: July 07, 2013, 05:40:01 pm »
These are files created by the backup program Retrospect.

I want to copy the backups to a FlexRaid drive on an offsite computer as a secondary backup. Retrospect does have the ability to create segmented files but if you don't restore the copy of the backup to the same path, you have to tell it where each of several hundred, to a few thousand, files are one at a time. Doable, but a pain it its own way. I have never actually lost data but I have lost a backup before – fortunately the original data was still good at the time – so I know I too am vulnerable. I go back to the days of the 8" floppy/ PDP-11 and disk drives the size of a washing machine. I have seen a lot of pain from failure of backup plans.

I will be switching to the segmented way so that flexraid does not puke on them.

Thanks for the help. I now know how to manage this problem.

Consider this issue closed.

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: Fail to copy a large file
« Reply #4 on: July 07, 2013, 07:11:41 pm »
If you are copying usine Window Explorer, I am surprise that you would have any issue.
Explorer normally "announces" the size of the file it is about to copy such that FlexRAID is able to route the file to a drive that can contain it or through out of space error.

Also, for it to be running for two weeks? To say something is off is an understatement.

Online vestige

  • Hero Member
  • *****
  • Posts: 3,573
  • Karma: +0/-0
    • View Profile
Re: Fail to copy a large file
« Reply #5 on: May 17, 2019, 08:30:08 am »