Author Topic: (RT RAID) How to shutdown safely?  (Read 6930 times)

Offline mikk

  • Newbie
  • *
  • Posts: 12
  • Karma: +1/-0
    • View Profile
(RT RAID) How to shutdown safely?
« on: January 13, 2013, 12:27:34 pm »
Hello,

as i understand it the Flexraid pool/service should be stopped prior to shutting down the computer.
But depending on what one does this is not really possible very soon i think?
I imagine the worst case example: i delete some 100GB  (or TB) of data on the pool and then wanna shutdown the computer.
As FR has to recompute the Parity for the data in the background this can take many hours before the computer actually is ready to shutdown .. did i get this right?

If yes .. how can i initiate/schedule a shutdown of the PC for right after FlexRaid finishes its work?

thanks in advance!
« Last Edit: February 04, 2013, 02:49:29 pm by Brahim »

Offline lrissman

  • Full Member
  • ***
  • Posts: 119
  • Karma: +6/-1
    • View Profile
Re: How to shutdown safely?
« Reply #1 on: January 13, 2013, 01:37:18 pm »
A) Flexraid runs as a service.   Windows stops services before shutting down, so flexraid will have been stopped
B) Flexraid will not update parity unless told to do so (not even on a shutdown)
C) with Expression scripts you can write your own pre and post scripts to do stuff around updates.   shutdown could be achieved with a simple .cmd executed after the update with "Shutdown /p"   or reboot "Shutdown /R /t 0"

Offline mikk

  • Newbie
  • *
  • Posts: 12
  • Karma: +1/-0
    • View Profile
Re: How to shutdown safely?
« Reply #2 on: January 13, 2013, 01:46:18 pm »
for A) .. exactly thats the problem .. thats why flexraid should be done with its updates before the shutdown, else it will be killed by windows and for sure mess up the data sooner or later (pretty much sooner i guess)

for B) .. im talking about the Realtime version .. thats the main difference opposed to the snapshot one i think: automatic updates on every write/delete you do on the pool

for C) again that applies to the snapshot type i guess. in realtime raid you dont initiate the update manually .. so you can not add a pre or post script either ... or am i wrong here?

Offline lrissman

  • Full Member
  • ***
  • Posts: 119
  • Karma: +6/-1
    • View Profile
Re: How to shutdown safely?
« Reply #3 on: January 13, 2013, 03:31:50 pm »
My apologies,  I was using the "unread posts since last visit" and missed that you posted in Real-time raid.

honestly I am not sure if flexraid will finish an update; however, realtime raid will update parity in realtime, so if you have finished writing 100G of data, parity should be done very very shortly after as it is writing in realtime.   

Thus you will probably find it is quite difficult to shutdown windows in the middle of a write of parity. 

I'll look to those with more experience in RT-raid to confirm though.

Offline mikk

  • Newbie
  • *
  • Posts: 12
  • Karma: +1/-0
    • View Profile
Re: How to shutdown safely?
« Reply #4 on: January 13, 2013, 04:12:20 pm »
yeah i agree on the writing side .. but when you delete a big amount of data the parity has to be updated aswell...
so if you delete 100gb on the storage pool this will also mean that 100gb on the parity disks will have to be updated/written (probably even worse .. first read then write as the old state is probably part of the calculation for the new state) .. well just guessing either about the datails as far as how i understand parity

but the problem is right there .. lets say i delete those 100GB .. with a parity update speed of about 30mb/s (thats what i had in my last benchmark) this would mean that Flexraid needs about 1 hour to get the PPU updated ... whereas the user is free to shutdown anytime during that period and with this probably kill the update process and leave the raid in an corrupted state

either this .. or i didnt understand about the details really either ;)

Offline bigbob

  • Newbie
  • *
  • Posts: 36
  • Karma: +0/-0
    • View Profile
Re: How to shutdown safely?
« Reply #5 on: February 04, 2013, 07:00:50 am »
Bump. 

Brahim?

I would love to know the answer to this as well.  After installing FR Realtime and pooling drives full of pics, dwlds, etc, the first thing I want to do is scan the pool for dupes and delete (hopefully) gigabytes of duplicate files. 

What is the Realtime Raid's ability to update parity in a gigabyte per hour sense after a major change like that?  Is RealTime's rate on parity update the same as the original create parity function?

In other words, if I delete or add 100GB of data, and 5 min later one of the drives dies, or there is a power outage and then a drive dies at reboot, etc, is there a chance some of the old or new data can't be recovered?
« Last Edit: February 04, 2013, 07:05:54 am by bigbob »

Offline b-earl

  • Hero Member
  • *****
  • Posts: 652
  • Karma: +13/-1
    • View Profile
Re: How to shutdown safely?
« Reply #6 on: February 04, 2013, 02:35:37 pm »
Hi all
Realtime does the update in realtime so if you wait some time and the manually stop the pool before shuting down the computer it should be safe from what i read in the wiki.

Server HW: Chenbro RM41416 case | Supermicro X10SLM-F + LSI SAS 9305-16i | Xeon E3-1231 v3 | 16 GB DDR3 ECC Ram
Server OS:   Windows Server 2016 (UEFI) on 250 GB Samsung 850 Evo ssd
Transparent RAID 1.1.0 2017.02.11
Backupserver: Supermicro X9SCM-F UEFI + LSI SAS9211-8i IT FW
Server OS: Win 2016

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: (RT RAID) How to shutdown safely?
« Reply #7 on: February 04, 2013, 02:59:21 pm »
The issue with having to stop the FlexRAID array from the WebUI when using RT RAID is a Windows only issue.

Windows will kill any service that does not stop within a given delay period, which is as little as 5 seconds on some system.
This short time frame does not allow RT RAID to properly complete some pending operations in time before it is crashed by Windows.

It is really pending delete operations that are affected too.
So, if you wait a bit after deleting files (long enough for those deletes to be processed), you don't have to stop FlexRAID from the WebUI and you can just reboot or stop the service as you please.
Note that FlexRAID can suspend a delete operation and resume it, but again, Windows crashing it does not give it a chance to even suspend the operation.

So, the blanket statement about stopping the pool first from the WebUI is essentially a general guideline for RT RAID users on Windows.

Snapshot RAID users are not affected. Same as users only using the pool.
This is only an RT RAID on Windows issue.

Killing FlexRAID on Linux with a CTRL+C for instance does the right thing as it at least gives FlexRAID a chance to properly exit.


Offline bigbob

  • Newbie
  • *
  • Posts: 36
  • Karma: +0/-0
    • View Profile
Re: (RT RAID) How to shutdown safely?
« Reply #8 on: February 04, 2013, 03:48:50 pm »
Can you quantify a little more what "So, if you wait a bit after deleting files (long enough for those deletes to be processed)," means? 

Here's a hard example.  I scanned my pool for very large files and found about 140GB of dupes.  If I delete 140GB of dupes, how would that translate into "wait a bit" time wise? 5 min, 10 min, etc?


Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: (RT RAID) How to shutdown safely?
« Reply #9 on: February 04, 2013, 05:25:29 pm »
It would translate into the time it takes to write 140GB x 2 (most likely less but x2 to be safe).

Offline vletroye

  • Hero Member
  • *****
  • Posts: 714
  • Karma: +7/-0
    • View Profile
Re: (RT RAID) How to shutdown safely?
« Reply #10 on: February 05, 2013, 01:20:15 pm »
I didn't find in the developer resources how to query FlexRaid (using FlexRaidCmd.exe on Windows) to know if there is pending/running updates.

Is it possible ? I would add a test on that in my "Shutdown Task" to wait until completion before proceeding with the shutdown...


Do I have to use the <list> command to get the reference codes of each task then use the <status> command on each of those reference codes to see if one is still running ???


Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: (RT RAID) How to shutdown safely?
« Reply #11 on: February 05, 2013, 05:18:46 pm »
The "list" and "status" commands are your only options.

Offline vletroye

  • Hero Member
  • *****
  • Posts: 714
  • Karma: +7/-0
    • View Profile
Re: (RT RAID) How to shutdown safely?
« Reply #12 on: February 05, 2013, 05:20:32 pm »
Works fine  8)

Could you just confirm that process ID are always in a sequence from 1 to n ?
I could then simply loop until I get the messageCode "errorNoSuchProcessId" from "FlexRaidCmd localhost -- status i"

Otherwise, I will really have to convert these JSON responses into PowerShell objects... (Should do that anyway... the scripts are so ugly now  :-[)


I will update my script asap to abort the shutdown if any process is in a "status" != "STATUS_COMPLETED"
« Last Edit: February 06, 2013, 06:03:29 pm by vletroye »

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: (RT RAID) How to shutdown safely?
« Reply #13 on: February 07, 2013, 02:58:47 pm »
The IDs are in a simple sequence from 1 to n indeed.

However, you might have to loop through.
Doing a simple "status" command without specifying a status ID will get you the status of the last task that has been run.
Now, the last task might not be the only task and they might be other prior tasks still running, but you have the option of caring only about the last task.

Offline vletroye

  • Hero Member
  • *****
  • Posts: 714
  • Karma: +7/-0
    • View Profile
Re: (RT RAID) How to shutdown safely?
« Reply #14 on: February 08, 2013, 03:46:09 pm »
Here is my powershell script - for information purpose ;)

Code: [Select]
$flexCmd = "C:\Program Files (x86)\FlexRAID 2.0 Client\FlexRAIDCMD.exe"

function ExecuteFRCmd([string]$cmd, [string]$hostname="localhost", [string]$port="-", [string]$timeout="-")
{
$error.clear()

$pinfo = New-Object System.Diagnostics.ProcessStartInfo
$pinfo.FileName = $flexCmd
$pinfo.RedirectStandardError = $true
$pinfo.RedirectStandardOutput = $true
$pinfo.UseShellExecute = $false
$pinfo.Arguments = "$hostname $port $timeout $cmd"
$p = New-Object System.Diagnostics.Process
$p.StartInfo = $pinfo
$p.Start() | Out-Null
$p.WaitForExit()
$output = $p.StandardOutput.ReadToEnd()

if ( $error.count -eq 0)
{
# Clean the JSON message (remove the trailing text (=> blabla))
$output = $output -replace '(?<First>.*)=>.+', '${First}'
}
else
{
$output = $null
}

return $output
}

function IsAnyTaskRunning()
{
$running = $FALSE

$state = ExecuteFRCmd("status")

if ($state -eq $null)
{
throw "Command failed to execute"
}
else
{
#Write-Host "States: $state"

$process = $state | ConvertFrom-Json

$message = $process.commandMessages.messageCode
if ($message -eq "successNoProcessSinceServerStartup")
{
#Write-Host "No Process started since Server startup"
}
else
{
$processID = $process.status.referenceCode

do {
$state = ExecuteFRCmd("status "+$processID)
$process = $state | ConvertFrom-Json

switch ($process.status.status)
{
{ @("STATUS_STARTED", "STATUS_PROCESSING", "STATUS_PAUSING", "STATUS_RESUMED", "STATUS_RESUMING") -contains $_ }
{
#Write-Host task $processID - $process.status.task - is running
$running = $TRUE
}
{ @("STATUS_COMPLETED", "STATUS_ABORTED", "STATUS_ABORTING", "STATUS_PAUSED") -contains $_ }
{
#Write-Host task $processID - $process.status.task - is not running
}
}

$processID -=1
} while (($processID -gt 0) -and ($running -eq $FALSE))
}
}

return $running
}

I loop from the latest process to the first one and stop as soon as one is not "stopped" (taking the various possible states into account)

I will now call it from my ShutDown Task to abort the shutdown.::

Code: [Select]
$running = IsAnyTaskRunning
if ($running -eq $TRUE)
{
    cmd /c shutdown -a
}

Or would you recommend to "pause" the current tasks and proceed with the shutdown ?
« Last Edit: February 08, 2013, 03:49:40 pm by vletroye »