Author Topic: "Total process size" what does it mean?  (Read 2563 times)

Offline Fastrack

  • Jr. Member
  • **
  • Posts: 63
  • Karma: +2/-0
    • View Profile
"Total process size" what does it mean?
« on: October 10, 2011, 10:24:01 am »
Hi,

I have 5 DRU's and 2 PPU's.  2TB each.  I recorded 16GB of shows last night on my PVR.  Flexraid in the logs said the total process size was:
[2011-10-10 00:17:05,546] INFO : Total process size = 78053099520

There are a few backups as well (about 2GB).  So total changed files was around 17-18GB.

Is the process size (new,deleted,changed) * # of DRU's??  I'm just trying to determine how the process size can be so high.

I did a search on all drives for Date Modified between 10/9 and 10/10 and I did not see anything else then what I have above.

Someone mentioned Hard Links on these drives as a cause of another issue I was having (I'm 99.99% sure there are not any).   But how can I check??
Ben
FlexRaid 2.1 Final (2015-11-01) | RAID6 T2 engine | Windows 7 64bit | C2D 6600 @ 2.40GHz | 4GB DDR3 | SageTV v9 | 5 x Hitachi 2TB DRU | 1 x Seagate 2TB DRU | 1 x Seagate 1TB DRU | 1 x Hitachi 2TB PPU

Offline jasonwc

  • Full Member
  • ***
  • Posts: 197
  • Karma: +5/-0
    • View Profile
Re: "Total process size" what does it mean?
« Reply #1 on: October 11, 2011, 09:20:28 am »
It's the amount of memory being used by the FlexRAID process (which you can confirm in Task Manager). It has nothing to do with the size of your RAID array or the changes you have made recently.
Norco RPC-4224 4U Rackmount Case (24 Hotswap Bays) | Asus P8B Workstation Motherboard (four PCI-E 16x slots @ 8x/8x/4x/4x and dual 82574L Intel server NICs) | Core i3-2100 | Supermicro AOC-SASLP-MV8 SAS Controller | Corsair TX650v2 PSU | Currently 16 TB array, 2 TB parity | WHS 2011

Offline Fastrack

  • Jr. Member
  • **
  • Posts: 63
  • Karma: +2/-0
    • View Profile
Re: "Total process size" what does it mean?
« Reply #2 on: October 11, 2011, 02:56:02 pm »
It's the amount of memory being used by the FlexRAID process (which you can confirm in Task Manager). It has nothing to do with the size of your RAID array or the changes you have made recently.

Since it's 75GB it doesn't make sense.  Just thought it was odd.  I know when I added my D: drive which is 27GB it said process size was 27GB (in bytes of course)

I guess I'll have to enable debug to see what it's really doing.
Ben
FlexRaid 2.1 Final (2015-11-01) | RAID6 T2 engine | Windows 7 64bit | C2D 6600 @ 2.40GHz | 4GB DDR3 | SageTV v9 | 5 x Hitachi 2TB DRU | 1 x Seagate 2TB DRU | 1 x Seagate 1TB DRU | 1 x Hitachi 2TB PPU

Offline jasonwc

  • Full Member
  • ***
  • Posts: 197
  • Karma: +5/-0
    • View Profile
Re: "Total process size" what does it mean?
« Reply #3 on: October 11, 2011, 03:11:52 pm »
Ah, apparently you are correct. My update from last night shows a size of 62 GB but I only added 42 GB of new movies. The process size seems consistently larger than the amount of data added/deleted, but it does seem to be correlated with the amount of changes since the last update. Sorry for the misinformation. I thought I read that the process size referred to the FlexRAID process itself.
Norco RPC-4224 4U Rackmount Case (24 Hotswap Bays) | Asus P8B Workstation Motherboard (four PCI-E 16x slots @ 8x/8x/4x/4x and dual 82574L Intel server NICs) | Core i3-2100 | Supermicro AOC-SASLP-MV8 SAS Controller | Corsair TX650v2 PSU | Currently 16 TB array, 2 TB parity | WHS 2011

Online Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,541
  • Karma: +204/-16
    • View Profile
Re: "Total process size" what does it mean?
« Reply #4 on: October 11, 2011, 10:00:47 pm »
It's the amount of memory being used by the FlexRAID process (which you can confirm in Task Manager). It has nothing to do with the size of your RAID array or the changes you have made recently.

No, it is the amount of data that will be processed.

Offline someone

  • Jr. Member
  • **
  • Posts: 81
  • Karma: +0/-0
    • View Profile
Re: "Total process size" what does it mean?
« Reply #5 on: October 11, 2011, 10:49:28 pm »
No, it is the amount of data that will be processed.
Meaning amount of data add/delete/modified?

Offline Fastrack

  • Jr. Member
  • **
  • Posts: 63
  • Karma: +2/-0
    • View Profile
Re: "Total process size" what does it mean?
« Reply #6 on: October 12, 2011, 08:20:34 am »
Meaning amount of data add/delete/modified?

So I ran an update last night with DEBUG enabled after recording 2x OTA HD shows totaling 16.22GB.  The total process size was 50.24GB.  I can see in the DEBUG log FlexRaid is processing other TV Shows - but those far exceed the 50.24GB.

Based on seeing this output the 50.24GB quote is the amount of data, bits and pieces FlexRaid will have to grab/use from other files based on the 16.22GB being added to the array.
Ben
FlexRaid 2.1 Final (2015-11-01) | RAID6 T2 engine | Windows 7 64bit | C2D 6600 @ 2.40GHz | 4GB DDR3 | SageTV v9 | 5 x Hitachi 2TB DRU | 1 x Seagate 2TB DRU | 1 x Seagate 1TB DRU | 1 x Hitachi 2TB PPU

Online Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,541
  • Karma: +204/-16
    • View Profile
Re: "Total process size" what does it mean?
« Reply #7 on: October 12, 2011, 08:44:37 am »
So I ran an update last night with DEBUG enabled after recording 2x OTA HD shows totaling 16.22GB.  The total process size was 50.24GB.  I can see in the DEBUG log FlexRaid is processing other TV Shows - but those far exceed the 50.24GB.

Based on seeing this output the 50.24GB quote is the amount of data, bits and pieces FlexRaid will have to grab/use from other files based on the 16.22GB being added to the array.

1. Make sure no other program is changing the existing shows such as their last modified date.
2. The next release will do better with processing differential changes. I discussed of this in another thread (forgot which one though).
3. As stated above, some of the files are processed purely for metadata information only.

For now, try to increase the delay prior to which new files need to be picked up (see advanced properties).