Author Topic: Bad performance? Post here!  (Read 40043 times)

Offline Skirge01

  • Full Member
  • ***
  • Posts: 140
  • Karma: +2/-0
    • View Profile
Re: Bad performance? Post here!
« Reply #15 on: October 06, 2013, 03:30:49 pm »
Here are the screenshots I believe you need for this.  The first set of shots include some additional benchmarks of other drives because 2 of the 3 are fragmented above 20% (top right) or 30% (top left), but I'm having issues with defrag seemingly stopping at around 7% of the "pass 1-consolidating" step.  As a result, I benchmarked a drive with 0% fragmentation (bottom left) to show that there's almost no difference in the benchmark results.  Bottom right is the PPU.

The next screen is obviously my tRAID parameters after following part 1 & 2 of the optimization guides.  I noticed zero change at each step.

The next set of screens is for the copy of a 26GB file from the C drive of the OS running tRAID (with 12GB of RAM) to the X drive, which is disk 4_4 from the array.  EDIT:  Note that I apparently swapped the top two screens when I pasted them into Word.

The final screen is for the copy of a 1GB file from the same C drive to the same X drive.  I could only get 1 screenshot before the copying was done.

P.S.  Limiting us to only 4 photos per post is a bit cumbersome for this thread.  ;-)

------------------------------
System Specs:
Supermicro X9SAE-V
Intel i7-3770T @ 2.5GHz
32GB DDR3 1600
LSI 9211-8i (IBM M1015 flashed to IT mode; all tRAID drives attached to this card)
Running ESXi 5.0.0-20120302001 via flash drive

tRAID running in a VM with the following specs:
VM version 8
Windows 7 Pro 64-bit
1 vCPU
12GB Ram
Plextor M5S 120GB for C: drive (attached to mobo)
Seagate ST3000DM001 3TB drives in the array

I normally have 4 VMs running, but turned 2 off during benchmarking.  The only VM I kept running was pfSense.
« Last Edit: October 06, 2013, 03:34:05 pm by Skirge01 »

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,525
  • Karma: +202/-16
    • View Profile
Re: Bad performance? Post here!
« Reply #16 on: October 06, 2013, 06:46:27 pm »
@Skirge01
Your performance results are fine. Small to medium file copies are super fast thanks to TCQ, and large file copies are above 50MB/s at their lowest.
This is inline with what I have seen from other setups.

If you need more performance, you will need to try the next build which will have the option for OS Caching.

Offline Skirge01

  • Full Member
  • ***
  • Posts: 140
  • Karma: +2/-0
    • View Profile
Re: Bad performance? Post here!
« Reply #17 on: October 06, 2013, 10:12:36 pm »
Thanks for the feedback, Brahim.  While it wasn't exactly what I was hoping to hear, it's definitely good to know that my system is not the bottleneck and I'm getting "expected" performance!  I hope the next build will get me the necessary performance then, since tRAID is not able to live stream TV shows while they're recording and that's a major issue for my setup.  I'm currently recording to a drive outside of tRAID and then copying them over afterwards, but this is obviously not how I can operate long term.  I don't mean this as a "dig" against tRAID, since it obviously has much more overhead to provide the peace of mind it does, but my 3 year old WD Green drive running at 5400rpm is able to keep up with the live streaming.  Quite honestly, that surprised me.

I'll keep my eyes peeled for the next release.  :)

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,525
  • Karma: +202/-16
    • View Profile
Re: Bad performance? Post here!
« Reply #18 on: October 06, 2013, 10:20:19 pm »
What do you mean you cannot stream while recording?

This has nothing to do with performance.
For one, we are only discussing of write speed in this thread as read speed should be at native speed. So, even the worst write performance should still not prevent a good read performance.

Open a different thread on it in the bug forum as that is not performance related.

Offline Gaevonn

  • Newbie
  • *
  • Posts: 29
  • Karma: +0/-0
    • View Profile
Re: Bad performance? Post here!
« Reply #19 on: October 06, 2013, 11:06:42 pm »
Brahim,
Sorry this took so long.  I had to make some changes after loading the TRAID RC8-10 caused the Blutooth Radio on my Windows 7 Pro 64-bit Server to stop working.  The Motherboard I am using on my servers has a micro card in it that supplies Intel Wireless and Bluetooth Functions.  After the RAID was built in the Web client and the Server rebooted, the Bluetooth would no longer start.  As a result, i switched to an alternate Server, using the same Motherboard, minus the micro card and running Windows 2008 R2 SP1.  Hopefully, I have captured what you are asking for.  Sadly, I think there may be a problem in the Tower-RAID enclosures I am using and there seems to be a big difference between the Sequential and Random Write times.  I also did not seem to see a difference between TCQ and OS Cache copy times. 

If needed I can change the Test Server config somewhat, but the Tower RAID Enclosure is the only option I have for holding the drives at the moment.  Let me know if more is needed.  Functionally it seems to work fine and supplies me with the features I am needing, though it does seem at times that drive access slows and then picks up again when streaming large files.

I could not figure out how to insert more than 4 Images, so I put them all in a PDF.  I hope this works. Viewing at 200% seems to work.

System Specs: Server is running the Host and Client RC10.
- Gigabyte Z87N-WiFi - minus the Micro card that supplies WiFi and Bluetooth.
-Intel Core i5 4570S @ 2.90Ghz (4-Cores)
- 8GB of DDR3 1600 Memory (Dual 4GB Sticks)
- LSI 9206-16E PCIe 3.0 HBA connected to a SANS Digital TowerRAID TR8X+ - 8 Bay SAS / SATA JBOD Storage Enclosure.
- 4x2TB Seagate Barracuda ST2000DM001 Drives configured as TRAID.
- 5x2TB Seagate SV-35 configured as standalone drives (4 in Enclosure and 1 internal OS)
- Windows 2008 R2 SP1 w/all current Microsoft patches.
-NO AV client installed on Test server.


I Work; therefore, I Live.
;-)

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,525
  • Karma: +202/-16
    • View Profile
Re: Bad performance? Post here!
« Reply #20 on: October 07, 2013, 12:24:58 am »
@Gaevonn
Your numbers with TCQ look consistent with other systems.
Small file copies are bottlenecked by the source copying disk and large copies start strong, bottom out, but stay above 50MB/s.

Your numbers with OS Caching on are also bottlenecked by the source copying disk.

Offline Gaevonn

  • Newbie
  • *
  • Posts: 29
  • Karma: +0/-0
    • View Profile
Re: Bad performance? Post here!
« Reply #21 on: October 07, 2013, 12:49:10 pm »
Brahim,

Thank you.  I was worried that I was going to have to buy new enclosures.  So then things look "normal"?  I am transfering files to stress the system all day.  Is there a check I should do to verify parity updates or error messages?
I Work; therefore, I Live.
;-)

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,525
  • Karma: +202/-16
    • View Profile
Re: Bad performance? Post here!
« Reply #22 on: October 07, 2013, 01:53:00 pm »
Brahim,

Thank you.  I was worried that I was going to have to buy new enclosures.  So then things look "normal"?  I am transfering files to stress the system all day.  Is there a check I should do to verify parity updates or error messages?
Transparent RAID gets you at best close to 65% (higher for SSDs) of the speed of a single disk on write and without caching.
Reads are parallel and so you can get a combined effect close to RAID 0 speed with parallel threads each going to a separate disk.

With OS Caching, the disk you are copying from is usually the bottleneck when copying files up to the size of your cache.
For files larger than your cache, the copy will start out strong until your cache is exhausted and will bottom at your raw speed.
Outside of the occasional large file copy, you will not notice the parity cost of the RAID when OS Caching is on.

RAID verification are done with the Verify/Verify+ tasks.

Offline Gaevonn

  • Newbie
  • *
  • Posts: 29
  • Karma: +0/-0
    • View Profile
Re: Bad performance? Post here!
« Reply #23 on: October 08, 2013, 05:59:11 pm »
Thanks for the info.  I have transferred approximately 1/3 of the 5TB of data on the TRAID drive to other locations and back with no issue.  Streaming from Plex with and without transcoding of the media files is working fine.  Sickbeard, Sabnzb and Plex are updating folders just fine.  I have not seen any hiccups yet.

Thank You again for a great product.

As a side question and one I probably already know your answer to.....  I am only running the storage servers at 8GB of RAM.  The MB currently supports up to 16GB (2 slots) so would it improve any performance to increase the RAM to Max of 16GB?  I would guess that more is always better but if there would be no noticible difference with TRAID performance, I would just wait.
I Work; therefore, I Live.
;-)

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,525
  • Karma: +202/-16
    • View Profile
Re: Bad performance? Post here!
« Reply #24 on: October 08, 2013, 06:59:30 pm »
If you are going to use OS Caching, then yes, more RAM is better.
There are going to be future optimizations that will take advantage of more RAM too. With RAM being cheap, I'd say max it out. :)

Offline johnzered

  • Jr. Member
  • **
  • Posts: 94
  • Karma: +0/-0
    • View Profile
Re: Bad performance? Post here!
« Reply #25 on: October 12, 2013, 04:07:54 am »
Did a few tests yesterday again, but still with very bad performance.
The test are from a Windows 7 machine to my 2012 Essentials R2 Preview VM were tRAID is installed.

OS Caching on
  11GB
    starts at: 75 MB/s ends at: 8,5 MB/s
  1GB
    around 76 MB/s

OS Caching off
  11GB
    starts at: 51 MB/s ends at: 47 MB/s
  1GB
    starts at: 75 MB/s ends at: 52 MB/s

Outside tRAID (source disk online)
  11GB
    starts at: 76 MB/s ends at: 74 MB/s
  1GB
    around 76 MB/s

I also tested copying a 18GB file locally on the server from a spinning disk outside of tRAID with OS Caching on (screenshot attached) and locally it looks like I do not get as bad performance as when copying over network. It started around 75 MB/s I think and I would say it averaged between 46-58 MB/s a few times it dropped to 18 MB/s but not lower it also went up to 65 MB/s a couple of times. Generally when I have OS Caching on the transfer speed fluctuates quite badly, actually when the screenshot is taken there's such a peek in performance.

@gasutora
How are your tests going?

@Skirge01
Are your benchmarks with the source disks offline? Also could you upgrade to RC10 and enable OS Caching?
As I'm getting very bad transfer speeds it would be interested to see your results as you're also running inside
ESXi. Do you have the LSI 9211-8i card set to passthrough or how are your disks presented to your Windows 7 VM?
Plextor M5S is you're datastore on which the VM is installed right?

@Gaevonn
Are your benchmarks with the source disks online?
And sorry if I missed it but in your copy tests what kind of disk is your source disk (c:), ssd or spinning disk?

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,525
  • Karma: +202/-16
    • View Profile
Re: Bad performance? Post here!
« Reply #26 on: October 12, 2013, 07:46:13 am »
@johnzered
You are not getting "very bad performance". Remember that, TCQ gets you decent speeds. So, don't get too obsessed with OS Caching.
There is a reason why I have both. Depending on resource constraints, one might do better over the other.

1. Your network speeds are for you to troubleshoot. Our only focus is local speed. Eliminate all the extra variables as it is not like the array decides to behave differently base on where the data is coming from.

2. I have noticed that you have only 1 core allocated to the VM. Maybe this is what is impacting OS Caching. There might be more threads at work with OS Caching. So, give an extra core to that VM.

3. Also, just as a test try bumping the VM's RAM to 8GB from 4GB and report the effect.

Offline Skirge01

  • Full Member
  • ***
  • Posts: 140
  • Karma: +2/-0
    • View Profile
Re: Bad performance? Post here!
« Reply #27 on: October 15, 2013, 02:01:24 pm »
@Skirge01
Are your benchmarks with the source disks offline? Yes, they were all still in the array, but I assigned one a drive letter of X for testing.
Also could you upgrade to RC10 and enable OS Caching?  Planning to this weekend.
As I'm getting very bad transfer speeds it would be interested to see your results as you're also running inside
ESXi. Do you have the LSI 9211-8i card set to passthrough or how are your disks presented to your Windows 7 VM?  Yes.
Plextor M5S is you're datastore on which the VM is installed right?  Yes.

Answers are in bold.  Are you running the same version of ESXi that I am?  Also, note that I have 2 of my VMs alone on different drives (an XP VM and a W7 VM which has tRAID); pfSense and WHS2011 share the same drive.

Offline johnzered

  • Jr. Member
  • **
  • Posts: 94
  • Karma: +0/-0
    • View Profile
Re: Bad performance? Post here!
« Reply #28 on: October 20, 2013, 10:44:38 am »
@Brahim

1. I understand that my network is out of the scope of tRAID. But still the only variation in the copy tests was with tRAID on and off, and I have tested several times.

2. Added a second core but no difference.

3. Tried with 4, 6 and 8 GB of RAM and I do not feel it makes any difference.

After updating to the official RC10 I do feel I get better performance with OS Caching than with TCQ. Not that OS caching has improved but rather that TCQ is not performing as good as before. Think I will go with OS caching and SWO 16 for the time being and focus on other final testings.

Offline johnzered

  • Jr. Member
  • **
  • Posts: 94
  • Karma: +0/-0
    • View Profile
Re: Bad performance? Post here!
« Reply #29 on: October 20, 2013, 10:46:18 am »
Answers are in bold.  Are you running the same version of ESXi that I am?  Also, note that I have 2 of my VMs alone on different drives (an XP VM and a W7 VM which has tRAID); pfSense and WHS2011 share the same drive.

I'm running ESXi 5.1.0 build 1157734. During my tests I only have one VM turned on, the one with tRAID installed.
The VM is installed (Datastore) on a Seagate Momentus hybrid drive.

I suppose you get what you pay for when you buy a cheap SATA-controller card instead of a IBM M1015.
I've been thinking about getting a IBM M1015 myself and flash it to IT-mode. But I read somewhere that the card doesn't provide SMART info and cannot spin down disks, is that true?