Author Topic: small files in the pool get corrupted when data is appended  (Read 7008 times)

Offline uhthome

  • Newbie
  • *
  • Posts: 42
  • Karma: +0/-0
    • View Profile
Re: small files in the pool get corrupted when data is appended
« Reply #30 on: September 27, 2015, 07:37:54 am »
I have installed the latest new release announced sept 18th and repeated the test. I am sorry to say that the issue was readily reproduced. Still, it occurs only in the pool drive.

I think a main feature of my logging program is the very short delay between opening a file, modifying it and closing it. With an additional artificial delay in the program the issue does not occur, as it is prevented by disabling SWO or TCQ.

After opening a file, it is in my view important that the file buffer is filled with the correct data before further steps are performed. In this case, due to the speed an simplicity of the program, modifying the file and closing it occurs even before the file has acutally been loaded into the file buffer. This leads to erroneously writing data from other, previously processed, files into the file.

Therefore, the problem seems to be that the pool drive allows modifying access to the file buffer and closing the file with write back to the disk before the buffer has even been filled with the corrrect contents. Better, these actions would be put on hold until the file buffer ist filled. This seems to happen correctly on all other file locations on my server and in my network, except for the pool. Speed of drive access cannot be the problem, since some of these drive locations are really slow. Are the write optimizations (TCQ and SWO) too smart in optimizing command orders through the pool, delaying the file buffer read after the file buffer write and circumventing any safeguard against wrong order of file access commands??

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: small files in the pool get corrupted when data is appended
« Reply #31 on: September 27, 2015, 10:49:34 am »
It is typical for optimizations to sometimes affect fringe cases.

I have tried hard to reproduce the issue you are reporting without much success. Furthermore, in the real world, I don't know of any application that does not write data in 512 bytes alignments. So, your use case is definitely unconventional.

With multiple layers of optimization (Windows cache, TCQ, SWO), I can see where there could be collisions (whether timing related or not).
As per your report, it is the strict combo of TCQ + SWO + Storage Pool that is causing the issue. Take one out and things work fine.

I am personally curious as to what in the combo is creating the collision in your setup. If I could replicate it, we would at least know whether it is fixable or not and document it if not.
I wonder if you could fire up a VM, setup tRAID on Linux, and report if the issue is also there.

Offline uhthome

  • Newbie
  • *
  • Posts: 42
  • Karma: +0/-0
    • View Profile
Re: small files in the pool get corrupted when data is appended
« Reply #32 on: November 23, 2015, 03:22:05 pm »
Unfortunately, I was not able to do the requested test on a Linux VM. I am unfamiliar with Linux and would need help here, which I was not able to obtain.

Today, I just want to let you now that the issue is still present in the latest version. One thing has changed though: the issue is no longer prevented by disabling SWO. Now, it can only be prevented by disabling TCQ.

Offline Brahim

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 8,547
  • Karma: +204/-16
    • View Profile
Re: small files in the pool get corrupted when data is appended
« Reply #33 on: November 23, 2015, 04:25:50 pm »
What virtualization software do you have? VMware or VirtualBox?
You can:
1. download images in places like http://www.osboxes.org/ubuntu/
2. Add some virtual disks
3. Follow this guide: http://wiki.flexraid.com/2014/06/22/ultimate-linux-guide-to-transparent-raid/