Mean ZFS Performance issues / problem

Status
Not open for further replies.

djiceman

Dabbler
Joined
Jan 1, 2012
Messages
11
Hey guys,

for the last couple of days, I've been trying to get a FreeNAS system with some ZFS storage up and running. Unfortunately I'm experiencing some mean performance issues on ZFS. Actually I wouldn't really call it a performance issue, it's more like a serious problem:

Initial performance over the network through AFP or rsync is about 100 - 115 MB/s - Nice. Unfortunately after a while the performance decreases rapidly. The file transfer seems to get stuck for a few seconds and I can hear the harddisks running high. After a couple of seconds the transfer resumes at a much lower speed, builds up again and gets stuck again. After a while the performance doesn't increase anymore, but remains at about 15MB/s - crap.
I tried local file transfer from a single SATA drive to the ZFS using rsync - same issue! So it's not a network problem.

But let me give you some details about what I'm trying to do and what steps I took trying to solve this.
The goal is to set up a FreeNAS system with 16 2TB drives configured as 1 ZFS Pool consisting of two raidz (8 disks each).
The used HW looks like this:

- Mainboard Gigabyte GA-EX58-UD5
- Intel Core i7 2,66 Ghz
- 6GB RAM
- 3ware 9650SE-12ML
- 16x Samsung 2TB HDDs
- 2GB Scandisk Cruzer Flashdrive
- FreeNAS 8.0.2 64-bit

- Drive cache on the 3ware is enabled!

So here some of the steps I took:

- set up 8 disk raidz on 3ware controller (single) - no change
- set up 8 disk raidz on 3ware controller (JBOD) - no change
- set up 4 disk raidz on onboard controller - no change
- removed 3ware completely and set up 4 disk raidz on onboard controller - no change
- installed 18GB of RAM - no change
- replaced the Mainboard - no change
- configured 8 disk RAID5 UFS on 3ware Controller - no issues, write speed is constantly well above 100MB/s.
- tried some ZFS tuning - no change.
- tried FreeNAS 8.0.3 RC2 - no change

When I had the 18GB of RAM installed, I noticed that it took longer for the issue to appear. So I tried again and kept an eye on the memory consumption. It seems that the trouble starts as soon as the system has all available memory reserved.

I'm running out of ideas what else to try to solve this. The only thing I can think of would be a caching SSD. I'm gonna try that tomorrow.

If any of you guys have an idea what could cause this and what I could do to solve it, please help me out here!

I would really like to have this set up on ZFS, but the way it looks right now, I might have to settle for a "good old" 12-disk RAID6 Hardware Raid. :(

Any suggestions would be really appreciated.

Cheers,

Ice


EDIT:

- added a 120GB SSD drive as cache to the pool - no change
- configured a 4+1 raidz - no change

Well, I'm officially out of ideas now. I think it must be an issue with the hardware, but I just can't figure it out.
I already replaced the MB, RAM, CPU, Controller. The disks are running perfectly fine as a hw raid. Damn, this is driving me crazy.

EDIT:

- tried FreeNAS 0.7.5.8854 - no change
 

djiceman

Dabbler
Joined
Jan 1, 2012
Messages
11
Usually I don't, but I tried it once yesterday and it seemed to fix things at first. At least it took a bit longer for the problem to reoccur, but unfortunately it did eventually. :(
 

djiceman

Dabbler
Joined
Jan 1, 2012
Messages
11
William's hint made me think about my harddrives a little more. So I had a closer look and noticed that I have a mix of older and newer Samsung models:

SAMSUNG HD203WI - definitely a 512 drive
SAMSUNG HD204UI - 4K drive internally - 512 emulation

So I figured, that might have something to do with the strange behaviour I am experiencing on my system. I went ahead and configured a 4 drive raidz with only HD204UI drives.
Unfortunately it didn't solve my problem. The behaviour remains the same. :(
I tried both: 512 and 4K setup, but none solved my problem. At least I got something out of it. I noticed that 4K gives me a much better performance on these drives. Something I wouldn't have expected.

For my FreeNAS configuration I've given up on zfs and instead configured a 12-disk RAID6 hw raid on the 3ware Controller.
This doesn't give me nearly the diskspace I had hoped for with ZFS, but it'll have to do for now.
I just moved about 11T to the system and the performance with the hw raid is stable and abot 110MB/s over the network.

I'm gonna set up a separate machine to play around a little more. I'd really like to get to the bottom of this.
So if you have any ideas or suggestions, feel free to share. ;) I'll be happy to try anything.

Cheers,

Ice
 
Status
Not open for further replies.
Top