Write speed 20MB/sec with a read speed of 60-80MB/sec

[Roi]

Hello,

I am experiencing strange problems here with write performance on my FreeNAS box. First of all some basics about my hardware:

AMD Athlon II X4 605e CPU
8GB DDR2 RAM
Gigabyte GA-MA78G-DS3H

HDDs are attached via SATA, I run a ZFS array with 5*3TB and one with 3*2TB.

As in the thread subject, the read speed is great, but the write speed is very poor. Interesting that it was better in the beginning when I migrated from NAS4Free to FreeNAS and imported an old ZFS array to the new box. It complained about a wrong sector size (512/4096 which means less performance) but it was a lot better. Then I created a new array with the volume manager and also created a second new array with three of the old disks. I see the write speed on all arrays, strange...

The CPU (using "top") is at 1-5%, also I do not see any IOWAIT and other stuff.

BTW: The write speed is good with single disks. I get 130MB/sec (local rsync job). The 60-80MB/sec read speed was measured over Gbit ethernet btw.

Some hints for me? A first search here and on Google did not turn up with much. Lots of very old hits.

Cheers,
Roi

 

[hugovsky]

Hi and welcome.

About your problem, best is to backup your data and start over. Read cyberjock's guide here. Your lack of perfomance is a combination of factors. You don't have a right cpu, you don't have a right board, 8GB is bare minimum ram and you have wrong sector size. And what is your vdev type?

 

[Roi]

Hi and thank you for your reply.

Well, I am pretty happy with the hardware as it does the job - also for quite a while now starting with NAS4Free and switching to FreeNAS (main reason was the jail plugins like the unofficial UrBackup one). At least for what I expect of a NAS box to do here. Sure I know about the risks not using ECC RAM. Also about the risks not using RAIDZ2. And yeah, I would use more than 8 GB of RAM, but I would need to buy it so I started with the 8 GB to have a look.

My vdev types are RAIDZ2 (first) and RAIDZ (second). The second array is just for backing up the first.

But I need to take a closer look. Maybe I fooled myself, shame on me. I just restore stuff from a old single disk to the second array. With USB 2... Just tested how much speed I get if I transfer from array one to two. Over 100MB/sec. Hm, I will test network write speed when the restore is finished.

 

[gpsguy]

An Intel Pro/1000 CT OEM version costs about $30 USD. It's a much better NIC than your Realtek.

As hugovsky said, you need more RAM for the amount of storage you are using and to use the plugins.

While your system may have been fine for NAS4free, FreeNAS is tuned for systems with more horsepower.

 

[Roi]

Thank you for your reply.

Yes, a RAM or system upgrade is in the pipeline. But it is not that always easy as it might be for others. For example power costs are very expensive here so you start looking for hardware which is not using that much - as it pays out over the months and years. Yes, with a bit less performance. So, now I need complete new hardware (mainboard, CPU, RAM) or old and expensive RAM. We'll see.

Do you have any suggestions according to power consumption, system performance and "normal" costs?

But, memory usage at the moment:

Code:

Mem: 284M Active, 704M Inact, 5720M Wired, 1066M Free

I use a dual head Broadcom NetXtreme II BCM5706 which I use in a LACP channel. I tried to bundle it with the onboard Realtek to a channel out of three NICs which at least worked for a few days without any known problems or speed decreases.

 

[Bidule0hm]

As you said it was better in the beginning I'm thinking about something: how full your pools are? (in percentage)

There is always some free RAM, it's not an indicator you need more RAM (but generally performance issue is...) :)

 

[jgreco]

Write speeds are closely intertwined with the amount of RAM, the size of the pool, and the amount of space free on the pool. ZFS performs poorly as it runs out of free resources.

 

[Roi]

My pools are filled 55% (first) and 25% (second).

@jgreco: Interesting, this is also the reason why more than 80% usage is giving out a warning message?

It seems that I need to watch over some memory modules on ebay - or buy completly new hardware. Any suggestions? ;-)

 

[Bidule0hm]

Well, it's not even close to 80%. So it must be the others reasons already mentionned ;)

 

[jgreco]

@jgreco: Interesting, this is also the reason why more than 80% usage is giving out a warning message?

Yes, but the 80% is kind of an arbitrary number. We know things get worse for most people out past that.

Well, it's not even close to 80%. So it must be the others reasons already mentionned ;)

"must be"? Seriously, I can cause a pool pain far before 80%. This is not a justifiable conclusion.

 

[Bidule0hm]

Yeah, not well written. I think that 95 % of his perfomance problem is because of the others reasons so he should not worry about the pool usage now and solve the other things first (RAM and NIC), and only then, if he has still performance problems he can see what to do about the pool usage ;)