Really slow transfer speeds, even when dd'ing on the local disk

[crisco]

Hi,
I built a little Intel Atom NAS and i'm having some trouble getting it to run decently. Basic specs are:

Intel Atom 1.66 Dual-core
4gb RAM (the max my motherboard supports)
80gb standard HDD for OS (a waste, i know)
4x 1.5TB WD Green drives in a RAIDZ2
FreeNAS-8.3.0-RELEASE-p1-x64 (r12825) -- Maybe i should have stuck with 32-bit?


Anyways, i have one CIFS share on my only volume, and i'm getting really slow speeds to it. My NAS is connected directly to the router with a 1g copper link, while my PC uses wireless-N, and i'm seeing ~4mb/s peak over the network.
I did a lot of reading around here and other places and i initially looked at the network connection, but i've now noticed that the copy speeds are even slow when i copy locally on the disk. For example, if i 'dd' a 1.6gb ISO i'm seeing the following:
1735393280 bytes transferred in 297.729026 secs (5828768 bytes/sec)

So, not my network i'm presuming. I'm not expecting this to be a fast NAS but i think my speeds are still about 10x slower than they should be. I tried enabling the "Autotune" but it had no noticeable effect. Where do i go from here?
Thanks in advance! (And BTW, FreeNAS is really an impressive product!)

 

[ProtoSD]

Hi,
I built a little Intel Atom NAS and i'm having some trouble getting it to run decently. Basic specs are:

Intel Atom 1.66 Dual-core
4gb RAM (the max my motherboard supports)
80gb standard HDD for OS (a waste, i know)
4x 1.5TB WD Green drives in a RAIDZ2
FreeNAS-8.3.0-RELEASE-p1-x64 (r12825) --

Thanks for posting your specs, saves us a lot of extra questions! :)

Maybe i should have stuck with 32-bit?

Nope, 64-bit is the correct version, I have a 1.6Ghz Atom also.

Anyways, i have one CIFS share on my only volume, and i'm getting really slow speeds to it. My NAS is connected directly to the router with a 1g copper link, while my PC uses wireless-N, and i'm seeing ~4mb/s peak over the network.

Bytes or Bits?? If this is bytes, that sounds about right for wifi.

I did a lot of reading around here and other places and i initially looked at the network connection, but i've now noticed that the copy speeds are even slow when i copy locally on the disk. For example, if i 'dd' a 1.6gb ISO i'm seeing the following:
1735393280 bytes transferred in 297.729026 secs (5828768 bytes/sec)

6MB/s ~5min, that is a bit slow. I don't remember my speeds, I know I posted them here before. I have 8GB of RAM in mine, so that definitely made a difference, but I don't think its all of your problem. Maybe there's a disk having trouble, try running a smartctl LONG test on all of your disks. It'll take a few hours on each disk, so start them before bed so there's not activity other than the tests.

So, not my network i'm presuming. I'm not expecting this to be a fast NAS but i think my speeds are still about 10x slower than they should be. I tried enabling the "Autotune" but it had no noticeable effect. Where do i go from here?
Thanks in advance! (And BTW, FreeNAS is really an impressive product!)

smartctl LONG test on your disks and post the results. You'll need to use ssh to connect so you can copy & paste the results.

Don't mess with Autotune, I'd delete any settings it has. It won't do much with only 4GB of RAM anyway.

Oh, also, make sure you have "prefetch" disabled, it's in the loaders section of the GUI. This is one of those counter-intuitive double-negative variables ;)

vfs.zfs.prefetch_disable = 1

 

[crisco]

Thanks for the help!
Wifi speeds are bytes, so it really isn't that terrible. In reading around it seems like everyone is seeing disk speeds in the 40mb/s+ range so that's what i was hoping for!
This guy has a really similar setup (i have the Zotac NM10 as well): http://forums.freenas.org/showthread.php?238-Poor-performance-with-2TB-4k-HDDs-and-Raid-Z

I do have prefetch disabled. I will get rid of the autotune additions.

I ran "smartctl --test=long /dev/ada[x]" for each of 0-3, which are the 4 drives in my array (4 is my OS disk). ADA0 gives this error:

[root@frumpy] ~# smartctl --test=long /dev/ada0
smartctl 5.43 2012-06-30 r3573 [FreeBSD 8.3-RELEASE-p5 amd64] (local build)
Copyright (C) 2002-12 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF OFFLINE IMMEDIATE AND SELF-TEST SECTION ===
Warning: device does not support Self-Test functions.

Sending command: "Execute SMART Extended self-test routine immediately in off-line mode".
(pass0:ahcich0:0:0:0): SMART. ACB: b0 d4 02 4f c2 40 00 00 00 00 00 00
(pass0:ahcich0:0:0:0): CAM status: ATA Status Error
(pass0:ahcich0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT )
(pass0:ahcich0:0:0:0): RES: 51 04 02 4f c2 40 00 00 00 00 00
Command "Execute SMART Extended self-test routine immediately in off-line mode" failed: No error: 0

Any idea what this means? When i go to "Volume Status" it shows ADA0p2 as "Online"

 

[crisco]

The "long" test probably finished, but i never saw results. How do i get those?

At any rate i did some more digging through dmesg and i found out that only 1 of 4 of my drives is showing SATA II compatibility (all 4 should be!). The one that shows that it handles SATA II also happens to be the one displaying the error above. I'm going to dig around and see if i can find a firmware update and go from there.

 

[crisco]

My journey continues...
After some searching it seems that the SMART functionality is slightly screwed up on my onboard Jmicron controller JMB363, and others with my board (Zotac NM10) have had similar issues. Basically the onboard controller drops speed to near zero when you connect 3+ drives. Here is the forum i found this info in: http://www.zotacusa.com/forum/topic/2682-nm10-dtx-sata-raid-configuration-issues/page__st__60

 

[cyberjock]

Now you know why I never ever buy Zotac. Not built with the best quality stuff...

 

[ProtoSD]

My journey continues...
After some searching it seems that the SMART functionality is slightly screwed up on my onboard Jmicron controller JMB363, and others with my board (Zotac NM10) have had similar issues. Basically the onboard controller drops speed to near zero when you connect 3+ drives. Here is the forum i found this info in: http://www.zotacusa.com/forum/topic/2682-nm10-dtx-sata-raid-configuration-issues/page__st__60

Sorry to hear that, glad you narrowed it down. Does your board have an available PCI slot? You could just throw a different controller in and disable the onboard one.

 

[crisco]

The problem is not Zotac, it's this crappy Jmicron stuff (which seems to be everywhere) and a lack of drivers for FreeBSD. Although, i suppose it *was* Zotac that put this crappy stuff on their boards...

It does have an available PCI slot, so now the question is: where do i find a good inexpensive SATA controller that doesn't have the same problems? :)

 

[ProtoSD]

I'm not sure about inexpensive, but take a look at this post:

Will-this-HBA-work-with-my-setup&p=43756&viewfull=1#post43756

Of course it might just be cheaper to get a better motherboard.

 

[crisco]

Great, thanks protosd!

I had a thought this morning--the motherboard does have 2 intel channels that apparently don't suffer from the same problems. Any thoughts on whether it would help to use the 2 intel channels and then only 2 of the crappy Jmicron ports? I think i might give that a shot. I'll need to switch to an SD card to do it, but it's a lot cheaper than a new card or motherboard...