I have setup my first FreeNAS (FreeNAS 11) box with the components listed here.
I have configured a single RAIDz2 volume for the pool out of my 8x4TB WD reds. My current issue, I believe, is around network speeds, but if someone more knowledgeable feels otherwise, I am all ears. Currently I am experiencing proper gigabit speeds when reading from the server, but when writing to the server I am lucky to hit 120 Mbits/sec.
Here's the output from iperf
Which shows pretty sad write speeds if I have interpreted this test correctly. I have done the following to also troubleshoot my problem
Note: Both non-nas systems are running Windows 7. One has an Intel gigabit NIC (built into motherboard), and the other has a Killer gigabit NIC (built into motherboard). I tried both as I've read Killer doesn't have the best reputation, but same results on both.
Unrelated to iperf tests, I did also run local speed tests against the machine to ensure it wasn't my pool. I created an uncompressed dataset and ran the following commands
Lastly, I ran CrystalDiskMark5 against the NAS when mounted as a drive. The results of this are attached
Other than the tests I've ran above, I am not sure where to begin to find the root cause of the issue. Any help or direction is greatly appreciated.
Update 1:
----------------------------------------------------------------------------------------------
This morning, after doing nothing to the system I am seeing faster speeds. Large transfers, ~20GB start very fast, 120MB/s then taper off around 70-88MB/s over SMB. The iperf stats still remain consistent (I am not sure how that's possible). Also I have run tcpdump and see the following
.119 is the NAS, and .180 is my Windows 7 machine.
From what I have researched online the SMB problem is not unique to me, but I am unsure if it could be useful info to help find the root cause.
Update 2:
----------------------------------------------------------------------------------------------
I have attempted to transfer a large file, ~10GB over FTP to the service on the local network. It seems both the read and write speeds over FTP have fully saturated the gigabit link. I have attached these screenshots to the thread, but long standing transfers of multiple large files the speeds tapers off to 60-70MB/s range again... It seems the SMB protocol is just slow (which without fine tuning is expected?), but doesn't answer why iperf is reporting such slow speeds for writes.
I have configured a single RAIDz2 volume for the pool out of my 8x4TB WD reds. My current issue, I believe, is around network speeds, but if someone more knowledgeable feels otherwise, I am all ears. Currently I am experiencing proper gigabit speeds when reading from the server, but when writing to the server I am lucky to hit 120 Mbits/sec.
Here's the output from iperf
Code:
[root@freenas ~]# iperf -s ------------------------------------------------------------ Server listening on TCP port 5001 TCP window size: 64.0 KByte (default) ------------------------------------------------------------ [ 4] local 192.168.1.118 port 5001 connected with 192.168.1.180 port 57641 [ ID] Interval Transfer Bandwidth [ 4] 0.0-10.1 sec 83.4 MBytes 69.3 Mbits/sec [root@freenas ~]# iperf -c 192.168.1.180 ------------------------------------------------------------ Client connecting to 192.168.1.180, TCP port 5001 TCP window size: 32.8 KByte (default) ------------------------------------------------------------ [ 3] local 192.168.1.118 port 44816 connected with 192.168.1.180 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 1.10 GBytes 941 Mbits/sec
Which shows pretty sad write speeds if I have interpreted this test correctly. I have done the following to also troubleshoot my problem
- Swapped out the cable (cate5e and cat6) between the nas and my router (Linksys WRT3200ACM), with the same result as above
- Ran the same iperf test between a different computer and the nas, with the same results as above.
- Ran iperf between the two computers that are not the nas, and this resulted gigabit speeds in both directions (I can post this output if necessary, but did not feel it added any benefit)
- Added an additional gigabit switch to the router, and connected the nas and other two machines to this switch. When running the iperf test from each computer to the nas, the same problematic results occurred
- ifconfig on the nas shows the following
Code:
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCS UM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6> ether ac:1f:6b:1d:24:2a nd6 options=9<PERFORMNUD,IFDISABLED> media: Ethernet autoselect status: no carrier igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCS UM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6> ether ac:1f:6b:1d:24:2b inet 192.168.1.119 netmask 0xffffff00 broadcast 192.168.1.255 nd6 options=9<PERFORMNUD,IFDISABLED> media: Ethernet autoselect (1000baseT <full-duplex>) status: active
Note: Both non-nas systems are running Windows 7. One has an Intel gigabit NIC (built into motherboard), and the other has a Killer gigabit NIC (built into motherboard). I tried both as I've read Killer doesn't have the best reputation, but same results on both.
Unrelated to iperf tests, I did also run local speed tests against the machine to ensure it wasn't my pool. I created an uncompressed dataset and ran the following commands
Code:
[root@freenas /mnt/test_vol/uncompressed]# dd if=/dev/zero of=/mnt/test_vol/uncompressed /test.dd bs=2048k count=50k 51200+0 records in 51200+0 records out 107374182400 bytes transferred in 174.633327 secs (614855045 bytes/sec) [root@freenas /mnt/test_vol/uncompressed]# dd of=/dev/null if=/mnt/test_vol/uncompressed /test.dd bs=2048k count=50k 51200+0 records in 51200+0 records out 107374182400 bytes transferred in 114.284731 secs (939532176 bytes/sec)
Lastly, I ran CrystalDiskMark5 against the NAS when mounted as a drive. The results of this are attached
Other than the tests I've ran above, I am not sure where to begin to find the root cause of the issue. Any help or direction is greatly appreciated.
Update 1:
----------------------------------------------------------------------------------------------
This morning, after doing nothing to the system I am seeing faster speeds. Large transfers, ~20GB start very fast, 120MB/s then taper off around 70-88MB/s over SMB. The iperf stats still remain consistent (I am not sure how that's possible). Also I have run tcpdump and see the following
Code:
tcpdump -i igb1 -nn -vv host 192.168.1.180 > /mnt/vol_name/tcpdump.txt 11:06:23.339944 IP (tos 0x0, ttl 128, id 14642, offset 0, flags [DF], proto TCP (6), length 132) 192.168.1.180.49548 > 192.168.1.119.445: Flags [P.], cksum 0x6bd7 (correct), seq 882:974, ack 941, win 16079, length 92 SMB-over -TCP packet:(raw data or continuation?) 11:06:23.340114 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 168) 192.168.1.119.445 > 192.168.1.180.49548: Flags [P.], cksum 0x8516 (incorrect -> 0x6144), seq 941:1069, ack 974, win 1026, length 128 SMB-over-TCP packet:(raw data or continuation?) 11:06:23.340765 IP (tos 0x0, ttl 128, id 14643, offset 0, flags [DF], proto TCP (6), length 261) 192.168.1.180.49548 > 192.168.1.119.445: Flags [P.], cksum 0xaf41 (correct), seq 974:1195, ack 1069, win 16425, length 221 SMB-o ver-TCP packet:(raw data or continuation?) 11:06:23.340942 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 244) 192.168.1.119.445 > 192.168.1.180.49548: Flags [P.], cksum 0x8562 (incorrect -> 0x67b5), seq 1069:1273, ack 1195, win 1026, leng th 204 SMB-over-TCP packet:(raw data or continuation?)
.119 is the NAS, and .180 is my Windows 7 machine.
From what I have researched online the SMB problem is not unique to me, but I am unsure if it could be useful info to help find the root cause.
Update 2:
----------------------------------------------------------------------------------------------
I have attempted to transfer a large file, ~10GB over FTP to the service on the local network. It seems both the read and write speeds over FTP have fully saturated the gigabit link. I have attached these screenshots to the thread, but long standing transfers of multiple large files the speeds tapers off to 60-70MB/s range again... It seems the SMB protocol is just slow (which without fine tuning is expected?), but doesn't answer why iperf is reporting such slow speeds for writes.
Attachments
Last edited: