- Joined
- May 28, 2011
- Messages
- 10,995
NAS Performance Testing with RealTek 8111 NICs vs. Intel NICs
And SSD ZIL and L2ARC Performance Testing
The purpose of this test is to show the performance difference between the motherboard installed RealTek NIC’s which are very predominate in our hardware and the add-on card Intel NIC. I will not make any assumptions that a motherboard installed Intel NIC would product the same results as the add-on card NIC.
The tests will be conducted while there is no other network traffic on the network. The LAN connections will plug into a single Rosewill 8-port Gigabit switch (RC-410LX) with no other connection to the switch. I will also connect a single Ethernet cable between the two devices to see if the switch has any effect on throughput.
Testing will consist of the following tests:
1) dd for baseline FreeNAS internal speed test. This will be done once only just to show there are no slowness problem plaguing the system.
2) Intel NAS Performance Toolkit (NASPT) which can easily be located on the Intel website, just Google search it. (Between each test the created test data will be deleted)
3) Crystal DiskMark version 3.0.1 (32-bit)
4) Copy a large single file from the NAS using Windoze.
5) Copy a large single file to the NAS using Windoze.
6) Reboot both systems and rerun the tests.
7) Install Intel NIC into main computer, rerun the tests.
8) Install a second Intel NIC into the FreeNAS computer, rerun the tests.
9) Install the SSD into the FreeNAS computer, set it up as ZIL, rerun the tests.
10) Setup the FreeNAS SSD to also be an L2ARC, rerun the tests.
System Configuration:
Main Computer:
Motherboard: Gigabyte X58-USB3
CPU: Intel i7 950 (3.06 GHz)
RAM: 24GB
MB NIC: RealTek 8111L
Hard drives: SSD (two) Crucial M4 256GB as individual drives, one 2TB Samsung HD204UI.
OS: Windows 7 w/Service Pack 1
FreeNAS Computer:
Motherboard: Gigabyte P45T-ES3G
CPU: Intel E8500 (3.16 GHz)
RAM: 8GB
MB NIC: RealTek 8111D
RAIDZ1: four WD Red 2TB + one Samsung 2TB HD204UI (no SWAP space on drives).
FreeNAS Configuration: 8.3.0-RELEASE, CIFS, DDNS, SSH, Plugins (MiniDLNA).
Network:
CAT5e cables
Rosewill 8-port Gigabit switch (RC-410LX)
Upgraded Hardware:
Intel EXPI9301CTBLK PCI Network Card (two cards)
Corsair P120 SSD (three partitions, 8GB, 100GB, 20GB)
Test 1
The dd test was conducted to show there are no internal write/read speed problems within the FreeNAS computer. The results are consistent for my SATA II connections, no complaints at all.
Code:
dd if=/dev/zero of=tmp.dat bs=2048k count=50k 51200+0 records in 51200+0 records out 107374182400 bytes transferred in 336.745898 secs (318858175 bytes/sec) dd of=/dev/zero if=tmp.dat bs=2048k count=50k 51200+0 records in 51200+0 records out 107374182400 bytes transferred in 287.611204 secs (373331014 bytes/sec)
Test 2
Intel NAS Performance Test Toolkit
I selected all testing cases to be performed. This does take a rather long time to run the testing but because I plan to also include a ZIL and L2ARC, it was just smart to see where these will shine, or not.
The Rosewill Gigabit Switch is connected between the two computers.
Code:
Testname Run1 Run2 Run3 Run4 Run5 Median HD Video Playback 50.3 51.5 51.8 51.5 51.9 51.5MB/s 2x HD Playback 51.8 82.7 52.4 52.4 52.8 52.4MB/s 4x HD Playback 83.7 66.4 66.8 67.4 78.8 67.4MB/s HD Video Record 203.7 175.9 182.7 176.6 191.2 182.7MB/s HD Playback & Record 66.5 62.0 65.0 67.0 68.5 66.5MB/s Content Creation 10.8 10.7 11.2 11.1 9.3 10.8MB/s Office Productivity 49.2 47.8 49.6 49.0 49.1 49.1MB/s File copy to NAS 72.7 77.5 72.4 76.5 73.3 73.3MB/s File copy from NAS 46.1 45.2 46.8 44.5 45.5 45.5MB/s Dir copy to NAS 4.9 4.8 4.9 5.0 4.8 4.9MB/s Dir copy from NAS 19.1 16.8 17.7 18.0 18.2 18.0MB/s Photo Album 18.9 18.2 18.2 18.1 18.1 18.2MB/s
Test 3
Intel NAS Performance Test Toolkit
Reboot both FreeNAS and main computer. Selected all testing cases to be performed.
The Rosewill Gigabit Switch is dis-connected and the computers are connected directly to each other.
Code:
Testname Run1 Run2 Run3 Run4 Run5 Median HD Video Playback 53.4 53.5 53.4 53.3 53.4 53.4MB/s 2x HD Playback 53.3 53.3 53.3 52.8 53.4 53.3MB/s 4x HD Playback 76.5 53.6 54.3 65.6 53.7 54.3MB/s HD Video Record 194.1 195.8 189.7 185.3 184.3 189.7MB/s HD Playback & Record 61.7 90.9 66.9 67.6 70.1 67.6MB/s Content Creation 10.7 11.3 10.3 10.8 10.3 10.7MB/s Office Productivity 50.6 49.0 49.6 49.2 49.0 49.2MB/s File copy to NAS 76.9 78.7 78.3 79.2 76.0 78.3MB/s File copy from NAS 47.7 77.5 43.8 78.7 47.3 47.7MB/s Dir copy to NAS 4.8 4.6 3.1 4.9 4.8 4.8MB/s Dir copy from NAS 17.8 17.5 18.0 17.9 17.9 17.9MB/s Photo Album 17.1 17.1 17.1 17.2 17.2 17.1MB/s
*Based on the results from running Test 2 and Test 3 the Gigabit switch has some might have a minor influence to our testing so I will use a direct connection between the two devices for all future testing.
Test 4
Crystal DiskMark
I have setup the settings as indicated in the screen capture.
Number of test runs = 2, Test Size = 100MB, Target = Mapped FreeNAS dataset.
Code:
Read Write Seq: 56.79 Seq: 102.4 512K: 56.98 512K: 98.31 4K: 9.504 10.23 4KQD32: 67.56 64.76
Test 5
Copy a large video file from FreeNAS. Since I’m using SSD’s on my main computer, there should be negligible impact cause by my computer. During the large file copy tests each file is a different 3.x GB .vob video file and are not used more than one after a reboot unless specified so they are not cached which would affect the testing.
NOTE: All Copy Video Files are using Windoze 7 to cut and paste the files.
84.0 MB/sec
Test 6
Copy a second large video file from FreeNAS.
82.7 MB/sec
Test 7
Copy a third video to FreeNAS.
94.1 MB/sec
Test 8
Copy a fourth video to FreeNAS.
106 MB/sec
Testing is complete using the original NIC equipment. At this point we will install the Intel NIC into the main computer and then repeat the testing we previously performed.
Test 9
Intel NAS Performance Test Toolkit
Selected all testing cases to be performed.
Code:
Testname Run1 Run2 Run3 Run4 Run5 Median HD Video Playback 51.6 60.5 60.3 60.2 60.4 60.3MB/s 2x HD Playback 50.9 59.3 59.9 59.6 60.1 59.6MB/s 4x HD Playback 97.2 61.6 73.2 70.1 85.7 73.2MB/s HD Video Record 179.1 181.1 201.1 183.4 198.0 183.4MB/s HD Playback & Record 72.1 74.0 75.8 76.5 74.2 74.2MB/s Content Creation 12.2 10.6 9.1 10.2 10.6 10.6MB/s Office Productivity 52.0 51.2 48.4 51.2 50.1 51.2MB/s File copy to NAS 82.7 83.2 84.1 80.8 86.0 83.2MB/s File copy from NAS 52.3 52.4 53.6 53.7 53.8 53.6MB/s Dir copy to NAS 5.0 5.1 5.0 4.9 5.2 5.0MB/s Dir copy from NAS 21.0 20.6 21.8 21.8 22.5 21.8MB/s Photo Album 25.9 25.1 26.1 25.3 25.5 25.5MB/s
Test 10
Crystal DiskMark
I have setup the settings as indicated in the screen capture.
Number of test runs = 2, Test Size = 100MB, Target = Mapped FreeNAS dataset.
(Forgot to capture the image, sorry)
Test 11
Copy a large video file from FreeNAS.
96.2 MB/sec
Test 12
Copy a second large video file from FreeNAS.
82 .0 MB/sec
Test 13
Copy a third video to FreeNAS.
102 MB/sec
Test 14
Copy a fourth video to FreeNAS.
104 MB/sec
At this point we will install the Intel NIC into the FreeNAS computer and then repeat the testing we previously performed.
Test 15
Intel NAS Performance Test Toolkit
Selected all testing cases to be performed.
Code:
Testname Run1 Run2 Run3 Run4 Run5 Median HD Video Playback 51.8 65.6 66.4 66.4 66.3 66.3MB/s 2x HD Playback 55.2 64.3 100.1 99.1 66.1 66.1MB/s 4x HD Playback 104.9 80.6 81.6 66.8 67.1 80.6MB/s HD Video Record 207.0 207.2 202.2 205.8 188.6 205.8MB/s HD Playback & Record 70.6 79.2 78.1 74.1 79.1 78.1MB/s Content Creation 9.7 11.7 12.0 11.1 11.7 11.7MB/s Office Productivity 48.6 50.8 51.4 51.2 51.4 51.2MB/s File copy to NAS 85.1 81.9 80.9 80.3 78.4 80.9MB/s File copy from NAS 54.1 54.3 55.0 54.9 54.9 54.9MB/s Dir copy to NAS 4.8 5.0 4.9 4.8 5.0 4.9MB/s Dir copy from NAS 20.5 21.3 20.7 21.3 20.9 20.9MB/s Photo Album 23.4 23.7 23.6 23.5 23.6 23.6MB/s
Test 16
Crystal DiskMark
I have setup the settings as indicated in the screen capture.
Number of test runs = 2, Test Size = 100MB, Target = Mapped FreeNAS dataset.
Code:
Read Write Seq: 71.72 Seq: 109.1 512K: 71.19 512K: 104.4 4K: 12.76 10.27 4KQD32: 66.85 64.55
Test 17
Copy a large video file from FreeNAS.
98.8 MB/sec
Test 18
Copy a second large video file from FreeNAS.
100 MB/sec
Test 19
Copy a third video to FreeNAS.
93.7 MB/sec
Test 20
Copy a fourth video to FreeNAS.
110 MB/sec
Results of adding the Intel NIC’s
Installing the Intel NIC’s did increase throughput but it wasn’t that significant in my opinion. Here are the differences of Test 2 to Test 15:
Code:
Testname Test 2 Test 15 Diff HD Video Playback 53.4 66.3 12.9MB/s 2x HD Playback 53.3 66.1 12.8MB/s 4x HD Playback 54.3 80.6 26.3MB/s HD Video Record 189.7 205.8 16.1MB/s HD Playback & Record 67.6 78.1 10.5MB/s Content Creation 10.7 11.7 1.0MB/s Office Productivity 49.2 51.2 2.0MB/s File copy to NAS 78.3 80.9 2.6MB/s File copy from NAS 47.7 54.9 7.2MB/s Dir copy to NAS 4.8 4.9 0.1MB/s Dir copy from NAS 17.9 20.9 3.0MB/s Photo Album 17.1 23.6 6.5MB/s
Adding the ZIL
I am adding an 8GB ZIL to this already speedy setup for home use. The ZIL hopefully will increase the writing speed of the FreeNAS computer.
Test 21
Intel NAS Performance Test Toolkit
Selected all testing cases to be performed.
Code:
Testname Run1 Run2 Run3 Run4 Run5 Median HD Video Playback 52.8 65.3 65.0 64.2 65.1 65.0MB/s 2x HD Playback 93.3 98.9 64.1 64.5 64.6 64.6MB/s 4x HD Playback 110.2 79.9 79.4 78.5 66.0 79.4MB/s HD Video Record 185.1 195.7 153.1 194.5 203.4 194.5MB/s HD Playback & Record 79.6 69.2 77.9 70.9 79.2 77.9MB/s Content Creation 12.1 9.4 12.1 11.8 9.7 11.8MB/s Office Productivity 51.3 49.6 50.9 48.7 50.4 50.4MB/s File copy to NAS 77.4 83.5 81.1 75.4 78.8 78.8MB/s File copy from NAS 55.8 55.5 52.6 55.1 54.8 55.1MB/s Dir copy to NAS 11.4 12.8 10.5 12.2 12.8 12.2MB/s Dir copy from NAS 19.9 20.8 20.1 20.0 20.3 20.1MB/s Photo Album 22.2 22.7 22.2 22.4 22.6 22.4MB/s
Test 22
Copy a video to FreeNAS.
109 MB/sec
Test 23
Copy another video to FreeNAS.
105 MB/sec
Adding the L2ARC
In a home environment I do not see the L2ARC being a huge benefit but since I have so much unused SSD space I will be giving 100GB to this test and this gives me ~20GB of SSD space to use as I will, likely for my plugins and scripts.
Test 24
Intel NAS Performance Test Toolkit
Selected all testing cases to be performed.
Code:
Testname Run1 Run2 Run3 Run4 Run5 Median HD Video Playback 55.5 65.6 65.5 65.5 65.2 65.5MB/s 2x HD Playback 55.8 63.1 63.7 65.4 64.4 63.7MB/s 4x HD Playback 80.1 64.7 66.0 66.7 78.5 66.7MB/s HD Video Record 190.0 183.9 185.5 135.2 182.8 183.9MB/s HD Playback & Record 90.8 75.7 70.1 80.8 75.6 75.7MB/s Content Creation 11.3 9.9 12.2 11.6 11.9 11.6MB/s Office Productivity 48.7 48.3 48.1 49.5 48.1 48.3MB/s File copy to NAS 80.1 81.2 73.9 67.6 79.3 79.3MB/s File copy from NAS 51.5 51.8 54.3 54.3 53.9 53.9MB/s Dir copy to NAS 11.4 11.2 13.1 10.5 11.9 11.4MB/s Dir copy from NAS 19.0 17.7 16.8 20.9 20.4 19.0MB/s Photo Album 22.2 22.6 22.4 22.0 22.8 22.4MB/s
Test 25
Crystal DiskMark
I have setup the settings as indicated in the screen capture.
Number of test runs = 2, Test Size = 100MB, Target = Mapped FreeNAS dataset.
Code:
Read Write Seq: 70.45 Seq: 108.5 512K: 70.00 512K: 103.6 4K: 12.29 9.990 4KQD32: 69.50 64.67
Test 26
Copy the first large video file from FreeNAS.
96.3 MB/sec
Test 26
Copy the first large video file from FreeNAS again, second time to see if the L2ARC cached anything.
116 MB/sec
Test 27
Copy the second large video file from FreeNAS again.
97.3 MB/sec
Test 28
Reboot both computers and copy the second large video file from FreeNAS again.
69.7 MB/sec
Overall Results
Here are the results of the Intel NAS Performance Test Toolkit listing the highlight of each test.
The addition of a ZIL made a significant change only to the “Dir copy to NAS” test and nothing else. I honestly didn’t expect much from a ZIL using CIFS.
The addition of the L2ARC made no changes either and again I believe it’s due to the use of CIFS or my ARC was large enough for these tests.
Code:
Testname Start NICs ZIL L2ARC HD Video Playback 53.4 66.3 65.0 65.5MB/s 2x HD Playback 53.3 66.1 64.6 63.7MB/s 4x HD Playback 54.3 80.6 79.4 66.7MB/s HD Video Record 189.7 205.8 194.5 183.9MB/s HD Playback & Record 67.6 78.1 77.9 75.7MB/s Content Creation 10.7 11.7 11.8 11.6MB/s Office Productivity 49.2 51.2 50.4 48.3MB/s File copy to NAS 78.3 80.9 78.8 79.3MB/s File copy from NAS 47.7 54.9 55.1 53.9MB/s Dir copy to NAS 4.8 4.9 12.2 11.4MB/s Dir copy from NAS 17.9 20.9 20.1 19.0MB/s Photo Album 17.1 23.6 22.4 22.4MB/s
Conclusion
My system does not benefit much from the addition of the Intel NIC’s over the internal RealTek NIC’s and was not worth the additional cost of $60.00 for two NICs. Maybe it’s because my CPUs are fast, I’m not sure but these are the facts of my system. Also I was unable to do any testing using NFS becasuse Windoze 7 Professional and below does not have NFS client included and I didn't find a free NSF client to add to Windoze 7 that wasn't just terribly slow, I mean crawling slow.
If NFS protocol could be used, I understand it would be faster and the ZIL and L2ARC might actually do some remarkable improvements.
And I did do another test with my network switch installed at the end of this and it has no real impact on the testing so I guess it's a good switch for home use.