adamjs83
Dabbler
- Joined
- Sep 3, 2015
- Messages
- 40
I am totally new to freenas and after reading this forum and the manual extensively I ordered my parts and assembled my machine which I have been testing the last few weeks. The full specs can be found in my signature below. This NAS is for home and home lab use exclusively.
I have one volume of spinning disks in raidz2 which I am able to write to over CIFS and get around 120MBps, better than I had hoped and the 1Gb link is clearly being saturated.
My other volume is 4 SSDs and is connected to ESXI using iSCSI over 2 multipathed links with and MTU of 9000 the entire way. With sync=always I get about 113MBps read and 40MBps write and 113/113 MBps with sync=standard. Setting all devices to the default MTU of 1500 slightly reduces all of those speeds These results are from a windows server 2012 R2 vm using CrystalDiskMark. The freenas gui reporting feature shows that each of my two Ethernet are splitting the traffic at about 450Mbps each, clearly not saturating the gigabit links.
In order to diagnose why the two iSCSI ethernet connections were not getting higher throughput I wanted to test the performance of the pool directly. My research pointed me to use DD. Below are the results.
To save time that's read of .23 MBps write and .57 GBps write
The read performance is ok for my needs but the write performance is abysmal and significantly slower than even the throughput reported in Crystal.
While the DD write is running this is the typical output of gstat for one of the 4 drives, the others are about the same.
I ran the same tests on my spinning pool and the results were .22 GBps write and .11GBps read. Not sure why those aren't reversed but plenty fast for my needs.
Does anyone have any ideas as to why I am getting this abysmal write performance from the SSDs on DD and how troubleshoot?
Additionally am I on the right track trying to figure out why the iSCSI is <500Mbps on each of the two Gb links?
I have one volume of spinning disks in raidz2 which I am able to write to over CIFS and get around 120MBps, better than I had hoped and the 1Gb link is clearly being saturated.
My other volume is 4 SSDs and is connected to ESXI using iSCSI over 2 multipathed links with and MTU of 9000 the entire way. With sync=always I get about 113MBps read and 40MBps write and 113/113 MBps with sync=standard. Setting all devices to the default MTU of 1500 slightly reduces all of those speeds These results are from a windows server 2012 R2 vm using CrystalDiskMark. The freenas gui reporting feature shows that each of my two Ethernet are splitting the traffic at about 450Mbps each, clearly not saturating the gigabit links.
In order to diagnose why the two iSCSI ethernet connections were not getting higher throughput I wanted to test the performance of the pool directly. My research pointed me to use DD. Below are the results.
Code:
adamjs83@sto01:/mnt/SAN-VOLUME % sudo dd if=/dev/zero of=testfile bs=1024 count=50000 50000+0 records in 50000+0 records out 51200000 bytes transferred in 220.526250 secs (232172 bytes/sec) sudo dd if=testfile of=/dev/zero bs=1024 count=50000 50000+0 records in 50000+0 records out 51200000 bytes transferred in 0.089598 secs (571441707 bytes/sec)
To save time that's read of .23 MBps write and .57 GBps write
The read performance is ok for my needs but the write performance is abysmal and significantly slower than even the throughput reported in Crystal.
While the DD write is running this is the typical output of gstat for one of the 4 drives, the others are about the same.
Code:
L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 1 340 0 0 0.0 114 456 0.0 69.0| gptid/10c674da-5e3c-11e5-93d7-0cc47a6ae2c8
I ran the same tests on my spinning pool and the results were .22 GBps write and .11GBps read. Not sure why those aren't reversed but plenty fast for my needs.
Does anyone have any ideas as to why I am getting this abysmal write performance from the SSDs on DD and how troubleshoot?
Additionally am I on the right track trying to figure out why the iSCSI is <500Mbps on each of the two Gb links?