SSD ZPOOL vs HDD ZPOOL

[BKdilse]

Hi,

I've had a read through a few posts with similar setups, but just wanted to run this by the community, with a hope for some opinions...

I have 2 FREENAS servers setup for ISCSI Storage for use with ESXI6. 0

Both systems have the max ram installed (16GB), Quad port NIC for ISCSI Multipath.

1st system has 4 x 512GB SSDs in RAIDZ1 and 2nd system had 4 x 500GB HDDs in RAIDZ1.

1st system is superquick running around 20 Windows Server and Workstation VM's. 2nd system is sluggish after around 5-6 VM's.

I know RAIDZ1 isn't the best for performance, but I don't want to loose anymore space especially on the SSDs.

I also read that TRIM is not for ZPOOLs.


Any thoughts/suggestions on the these 2 setups?

Thanks,

BK

 

[Robert Trevellyan]

Any thoughts/suggestions on the these 2 setups?

It doesn't seem at all surprising to me that the SSD pool works a lot better than the HDD pool. If I understand what I've read correctly, the best you can expect for IOPS from a single RAIDZ vdev is whatever the slowest drive can offer, and IOPS are where SSDs tend to excel.

As a more general point, your boxes seem a bit under-specced for the workload.

I also read that TRIM is not for ZPOOLs.

Where?

 

[mav@]

Using RAIDZ1 of HDDs for VM is bad idea. Striped mirrors of 1TB drives instead of RAIDZ1 of 500GB drives would give you 2-4 times better performance, better reliability and probably not so different price per gigabyte.

 

[BKdilse]

It doesn't seem at all surprising to me that the SSD pool works a lot better than the HDD pool. If I understand what I've read correctly, the best you can expect for IOPS from a single RAIDZ vdev is whatever the slowest drive can offer, and IOPS are where SSDs tend to excel.

As a more general point, your boxes seem a bit under-specced for the workload.

Where?

Thanks for your feedback.

It's only a Dev lab so the spec isn't too much of a concern, and freenas reporting doesn't show any workload issues.

As for SSD Trim. I read a comment by on this thread:

https://forums.freenas.org/index.php?threads/ssd-zpool.21608/

 

[BKdilse]

Using RAIDZ1 of HDDs for VM is bad idea. Striped mirrors of 1TB drives instead of RAIDZ1 of 500GB drives would give you 2-4 times better performance, better reliability and probably not so different price per gigabyte.

As its a Dev system, I don't want to waste any further space. I know the best performance would be RAID0.

 

[Robert Trevellyan]

I read a comment by on this thread

Seems ambiguous to me. Only @cyberjock suggests there might be a bug with TRIM support, and he doesn't say that it would cause problems.

 

[BKdilse]

Seems ambiguous to me. Only @cyberjock suggests there might be a bug with TRIM support, and he doesn't say that it would cause problems.

Hmm, I couldn't find a way to actually test if TRIM is working, but the system does perform very well (I spose it will as it's got fast storage).
I know TRIM does not work in Windows if you are using the disks in a RAID1 array. It also does not work in Synology NAS, if you are using it as Disk LUN mode, but does on File Mode

 

[Robert Trevellyan]

I couldn't find a way to actually test if TRIM is working, but the system does perform very well

They way I understand it, TRIM is most beneficial for maintaining write performance on mostly full drives under heavy write load. Since you'll be keeping your pool below 80% full if you follow FreeNAS best practice, TRIM support should not be critical to performance. If you were using an SSD as an L2ARC or dedicated ZIL device, I would expect TRIM support to be more important.

 

[BKdilse]

They way I understand it, TRIM is most beneficial for maintaining write performance on mostly full drives under heavy write load. Since you'll be keeping your pool below 80% full if you follow FreeNAS best practice, TRIM support should not be critical to performance. If you were using an SSD as an L2ARC or dedicated ZIL device, I would expect TRIM support to be more important.

Very true. Same goes with Windows, recommended free space is 25%

 

[BKdilse]

Just a quick update... I've now reconfigured the HDD system to use RAID10 instead of RAIDZ1 and it is a little more snappier...

I'll be using the SSD NAS for machines I want fast responses from and the HDD NAS for the ones I don't. Also I thought it would be a good idea to split the windows domain controllers across the two NAS boxes.

 

[Peter Jakab]

Hi BKdilse,

Could you share us same performance stat for the two pool? Any thoughbut MByte/sec or IOPS tested on SSD drives.

Bye,
Jackson

 

[jgreco]

Since you'll be keeping your pool below 80% full if you follow FreeNAS best practice, TRIM support should not be critical to performance.

60% for VM usage.

 

[BKdilse]

60% for VM usage.

Too much loss of expensive SSD storage :(

 

[BKdilse]

Hi BKdilse,

Could you share us same performance stat for the two pool? Any thoughbut MByte/sec or IOPS tested on SSD drives.

Bye,
Jackson

Hi Jackson,

I didn't grab the IOPS but I did a quick file copy test from a Windows machine, copying a 60GB file...

HDD RAIDZ1 = 70-110MB/s
FreeNAS GUI Reporting on Disk = 29MB/s accross all 4 disks

HDD RAID10 = 70-110MB/s
FreeNAS GUI Reporting on Disk = 46MB/s accross all 4 disks

I have not got the SSDs setup in RAID10 due to a massive loss of disk spave. RAIDZ1 is working well for me, as my VM's are flying.
FreeNAS GUI Reporting on Disk = 28MB/s accross all 4 disks, with VMware.

I'll try some Crystal Disk tests in a VM, when I get a chance.

 

[jgreco]

Too much loss of expensive SSD storage :(

Yes, well, it is what it is. There's a good chance that there are some ways to make some tradeoffs that would help. Fragmentation probably isn't as serious a problem w.r.t. read speeds on an SSD pool, it's still the writing that's likely to be a killer.

 

[Robert Trevellyan]

reconfigured the HDD system to use RAID10 instead of RAIDZ1

Are you using hardware RAID with FreeNAS, or are you using striped mirrors but calling it RAID10?