Hardware in hand, questions on config...

[pageb018]

Just trying to find the right way to build out my FreeNAS box, I played around a little and I am here with a few questions before finalizing.

Current hardware:

Dell r510 12 Bay 3.5 + 2 bay 2.5 internal

96gb RAM

12x 2TB HDD

2x 1TB Samsung EVO SSD

1x 16gb Optane (in PCIE Adapter)

H200 Card in IT Mode

10gb SFP+ Card (running to 10gbe Switch)

2x - XCP-NG Hosts with connections to main 1gig network and 10gig storage network.

My use case will be VM disk storage and general file sharing and backup. I currently have an unRAID server that houses plex/emby media and other file storage. The end goal would be to keep mostly the media on the unRAID box and move all other file sharing and storage to the FreeNAS box. I would also backup critical files between them as needed.
I think for the non ssd pool I will go with 6x or 5x striped mirrors. This is mostly so I can upgrade the vdevs easier down the line. I don't care too much about performance on the non-ssd pool. I am also trying to decide if I should keep two drives open to expand the SSD pool (more on that later).
For the SSD pool, I currently have the two 1TB's in a mirror, but I think I would like more storage, so I may extend to add another mirror.
Ok, so questions:

1. Besides the obviously size increase, will I see a significant performance increase in adding another mirror to the SSD pool? I am running up against the physical capacity of the server, and while I could expand to another chassis later on, I am not ready for that now. So space is a little limited.
2. I currently running the server off of mirrored usb drives. The server has been up for about a week and one usb already died. I am worried about this. But the usb drives are cheap enough. Which brings me to my next question...
3. What to do with the optane? I currently have it as a SLOG on the SSD pool, but I am reading that this doesn't really help. Should I make the optane the SLOG for the hdd pool? Or should I use it as a boot drive. If I use it as a boot drive, I would add another to mirror, but at that point I would also be out of PCIE slots. Which isn't a problem now, but would be down the line if I wanted to use a SAS expander for more drives.

So...what is my best bet here? So many options. I may just be overthinking it. But I hope you can help so I have a great solid foundation for this storage server.

Thank you.

 

[artlessknave]

1. relatively modern SSD speeds are limited by the interface, not the drives. unless they are SAS3/NVME SSD's, you will never get more than 6gb/s from them, but prececely because the are SSD's, you will get close to that 6gb/s continuously. because of this, the IOPS can mostly be ignored as well, cuz its gonna be MULTIPLES of 10000 or 100000,

Show : IOPS

Intel X25-M G2 (MLC)	SSD	~8,600 IOPS[11]	SATA 3 Gbit/s
Intel X25-E (SLC)	SSD	~5,000 IOPS[13]	SATA 3 Gbit/s
G.Skill Phoenix Pro	SSD	~20,000 IOPS[16]	SATA 3 Gbit/s
OCZ Vertex 3	SSD	Up to 60,000 IOPS[17]	SATA 6 Gbit/s
Corsair Force Series GT	SSD	Up to 85,000 IOPS[18]	SATA 6 Gbit/s	555 MB/s sequential read 525 MB/s sequential write)
Samsung SSD 850 PRO	SSD	100,000 read IOPS 90,000 write IOPS[19]	SATA 6 Gbit/s	550 MB/s sequential read 470 MB/s sequentialwrite

IOPS - Wikipedia

en.wikipedia.org

while spindle drives max around 200, meaning that having more vdev's is mostly irelevant for SSD's.
2. all USB drive are not created equally, most have zero wear leveling and die REALLY fast with zfs. consider getting sandisk cruzer fit 2.0 or 3.0. (sandisk usb drives usually have wear leveling) or other SSD USB drives since they usually have wear leveling.
3. SLOG NEEDS power safety at the drive level. if it doesn't have that the whole pool is at risk. SLOG also only applies to synchronous writes. unless you have sync writes set for the whole pool or are using ESXi NFS (forces sync writes for VM files, most of the time the SLOG will do nothing. the PCIE slot is probably more useful for something else. the drive isn't really big enough for L2ARC, since you have more RAM anyway.

if I should keep two drives open to expand the SSD pool

do you mean to keep 2 drive *bays* open instead of 2 drives? (otherwise you're...going to expand an SSD pool with spindle disks? if so, that's.....silly. the SSD's main bonus (speed) will be negated.)