Flashed DELL H310 mini and SSD disks

[poldas]

Hi,

I completed my storage:

DELL R720xd
H310 mini flashed to 9211-8i IT
64 GB RAM
9 x Intel SSD D3-S4510 2.5" - 1.92TB
Intel 2x 10 Gbit ethernet

I want to use storage for VMware VM disks and databases.

1. I'am planing to create 2 pools RAID-Z1, one for VM and second for dabases each with 4 disks (9 disk for spare). Is the optimal constelation?
2. Is the necessary export SLOG to extened storage? If yes, is the Intel P3700 400GB good choice?
3. Is H310 mini enough for SSD disks?

 

[HoneyBadger]

1. I'am planing to create 2 pools RAID-Z1, one for VM and second for dabases each with 4 disks (9 disk for spare). Is the optimal constelation?
2. Is the necessary export SLOG to extened storage? If yes, is the Intel P3700 400GB good choice?
3. Is H310 mini enough for SSD disks?

1. For VM disks/databases where a lot of random access is driven out, even SSDs are best configured as mirror vdevs. This may mean you need more drives to get the same amount of usable space.
2. Even on all-SSD vdevs, an SLOG device can improve latency if it is significantly faster (which the P3700 400GB is) - this is a good choice. The Optane P-series is also acceptable. If you are using iSCSI, you must set sync=always on the ZVOLs used - NFS default settings do use sync so it can be left at the default of "standard."
3. The H310 mini can now be flashed to LSI IT firmware; the stock firmware will not be enough due to low factory queue depth, so it must be flashed. There is a separate process that is required for the Mini/Mono cards. Once flashed it should be acceptable, but it is an older card and may have some limitations around performance.

 

[Chris Moore]

I would suggest putting all the drives in a single pool so IOPS can be spread across all drives, that will provide more performance and the SLOG device can serve the single pool. The storage can still be separated logically instead of making two separate pools.

 

[poldas]

3. The H310 mini can now be flashed to LSI IT firmware

I have done it H310 mini now is 9211-8i IT

I would suggest putting all the drives in a single pool so IOPS can be spread across all drives

Now I have 2 pools. Do you recomend to put all disks to one pool and make 2 datasets - one for VM's and second for DB's?

To be honest I have 9 disks, 8 for storage and 1 for spare. What do you recommend:
RAID 1 (8 disks) + spare (1 disk) or RAIDZ2 (9 disks) without spare?

Even on all-SSD vdevs, an SLOG device can improve latency if it is significantly faster (which the P3700 400GB is) - this is a good choice.

Can I use PLEXTOR 256GB PCle Gen3x4 PX-256M9PeY instead Intel P3700 400GB? Does 256 GB is enough for ~12 GB storage?

 

[HoneyBadger]

I have done it H310 mini now is 9211-8i IT

Perfect.

Now I have 2 pools. Do you recomend to put all disks to one pool and make 2 datasets - one for VM's and second for DB's?

View attachment 32074

To be honest I have 9 disks, 8 for storage and 1 for spare. What do you recommend:
RAID 1 (8 disks) + spare (1 disk) or RAIDZ2 (9 disks) without spare?

Yes, that's the suggestion - all of the disks in one pool, using mirrors ("RAID1" in your example) with one as a spare.

Can I use PLEXTOR 256GB PCle Gen3x4 PX-256M9PeY instead Intel P3700 400GB? Does 256 GB is enough for ~12 GB storage?

The Plextor is not a good substitute as it does not have the power loss protection offered by the Intel P3700 - SLOG devices require "confirmation" that the writes were successful to stable storage, and devices without PLP circuitry need to actually commit the bits all the way to NAND. This makes them incredibly slow to do proper sync writes. Devices with PLP can return "complete" because they can consider their DRAM cache as non-volatile.

256GB of RAM for 12TB is a good ratio. ZFS loves memory, and generally speaking will use as much as it can for read caching.

 

[poldas]

Yes, that's the suggestion - all of the disks in one pool, using mirrors ("RAID1" in your example) with one as a spare.

Sorry I thought:

RAIDZ1 not RAID1 (8 disks) + spare (1 disk) or RAIDZ2 (9 disks) without spare?

Should I use only mirror constellation for SSD storage or can I use RAIDZ also?

256GB of RAM for 12TB is a good ratio. ZFS loves memory, and generally speaking will use as much as it can for read caching.

I mean 256 GB SLOG storage :) Does it enough for 12TB storage?

Now I have 64 GB RAM in my R720xd

 

[HoneyBadger]

Sorry I thought:

RAIDZ1 not RAID1 (8 disks) + spare (1 disk) or RAIDZ2 (9 disks) without spare?

Should I use only mirror constellation for SSD storage or can I use RAIDZ also?

Mirrors is the highest performance configuration, and with an R720xd chassis you don't have a shortage of bays to fill up with extra drives if you need additional capacity. RAIDZ is possible certainly but will offer lower overall performance.

I mean 256 GB SLOG storage :) Does it enough for 12TB storage?

Yes, in fact the default tunables limit the SLOG usage to 4GB maximum - but in either scenario, the Plextor is not an appropriate device for SLOG. Stick with the Intel DC P3700 or an Optane P4801X for the reasons previously mentioned with respect to power-loss-protection and write performance.

Now I have 64 GB RAM in my R720xd

I'd suggest increasing that if possible, especially if this is going to be servicing multiple hypervisor hosts and high I/O DBs. The more RAM you have, the more data ZFS can keep cached there - SSDs might be fast, but RAM is faster. What is the processor in the unit? You don't need a lot of cores.

 

[poldas]

What is the processor in the unit? You don't need a lot of cores.

2x Intel® Xeon® SixCore E5-2620 (15M Cache, 2,0GHz, 7,2GT/s Intel® QPI)

Optane P4801X

I can't use it in my DELL R720xd

 

[HoneyBadger]

2x Intel® Xeon® SixCore E5-2620 (15M Cache, 2,0GHz, 7,2GT/s Intel® QPI)

Might be a little slow on the per-core speed, and you've probably never need all twelve cores - keep an eye on the CPU usage during heavy reads/writes.

I can't use it in my DELL R720xd

There's an option for a 4x2.5" NVMe hot-swap bay at the front of the system, or you can use an internal U.2 to PCIe adapter card. Or seek out the older Optane P4800X 375GB in PCIe form factor.