There's a new SLOG champion in town (Intel Optane 900p SSD)

[Chris Moore]

Do we have any idea of how many of these are being used for SLOG and if they have been tested for this purpose?

https://www.servethehome.com/exploring-best-zfs-zil-slog-ssd-intel-optane-nand/

 

[c32767a]

https://www.servethehome.com/exploring-best-zfs-zil-slog-ssd-intel-optane-nand/

I have one in a test array. the 900P 280G NVMe version.

Code:

smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.1-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:					   INTEL SSDPED1D280GA
Firmware Version:				   E2010325
PCI Vendor/Subsystem ID:			0x8086
Controller ID:					  0
Number of Namespaces:			   1
Namespace 1 Size/Capacity:		  280,065,171,456 [280 GB]
Namespace 1 Formatted LBA Size:	 512
Local Time is:					  Sat Mar 24 19:22:49 2018 EDT
Firmware Updates (0x02):			1 Slot
Optional Admin Commands (0x0007):   Security Format Frmw_DL
Optional NVM Commands (0x0006):	 Wr_Unc DS_Mngmt
Maximum Data Transfer Size:		 32 Pages

Supported Power States

St Op	 Max   Active	 Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +	18.00W	   -		-	0  0  0  0		0	   0

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +	 512	   0		 2


=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02, NSID 0xffffffff)
Critical Warning:				   0x00
Temperature:						43 Celsius
Available Spare:					100%
Available Spare Threshold:		  0%
Percentage Used:					0%
Data Units Read:					140,909 [72.1 GB]
Data Units Written:				 87,693,074 [44.8 TB]
Host Read Commands:				 2,545,885
Host Write Commands:				488,824,861
Controller Busy Time:			   345
Power Cycles:					   6
Power On Hours:					 1,075
Unsafe Shutdowns:				   1
Media and Data Integrity Errors:	0
Error Information Log Entries:	  0

Error Information (NVMe Log 0x01, max 64 entries)
No Errors Logged

I haven't pulled the plug on it to test the quasi-PLP-esqeness of it yet, but our boxes are all fed dual rail UPS power, so I'm not too worried about PLP.

If we bind it to a volume as an slog, it has the same effect on performance as turning off sync writes. I'm really impressed with it so far.
We may actually go with a bigger card and use it for L2ARC as well.

 

[Stux]

The biggest concern with the 900p is this unresolved issue with ESXi psssthrough into FreeBSD

https://redmine.ixsystems.com/issues/26508

 

[Eric Schrauth]

The biggest concern with the 900p is this unresolved issue with ESXi psssthrough into FreeBSD

https://redmine.ixsystems.com/issues/26508

I read that before. I don't quite understand the issue. Can you further explain?

 

[Stux]

I read that before. I don't quite understand the issue. Can you further explain?

There’s currently incompatibility between various Unix like kernels and the 900p when it is passed through from ESXi. The indication is it’s a timing issue, and perhaps is fixed in FreeBSD 12

 

[minez]

There’s currently incompatibility between various Unix like kernels and the 900p when it is passed through from ESXi. The indication is it’s a timing issue, and perhaps is fixed in FreeBSD 12

Seems like passthrough won't work currently for an AIO setup, but what about the option for creating an ESXi datastore on the Optane drive and using this datastore in FreeNAS for the slog/l2arc/swap? Seems this was tested and got fairly good speeds : http://napp-it.org/doc/downloads/optane_slog_pool_performane.pdf.

In this case FreeNAS wouldn't have direct access to the drive. Are there any major risks in doing this (at least until the passthrough issue is resolved)?

 

[Ericloewe]

I imagine latency would not be acceptable for SLOG. As for risks, it depends on how well sync writes are honored.

 

[Stux]

Seems like passthrough won't work currently for an AIO setup, but what about the option for creating an ESXi datastore on the Optane drive and using this datastore in FreeNAS for the slog/l2arc/swap? Seems this was tested and got fairly good speeds : http://napp-it.org/doc/downloads/optane_slog_pool_performane.pdf.

In this case FreeNAS wouldn't have direct access to the drive. Are there any major risks in doing this (at least until the passthrough issue is resolved)?

I’m not sure how reliably the slog is via that method.

Considering Samsung Evo drives perform admirably via ESXi but I think terribly natively (not confirmed) that leads me to suspect that sync writes to datastores might not be 100% safe, but my research indicates that ESXi does preserve sync writes.

So, the safety is not confirmed.

 

[minez]

Considering Samsung Evo drives perform admirably via ESXi but I think terribly natively (not confirmed) that leads me to suspect that sync writes to datastores might not be 100% safe, but my research indicates that ESXi does preserve sync writes.

Based on my understanding (and correct me if this is wrong!), ESXi forces all NFS writes to be synchronous writes. So ESXi is writing synchronously to the FreeNAS VM, and those writes hit the slog and are then written asynchronously to the HDDs. From the FreeNAS VMs perspective, the slog (which in this case is a datastore) appears as a normal drive which FreeNAS will distribute those writes to in the same way as it would if it was passed through. I would have thought that far as ESXi is concerned, it is just writing back to FreeNAS over the storage network and then it is the responsibility of the FreeNAS VM to decide what to do with that data - it doesn't seem logical to me in this case that ESXi would change the way it writes based on whether it is writing to a datastore or not.

Or are you suggesting that any sync write to a datastore is potentially unsafe due to the way datastores work in general?

Assuming that the 900p does have adequate PLP, the other question is whether ESXi handles remounting the datastore correctly to the FreeNAS VM after a power loss event to allow the data to be written correctly. Do you have any thoughts on this?

Apologies if some of my assumptions are incorrect here - still learning the ins and outs!

 

[Stux]

800p/M10 M.2 Optane disk with about 50% of the performance of the 900p... may be an interesting option.

https://forums.freenas.org/index.php?threads/intel-optane-m10-800p-for-slog.63762/

 

[foo]

Does anyone know if this will fit inside a HP MicroServer Gen G8 (my current hardware)? It seems the Optane 900P comes with a low profile backplate but I'm not sure if the length of the card will fit inside the case. If it does, I'll get one and see how it works as a SLOG.

 

[Sisyphe]

I found a fix for passing-through the Optane 900P to Freenas in ESXi, see here:
https://redmine.ixsystems.com/issues/26508