Anyone Else Mess Around with NVME Yet?

[JWTech]

I threw some NVME drives into one of my work's development servers as zil/l2arc/swap, and so far I'm damn impressed with them.
Biggest issue in the dev server was "platter latency" from the drives being sluggish when working with massive amounts of smaller files, or a large amount of users copying files simultaneously.
I grabbed two of those OCZ/Toshiba 256GB RD400 NVME drives (2.6GB/s read, 1.2GB/s write each), partitioned them accordingly and added them to the test-pool.

I don't think I've ever noticed more consistent performance out of a FreeNAS machine before.

Anyone else mess around with any of those yet? What was your experience with them?

 

[Borja Marcos]

I've used Intel P3500 drives on FreeBSD, which should be similar. I found several caveats:

- Trim performance was abysmal until the last firmware version came out. By abysmal I mean, deleting a large file caused a severe I/O starvation for several minutes.

- The drives come formatted for 512 byte blocks, performance increased about 20% by formatting them for 4 KB blocks.

- Hot plug PCIe not supported for them. This is not a SATA or SAS drive you can replace without powering down the system.

Software support on FreeBSD is quite immature yet. Rather than relying on CAM, NVMe drives live in their own stack. That means, they don´t benefit from some important CAM improvements like scheduling and TRIM coalescencing.

Other than that, performance is excellent.