NVMe...4K?

[kspare]

We primarily use P3700 nvme cards in our storage servers. I've been trying to get a grasp on the reason to do this. It's popped up again now that we're experimenting with meta drives as well.

I'm happy to do some testing for the community if I can get some ideas on how to actually test the effectivness of running all 4k, or maybe a combination of 4k and 512?

I generally run 2 400gb p3700 OP to 100gb for SLOG, 2 800gb P3700 OP to 400gb for Meta, and 1 2TB P3700 for L2Arc.

I only use freenas for an NFS 3 store for my vmware cloud.

 

[HoneyBadger]

The logic behind it is that since you're never going to be writing a block smaller than 4K (your ashift size) you might as well have the drive optimize itself to represent that as "single 4K write" versus "eight 512b writes" - there's potential improvement by reducing the number of back-end I/Os hitting the vdevs.

 

[kspare]

Is there anything to consider with the actual pool itself or leave it to the default of 128k?

What you say makes sense, I guess I just need to give it a try.

 

[HoneyBadger]

No, it's more related to ashift than recordsize. ashift detemines the minimum, recordsize is the maximum.

 

[kspare]

I'll pull all the nvme drives and format them as 4k, when I recreate the pool are there some special steps to take there as well?

 

[HoneyBadger]

I'll pull all the nvme drives and format them as 4k, when I recreate the pool are there some special steps to take there as well?

Nope. FreeBSD understands 4Kn drives and will handle the intermediate steps for you.

 

[Herr_Merlin]

Plus ashift 12 is default for FreeNAS.. so nothing to change / optimize for the 4K