Register for the iXsystems Community to get an ad-free experience

NVMe Driver Configuration for Consumer Drivers

Western Digital Drives - The Preferred Drives of FreeNAS and TrueNAS CORE
Joined
Sep 20, 2021
Messages
2
Hey, guys.

I bought an adapter card to plug my NVMe driver into a PCI-e slot (adapter image is attached). This board has a super capacitor, capable of keeping the driver on for seconds in case of power failure. Enough time to write any outstanding data.

It works as a UPS directly connected to the NVMe driver. In theory, it works as a Power Loss Protection (PLP) added to a consumer NVMe driver. So I would like to use a consumer NVMe to accelerate writes with fsync type calls.

In theory writes with fsync-like calls are dumped directly into flash without making use of driver cache memory. This makes writing very slow, especially for many small files.

We know that only enterprise drivers are able to be fast in these scripts, because only they have supercapacitors inside them. Therefore, they are prepared to ignore fsync calls, keeping the data in cache memory. But they are too expensive, out of my budget. So I'm looking for a cheaper alternative.

This Chinese adapter with supercapacitor is very cheap. As well as consumer NVMe drivers they are also very cheap compared to enterprise drivers.

But as we all know, consumer NVMe's are not able to bypass the fsync-like calls used by many storage software, databases, etc. This makes the consumer NVMe very slow, especially on small files.

Is there any way to configure the NVMe OS-driver so that it replaces or not sends fsync-type calls from applications to the physical nvme driver? It would be interesting to be able to perform this configuration indicating the driver that we know is protected with the super capacitor.

Or would it be possible to place this instruction directly in the driver firmware?

Thus, he could apply a flush to the data only when the node was turned off. Because it would be guaranteed that the driver would have the energy and time to write the data.

Any idea?

Grateful,
 

Attachments

  • supercapacitor_nvme.jpg
    supercapacitor_nvme.jpg
    54.9 KB · Views: 142

jgreco

Resident Grinch
Moderator
Joined
May 29, 2011
Messages
16,531
This seems immensely more complex than simply setting sync=none, while giving you the same benefit.
 
Joined
Sep 20, 2021
Messages
2
Then,
But that would be the idea. From simply configuring the drive so that, when it receives a "sync" instruction, it leaves the data in the cache to be physically written without any priority. I understand that this is what makes a business SSD, those that have supercapacitors such as the Seagate Nytro XP960LE30002, the Hynix HFS960GD0MEE-5410A or even the Kingston DC1000.

What could make it so complex?
 

jgreco

Resident Grinch
Moderator
Joined
May 29, 2011
Messages
16,531
Because it's a Rube Goldberg solution to a simple problem that already has a simple answer.
 

HoneyBadger

Mushroom! Mushroom!
Joined
Feb 6, 2014
Messages
4,032
Is there any way to configure the NVMe OS-driver so that it replaces or not sends fsync-type calls from applications to the physical nvme driver? It would be interesting to be able to perform this configuration indicating the driver that we know is protected with the super capacitor.

This was done on very old hardware that didn't properly represent its PLP capability, like circa 2000's Sun flash cards. Those cards had a high level of QA/QC, their own internal firmware was capable of understanding when a power loss event occurred, and could then flush their cache internally. A modern "consumer" drive has none of that, and I'd be extremely hesitant to throw a random "supercapacitor on a card" into my machine unless I was wanting to see how much Magic Blue Smoke I could generate.

We know that only enterprise drivers are able to be fast in these scripts, because only they have supercapacitors inside them. Therefore, they are prepared to ignore fsync calls, keeping the data in cache memory. But they are too expensive, out of my budget. So I'm looking for a cheaper alternative.

Buy an Optane M10 16/32GB card. It's likely to provide similar endurance to a consumer SSD as well as blow its doors off for SLOG workloads - the 16GB unit is good for about 1Gbps of sync-writes in small records, 32GB is good for 2Gbps. See the link in my signature for an example of SLOG benchmarks.
 

Ericloewe

Not-very-passive-but-aggressive
Moderator
Joined
Feb 15, 2014
Messages
18,118
Even if SSD firmware were typically open-source (it literally never is) and it were not a complete spaghetti mess (I'd bet it is a mess) and you could setup the development toolchain and you could upload the new firmware to your drive and you could debug this when it inevitably became necessary and you targeted a specific model or small range of drives, this would still be a bad idea unless you were looking to start a business selling these. It would still be a terrible business model, but it would begin to make sense from a time investment perspective.
 

HoneyBadger

Mushroom! Mushroom!
Joined
Feb 6, 2014
Messages
4,032
This isn't about rewriting firmware, but rewriting a driver to change the behavior of the FreeBSD NVMe driver, making it not forward the O_DSYNC/fsync/etc commands to the physical device.

I'd qualify this as a Bad Idea unless you were 100% confident in the hardware underpinning it, such as the aforementioned Sun F-series devices.


(The lack of the ability to adjust this setting outside of Solaris is also why sync-write performance on those F-series cards tends to suck outside of Solaris as well.)
 
Top