Quad m.2 card adapter compatibility

maniyer

Dabbler
Joined
Jun 26, 2019
Messages
19
I’m posting this because I haven’t found much information online or in the forums regarding the compatibility of quad m.2 adapter cards with FreeNAS. Considering the growing m.2 adoption on boards, it makes sense to discuss them regardless of how they are used, also since the newer Gen4x4 are just being released.

For the sake of this discussion, let’s assume the motherboard supports 4x4x4x4 bifurcation (which mine does, and many do now), or that you’re using a bridged adapter which doesn’t require bifurcation (although they are more expensive and possibly slower).

The main concern to me is will FreeNAS see the drives if one of the cards is used. If the motherboard recognises the board and the drives in the bios, will FreeNAS automatically detect them as well? Do any of them need drivers for FreeNAS to recognise them, and are there any drivers already in FreeNAS to support them?

If anyone has one running on their system, please let me know if it works and if anything needed to be done for them to work normally with four m.2 drives.

The few cards I’ve found online (although there are many more):

MSI M.2 Xpander-Aero
Dell Ultra-Speed Drive Quad NVMe M.2
Asus Hyper M.2 Card V2
ASRock Quad M.2 Gen 3
Gigabyte CMT4034
Aplicata Quad 410
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
I can confirm that on a Mobo without the support, one of those cards correctly presents the first NVME drive (and no others) to the OS.

I assume the only barrier there to additional drives is the lack of bifurcation support.
 

maniyer

Dabbler
Joined
Jun 26, 2019
Messages
19
Thanks for the reply, but I did say in my original post that my board has 4x4x4x4 bifurcation, so that shouldn't be an issue. The question really comes down to whether I need further drivers for FreeNAS to recognise them. I've seen plenty of scenarios where the BIOS recognises all the drives, but FreeNAS doesn't. So just wondering if there is a need for further drivers for FreeNAS to recognise any of these cards or if it will just work out of the box. Looking for anyone who might have tried or tested this scenario with any of the many available cards.
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
I was just sharing the experience I was able to have with mine to confirm to the extent that I can... I get that someone with a bifurcation supporting Mobo and one of those cards with 2+ NVME drives needs to confirm it for real.
 

maniyer

Dabbler
Joined
Jun 26, 2019
Messages
19
Thanks, I appreciate that. Anyone else out there have any experience with any of these cards interacting with FreeNAS?
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
You should probably clarify your post to indicate you're talking about m.2 NVMe. Most of the m.2 SATA stuff - definitely including dual and quad adapter stuff - works fine (generally for the obvious reasons).
 

maniyer

Dabbler
Joined
Jun 26, 2019
Messages
19
Yes, agreed. My apologies. The cards I'm referring to are NVMe. Thanks for clarifying.
 

MLandgraf

Cadet
Joined
Aug 7, 2019
Messages
1
Hi,
I am new to NAS (had always MS servers from 2003 up). In a former life I have programmed drivers for UNIX (in pre-linux times).

I now intend to play with a FreeNAS configuration - goal is to mirror a MS server to the NAS, which is 400km apart.

I understand from this thread, that with a quad NVMe card under Windows a driver is needed to make drives #2 to 4 known to the system.
(I am not even asking if such a driver is present in FreeNAS :smile: ).

I have read in a different thread about the use of a similar card from Supermicro (AOC-SHG3-4M2P).
It supports Windows and Linux, but is supports only 2 lanes per SSD (total 8).

I have at hand a ASRock Quad M.2 Gen 3 (NVMe) , which supports 4 lanes per SSD (total 16). It definitely only has Win drivers.

Alternately: Does anyone happen to know if for any of the above by maniyer mentioned cards a LINUX (open) Driver available which could get adopted to FreeNAS? Which I could tweak?

Or does anyone have a solution for driving four 2TB M.2 NVMe SSDs from a AsRock X470D4U2-2T?
(the two onboard M.2 slots only support 2 3.0 lanes!

Thank you!
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
I would hope it wouldn't be a driver thing unless the PCIe switch needs some sort of setup. More common is that the host platform may not support NVMe well, or mishandles the PCIe switch. My offhand suspicion is that it would work fine as long as it's in a recent Supermicro board, which probably has any necessary setup support baked into it.
 

maniyer

Dabbler
Joined
Jun 26, 2019
Messages
19
I think having only 2 lanes per card defeats the purpose of having such fast NVMe cards, along with the fact that the Supermicro card is so much harder to find and also the ones I have found are also 4 or 5 times the price of any of the others mentioned. But at the end of the day, if it's the only one that works with FreeNAS, then there isn't much choice. Let me know if anyone has tested it.
 

jgreco

Resident Grinch
Joined
May 29, 2011
Messages
18,681
I think having only 2 lanes per card defeats the purpose of having such fast NVMe cards, along with the fact that the Supermicro card is so much harder to find and also the ones I have found are also 4 or 5 times the price of any of the others mentioned. But at the end of the day, if it's the only one that works with FreeNAS, then there isn't much choice. Let me know if anyone has tested it.

Harder to find? Well Supermicro doesn't have a ton of retail outlets but it's available at some of the common SM retailers, WiredZone etc.

Two lanes isn't a serious issue for a NAS. That's still about 2GBytes/sec per SSD, and that means you could be blasting data from a single SSD out two 10Gbps ethernet ports and only be barely grazing that limit. I've got faster NVMe SSD's than that, but in practice even in a non-NAS situation they rarely seem to outrun 2GBytes/sec, so this may not be a showstopper.

More important is that it's an x8 card. The more common x16 cards are ~~stupid for a NAS because there's no way you're going to take advantage of that capacity, and many server boards don't even have an x16 slot.

Sorry to say that I haven't had a reason to test that particular card.
 

maniyer

Dabbler
Joined
Jun 26, 2019
Messages
19
The one place you did mention, WiredZone, don't have the SuperMicro card in stock, sorry. I've found it on eBay for $200, but that's a lot more more money than the other cards mentioned above. I understand the speeds probably won't make much difference, even at 10Gbps ethernet, but since the technology is available, I felt it was worth exploring. The bottom line is compatibility. No one has come forward who has tested any of the mentioned cards, Supermicro or any other for that matter.
 

maniyer

Dabbler
Joined
Jun 26, 2019
Messages
19
I have been in conversation with ASRock, and this is what they have to say so far:

Me:

Does this board (x5870m Pro 4) support 4x4x4x4 bifurcation on it's primary PCIe 16x connector. I'm looking to purchase one of the quad m.2 adaptor card and wanted to make sure all 4 m.2 cards are recognised before I start purchasing all the components. Will having cards in the other three slots affect this? Is there a CPU requirement for this to work? The idea is to use a 1x slot for a cheap graphics card so the CPU doesn't require graphics integration. Will using the other 16x slot (4x) affect the first slot? Any further information will be appreciated. Where do I change this in the BIOS if necessary?

ASRock:

Hello,
Good news!
X570 in combination with a Matisse CPU can support x4/x4/x4/x4 bifurcation.
The Ultra Quad M.2 Card is also supported, and you can create a bootable RAID J
Please set BIOS\Advanced\AMD PBS\NVMe RAID mode to Enabled
Then set PCIe x16 Switch to 4X4
After saving these settings you can enter BIOS setup again, go to Advanced, and then RAID2Xpert

Me:

When using the ultra quad m.2, will the drives be recognised and usable without using RAID? I will be using software level raid (ZFS FreeNAS), so want the drives to be recognised individually to avoid any issues.

ASRock:

Hello,
FreeNAS is not officially supported. Maybe you can check with the community if this will work with X570 & Zen2.
I did run a test with Windows 10 version 1903 today. NVMe RAID mode must be activated in order for the 4x4 bifurcation option to be available. Once this was set, all 4 M.2 drives were available as individual drives in Windows.
 

maniyer

Dabbler
Joined
Jun 26, 2019
Messages
19
Once this was set, all 4 M.2 drives were available as individual drives in Windows.

At least for me, this sounds promising, but not sure until someone has actually tested one of these cards.
 

racom

Cadet
Joined
Sep 11, 2014
Messages
1
I've got bifurcation working with ASRock X570m Pro4.
https://www.asrock.com/mb/AMD/X570M Pro4/

1x16 -> 4x4

The setting / support for bifurcation was introduced with BIOS version 2.21 2019/11/25.
The board also supports ECC memory (unbuffered!).

I use it now in combination with this card.
https://www.asrock.com/mb/spec/product.asp?Model=ULTRA QUAD M.2 CARD

as far as I can tell, this feature is operating system agnostic.

Pictures:
1575880149602.png


1575880167877.png
 

bvalente

Cadet
Joined
Feb 17, 2020
Messages
1
This totally saved my bacon on this asrock ultra quad m2 setup - thx

instructions for setting up that card are non-existent
 

VAB

Cadet
Joined
Apr 5, 2020
Messages
2
Sorry to jump into your dialoge but my case is different (at least in HW used).
I have a Arock x299 Taichi xe on which I installed in the PCIe 3 the ultra quad M2 card.
The uility provided by Asrock works, allowing me to controll the fan and deteting the goutr SSD M2 I put on the card.
But when I go to the Windows "computer management" utility, non memory is shown and I cannot use them at all.
I put on the card four Samsung 970 EVO (a couple of the are 970 EVO Plus).
Any suggestion to have this solved
Thanks
 

VAB

Cadet
Joined
Apr 5, 2020
Messages
2
BTW my motherboard bios does show any ADVANCED\AMD PBS screen (obiously being it equipped with an intel CUP)
my CPU is a i9 9960
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,702
But when I go to the Windows "computer management" utility,
I think you came to the wrong forum.

Were you planning to use FreeNAS?
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,175
I'll keep those posts here, even if they are strictly speaking off-topic.

From what I can tell, the ASRock Ultra quad M.2 adapter is a quirky little thing. It's a standard passive 4x PCIe x4 PCIe to M.2 adapter as far as the SSDs are concerned (although the power delivery circuitry seems excessive for a handful of SSDs and a crappy little fan, you'd think the slot would suffice...). The fan, however, seems to be controlled by a Nuvoton IC of some description that hangs off the SMBus that happens to exist in the PCIe slot (might be pulling double duty for PCIe lane configuration stuff and fan control) - since different motherboards may have devices at certain SMBus addresses, they add some dip switches to choose a different address. A very 80s approach to something that deals in very fast storage...

Anyway, VAB, what matters to you is this: Fan control is completely independent of the SSDs working. It is very likely that your motherboard does not support the requisite bifurcation (tetrafurcation?) of the x16 slot into x4 OR you need to manually configure it in the BIOS setup (see example in post #15 above) OR the card identifies itself in a way which the system firmware does not like (in which case, the setup option might still help).
 
Top