U.2 drives noob questions

Sasquatch

Explorer
Joined
Nov 11, 2017
Messages
87
I was told my 6x hdd was too loud and admittedly it was audible at night in adjacent bedroom(through the wall!)
switched to 2x HDD and its much better, but capacity is running low and im eying 4x 8TB all flash options.
used 7.68 TB u.2 drives cost about th same as qvo 8TB with approx 10x more life in them write-wise.

I dont need nvme speed, but enterpise/server grade sata SSDs in 8TB size range are not availiable used
nand new are out of my budget range.
Now the questions:
Is there u.2 drive that will operate under SATA? my options are intel dc p4510 or koxia CD6,
I bought cheap samsung pm983 but it won't get recognized when pulgged into SATA backplane with SAS connectors.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,194
Is there u.2 drive that will operate under SATA?
No, of course not. The whole point is to use PCIe. In fact, they're incompatible at every layer, from the connector to the protocol they speak.
cheap samsung pm983 but it won't get recognized when pulgged into SATA backplane with SAS connectors.
Please don't do that, you may damage the connectors on either end.
 

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,703
In order to connect those, you would need a Tri-mode HBA like this one:


And the appropriate cable(s) for the U.2 drives.
 

Sasquatch

Explorer
Joined
Nov 11, 2017
Messages
87
Yes I've seen 9500-8i but I struggle to see how it can serve 32x nvme drives through one sff8654, backplane with multiplexer is used?
Also can't find cable sff8654 -> 8x sff8639 for it...

I actually considered this: https://www.aliexpress.us/item/3256....0.0.c6d638daVXWwzn&mp=1&gatewayAdapt=glo2usa
with sff8654 to 2x sff8639 cables. but as with all things Chinese quality and reliability remains a big question.

there is also https://www.supermicro.com/en/products/accessories/addon/AOC-SLG3-8E2P.php
with https://store.supermicro.com/supermicro-55cm-oculink-to-u-2-pcie-with-power-cable-cbl-sast-0956.html
might be actually better idea IF i find below $300, lol
 
Last edited:

sretalla

Powered by Neutrality
Moderator
Joined
Jan 1, 2016
Messages
9,703
I don't have personal experience with these in use together with TrueNAS, but as I understand it, you're effectively "switching" PCI to allow oversubscription of the PCI slot beyond the available lanes.

The device you linked to may be able to do the same (but it seems to mention slot bifurcation, so maybe not).

Don't feel like you need to be limited by the PCI lanes if you're using the proper switching card like the LSI one... you're extremely unlikely to be doing anything with data that can't arrive or leave on 16 lanes in total in reasonable time since you can't push or pull it in or out with a NIC that's only got a maximum of 16 lanes too.
 

Ericloewe

Server Wrangler
Moderator
Joined
Feb 15, 2014
Messages
20,194
In order to connect those, you would need a Tri-mode HBA like this one:


And the appropriate cable(s) for the U.2 drives.
You probably don't want a tri-mode HBA - the disks end up exposed as SCSI disks, so you can throw many of the latency benefits out the window. Also, the cost is ludicrous. Also, you need a tri-mode expander to use more than two/four disks, which is priced up to plaid.

The only reasonable option, in my view, is to go straight-up PCIe. If you have enough lanes available on your system, the adapters are relatively simple, though you may need retimer cards instead of passive ones, depending on cable lengths. If you don't have enough lanes, you would need PCIe switches, such as the AOC-SLG3-8E2P you linked.
 

Sasquatch

Explorer
Joined
Nov 11, 2017
Messages
87
You probably don't want a tri-mode HBA - the disks end up exposed as SCSI disks, so you can throw many of the latency benefits out the window. Also, the cost is ludicrous. Also, you need a tri-mode expander to use more than two/four disks, which is priced up to plaid.

The only reasonable option, in my view, is to go straight-up PCIe. If you have enough lanes available on your system, the adapters are relatively simple, though you may need retimer cards instead of passive ones, depending on cable lengths. If you don't have enough lanes, you would need PCIe switches, such as the AOC-SLG3-8E2P you linked.
exactly my thoughts, sadly i have only one x16(bifubication possible to x4x4x4x4)
and 2x m2 x4 slots available after 10G NIC(x4 slot) and transcoding GPU(x1 slot) are installed.
so its multiplexer/switch card or 6x instead of 8x wide pool
 

Davvo

MVP
Joined
Jul 12, 2022
Messages
3,222
6x is plenty, your initial idea was using 4x SSDs; contrary to HDD practice, a NVMe pool of 4 drives in RAIDZ1 is safe enough.
Use something like this in the x16 slot and you are done without a multiplexer.​
 

Sasquatch

Explorer
Joined
Nov 11, 2017
Messages
87
I don't have personal experience with these in use together with TrueNAS, but as I understand it, you're effectively "switching" PCI to allow oversubscription of the PCI slot beyond the available lanes.

The device you linked to may be able to do the same (but it seems to mention slot bifurcation, so maybe not).

Don't feel like you need to be limited by the PCI lanes if you're using the proper switching card like the LSI one... you're extremely unlikely to be doing anything with data that can't arrive or leave on 16 lanes in total in reasonable time since you can't push or pull it in or out with a NIC that's only got a maximum of 16 lanes too.
Both cards do the same the AliExpress one says motherboard bifubication not needed and it uses briadcom multiplexer/switch chip.
Yes even with 40G NIC 8 lanes(3.0) is more than enough to saturate it.
 

Sasquatch

Explorer
Joined
Nov 11, 2017
Messages
87
6x is plenty, your initial idea was using 4x SSDs; contrary to HDD practice, a NVMe pool of 4 drives in RAIDZ1 is safe enough.
Use something like this in the x16 slot and you are done without a multiplexer.​
Thing is with 4x SSD I get 15TB usable storage in mirrored or z2 pool. u.2 ssds above 7.68 tb are still prohibitively expensive per tb for me.
An I'm running out of space on 2x 18tb HDD pool
If I get 8x I'll have 45TB and possibly to double it in 5 years when 16TB u.2 prices drop.
 

Davvo

MVP
Joined
Jul 12, 2022
Messages
3,222
Confused QVOs for m.2 drives; they are SATA. The AliExpress link you posted doesn't work anymore.

My bad, please continue with your suggested U.2 path.
 

Sasquatch

Explorer
Joined
Nov 11, 2017
Messages
87
Confused QVOs for m.2 drives; they are SATA. The AliExpress link you posted doesn't work anymore.

My bad, please continue with your suggested U.2 path.
No probs, m.2 to u.2 adapter cables would work though so not a bad shout.

They won't ship to Europe anymore, if you pretend to be in the us it still works ;).
 

firesyde424

Contributor
Joined
Mar 5, 2019
Messages
155
In my experience, the tri-mode adapters are an exercise in too many compromises. As far as enterprise grade flash drives, unless you are after things like power loss protection(which you shouldn't need if your pool is configured properly) or you are consistently writing tremendous amounts of data, there's really not much benefit to the average home user for the extra cost.

If you don't need the performance of an NVME drive, there are any number of cheap SATA SSDs that can get you the capacity you need for far less money. The Samsung QVO drives you mentioned are one example. Teamgroup and Silicon Power both make cheap SATA SSDs as well, with Crucial not far behind.
 

Sasquatch

Explorer
Joined
Nov 11, 2017
Messages
87
In my experience, the tri-mode adapters are an exercise in too many compromises. As far as enterprise grade flash drives, unless you are after things like power loss protection(which you shouldn't need if your pool is configured properly) or you are consistently writing tremendous amounts of data, there's really not much benefit to the average home user for the extra cost.

If you don't need the performance of an NVME drive, there are any number of cheap SATA SSDs that can get you the capacity you need for far less money. The Samsung QVO drives you mentioned are one example. Teamgroup and Silicon Power both make cheap SATA SSDs as well, with Crucial not far behind.
I do agree I do not need the nvme performance, with 10Gbe NIC 8 wide z2 pool will saturate it with spinners on read. Write wise Sata ssds should give me half the bandwidth. But.
I can get 7.68tb koxia cd6 with 2PB written( about eight of rated life) for £280 or intel DC p4510 for £300 with well over 10PBW left. Cheapest new consumer grade data SSD would set me back £340. Qvo being the cheapest. Temgroup has 8 TB Sata on offer but only in tlc at £600+.
If I go with kioxia I would have £480 to spend on cabling and controller to get full name storage at the same price point as QVOs, and 8TB qvo is rated only at 2.8PBW.
On a plus side I could saturate 40GB NIC with nvme pool both read and write....no need for it but I could ;)

Do you see my point now? Or am I crazy anyway?

Edit:
Temgroup 8TB is £800+ and qvo offer ended and is £450+
And even though 8x 4TB would be 6tb larger then current pool I would most likely run out in a year or so
 
Last edited:

firesyde424

Contributor
Joined
Mar 5, 2019
Messages
155
Hol
I do agree I do not need the nvme performance, with 10Gbe NIC 8 wide z2 pool will saturate it with spinners on read. Write wise Sata ssds should give me half the bandwidth. But.
I can get 7.68tb koxia cd6 with 2PB written( about eight of rated life) for £280 or intel DC p4510 for £300 with well over 10PBW left. Cheapest new consumer grade data SSD would set me back £340. Qvo being the cheapest. Temgroup has 8 TB Sata on offer but only in tlc at £600+.
If I go with kioxia I would have £480 to spend on cabling and controller to get full name storage at the same price point as QVOs, and 8TB qvo is rated only at 2.8PBW.
On a plus side I could saturate 40GB NIC with nvme pool both read and write....no need for it but I could ;)

Do you see my point now? Or am I crazy anyway?

Edit:
Temgroup 8TB is £800+ and qvo offer ended and is £450+
And even though 8x 4TB would be 6tb larger then current pool I would most likely run out in a year or so
If you can get the 7.68TB drives for those prices, I'd say go for it. It's probably going to be a while before you start to see large numbers of 15.36TB NVME drives on the used market.

You might also look for Micron 5200 Eco or Micron 5210 Ion drives. Those are older SATA SSDs and you should start to see larger numbers of those hitting the secondary markets soon, if not already. It might take some patience, but I think you can find what you are looking for.
 

Sasquatch

Explorer
Joined
Nov 11, 2017
Messages
87
Hol

If you can get the 7.68TB drives for those prices, I'd say go for it. It's probably going to be a while before you start to see large numbers of 15.36TB NVME drives on the used market.

You might also look for Micron 5200 Eco or Micron 5210 Ion drives. Those are older SATA SSDs and you should start to see larger numbers of those hitting the secondary markets soon, if not already. It might take some patience, but I think you can find what you are looking for.
Thanks for the micron tip, wasn't aware of those.
 
Top