U.2 drives noob questions

[Sasquatch]

I was told my 6x hdd was too loud and admittedly it was audible at night in adjacent bedroom(through the wall!)
switched to 2x HDD and its much better, but capacity is running low and im eying 4x 8TB all flash options.
used 7.68 TB u.2 drives cost about th same as qvo 8TB with approx 10x more life in them write-wise.

I dont need nvme speed, but enterpise/server grade sata SSDs in 8TB size range are not availiable used
nand new are out of my budget range.
Now the questions:
Is there u.2 drive that will operate under SATA? my options are intel dc p4510 or koxia CD6,
I bought cheap samsung pm983 but it won't get recognized when pulgged into SATA backplane with SAS connectors.

 

[Ericloewe]

Is there u.2 drive that will operate under SATA?

No, of course not. The whole point is to use PCIe. In fact, they're incompatible at every layer, from the connector to the protocol they speak.

cheap samsung pm983 but it won't get recognized when pulgged into SATA backplane with SAS connectors.

Please don't do that, you may damage the connectors on either end.

 

[sretalla]

In order to connect those, you would need a Tri-mode HBA like this one:

HBA 9500-8i Tri-Mode Storage Adapter

Designed for enterprise-level storage enclosures or servers, the 9500-8i PCIe Gen4 Tri-Mode HBA is ideal for increased connectivity and maximum performance for data center flexibility.

www.broadcom.com

https://docs.broadcom.com/doc/BC-0510EN

And the appropriate cable(s) for the U.2 drives.

 

[Sasquatch]

Yes I've seen 9500-8i but I struggle to see how it can serve 32x nvme drives through one sff8654, backplane with multiplexer is used?
Also can't find cable sff8654 -> 8x sff8639 for it...

I actually considered this: https://www.aliexpress.us/item/3256....0.0.c6d638daVXWwzn&mp=1&gatewayAdapt=glo2usa
with sff8654 to 2x sff8639 cables. but as with all things Chinese quality and reliability remains a big question.

there is also https://www.supermicro.com/en/products/accessories/addon/AOC-SLG3-8E2P.php
with https://store.supermicro.com/supermicro-55cm-oculink-to-u-2-pcie-with-power-cable-cbl-sast-0956.html
might be actually better idea IF i find below $300, lol

 

[sretalla]

I don't have personal experience with these in use together with TrueNAS, but as I understand it, you're effectively "switching" PCI to allow oversubscription of the PCI slot beyond the available lanes.

The device you linked to may be able to do the same (but it seems to mention slot bifurcation, so maybe not).

Don't feel like you need to be limited by the PCI lanes if you're using the proper switching card like the LSI one... you're extremely unlikely to be doing anything with data that can't arrive or leave on 16 lanes in total in reasonable time since you can't push or pull it in or out with a NIC that's only got a maximum of 16 lanes too.

 

[Ericloewe]

In order to connect those, you would need a Tri-mode HBA like this one:

HBA 9500-8i Tri-Mode Storage Adapter

Designed for enterprise-level storage enclosures or servers, the 9500-8i PCIe Gen4 Tri-Mode HBA is ideal for increased connectivity and maximum performance for data center flexibility.

www.broadcom.com

https://docs.broadcom.com/doc/BC-0510EN

And the appropriate cable(s) for the U.2 drives.

You probably don't want a tri-mode HBA - the disks end up exposed as SCSI disks, so you can throw many of the latency benefits out the window. Also, the cost is ludicrous. Also, you need a tri-mode expander to use more than two/four disks, which is priced up to plaid.

The only reasonable option, in my view, is to go straight-up PCIe. If you have enough lanes available on your system, the adapters are relatively simple, though you may need retimer cards instead of passive ones, depending on cable lengths. If you don't have enough lanes, you would need PCIe switches, such as the AOC-SLG3-8E2P you linked.

 

[Sasquatch]

You probably don't want a tri-mode HBA - the disks end up exposed as SCSI disks, so you can throw many of the latency benefits out the window. Also, the cost is ludicrous. Also, you need a tri-mode expander to use more than two/four disks, which is priced up to plaid.

The only reasonable option, in my view, is to go straight-up PCIe. If you have enough lanes available on your system, the adapters are relatively simple, though you may need retimer cards instead of passive ones, depending on cable lengths. If you don't have enough lanes, you would need PCIe switches, such as the AOC-SLG3-8E2P you linked.

exactly my thoughts, sadly i have only one x16(bifubication possible to x4x4x4x4)
and 2x m2 x4 slots available after 10G NIC(x4 slot) and transcoding GPU(x1 slot) are installed.
so its multiplexer/switch card or 6x instead of 8x wide pool

 

[Davvo]

6x is plenty, your initial idea was using 4x SSDs; contrary to HDD practice, a NVMe pool of 4 drives in RAIDZ1 is safe enough.
Use something like this in the x16 slot and you are done without a multiplexer.​

 

[Sasquatch]

I don't have personal experience with these in use together with TrueNAS, but as I understand it, you're effectively "switching" PCI to allow oversubscription of the PCI slot beyond the available lanes.

The device you linked to may be able to do the same (but it seems to mention slot bifurcation, so maybe not).

Don't feel like you need to be limited by the PCI lanes if you're using the proper switching card like the LSI one... you're extremely unlikely to be doing anything with data that can't arrive or leave on 16 lanes in total in reasonable time since you can't push or pull it in or out with a NIC that's only got a maximum of 16 lanes too.

Both cards do the same the AliExpress one says motherboard bifubication not needed and it uses briadcom multiplexer/switch chip.
Yes even with 40G NIC 8 lanes(3.0) is more than enough to saturate it.

 

[Sasquatch]

6x is plenty, your initial idea was using 4x SSDs; contrary to HDD practice, a NVMe pool of 4 drives in RAIDZ1 is safe enough.
Use something like this in the x16 slot and you are done without a multiplexer.​

Thing is with 4x SSD I get 15TB usable storage in mirrored or z2 pool. u.2 ssds above 7.68 tb are still prohibitively expensive per tb for me.
An I'm running out of space on 2x 18tb HDD pool
If I get 8x I'll have 45TB and possibly to double it in 5 years when 16TB u.2 prices drop.

 

[Davvo]

Confused QVOs for m.2 drives; they are SATA. The AliExpress link you posted doesn't work anymore.

My bad, please continue with your suggested U.2 path.

 

[Sasquatch]

Confused QVOs for m.2 drives; they are SATA. The AliExpress link you posted doesn't work anymore.

My bad, please continue with your suggested U.2 path.

No probs, m.2 to u.2 adapter cables would work though so not a bad shout.

They won't ship to Europe anymore, if you pretend to be in the us it still works ;).

 

[firesyde424]

In my experience, the tri-mode adapters are an exercise in too many compromises. As far as enterprise grade flash drives, unless you are after things like power loss protection(which you shouldn't need if your pool is configured properly) or you are consistently writing tremendous amounts of data, there's really not much benefit to the average home user for the extra cost.

If you don't need the performance of an NVME drive, there are any number of cheap SATA SSDs that can get you the capacity you need for far less money. The Samsung QVO drives you mentioned are one example. Teamgroup and Silicon Power both make cheap SATA SSDs as well, with Crucial not far behind.

 

[Sasquatch]

In my experience, the tri-mode adapters are an exercise in too many compromises. As far as enterprise grade flash drives, unless you are after things like power loss protection(which you shouldn't need if your pool is configured properly) or you are consistently writing tremendous amounts of data, there's really not much benefit to the average home user for the extra cost.

If you don't need the performance of an NVME drive, there are any number of cheap SATA SSDs that can get you the capacity you need for far less money. The Samsung QVO drives you mentioned are one example. Teamgroup and Silicon Power both make cheap SATA SSDs as well, with Crucial not far behind.

I do agree I do not need the nvme performance, with 10Gbe NIC 8 wide z2 pool will saturate it with spinners on read. Write wise Sata ssds should give me half the bandwidth. But.
I can get 7.68tb koxia cd6 with 2PB written( about eight of rated life) for £280 or intel DC p4510 for £300 with well over 10PBW left. Cheapest new consumer grade data SSD would set me back £340. Qvo being the cheapest. Temgroup has 8 TB Sata on offer but only in tlc at £600+.
If I go with kioxia I would have £480 to spend on cabling and controller to get full name storage at the same price point as QVOs, and 8TB qvo is rated only at 2.8PBW.
On a plus side I could saturate 40GB NIC with nvme pool both read and write....no need for it but I could ;)

Do you see my point now? Or am I crazy anyway?

Edit:
Temgroup 8TB is £800+ and qvo offer ended and is £450+
And even though 8x 4TB would be 6tb larger then current pool I would most likely run out in a year or so

 

[firesyde424]

Hol

I do agree I do not need the nvme performance, with 10Gbe NIC 8 wide z2 pool will saturate it with spinners on read. Write wise Sata ssds should give me half the bandwidth. But.
I can get 7.68tb koxia cd6 with 2PB written( about eight of rated life) for £280 or intel DC p4510 for £300 with well over 10PBW left. Cheapest new consumer grade data SSD would set me back £340. Qvo being the cheapest. Temgroup has 8 TB Sata on offer but only in tlc at £600+.
If I go with kioxia I would have £480 to spend on cabling and controller to get full name storage at the same price point as QVOs, and 8TB qvo is rated only at 2.8PBW.
On a plus side I could saturate 40GB NIC with nvme pool both read and write....no need for it but I could ;)

Do you see my point now? Or am I crazy anyway?

Edit:
Temgroup 8TB is £800+ and qvo offer ended and is £450+
And even though 8x 4TB would be 6tb larger then current pool I would most likely run out in a year or so

If you can get the 7.68TB drives for those prices, I'd say go for it. It's probably going to be a while before you start to see large numbers of 15.36TB NVME drives on the used market.

You might also look for Micron 5200 Eco or Micron 5210 Ion drives. Those are older SATA SSDs and you should start to see larger numbers of those hitting the secondary markets soon, if not already. It might take some patience, but I think you can find what you are looking for.

 

[Sasquatch]

Hol

If you can get the 7.68TB drives for those prices, I'd say go for it. It's probably going to be a while before you start to see large numbers of 15.36TB NVME drives on the used market.

You might also look for Micron 5200 Eco or Micron 5210 Ion drives. Those are older SATA SSDs and you should start to see larger numbers of those hitting the secondary markets soon, if not already. It might take some patience, but I think you can find what you are looking for.

Thanks for the micron tip, wasn't aware of those.