Thinking about building my own NAS

[StrangeOne]

Thanks a mill for a the replies! Really appreciate them!

I'd like to get within $/£/€3000 (including the drives).

You have a good budget then. Granted your hardware is likely more expensive depending on your country.

If you look at my system in signature. It only pulls 112 watts idle and peaks during loads around 250. I consider that pretty low power lol

Don't be scared of used enterprise equipment. But yes you should caution on who you buy it from.

I forgot to mention the power supplies are included in the case price I factored in. The supermicro case is usually paired with 2x920W platinum PSUs. I only leave one in since the second one is failover it does increase the power draw by 15 watts or so. So no point leaving it in.

Your main issue is limitations on what you can find in your country without paying absorbents amount of import taxes. ('m sure you know already lol)

- HBA (LSI 9207-8i - maybe 16i or another 8i in the future if needed): I am running the LSI 9207-8i myself. I did initially use it for breakouts to SATA for 8 SSDs in an older project but now its in my main rig attached to the SAS expander. This is where you want to see what your future plans are. Depending your hardware choices, you might not have the ability to add in another card.

10Gbe NIC- Stick with Intel if you can, mellonox is good as well.. An X520 or X540 works great with Unifi lineup. I run their switches and self host the unifi controller.

 

[Etorix]

I'd like to get within $/£/€3000 (including the drives).

This is going to fit. And you might be able to stretch to 2000 for the drives.

So for 1500, I could have roughly:
- 4x 20TB (80TB raw, but 50TB in RAIDZ1 or 40TB in mirror)
- 6x 12 (70TB raw, almost 50TB win RAIDZ2)

For safety, go for raidz2 over mirrors.
I count (all raidz2):
4*20 TB = 40 TB raw / 32 TB usable (80%)
6*12 TB = 48 TB raw / 40 TB usable
This shows that, while 20 TB may be the best price per TB per drive, the higher space efficiency of a wider vdev wins.
Think of the right width to expand your pool later with multiple vdevs, possibly leaving one or two slots free for spares or replacements, and see what it could give for 2000.

Let's put some numbers—taken from wherever is convenient for me, since I do not know what's the relevant market for you.

UDIMM:
E3C246D4U-2T from $250 (open box from China), add an inexpensive Core i3-8100/9100 (Xeon E-2100/2200 for more cores and/or >64 GB RAM)
X470D4U from 310 E (new), add your choice of ECC-supporting Ryzen CPU
in all cases, add 32-64 GB of ECC UDIMM, and a -8i HBA when needed (later)

RDIMM:
EPYC/Scalable opportunities will best come as complete servers, bought locally due to weight. @StrangeOne already provided a quote.
X10SDV-4C-7TP4F from $380 (used) low power, price includes CPU, 16i HBA and 2*SFP+ all onboard (with still 4*SATA and up to 5*M.2, with cheap bifurcating adapters in PCIe slots) — add 64-128 GB of cheap(*) second-hand RDIMM "because you can", a 60 mm fan on the CPU heatsink and something on the HBA heatsink and you're all set
(*) "cheap" as in 16 GB for $12 apiece, $48 for 64 GB

The issue with 'old' hardware I have is:
- if you buy used server equipment (from a dismantler, recycling centre or some random seller from China) - there is a risk that something won't work (even if the board itself is better quality and has better lifespan than a desktop motherboard) and any returns is painful. I don't have any contacts in this area that could lend me a board for a month and I'd buy it if I'm happy then. With new desktop boards there is a warranty that makes my life easier. New server stuff is expensive.

And that's why we love refurbished or second-hand from friendly forums… Refurbished servers have worked for years; if they are not damaged in shipping, they will just keep working. When they eventually break, you treat them as expendable and buy something else.
Or you can still buy a new X10SDV-4C-7TP4F for 1000 E, with all accessories and full warranty.

 

[ChrisRJ]

The issue with 'old' hardware I have is:
- if you buy used server equipment (from a dismantler, recycling centre or some random seller from China) - there is a risk that something won't work (even if the board itself is better quality and has better lifespan than a desktop motherboard) and any returns is painful.

Adding to what @Etorix wrote, the risk of a new board/CPU/RAM not working is actually higher, because they are not "burnt in".

 

[LJ247]

Thanks for making me even more confused now :) haha
But honestly, I think I'm now in a much better place than I was a week ago.

RAM/CPU: definitely ECC now (learnt that ZFS scubbing may kill the pool if a bit in RAM gets corrupted) so all the desktop i5 or i7 are out :)

CPU:
Should I worry about Xeon D/E/W/E3 or is this irrelevant in my case of light use? For a price of 'up to €300-350' I can get (specs from Intel):
- Xeon E-2356 (2021, 3.2GHz, 6c/12t, LGA1200, 128GB 3200MHz, 20 PCIe lanes)
- Xeon E-2224 (2019, 3.5GHz, 4c/4t, LGA1151, 128GB 2666MHz, 16 PCIe lanes)
- Xeon E-2174 (2018, 3.8GHz, 4c/8t, LGA1151, 128GB 2666MHz, 16 PCIe lanes)
- Xeon W-1250 (2020, 3.3GHz, 6c/12t, LGA1200, 128GB 2666MHz, 16 PCIe lanes)
- Xeon E3-1275V6 (2017, 3.8GHz, 4c/8t, LGA1151, 64GB 2400MHz, 16 PCIe lanes)
- Intel Pentium G4600 (2017, 3.6GHz, 2c/4t, LGA1151, 64GB 2400MHz, 16 PCIe lanes) - low power, cheap

Motherboards. Well, there is a lot to pick from, but all depends on the above. So for 'up to €400-450' I can get:
- ASRock E3C246D4U (C246, PCI x16/8/4, M.2, Xeon E2100/E2200, 128GB 2666MHz, 2x 1 GbE Intel i210)
- ASRock E3C246D4M-4L (C246, PCI x16/8/1/1/1, M.2, Xeon E2100/E2200, 2666MHz, 4x 1 GbE Intel i210/i219)
- ASRock E3C242D4U2-2T (C242, PCI x16/8/1), M.2, Xeon E2100/E2200, 128GB 2666MHz, 2x 10 GbE Intel X550-AT2)
- ASRock E3C252D4U (C252, PCI x16/4/1), M.2, Xeon E2300, 128GB 3200MHz, 2x 1 GbE Intel i210)
- ASRock W480D4U (W480, PCI x16/8/1), M.2, Xeon W1200/W1300, 128GB 2666MHz, 2x 1 GbE Intel i210)
- Supermicro X12SAE (W480, PCI x16/16/4/1, 2x M.2, Xeon W1200, 128GB 2666MHz, 1x 1 GbE + 1x 2.5 GbE (Intel i219/i225)
- Supermicro X11SCA-F (C246, PCI x16/16/4/1, 2x M.2, Xeon E2100/E2200, 128GB 2666MHz, 2x 1 GbE (Intel i219/i225)
- Supermicro X11SSH-F (C236), PCI x8/8/4, M.2, Xeon E1200, 64GB 2400MHz, 2x 1 GbE Intel i210)

I understand that 2x PCIe x16 means: x16/none or x8/x8.
I don't care about the onboard LAN (maybe except of the 2x 10Gb on ASRock E3C242D4U2-2T) - I'm going to put a 10Gb Intel NIC anyway.
I don't care about onboard SATA ports as I'm going to get LSI 9207 HBA anyway.
All the above support 64GB of RAM or more - also good for my use.
All the above have M.2 for a nice boot drive.

HBA:
- LSI 9207-8i (because it already has IT mode, PCIe gen3.0, 8 lanes)
- 16 port HBAs are expensive, so for future expansion I'd probably look for 2x 8 HBA (1x 8 now for the 6/8 HDD setup (thanks for convincing me! and the second one for future itilisation of my current desktop 4x3TB in a separate pool (or adding more disks)).
- having 2 HBAs I will need a motherboard with PCIe slots for 16/8 - this eliminates ASRock E3C252D4U
- by adding Intel's X520 NIC I'm out of PCI slots on any of the boards (or I could stick it to the Supermicro X12SAE PCI x4 port (it looks like the slot doesn't have end-cap, so the x8 card will be sticking out and running on half the lanes).

Now, looking at what I wrote, comparing the items and thinking out loud, looks like @Etorix suggestion of ASRock E3C246D4U2-2T was the best option as I could use the built-in 10GbE and use PCI slots for 2x 8-port HBA :)

Thanks again and regards

 

[Etorix]

You seems to be on a good track to make an informed decision.

Two remarks: LSI 9200 are actually PCIe 2.0 cards (but for HDDs they are fine), and you should actually care about SATA ports from the motherboard because 8 of them can save you from adding a second HBA, freeing a PCIe slot.

 

[LJ247]

Thanks @Etorix, but I found this (it says that the 9207 is a PCIe 3.0, even if most of the boards have the same controller :)):

hba_features - ServeTheHome

LSI HBA features (click to enlarge)

www.servethehome.com

The 'I don't care' was more likely it is not a key decision point. The more I get the better, of course, but I wouldn't ignore a good board (that would suit me perfectly) only because it has 4 SATA ports.