New build - am I overdoing it?

[calgarychris]

Hi,

I'm looking to build a new system as my Xeon E3-1231 system is about 8 years old and running at 77% full. I figure this would be a good time to try SCALE as I've only played around in CORE. Currently my system runs 3 jails for various bits and pieces (Sabnzbd etc., Plex, and Nextcloud) and I occasionally will set up a VM to play around in a new OS. The system is only for home use and has never batted an eye with cpu never going above 25% I'd say (probably lower tbh). What I'm looking at is as follows:

Supermicro X11SPL-F
128GB (4x 32GB DDR4-2666 ECC RDIMM)
Intel Xeon SILVER 4208 (BX806954208)
6 x WD Red Plus 10TB 3.5' NAS HDD SATA3 7200RPM
450-550W PSU (recommendations? Is it enough?) - my budget might be around $150 AUD ($100USD)
OS running on 2x Kingston SA400S37/240G AS400SSD 2.5inch 7mm SATA3 SSD
Fractal Design 7 XL case with 4-6 fans

The above was chosen primarily because it seems *really* difficult to get Supermicro boards here in Australia so my choices seem to be fairly limited (global skus, prices are outrageous). I'd like to future proof a little bit and ideally get the same amount of life out of this rig as the last one - the large case would allow for future expansion. I would simply put more drives in my current setup but 1) the case is too small 2) I don't have a backup system so this would give me an opportunity to move the old one to backup.

Is this total overkill? Any issues with the above setup?

Thanks

 

[samarium]

Interesting, I'm upgrading a E3-1231 X10SLM-F system too.

CPU/MEM up to you really, sounds OK. -F model good idea, IPMI. I picked up a EYPC 7302P and a SMC H11SSL-F motherboard second hand USD450, took the importing risk, and already had 256GB of LRDIMM albeit slow 2133, that I was planning on using in a X10SRL if I could find one. I wanted the more cores and more memory and more pcie slots and more pcie lanes. The X10SLM was always limiting, and I regretted not spending more money then and getting an E5 and a more expandable motherboard, back in the days when it wasn't that much more expensive, or I could have a case of rose colored rear vision goggles.

My existing Seasonic 550W M12 PSU is about 12 years old, so preemptively upgrading to a Gold 750W, bigger as maybe will get GPU for transcode/AI, and it will suck more power, and I it would be prudent to get a new PSU anyway given the age of my old one.

I'm just going to transplant the motherboard and a new PSU into the existing case, a Fractal Design Define R2, and reuse everything else, and slowly upgrade the disks, 1/year, so pool size will increase slowly. I have a Raspberry Pi running some external disks for nightly backup zfs send/recv, and it works. I am selective in my backups. Lots of stuff that doesn't really matter, as long as I have metadata to rebuild/redownload.

I agree SMC kit is ridiculously expensive in Oz, but so are most components. Check the second hand markets, especially overseas.

I'm trying to limit my HDDs to 3x mirror, sounds like you want more space. Not sure about the SSDs, make sure there aren't too many online horror stories for the model or relevant firmware, and getting a 2 different brands might protect you better, less chance of a bad batch or bad firmware, so look for different flash controller chips too I guess.

Some expansion room is good, but maybe flash will start to dominate and you won't need that much space, but probably not in 5 years I guess, not for reasonable prices anyway. If I could get 8TB flash for AUD200 ... would be time to start retiring HDDs I think.

Check the URE on the disks, you want 10^-15, not 10^-14, and 10TB are probably CMR not SMR, but check. You might want to check Annual TBW, IIRC someone was saying WD was making life difficult. You might like to canvas the other drive options if you are just going WD by habit. I assume you are RAIDZ2 4D+2P.

If you are adding fans, check out Arctic, good price/noise/performance.

Seems a fair middle of the road NAS server, and the memory will be useful for ARC and apps.

 

[calgarychris]

My existing Seasonic 550W M12 PSU is about 12 years old, so preemptively upgrading to a Gold 750W, bigger as maybe will get GPU for transcode/AI, and it will suck more power, and I it would be prudent to get a new PSU anyway given the age of my old one.

I'm just going to transplant the motherboard and a new PSU into the existing case, a Fractal Design Define R2, and reuse everything else, and slowly upgrade the disks, 1/year, so pool size will increase slowly. I have a Raspberry Pi running some external disks for nightly backup zfs send/recv, and it works. I am selective in my backups. Lots of stuff that doesn't really matter, as long as I have metadata to rebuild/redownload.

What are you running on your pi? I've got an Rpi3 that I could use for something similar, with lower power consumption I guess....

I agree SMC kit is ridiculously expensive in Oz, but so are most components. Check the second hand markets, especially overseas.

It's killing me. And just when I think I can find a decent board, the memory is extortionist...

Check the URE on the disks, you want 10^-15, not 10^-14, and 10TB are probably CMR not SMR, but check. You might want to check Annual TBW, IIRC someone was saying WD was making life difficult. You might like to canvas the other drive options if you are just going WD by habit. I assume you are RAIDZ2 4D+2P.

Yeah thanks - good thought. I got burnt on my last WD as we discovered the SMR problem after I bought the replacement drive. I think my current ones are Hitachi's from awhile back and I had two with issues off the bat. I've just replaced the WD SMR with a WD CMR drive, so time will tell if that one creates any other issues.

If you are adding fans, check out Arctic, good price/noise/performance.

Seems a fair middle of the road NAS server, and the memory will be useful for ARC and apps.

Thanks for the tip - I'll look into them. I'm going for middle of the road as I suspect my requirements are way below most people's so this'll give some room for future proofing from my perspective.

 

[samarium]

What are you running on your pi? I've got an Rpi3 that I could use for something similar, with lower power consumption I guess....

Pi4 4GB, ubuntu zerver, runs ZFS fine, and I use it as a ZFS replication target with a USB connected multi disk pack, which powers down all day until backups arrive at midnight. Not 100% happy with USB, would prefer eSATA, but that both requires the Pi to have a SATA adapater, and one that support Port Multiplier, and I don't like PM that much either ... probably would like a USB UAS multidisk enclosure over a USB 3.2 2x2 link or whatever it is called today, ie 20Gb/s, maybe USB4, either way not Pi4 compatible with only a PCIe V2.0 x1 link if you swapout the USB chip. So I'll settle for the USB 3.0 connection I have, it works OK for the 3 disks in the mirror in this enclosure. I have another disk which is part of the backup zpool, which I keep offline, and sync it up every 3 months or so, maybe by connecting both the pack and the extra disk to my server, or maybe I make the extra disk available over the network as a network block device using qemu-img. Not sure if you could do with TN anyway, maybe iSCSI would work. Anyway, I have a working solution, with some backup, and some offsite capability. I previously had a mini TV box with an Atom processor and 2GB RAM, and it just didn't work out as a ZFS backup target. TN replicates to the Pi4 with only one issue, which is it mounts the resultant file systems, and I want them unmounted on the backup server, so I have to follow along with a cron job and clean up it's messes, TN zettarepl should just leave the datasets in the state they were before it came and opened the gate.

It's killing me. And just when I think I can find a decent board, the memory is extortionist...


Yeah thanks - good thought. I got burnt on my last WD as we discovered the SMR problem after I bought the replacement drive. I think my current ones are Hitachi's from awhile back and I had two with issues off the bat. I've just replaced the WD SMR with a WD CMR drive, so time will tell if that one creates any other issues.


Thanks for the tip - I'll look into them. I'm going for middle of the road as I suspect my requirements are way below most people's so this'll give some room for future proofing from my perspective.

RAM you can probably get cheaper on taobao, aliexpress, ebay etc, with all the normal caveats, so YMMV. I got my RAM as part of a E5-2660v4 + cheap Chinese X99 motherboard, which was meant to be a stop gap until I found an X10SRL at a reasonable price, then the EYPC option came up, and I jumped at that option, since I could use the RAM. Time will tell if I succumb to speed greed and upgrade to 2933 RAM. Should keep me going for another 5 years or more either way.