Critique My Build and Questions (power, motherboard type)

[path]

This is the new build I'm trying to put together and I've purchased most of what I've need. Please do give me some critique

https://pcpartpicker.com/list/37BMJf

The purpose is mostly to act as a home media server and run plex or jellyfin (and hopefully get some hw transcoding).

Some questions
1. Do I have enough power with a 650W PSU? Should I be going bigger?
2. The Xeon W-1250 and W-1290 seem similarly priced. How do I choose between them if my machine is idle a lot. Is it mostly about the power when its idle?
3. The motherboard I've selected is hard to find. I see https://www.newegg.com/asrock-rack-...h-gen-intel-core-processors/p/N82E16813140049 which is quite similar (except for ipmi). Are there any other tradeoffs here?
4. Anything else i'm missing?

Thanks in advance!

 

[Heracles]

Anything else i'm missing?

Backups ? How do you intend to do your backups ? No single server, TrueNAS or other, can be more than a single point of failure.

About your drives, you have 4x 10TB and 4x 14TB. That means either :
4 mirrors (good performance and redundancy but may not be ideal for media server)
2x RaidZ1 (highly discouraged with drives that big)
2x RaidZ2 (as much space lost as mirrors and 2 vDev less but a little better redundancy)
1x RaidZ2 (14TB drives will be used as 10TB drives only ; wasting a lot of space)

So none of these design sounds great. I would go with either 8x 10TB or 8x 14TB and a single RaidZ-2 vdev for a media server.

 

[path]

Backups ? How do you intend to do your backups ? No single server, TrueNAS or other, can be more than a single point of failure.

Yes I'm still figuring out the backups story. Im thinking of getting a lower power nas sitting in a friends house for now.

About your drives, you have 4x 10TB and 4x 14TB. That means either :
4 mirrors (good performance and redundancy but may not be ideal for media server)
2x RaidZ1 (highly discouraged with drives that big)
2x RaidZ2 (as much space lost as mirrors and 2 vDev less but a little better redundancy)
1x RaidZ2 (14TB drives will be used as 10TB drives only ; wasting a lot of space)

So none of these design sounds great. I would go with either 8x 10TB or 8x 14TB and a single RaidZ-2 vdev for a media server.

I've been using the 4X10TB so far as mirrored vdevs. So my plan was to extend it and treat it as mirrored vdevs and give me a pool of 44TB usable space.

 

[Heracles]

I've been using the 4X10TB so far as mirrored vdevs. So my plan was to extend it and treat it as mirrored vdevs and give me a pool of 44TB usable space.

Ok. So mirrors are certainly good. So 2x 10TB mirrors + 2x 14TB mirrors = 48TB. You did not specified how much space you need day 1 neither how much you expect to add over time. It is million time easier and more efficient to put all the space you need day 1. Also, avoid going over 80% and it would be ideal to stay below 50%. Some people here lost their data when their pool reached 100%, so be sure not to join them.

 

[path]

Ok. So mirrors are certainly good. So 2x 10TB mirrors + 2x 14TB mirrors = 48TB. You did not specified how much space you need day 1 neither how much you expect to add over time. It is million time easier and more efficient to put all the space you need day 1. Also, avoid going over 80% and it would be ideal to stay below 50%. Some people here lost their data when their pool reached 100%, so be sure not to join them.

Good to know. I think if I start getting to this being full, I'll probably swap out some drives if this becomes an issue. :)

Any other concerns you see?

 

[Heracles]

32G of RAM is enough but on the low side. It is non-ECC, so not ideal. RAM is the only place where ZFS does not ensure integrity, so it is usually good to have ECC RAM to ensure integrity there.

7200 RPM are more noisy for not such a great speed increase. If noise is a concern for you, be careful here...

As for the power supply, it should be enough according to a simulator I used but that one did not offered to simulate your exact setup.

 

[path]

32G of RAM is enough but on the low side. It is non-ECC, so not ideal. RAM is the only place where ZFS does not ensure integrity, so it is usually good to have ECC RAM to ensure integrity there.

7200 RPM are more noisy for not such a great speed increase. If noise is a concern for you, be careful here...

As for the power supply, it should be enough according to a simulator I used but that one did not offered to simulate your exact setup.

ooh, I thought I had selected ECC ram. Good point. Could you point me to this simulator so I can understand how it works and add the specifics of my system?

 

[Heracles]

A simple Google search with "sizing power supply" should return you a few examples. Here, I used that one...

 

[path]

Hey folks, fwiw. I contacted Asrock Rack and they confirmed that the two motherboards are identical except for one having IPMI features and the other having onboard Audio. So if I can substitute IPMI, the M480 WS can be used in lieu of the other one.

 

[NugentS]

IPMI is well worth it

 

[joeschmuck]

You have had a lot of good responses.

IPMI is well worth it (as previously said) but it will likely take you time to figure out it's benefits if you have never used it before. Think, no keyboard, mouse, monitor and you can use a different computer to control it from. It really simplifies things.

ECC RAM in my book is a must, but if this is ONLY a media server then not so much a must but I would say highly desirable. And I'd shoot for 32GB minimum. I actually run TrueNAS in a VM on 16GB RAM, works like a champ. My bare metal system although has 64GB RAM so I can expand my VM as desired.

Transcoding, you mentioned it in your first post. you never specify what the transcoding will need to do, for example convert DVD to BlueRay quality feeding Raspberry PI's on 3 TV's simultaneously. The best option is to just save the original format as something each playing device can natively play, then you minimize transcoding overhead.

As for the VDEV's of hard drives. I think you would be better off running a 6 or 8 drive 10TB or 14TB VDEV in RAIDZ2. i say this because resilvering (rebuilding) times are measured in several days if you are at 50% full or more. Some folks have resilvered over a full week. Why is this important you ask? Because if your mirrored drive fails, all that data is gone. Even if you have all your data stored, recovery takes a long time and most people are impatient. So planning it out early is a good thing to do.

Tell you what, take a different approach instead and I have recommended this before and it worked out well.
1. Buy your basic system, making sure it's rock solid (Power Supply, Case, Motherboard, RAM), and burn these in properly. remember 1 thing, these parts are expected to live a long time, well beyond the life of several sets of hard drives and are likely less than the cost of 3 to 4 hard drives. think about that!
2. Buy a sampling of hard drives, maybe only four of the drives you think you want to use. Build your VDEV's/Pools and play with the setup. Find out if the drives are too loud, if they get too warm, fill the pool up with data (you can run a few commands to generate some very large files to suck up space) and then remove one of the drives. Format that drive in a different machine and reinstall it as a "NEW" drive so you can experience resilvering and how long it takes. You could do all this with a single mirror as well. Remember, the hard drives are Consumables. They will fail and if you treat them poorly (heat and/or bouncing around, powering up/down often, poor power), they will fail sooner, treat them nice and they are apt to last 5+ years.

After you have given yourself a few days or a few weeks, you will be able to make a better educated decision on what it right for you.

Good Luck and remember where your money is being spent, buy the best core parts you can afford, the hard drives are consumables even though they cost the most overall.

 

[Samuel Tai]

Note the C2550D4I has an AST2300 BMC driving the IPMI. This will require using the Oracle JRE to have the remote KVM experience. (AST2500 BMCs allow for a Java-less HTML5 remote KVM.) The BMC needs the JRE to accept TLS1.0, but recent Oracle JRE releases disable TLS1.0 by default. You can overcome this by editing the java.security file's jdk.tls.disabledAlgorithms statement to remove the ban on TLSv1, and also modifying the Java control panel's Advanced tab to enable TLSv1.0.

 

[path]

IPMI is well worth it

Makes sense. My previous machine (the C2550D4I infact) had IPMI and I did love it. But it seems that the motherboard I've zeroed in on is not available :(. So I plan to build a PI KVM https://pikvm.org

 

[path]

Note the C2550D4I has an AST2300 BMC driving the IPMI. This will require using the Oracle JRE to have the remote KVM experience. (AST2500 BMCs allow for a Java-less HTML5 remote KVM.) The BMC needs the JRE to accept TLS1.0, but recent Oracle JRE releases disable TLS1.0 by default. You can overcome this by editing the java.security file's jdk.tls.disabledAlgorithms statement to remove the ban on TLSv1, and also modifying the Java control panel's Advanced tab to enable TLSv1.0.

OOH. I never figured this out. I was constantly blocked by the JAVA security issue. I used the command line ipmi tool and that would usually be sufficient for me to atleast see the bios and send inputs.

 

[path]

You have had a lot of good responses.

Thanks for the really detailed response.

IPMI is well worth it (as previously said) but it will likely take you time to figure out it's benefits if you have never used it before. Think, no keyboard, mouse, monitor and you can use a different computer to control it from. It really simplifies things.

Yup. I've been a fan ever since my last board. But as I mentioned in this case sadly, availability of parts is an issue. I'm hoping pi KVM solves my issue for me.

ECC RAM in my book is a must, but if this is ONLY a media server then not so much a must but I would say highly desirable. And I'd shoot for 32GB minimum. I actually run TrueNAS in a VM on 16GB RAM, works like a champ. My bare metal system although has 64GB RAM so I can expand my VM as desired.

Transcoding, you mentioned it in your first post. you never specify what the transcoding will need to do, for example convert DVD to BlueRay quality feeding Raspberry PI's on 3 TV's simultaneously. The best option is to just save the original format as something each playing device can natively play, then you minimize transcoding overhead.

Honestly my transcoding need for today is pretty minor. I have various media files, some of which are in 4K and I'd like to be able to play them on my TV or my phone or tablet. It's one device at a time usually. However, my last hardware lasted me 5 years, so I figure I might as well allow for some future proofing. My thought was that if the hardware is capable of acceleration then the system wouldn't have to run too hot.

As for the VDEV's of hard drives. I think you would be better off running a 6 or 8 drive 10TB or 14TB VDEV in RAIDZ2. i say this because resilvering (rebuilding) times are measured in several days if you are at 50% full or more. Some folks have resilvered over a full week. Why is this important you ask? Because if your mirrored drive fails, all that data is gone. Even if you have all your data stored, recovery takes a long time and most people are impatient. So planning it out early is a good thing to do.

Tell you what, take a different approach instead and I have recommended this before and it worked out well.
1. Buy your basic system, making sure it's rock solid (Power Supply, Case, Motherboard, RAM), and burn these in properly. remember 1 thing, these parts are expected to live a long time, well beyond the life of several sets of hard drives and are likely less than the cost of 3 to 4 hard drives. think about that!
2. Buy a sampling of hard drives, maybe only four of the drives you think you want to use. Build your VDEV's/Pools and play with the setup. Find out if the drives are too loud, if they get too warm, fill the pool up with data (you can run a few commands to generate some very large files to suck up space) and then remove one of the drives. Format that drive in a different machine and reinstall it as a "NEW" drive so you can experience resilvering and how long it takes. You could do all this with a single mirror as well. Remember, the hard drives are Consumables. They will fail and if you treat them poorly (heat and/or bouncing around, powering up/down often, poor power), they will fail sooner, treat them nice and they are apt to last 5+ years.

After you have given yourself a few days or a few weeks, you will be able to make a better educated decision on what it right for you.

Good Luck and remember where your money is being spent, buy the best core parts you can afford, the hard drives are consumables even though they cost the most overall.

Great Advice. Burn in is not something I've done in the past when I've put my systems together. I plan to do so this time.

The raidz setup thing is interesting. I used to have a 7 drive setup in RaidZ2 for years and it worked fine. At one point when most of my drives were pretty old I lost two drives back to back and I did sweat bullets waiting for everything to rebuild. Thats when I came across this article https://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-not-raidz/ and switched to using 4 10 TB drives in a mirrored vdev formation. So my plan was to just extend it. But as you said, I do worry about losing 2 drives in the same mirror set back to back.

Am I correct in understanding that your recommendation is to instead build a large RaidZ2 setup which has more than enough room so that I remain below 50% utilization? So ideally I get the advantage of being able to lose any two drives AND faster resilvering in the case of disk failure?

 

[Samuel Tai]

You should also expand your MB search to the other major vendors:

1. MSI - https://eps.msi.com/product/99/56148. The D3020 and D3030 are both micro-ATX.
2. Asus - https://www.asus.com/us/Commercial-Servers-Workstations/P11C-M-4L/. The P11C-M-4L is micro-ATX.
3. Gigabyte - https://www.gigabyte.com/Enterprise/Server-Motherboard?fid=2287. Besides ASRock, they appear to have the widest range of micro-ATX server boards.

 

[joeschmuck]

Am I correct in understanding that your recommendation is to instead build a large RaidZ2 setup which has more than enough room so that I remain below 50% utilization? So ideally I get the advantage of being able to lose any two drives AND faster resilvering in the case of disk failure?

Yes an no. I really think this is a personal call and you should do your homework on the advantages and disadvantages of each way to build your pool. My "personal" preference for your use case is a RAIDZ2 and having a backup on maybe a mirror elsewhere. The thing is, as a media file server once you have loaded all your content, you will likely not be adding much new stuff often and thus a separate backup copy will only rarely need to be backed up, and you could back it up however you desire, maybe it's on a few large NTFS hard drives you pop into your Windoze machine. It doesn't matter so long as you like the outcome. Mirrors are fine but just for a media server, it's just not my preference but I'm sure there are others who would and do prefer it.

Do that homework and find out what is good and bad and then test it out. Figure out how much storage you will need and double it at least, shop for that. If you need 20TB, buy 40TB worth of formatted storage (remember for RAIDZ2 two of the drives do not add into the calculation). use a RAID Calculator, they work nice enough.

Also, 80% utilization unless you are using iSCSI which is 50% max utilization if i remember correctly, and I have no idea why you would use iSCSI for just a media server.

Good luck.