Advice on Mobo, CPU and memory for HW upgrade

[GeekGoneOld]

I am upgrading my Mobo since I have AMD cpu and no ECC (please don't judge - that's what I'm fixing!) though it has been running well for about 5 years. I am looking for reasonable choices to avoid overpaying.

I will keep the following as they are working very well (don't have exact model numbers handy, but can get them if necessary):

• Tower case with at least 8 drive bays and excellent cooling. ATX, mATX (don't remember brand)
• Corsair 650 PSU (don't remember exact model)
• 6xWD Red 3T in RaidZ2 (no encryption, no dedup)
• 2xIntel SSD 800MB in Raid 1 (SLOG)
• M1015 HBA in IT mode (may not need in new config)
• 2x Intel 1G NIC (don't remember model) (may not need in new config)
• 2x16GB USB2.0 in Raid 1 (boot)

My use is:

• File server over (mostly) SMB/1GBE for storing media files, backups etc.
• File server over NFS/1GBE for datastore for diskless ESXi computer running 4 VMs (Windows 10, OSx, linux pbx and pfSense)
• Plex Media Server with maximum 2 simultaneous transcodes of 1080P max as well as OTA recording using HDHomeRun (max 4 channels)

In reviewing all the relevant docs (Hardware Guide, forum posts, googling parts) it seems the SuperMicro X11SSM-F is very well suited. It seems that a reasonable processor is an E3 (E3 1230 V5?).

So...

1. Is the X11SSM-F a good choice?​

2. What processor will be reasonable for my load?​

3. What are the advantages to using i3 over E3 or vice-versa in my use case?​

4. I believe I am limited to UDIMM for this board, or is it this processor? Are there advantages to supporting RDIMM/LRDIMM? Use to be RDIMM was much cheaper but don't really see that now. Also I saw reference to RDIMM being faster with slightly more latency (???)​

5. What about V5 vs V7 or E3 V5 vs i3 V6 etc. It looks like a possibility is an older, high end processor (E3V3) may be an option over a newer lower end processor (i3V6). Trying to find the sweet spot (price/performance) is a bit tricky. Also, I don't know how important multi core (or hyperthread) are in a Freenas box running Plex.​

6. I heard that you need V2 of BIOS to run Kady Lake processors. If the Mobo is shipped with V1 BIOS, can I flash firmware through IPMI without a processor?​

7. Is there any need for a GPU?​

TIA
GGO

 

[PhiloEpisteme]

2x16GB USB2.0 in Raid 1 (boot)

Folks have been turning to SSDs or SATA DOMs for boot devices for the improved reliability. For the modest price of newer SSDs you may consider this route.

2x Intel 1G NIC (don't remember model) (may not need in new config)

I imagine you'd just use the one on the board rather than an additional card?

xIntel SSD 800MB in Raid 1 (SLOG)

Just for clarity, you're not actually going to use RAID1 here but instead Mirror these drives via ZFS?

Is the X11SSM-F a good choice?

I use it and it works just fine. If the price isn't prohibitive and you don't need more memory than it supports I think you're fine.

What processor will be reasonable for my load?

I'd suggest you check the passmark score of various CPUs and compare that to plex's passmark recommendations.

What are the advantages to using i3 over E3 or vice-versa in my use case?

I believe i3's will tend to be cheaper but have lower passmark scores for comparable E3s. For example, the Intel Xeon E3-1230 v6 @ 3.50GHz has a passmark score of ~9800 whereas the Intel Core i3-7100 @ 3.90GHz has a score of ~5800 despite the higher clock speed.

I believe I am limited to UDIMM for this board, or is it this processor? Are there advantages to supporting RDIMM/LRDIMM? Use to be RDIMM was much cheaper but don't really see that now. Also I saw reference to RDIMM being faster with slightly more latency (???)

The UDIMM refers to the memory supported by the board, from the supermicro site you'll see the board listed as supporting up to "64GB Unbuffered ECC UDIMM, DDR4-2400MHz, in 4 DIMM slots". AFAIK FreeNAS cares more about total memory available rather than latency. In this case you could probably just save money and stick with this board and the cheaper UDIMM memory. Someone will hopefully correct me if I'm wrong here.

What about V5 vs V7 or E3 V5 vs i3 V6 etc. It looks like a possibility is an older, high end processor (E3V3) may be an option over a newer lower end processor (i3V6). Trying to find the sweet spot (price/performance) is a bit tricky. Also, I don't know how important multi core (or hyperthread) are in a Freenas box running Plex.

Again, I'd just compare price vs passmark here.

I heard that you need V2 of BIOS to run Kady Lake processors. If the Mobo is shipped with V1 BIOS, can I flash firmware through IPMI without a processor?

I wasn't aware that was an issue on this board. I don't recall enough about how to flash it but if I were you I'd refer directly to supermicro's spec sheets for this board. Those will give you the most accurate information about what is supported and what is required for your use case.

Is there any need for a GPU?

Sorry, can't help you much here. Historically there wasn't much use but there has been talk on the forums of eventual support and use cases for a GPU or at a CPU with integrated graphics. You'd be better off reading the forums or waiting for someone more knowledgable than trusting any advice I'd give in this regard.

Other folks may have other advice etc but hopefully this is a good start.

 

[GeekGoneOld]

Folks have been turning to SSDs or SATA DOMs for boot devices for the improved reliability. For the modest price of newer SSDs you may consider this route.

Sounds like a good idea but I will already be using all 8 SATA ports. What are some options to use an SSD without resorting to using the HBA?

I imagine you'd just use the one [NIC} on the board rather than an additional card?

Yes, as long as by "the one" you mean "the three". Aren't there are 3 NIC ports on board (2 general purpose and one for IPMI)?

Just for clarity, you're not actually going to use RAID1 here but instead Mirror these drives via ZFS?

Yes, mirrored. Would I be correct in saying RaidZ1? Is there a difference between mirroring and RaidZ1?

I use it [X11SSM-F] and it works just fine. If the price isn't prohibitive and you don't need more memory than it supports I think you're fine.

Awesome. Does the 1151 processor socket limit me in any bad way (e.g. ruling out some other socket that is a better choice)?

I'd suggest you check the passmark score of various CPUs and compare that to plex's passmark recommendations.

I believe i3's will tend to be cheaper but have lower passmark scores for comparable E3s. For example, the Intel Xeon E3-1230 v6 @ 3.50GHz has a passmark score of ~9800 whereas the Intel Core i3-7100 @ 3.90GHz has a score of ~5800 despite the higher clock speed.

Can I trust passmark to be representative of CPU performance for my needs? If so, I will just make a chart of price vs passmark and plot a bunch of processors.

The UDIMM refers to the memory supported by the board, from the supermicro site you'll see the board listed as supporting up to "64GB Unbuffered ECC UDIMM, DDR4-2400MHz, in 4 DIMM slots". AFAIK FreeNAS cares more about total memory available rather than latency. In this case you could probably just save money and stick with this board and the cheaper UDIMM memory. Someone will hopefully correct me if I'm wrong here.

I saw some reference to UDIMM being MORE expensive than RDIMM. Perhaps that was a very temporary situation.

Again, I'd just compare price vs passmark here [v5 vs v6 vs v7...].

Should I be worried about using a v5 or v6 processor given that we are now (or almost?) at v10? What might I lose? Can I use a V9 with this board?

I wasn't aware that was an issue on this board. I don't recall enough about how to flash it but if I were you I'd refer directly to supermicro's spec sheets for this board. Those will give you the most accurate information about what is supported and what is required for your use case.

I will check with SM. Anyone know if you can flash BIOS on SM boards without a processor (i.e. just using the BMC)?

Sorry, can't help you much here. Historically there wasn't much use but there has been talk on the forums of eventual support and use cases for a GPU or at a CPU with integrated graphics. You'd be better off reading the forums or waiting for someone more knowledgable than trusting any advice I'd give in this regard.

Thanks for your candor. I suppose I could replace the processor later with an integrated CPU/GPU one if needed. Anyone know if I would care about having a GPU or video card?

Other folks may have other advice etc but hopefully this is a good start.

An awesome start. Thanks.

 

[PhiloEpisteme]

Sounds like a good idea but I will already be using all 8 SATA ports. What are some options to use an SSD without resorting to using the HBA?

A used HBA can be had for $50-$75 + ~$10 of cables. If you ever want to add more drives they are a great way to go.

Yes, as long as by "the one" you mean "the three". Aren't there are 3 NIC ports on board (2 general purpose and one for IPMI)?

You're right, there are two NICs. My point was just that you don't need to buy a new network card.

Yes, mirrored. Would I be correct in saying RaidZ1? Is there a difference between mirroring and RaidZ1?

Yes, RAIDZ1 is analogous to RAID5. Mirroring is analogous to RAID1. Check the terminology primer. And the various intro guides to get a good sense. :)

Awesome. Does the 1151 processor socket limit me in any bad way (e.g. ruling out some other socket that is a better choice)?

It limits the CPUs you can use. You'll be able to put xeons in it though so it isn't like you're missing a ton of power potential.

Can I trust passmark to be representative of CPU performance for my needs? If so, I will just make a chart of price vs passmark and plot a bunch of processors.

I'm not 100% sure how well that translated to all use cases. I'd say you could look around at other benchmarks etc or look at CPUs folks around here use for a similar setup.

Should I be worried about using a v5 or v6 processor given that we are now (or almost?) at v10? What might I lose? Can I use a V9 with this board?

Check the docs from supermicro for the specs. Newer CPUs tend to run cooler|at a lower wattage for a given clock speed.

 

[pro lamer]

IIRC Xeon Entry (E-2xxx) seem to be significantly faster than Xeon E3 v6 regarding passmark and don't seem to be very expensive. Edit: unless you can easily have a second hand one much cheaper.

You may consider this if you want to start with some Pentium or newer generation i3 to have upgrade path. This would influence motherboard choice.

Some SM motherboards are documented by SM to work with i3 v9 but others having the same chipset are not documented to support the new generation.

Newer generations may be more Spectre/meltdown/zombieland and similar (possibly the ones not known yet) vulnerabilities resistant. But documentation regarding motherboards support for them is not clear to me again...

Edit: IIRC Xeon E-21xx is still susceptible to the vulnerabilities mentioned above. At least 22xx may be "needed" (verify please) and motherboard support for them is the one that I have kept failing to learn...

Sent from my phone

 

[charlie89]

Anyone know if you can flash BIOS on SM boards without a processor (i.e. just using the BMC)?

Yes when the board has IPMI this should work fine, have done this on my X9DRi-F. Just be sure to insert a serial key to enable this special feature (either buy the key or calculate it based on the first lan mac address). Then it's just as simple as uploading the bios file into the ipmi web interface.

 

[GeekGoneOld]

Re: need for GPU, can I get away with no video whatsoever. For years the only time I used video was if boot failed but with IPMI I don't need that. Will the bios boot without any graphics? Will Plex server run?

Also, if I do get graphics, what is the difference between aCPU/GPU combo and a separate video card? is there a video port on the Mobo that is dead if there is no GPU?

 

[PhiloEpisteme]

Re: need for GPU, can I get away with no video whatsoever. For years the only time I used video was if boot failed but with IPMI I don't need that. Will the bios boot without any graphics? Will Plex server run?

I never use a video card in either of my systems. Both of my boards have very basic built-in graphics suitable for the boot sequence etc. Everything else is managed via the UI reached over HTTP or HTTPS.

calculate it based on the first lan mac address

Ah, this is a trick I was not aware of.

 

[GeekGoneOld]

One other question. I see that all the E3 V6 CPUs without video (E3 12x0 V6) don't support Intel Quick Sync Video. The ones with video (E3 12x5 V6) do. This is needed for Plex hardware acceleration of transcoding. Is there a way to use the hardware acceleration if an E3 12x5 V6 processor is used?

 

[GeekGoneOld]

So it appears (subject to the hardware acceleration answer) that I will choose:

X11SSM-F with E3 1230V6
OR
X11SCM-F with E2134

In either case, I will put in my HBA and boot off a new SSD.

The performance should be close, the top one is cheaper but the bottom one is newer. Am I likely to see any limitation in choosing the top one other than getting a newer, faster processor later (likely never to happen)? Specifically is a new release of FreeNAS or Plex likely to need a feature that is not going to be available in the first choice? Also, is the bottom one in use by anyone?

Can't wait to pull the trigger...

 

[pro lamer]

X11SSM-F with E3 1230V6

Brand new or second hand one?

These forums have some sticky/resources regarding a bit cheaper, pre owned stuff...

X11SCM-F with E2134

Not being sure of answers to other questions, if I were to buy X11SCx then I wouldn't buy E-2134. I'd buy either 2136 (15000 passmark instead of 10000) or a Pentium or i3. The latter gives me easier with upgrade path since I don't like selling :P

newer, faster processor later (likely never to happen)?

Why not? Sell the old one and in theory buy a newer generation one (I'm still not sure which motherboards support E-22xx CPUs)...

Specifically is a new release of FreeNAS or Plex likely to need a feature that is not going to be available in the first choice? Also, is the bottom one in use by anyone?

Maybe someone else will chime in...

Sent from my phone

 

[PhiloEpisteme]

X11SSM-F with E3 1230V6
OR
X11SCM-F with E2134

If you're looking at these generation boards I recommend the X11SSM-F over the other unless you have a specific need for the M.2 slot. The X11SSM-F has more lanes dedicated to PCIe so it is more flexible in that regard.

 

[pro lamer]

these generation boards

They are different generations. X11SCx are one generation newer. Some of them are documented by supermicro to even support i3 v9 but I can't say which ones from the top of my head and I don't know if they support E-22xx too...

Sent from my phone

 

[GeekGoneOld]

OK, so slight revision. Seems it is better to use a newer board as the basis.

Motherboard: X11SCH-F with all drives on the SATA ports except for two small kingston A400 disks as mirrored boot disk on my HBA

The current question is about CPU and GPU.

If I put a CPU/GPU on this (e.g. E2146G), is the video useless (since there is no video port for it and, therefore, maybe not even "activated"). The reason I am looking to use one with video is for hardware acceleration of transcodes.

Also, should I go with E21xx or with i3?. I'm thinking E2136/E2146G (depending on above question) vs i3 9100F/i3 9100. The performance of the i3 is shockingly good given it is 4C/4T at just over 9,000 PassMark whereas E2136 (6C/12T) is only about 16,000. Given my load is relatively small (2x1080p transcodes and serving NFS datastore for ESXi) am I just wasting money on the extra power of Xeon processor or do I get something noticeable out of it?

 

[GeekGoneOld]

I just got confirmation from Supermicro that a CPU/GPU in the X11SCH-F Mobo shows two display adapters in Windows: the ASPEED 2500 and the Intel UHD Graphics P630. Even though we are talking about FreeBSD, not windows, this means that the GPU IS there even though there is no video port. This is encouraging for using a GPU on this board for HW decode/encode. I think I will spend the extra $50

 

[HoneyBadger]

Not to rewind all the way to the beginning or anything, but ...

2xIntel SSD 800MB in Raid 1 (SLOG)

What exact make/model of drives are these? SLOG devices need certain characteristics in order to do their job well, and in 2019 there's a strong preference for NVMe devices over SATA/SAS.

(Also I assume there's a typo in the "800MB" and you mean either "800GB" or "80GB".)

 

[GeekGoneOld]

ATA INTEL SSDSC2BB08 0370

Intel® SSD DC S3500 Series
120GB, 2.5in SATA 6Gb/s, 20nm, MLC

Note the 800MB is actually 8GB as I limited the disk to 8GB only. Don't need more as I only have 8GB of RAM and this way the wear algorithm has lots to work with. I chose this model because it has better performance than the smaller one (80GB?) and has the supercap power hold.

What is NVMe and why the preference?

 

[HoneyBadger]

Note the 800MB is actually 8GB as I limited the disk to 8GB only. Don't need more as I only have 8GB of RAM and this way the wear algorithm has lots to work with. I chose this model because it has better performance than the smaller one (80GB?) and has the supercap power hold.

I would strongly recommend getting more RAM for your new build, especially given your desired use case. Regular SMB doesn't really require much; however, VM hosting and Plex will both need more. I'd consider 16GB as the bare minimum for that.

The S3500 is an acceptable SLOG device, but it's just considered slow nowadays. The logic behind wear-leveling and driving up performance (as well as supercap for power-loss-protection) is sound; you may just want to Go Faster.

What is NVMe and why the preference?

Non-Volatile Memory Express (NVMe) is an interface/protocol; it has a much lower latency and higher bandwidth than SAS/SATA. Check the link in my signature ( https://forums.freenas.org/index.php?threads/slog-benchmarking-and-finding-the-best-slog.63521/ ) where you can see users benchmarking and comparing their SLOG devices; you'll very quickly see the difference in performance that's driven out from a low-latency NVMe device like the Intel Optane or DC P-series cards, especially at the smaller record sizes that are common in VM hosting.

Speaking of record sizes, you'll also want the dataset being exported via NFS for your VM storage to use a smaller recordsize, to avoid a "partial record rewrite" penalty - if you have a 128K record in your VMDK file, but then write 4K - you'll have to read that 128K record, update the changed 4K, and then write back the 128K. Not a fun time. If you set the recordsize to 16K or 32K, you'll still have that penalty, but it won't be as pronounced.