Greetings: Server plan still missing some parts

[WillM]

I've been a longtime user of FreeBSD and for a while I did run FreeNAS 0.7x but I had a shoddy hardware platform back then and it stopped working years ago. I'm ready to dive in again, so I am building a box to host lots of audio and video media, plex transcodes, backups to amazon, nfs or iscsi mounts for use by ESXi, backups from client PCs (time machine backups for two macbook pros and a few others boxes in the house) and also several other jailed apps like an freeradius, nginx reverse proxy, webapp containers like tomcat, node.js, an hsqldb database, gitlab, jenkins, etc.

I did review the "read this first" advice on hardware, so hopefully there are no glaring omissions.

cpu: e5-1620 v3 4c/8t 3.50 GHz
http://ark.intel.com/products/82763/Intel-Xeon-Processor-E5-1620-v3-10M-Cache-3_50-GHz

mainboard: Supermicro X10SRA-F lga2011/c612 ATX - 10 SATA ports and 8 slots for up to 512gb RAM
http://www.supermicro.com/products/motherboard/Xeon/C600/X10SRA-F.cfm

memory: for now, just 1 stick of Samsung DDR4-2133 32GB/4Gx72 Load Reduced CL15 (supermicro recommended)
https://www.superbiiz.com/detail.php?name=D421LR32S1

discs: 3 x 4TB WS Red NAS ( I expect to config these as raidz1 for 8TB)

I have some spare SSDs I may put into this build also later on to provide low latency storage for some of the more demanding applications. Regarding the RAM, I do know that putting in an odd number of sticks does slow down memory access, but I'm hoping performance will still be plenty good enough for what I am doing initially. Later, once I get more jails running and such (and perhaps when RAM prices fall more), I expect I may add a second 32GB stick to complete the pair.

I am also still in search of a shallow depth case (15.25" or less). Affordability is important here, as it seems you can spend a ridiculous amount on good rackmount cases. I'm looking primarily for external hotswap 3.5" drive access through the front bezel (maybe 8 or 12 vertically oriented trayless slots seems ideal). It seems a 3U would be sufficient height for this, but it has been surprisingly difficult to find something like this. This item seems almost perfect, but its not short depth:

http://www.amazon.com/dp/B001OS0P8E/?tag=ozlp-20

Would appreciate your input and suggestions.

-Bill

 

[Ericloewe]

memory: for now, just 1 stick of Samsung DDR4-2133 32GB/4Gx72 Load Reduced CL15 (supermicro recommended)
https://www.superbiiz.com/detail.php?name=D421LR32S1

Nope, Xeon E5-16xx CPUs only support UDIMMs and RDIMMs, not LRDIMMs. You'll have a very hard time finding information about this, but experience backs up what little there is.

32GB RDIMMs should be fine, though.

discs: 3 x 4TB WS Red NAS ( I expect to config these as raidz1 for 8TB)

This is an odd juxtaposition. RAIDZ1 is generally discouraged. And the lowish amount of storage suggests you might be better off with a Xeon E3 system. You're limited to 32GB on E3 v3 and v4 and 64GB on E3 v5, but it doesn't seem to be a major limitation in your case.
What are your future expansion plans, exactly?

 

[WillM]

Nope, Xeon E5-16xx CPUs only support UDIMMs and RDIMMs, not LRDIMMs. You'll have a very hard time finding information about this, but experience backs up what little there is.

32GB RDIMMs should be fine, though.

Hmm, well when I look at the mainboard this is going into:

http://www.supermicro.com/products/motherboard/Xeon/C600/X10SRA-F.cfm

And click into the "Tested Memory List" I am offered this exact LRDIMM part as a valid option for that board in question. I think this has changed with the v3 and later CPU parts.

This is an odd juxtaposition. RAIDZ1 is generally discouraged. And the lowish amount of storage suggests you might be better off with a Xeon E3 system. You're limited to 32GB on E3 v3 and v4 and 64GB on E3 v5, but it doesn't seem to be a major limitation in your case.
What are your future expansion plans, exactly?

Yeah, I think agree with you on that. I now understand (from a recent read of the recommendations maybe an hour ago) that raidz1 is considered passe at this point. So I will probably end up getting an additional 4TB disc to go into that first vdev to get up to raidz2. My aim for this build was to arrive at a similar price as the FreeNAS Mini, but to choose components with more headroom for future growth (both in memory and cpu capability), since I had so many other uses in mind for the jails. Ultimately, I want this hardware to have a much longer life beyond being just a dedicated NAS-only device.

 

[WillM]

Good catch Ericloewe on that LRDIMM. I did find the post from April 2015 where another user hit the incompatibility of which you speak between E5-1620v3 and LRDIMM:

https://forums.freenas.org/index.ph...s-build-incl-plex-mb-a1sa7-2750f.26672/page-2

Thanks.

 

[Fuganater]

I have the X10SRL-F which can use you E5 chip. I have 32GB RDIMM in it with no issues. You may want to check it out.

Check out the 3U or 4U Rosewill chassis. It can do 12 HDD vertically mounted in the front. I may meet your dept requirement.

 

[WillM]

Thanks Fuganater, is that 32g chip an LRDIMM? That was the specific concern I had.

I did look at the Rosewills and for now am opting to slide a perfectly sized desktop case on a shelf sideways, as it gave me all I needed for the cost of a low priced shelf.

 

[WillM]

FWIW, I am doing a thorough burn in outside of the case on my test bench now using the following components:

cpu: e5-1620 v3 4c/8t 3.50 GHz
mainboard: Supermicro X10SRA-F lga2011/c612 ATX
memory: Samsung DDR4-2133 16GB/2Gx72 ECC/REG CL15 Server Memory (32g total)
discs: 4 x 4TB WS Red NAS

I'm temporarily using a newish Enermax powersupply (MODU82+ 525W) for testing, but I've ordered a gold rated Seasonic X-850 SS-850KM3 that is still on its way.

Running memtest86 6.2.0 and using IPMI to access it on this setup (spinning discs not yet plugged in) I'm drawing about 98 watts from the wall. Not too bad I guess. Better than the old PowerEdge 400sc I was using before with only a celeron that drew 165w at idle and 300w+ under load.

I also have in my possession the original, single 32G LRDIMM still unopened in the box, but I don't want to pay restocking fees to find out if the LRDIMM will work or not. Hoping Funganater will tell me he runs an LRDIMM just fine in his. :)

 

[Fuganater]

This is the RAM I bought.

 

[WillM]

Thanks, yep... and by now I've read more indicating LRDIMM doesn't work with the 16xx CPUs... bummer.

On a different topic, does anyone know: I experienced some frequent problems running memtest86 until I disabled C-states (Supermicro lists this as the "Energy Efficient" setting for the CPU). Is this a typical compromise that must be made to get FreeNAS working ok? Will I still at least have the benefit of C1 or C1E states with this disabled? I would like this to be as efficient as possible.