1st FreeNAS build

[Archaniel]

Hi guys,

I'm new here and try to build my first FreeNAS file storage server. I want a fast, reliable server able to stream Plex, while able to read/write at last at speeds of 40-50MB/s.

Eventually I'd like to use the same box with dual-booting Windows Media Center to push some BluRay 3D's through HDMI directly to telly (not being able to access the drives though, because encryption etc) - but this is just an optional idea, which I will discuss later.

My current thoughts are:
Supermicro X11SAE-M board
Pentium G4400 CPU (has AES-NI and ECC support)
2x8GB ECC Unbuffered RAM (Crucial or Kingston? - I read Kingston does not work well with Supermicro somewhere?)
2x 3TB WD Red Nasware 3.0 (already bought) drives in ZFS mirror raid, full disk encryption - for photo storage and sensitive docs I don't like to loose + some Bacula backups
1x 3TB "reused" drive - just for movies, and some disposable junk
1x 120GB M.2 SSD for cache (Adata - some cheap stuff prolly)
some old 2.5" 500GB HDD's I have - just for some extra storage of junk
USB-booting of FreeNAS

My question is - what's the fuss about X11SAE-M ? All people want to use it as a "workstation". My question is WHY? It has ECC support, 8x SATA3 support - isn't it good for storage? Is it due Xeon support? Ain't the Pentium G4400 quite enough for storage?

I understand I can run Plex in a jail to read all my movies I'll have on that single 3TB drive and stream and even maybe transcode up to 4K (if needed, direct play of 4K should be no problem with 16GB RAM and that G4400)

Also, will I get the requested read/write speeds? Ofc the box will be hooked to a GBIT network

optional thoughts:
Now regarding that WinMedia center - One thought I had was having a BluRay drive in the box to play BD-3D movies directly (and BD3D ISO images for example) directly to my telly - not yet sure if it works, need to somehow get to test it - maybe with nVidia 3DTV play (which would require hooking up a supported nvidia card to the Motherboard). But I was considering installing ESXi on the box, while piping directly the nVidia card out to the WMC and the drives directly to the FreeNAS VM. I think it could somehow work, but question is if it's feasible. Only reason, really, is BD3D playback... - did anyone here try to do such thing? It means also that the storage box would have to be connected via HDMI to the telly...

thanks for the insights

 

[BigDave]

1x 120GB M.2 SSD for cache (Adata - some cheap stuff prolly)

Don't waste your money on cache drives, buy more RAM, the RAM is faster than any SSD (especially a cheap one).

My question is - what's the fuss about X11SAE-M ? All people want to use it as a "workstation". My question is WHY? It has ECC support, 8x SATA3 support - isn't it good for storage? Is it due Xeon support? Ain't the Pentium G4400 quite enough for storage?

Most people in here don't build dual boot machines that run FreeNAS, they build storage appliances. Storage
appliances don't typically need stuff like HD graphics with HDMI output and 5.1 surround sound, so they would
never recommend hardware for FreeNAS that had any of that extra, unneeded hardware. You seem leaning toward
a dual boot machine, I will not try and talk you out of that, but I will say that for your use case, Windows Server would
be a much better choice for that. FreeNAS is great, but requires strict attention to using and maintaining the proven
hardware recommendations brought forth by this community. Most members in here don't worry too much about
losing their data, they would like you to have that same level of comfort.

 

[Archaniel]

Thanks for the tip. So I'll keep 3D BluRay streaming from my PC.

Update 1: instead the M.2 cache, I'll by more RAM
Update2: Pure FreeNAS install

What about that motherboard? Is X11SAE-M unsuitable for a storage appliance?

 

[Spearfoot]

What about that motherboard? Is X11SAE-M unsuitable for a storage appliance?

No, that board will work fine. It's just that, as @BigDave mentioned above, it has built-in graphics support that you don't need in a storage appliance.

 

[BigDave]

What about that motherboard? Is X11SAE-M unsuitable for a storage appliance?

unsuitable, no.
Ideal, again no.
The model is not something I would go for unless it was ALOT cheaper than a regular server board. I have
no idea how difficult sourcing hardware is in your country, but the X11SAE-M lacks one feature that I
consider to be of paramount importance, and that is BMC or IPMI interface. I would never attempt to
administer the health of a storage server without it. IPMI is a very valuable tool and once you have it,
you will never go without it. On the Supermicro boards, look for the "-F " at the end of the model number.
This indicates it has dedicated IPMI interface.

 

[maglin]

It really comes down to no IPMI on a workstation MB. You probably have never used it. It's worth the extra cost for it. Headless install and when you have to reboot because your plex jail had a run away process that has locked up your box you can do it from any machine in your house.


Sent from my iPhone using Tapatalk

 

[Archaniel]

OW! I totally missed that! I thought the SAE-M has IPMI as it comes like a standard feature one would expect on a server-ish motherboard like supermicro is.

I currently have options to order:
X11SSL-F (cheapest, but only 6 SATA slots. Though I have an ASUS U3S6 expansion card in my desktop, which I only use for USB3.0 support and could in worst case spare if I ever go above 6 drives)
X11SSM-F roughly 80EUR more costly than the SSL-F
X11SSAE-F - same price as SSM-F, I'm seeking a microATX board though - leaving this one out from the comparison

what are the biggest differences between SSM and SSL ? Is the C236 chipset faster compared to C232 in terms of significant performance indicators (IOPS perhaps?)

 

[Ericloewe]

what are the biggest differences between SSM and SSL ? Is the C236 chipset faster compared to C232 in terms of significant performance indicators (IOPS perhaps?)

https://forums.freenas.org/index.php?threads/so-you’ve-decided-to-buy-a-supermicro-x11-board.39549/

 

[Archaniel]

I'm sorry, but that's the thread I got my infos from in first place. It doesn't answer my question regarding performance difference between C232 and C236 chipsets, though.

 

[Archaniel]

ok update 3: Motherboard will be X11SSM-F

 

[Ericloewe]

I'm sorry, but that's the thread I got my infos from in first place. It doesn't answer my question regarding performance difference between C232 and C236 chipsets, though.

There is no performance difference. Why would there be? Of course, everything on the PCH shares 8x PCI-e lanes back to the processor.

 

[Archaniel]

Update 4: I decided to buy 2 more drives and reuse my older WD EZRX 30 green to go with RaidZ2 with 9TB storage space, i.e. my setup will be following:

Supermicro X11SSM-F (ordered)
Intel Pentium G4400 (ordered)
2x8GB Kingston ECC UDIMM (certified to run in the mobo) (ordered)
4x WD Red 3,0TB - 2 from vendor A, 1 from vendor B and 1 from vendor C to mix the batches (ordered)
1x WD Green 3,0TB - old drive, which runs quite ok. Guess I'll have to run wdidle with it (reuse)
2x old 1TB Drives in mirror for secondary backup of photos (reuse)
PSU: Fortran AURUM S 400 Gold (ordered)
Case: Fractal Design Define R5 (to be ordered)
Boot drive: I prefer USB, meaning I run either 9.10, or 9.30 with the XHCI support. (reuse) - in worst case scenario I'll use some old 500GB 2.5" drive

In the Jails I'll be running PLEX, Bacula, torrents - I'll eventually order 2x 8GB RAM to fully populate the banks to 32GB

I hope this setup will be OK-ish for my usecase. Any more tips for what should I be on lookout for?

 

[BigDave]

I'll eventually order 2x 8GB RAM to fully populate the banks to 32GB

We always recommend starting out with the largest RAM capacity available for your board/cpu configuration,
which in your case would be 16GB sticks. I now run my board (same as you have) with one 16GB stick. This is
obviously dependent upon this RAM being available in your homeland. Good luck!

 

[Archaniel]

I had the option to buy 1x16GB aswell, but ain't having dual channel better for performance as running only with 1 stick ?

 

[Scampicfx]

Whether you're running it in dual channel or single channel, it makes no feelable difference for your NAS.
There are performance tests in the web describing that there *is* a difference between single and dual channel configurations, however, it is *very* minor!

I definetly agree to the point that you should always start with the most largest RAM capacity, as long as it is economic.
I recently had a look to the supermicro mainboards X10 which support max 256 GByte RDIMM and 512 GByte LRDIMM. Stay away from LRDIMM since it is very expensive! I think 32 GByte RDIMM are a good choice at the moment! There is a huge difference in price between 32 GByte RDIMM and 64 GByte LRDIMM... I wouldn't go bigger than 32 GByte at the moment...
But anyway, in your case, definetly start with 1x 16GB UDIMM... It is cheap and gives you the most easiest upgrade options in future!

 

[Archaniel]

well the 2x8GB sticks are like ... 4EUR cheaper ... which in the case of how expensive this build went is bollocks. I'll try to swap it in then for a Crucial: CT16G4WFD8213 - but not sure if this one is compatible with the board :/ Will the stick work ?

 

[Scampicfx]

I just had a look to the tested memory list on the website of Supermicro:
https://www.supermicro.nl/support/resources/memory/display.cfm?sz=16.0&mspd=2.133&mtyp=82&id=39B8FAAF928AD46F59DF7C4CA9AE0DCE&prid=84941&type=DDR4 1.2V&ecc=1&reg=0&fbd=0

DDR4-2133 ECC supported:

In one 3d application server, I'm running X11SAE mainboard with 1x Samsung M391A2K43BB1-CPB 16 GByte module and it's a nice flawless running system :)
I don't have any experience with Kingston RAM. I've never used it so far (as far as I remember there are some negative reports about Kingston RAM).
I am running Crucial RAM in some of my desktop machines (and everything is okay so far), however, for server storage setup I would stick to the recommendations of the manufacturer and install a module listed on the tested memory list.

 

[BigDave]

Just let me add here that I have (in the past) run Crucial RAM for my Supermicro boards
using the Crucial Advisor tool on their website. I went with Samsung for my new X11 board
(see Show button my sig) and have not had trouble with the three platforms. Crucial offers
a compatibility guarantee so you could return it for a refund if it does not work.

EDIT:

CT16G4WFD8213 - but not sure if this one is compatible with the board :/ Will the stick work ?

I went back and failed to find that number on the "approved" list for the X11SSM-F board, sorry!

 

[Archaniel]

Really hard to get Samsung RAM sticks in here, sadly :(

I've called my dealer, he swapped me the 2x8 for 1x16 Crucial. Tomorrow should the mobo arrive and I'll be able to put the stuff together. The dealer also told me that the 16GB Stick I ordered runs fine with the motherboard.
How long should I run memtest after I first boot up? It's going to take quite a while I guess?

Have ordered also the last 2 WD RED drives from 2 different vendors (and different from the first one). Am now trying to free my 3TB drive somehow, so I'll have 5 drives ready to go.

Tomorrow is the big day of assembly /yey :)

 

[BigDave]

I run a minimum of 20 passes with what is now Passmark's version of the free version of MemTest 86.