New (to) FreeNAS - server build multiple purposes

[Victor0001]

I first want to thank the community and everyone else who supported the amount of resources to be found on the web for FreeNAS! Awesome.

Old situation:
I work in the IT business for quite some years as system administrator and a developer in my free/study time. So all the IT stuff at home will be done by me after a bit too much research some may say. (current research is only taking up 1 month so far) Currently I have a 4 bay QNAP 469-L(only 3 disks in use) running, my dad has a 2 bay NAS and we replicate eachothers data as off-site backup. However I'm running low on capacity and hardware/software capabilities. This is my second NAS and it seems that each 3 years the NAS needs to be replaced, that's a waste and I want a server that can be upgraded in 3 years with extra storage, memory and even CPU.

New situation:
My old NAS will move to my dad for the same previous reason and I will move to a server with FreeNAS to store data, run all my wishes and be upgradable.

I would like to use FreeNAS server for the following features:

• 8TB effective storage. My own data is about 3TB in which only 2 needs a backup. So I have enough growth for about 2 years. After that I could expand with another vdev.
• Fileserver for 3 users
• Sabnzbd, Sickbeard, Torrent, CouchPotato
• Minecraft server (on demand)
• Replacement for Vera Lite (Home Automation), probably moving to Domoticz
• ZoneMinder for security camera's
• Webdevelopment closed area including database (on demand)
• Possibility to stream video's (on demand)

I'm planning to start off with FreeNAS 10 if it will be released any time soon (within a month or 2) as I don't see any reason why I would invest my time to configure everything in FreeNAS 9 and after a couple of months upgrade to 10. Is this wise?

Hardware:

Code:

Intel Xeon E3-1230 v5
Supermicro X11SMM-F
WD Red WD30EFRX, 3TB (6x)
Fractal Design Define R5
Samsung M391A2K43BB1 16GB (2x)
Seasonic G-series 550watt
Transcend SSD420K 64GB (2x) (mirrored OS)

part(y)picker list:
https://pcpartpicker.com/list/DshnXH

De SSD's are a different type in de partpicker list as I couldn't find the type I can get in The Netherlands.
And I found these on ebay which seems to be Sandisk rebands, so perhaps saving a couple of bucks and going for 2 of these.
http://www.ebay.nl/itm/NEW-XUM-32GB...754916?hash=item360f3af0a4:g:36YAAOSwo4pYXSnK

Questions:
Supermicro has been recommended everywhere however when I check their list for tested memory I have only 3 types to pick from. Are there any other memory compatibility/tested lists to choose from? I don't see Samsung as a reliable brand for servers, but this is the only option available for me.
And in regards of the memory, is 32GB enough?

Already made a comparable comment in another thread but I'm wondering if an i3 6100 as CPU wouldn't suffice? Yes I'll run a couple of VM's, but for example the Minecraft OS would only run on demand.

I also wonder if I shouldn't go for an ESXI server and run FreeNAS in VM, however I already ran into problems running it in a Vmware Workstation in relation with jails not willing to install. And I'm not sure if I enjoy the pricing of ESXI server for a home solution.

Last but not least, are there solutions to have external boxes to add 6 hdd's if in the future I decide to expand with another set of disks instead of upgrading the existing to 6TB(for example).

Any advice, considerations are most welcome!

 

[lukyjay]

Hi

I have this motherboard and CPU, but I instead went with this RAM because it was cheaper for me and is working fine: KINGSTON 16GB KVR21E15D8/16 DDR4-2133MHz ECC CL15

I am having problems with my motherboard though, it seems to be a lemon....

Having those two SSDs isn't really necessary. FreeNAS runs fine from a USB drive.

Supermicro has been recommended everywhere however when I check their list for tested memory I have only 3 types to pick from. Are there any other memory compatibility/tested lists to choose from? I don't see Samsung as a reliable brand for servers, but this is the only option available for me.

I had this problem too. If you go to the memory manufacturers website, you can look up the compatibility that way. That's how I found the ram mentioned above.

The list on Supermicro isn't "these are the only ones that will work", they're just recommended and are guaranteed to be compatible. There are other options out there which tend to be cheaper.

Already made a comparable comment in another thread but I'm wondering if an i3 6100 as CPU wouldn't suffice?

Please double check but I'm pretty sure that CPU doesn't support ECC ram. Why wouldn't you just go for the Xeon? It's a server motherboard, server ram so why have a low end consumer CPU?

I also wonder if I shouldn't go for an ESXI server and run FreeNAS in VM,

Please read the stickies about this, it's not recommended by many. You said you're using FreeNAS 10, 10 supports running VMs quite elegantly through the interface. Also you could run in to problems with having docker apps if FreeNAS is installed as a VM (at least I did when I tested this). I think you should consider running FreeNAS as your host OS at first since that's how its designed, and reconsider down the track if things aren't to your liking.

Last but not least, are there solutions to have external boxes to add 6 HDD's if in the future I decide to expand with another set of disks instead of upgrading the existing to 6TB(for example).

Yes but they're crap. You're better getting a hot swap chassis if you intend to have a large (10+) drives.

Fractal Design Define R5

You should consider the NZXT h440. This case supports 11 3.5" hard drive bays and conceals them quite well. It also has two additional SSD bays at front if you decide to go down that path.

If you (or anyone) disagrees with anything in my reply, please let me know as I am also new to this and am still learning. If you find it helpful, please hit thanks :)

 

[Ericloewe]

Hardware:

Code:

Intel Xeon E3-1230 v5
Supermicro X11SMM-F
WD Red WD30EFRX, 3TB (6x)
Fractal Design Define R5
Samsung M391A2K43BB1 16GB (2x)
Seasonic G-series 550watt
Transcend SSD420K 64GB (2x) (mirrored OS)

part(y)picker list:
https://pcpartpicker.com/list/DshnXH

Looks good.

Please double check but I'm pretty sure that CPU doesn't support ECC ram.

It does support ECC.

 

[Victor0001]

Hi

I have this motherboard and CPU, but I instead went with this RAM because it was cheaper for me and is working fine: KINGSTON 16GB KVR21E15D8/16 DDR4-2133MHz ECC CL15
I had this problem too. If you go to the memory manufacturers website, you can look up the compatibility that way. That's how I found the ram mentioned above.

The list on Supermicro isn't "these are the only ones that will work", they're just recommended and are guaranteed to be compatible. There are other options out there which tend to be cheaper.

Thanks! I'll surely look it from the other perspective, didn't think of that.
However for me those Kingston ones are more expensive than the Samsungs SuperMicro recommends, but I prefer the Kingstons over the Samsungs.

I am having problems with my motherboard though, it seems to be a lemon....

What problems?

Having those two SSDs isn't really necessary. FreeNAS runs fine from a USB drive.

True! I just don't like the idea of booting a critical OS off an USB drive and I see others recommend SSD over USB.

Please double check but I'm pretty sure that CPU doesn't support ECC ram. Why wouldn't you just go for the Xeon? It's a server motherboard, server ram so why have a low end consumer CPU?

I'm definitely going for ECC. And the i3 6100 does support it, it has 4 cores just no hyperthreading. However the more and more I think of this server, the more appliances I find for it and the more I'm leaning to the Xeon.

Please read the stickies about this, it's not recommended by many. You said you're using FreeNAS 10, 10 supports running VMs quite elegantly through the interface. Also you could run in to problems with having docker apps if FreeNAS is installed as a VM (at least I did when I tested this). I think you should consider running FreeNAS as your host OS at first since that's how its designed, and reconsider down the track if things aren't to your liking.

Yeah I read it, I even had my own issues while attempting jails in VM Workstation as a test. Just having my thoughts about it and yes FreeNAS 10 is pretty nice with VM's! You are right.

Yes but they're crap. You're better getting a hot swap chassis if you intend to have a large (10+) drives.
You should consider the NZXT h440. This case supports 11 3.5" hard drive bays and conceals them quite well. It also has two additional SSD bays at front if you decide to go down that path.
If you (or anyone) disagrees with anything in my reply, please let me know as I am also new to this and am still learning. If you find it helpful, please hit thanks :)

h440 supports 11 bays indeed, That is actually just 1 more than the R5. On the other side it also has 8 2,5" bays, which could be nice for SSD volume But I don't expect that to in a decent price range within the next 8 years, comparing the normal drives. That is my expected lifecycle of this server.

 

[lukyjay]

What problems?

All fixed! You can see my other thread, but basically the onboard video equaled constant crashes. Turns out it was a PSU problem, not related to the motherboard.

True! I just don't like the idea of booting a critical OS off an USB drive and I see others recommend SSD over USB.

You can mirror the USB drives tho, so if one fails you have another to use :) Of course it's your server, your money, your choice. SSD would definitely be better, but it means you lose a harddrive bay, a SATA port, a SATA power plug and only 8gb~ of that drive is going to be utilized. The rest will be inaccessible. It'd definitely perform better and last longer though.

Just having my thoughts about it and yes FreeNAS 10 is pretty nice with VM's

Still a bit rusty though. Today I feel like throwing it against the wall as I can't get any of them to work :P I believe release is due end of Feb which isn't far away, and it aught to be fixed by then.

You mention price difference between Kingston and Samsung RAM, also the R5 and h440 case. In Australia its the complete opposite! The Kingston was almost $100 cheaper for me, and I cannot find the r5 anywhere. Interesting how things are so different on the other side of the world :)

 

[Victor0001]

Still a bit rusty though. Today I feel like throwing it against the wall as I can't get any of them to work :p I believe release is due end of Feb which isn't far away, and it aught to be fixed by then.

Yes and I might just wait it out, so I won't run FreeNAS 9 at all. :)

You mention price difference between Kingston and Samsung RAM, also the R5 and h440 case. In Australia its the complete opposite! The Kingston was almost $100 cheaper for me, and I cannot find the r5 anywhere. Interesting how things are so different on the other side of the world :)

Kingston memory is €132 while Samsung is €127 (for 1x16GB) :) (Netherlands)

Today I have been also researching if I might go for a 19 inch rack case but the prices will go way up compared to the R5 case and I would need to build some DIY rack storage at home. So no thanks :)
To add a new part to the list I found this UPS

Code:

Eaton Protection Station 800 DIN

I'm getting there!

 

[lukyjay]

I would need to build some DIY rack storage at home

Not really, you could sit it on a bench or something if that's the only rack thing you own. Just make sure it has enough cooling, those things get loud. A SAS backplane would be convenient with hot swap bays, but $$$$$$. Those are $600+ in Australia (for a 4u)

 

[Victor0001]

It would get as loud as you would build it. Some chassis are completely empty and you fill in the fans to make the cooling/noise as you please But I already thought it over... not gonna happen!

 

[Ericloewe]

And the i3 6100 does support it, it has 4 cores just no hyperthreading.

Nope, two cores plus hyperthreading.

 

[Victor0001]

Nope, two cores plus hyperthreading.

Oops you are right! Guess I weren't fully awake and focused this morning with 2 kids running and jumping around me.

I see in your signature Ericloewe that you have 1 single SSD as boot. Any reason why you didn't mirror it or didn't go for USB boot?

 

[Ericloewe]

Any reason why you didn't mirror it or didn't go for USB boot?

This was the reasoning:

• USB flash drives can be painfully slow (I was sick of that)
• Mirrored SSDs are rather excessive, since nothing of value is lost if the SSD hits the bitbucket.
• (In the pre-9.10 days, booting from USB on an X11SSM-F was impossible. Believe me, I tried, but couldn't even get the installer to boot.)

 

[Victor0001]

This was the reasoning:

• USB flash drives can be painfully slow (I was sick of that)
• Mirrored SSDs are rather excessive, since nothing of value is lost if the SSD hits the bitbucket.
• (In the pre-9.10 days, booting from USB on an X11SSM-F was impossible. Believe me, I tried, but couldn't even get the installer to boot.)

Thanks! Now i'm convinced to stick with SSD instead of USB :)
But I'm missing something regarding your second point. Isn't the configuration saved on the SSD? In case the SSD dies, you lose that and your whole NAS to operate. It would require a new SSD to be bought, installed and restore the configuration file. Wouldn't a mirror save you the hassle later on?

I found an SSD of 32GB, saving me 20 euros. But If I can save money by not mirroring because its just to easy to get things back up, I would surely save another 30 euros.

 

[lukyjay]

Wouldn't a mirror save you the hassle later on?

The likeliness of an SSD failing isn't very high. Since they don't have moving parts and the new ones have insanely high write amounts, it wouldn't be a wise way to spend money.

How long do you think the SSD will last? 10 years? Imagine the cost of the same SSD in 10 years time :)

 

[Ericloewe]

case the SSD dies,

Which is rather unlikely.

Wouldn't a mirror save you the hassle later on?

Possibly, but it would also occupy the last SATA port, which could otherwise be used for replacing HDDs without removing one first.

Since the hassle is small and unlikely, I don't bother with mirrored SSDs.

 

[Victor0001]

Possibly, but it would also occupy the last SATA port, which could otherwise be used for replacing HDDs without removing one first.

This convinced me . Of course! Thanks no more mirroring. Thank you

 

[GBillR]

But I'm missing something regarding your second point. Isn't the configuration saved on the SSD? In case the SSD dies, you lose that and your whole NAS to operate. It would require a new SSD to be bought, installed and restore the configuration file. Wouldn't a mirror save you the hassle later on?

I am failrly certain there is a way to backup the config files and therefore restore the configuration should the boot drive fail... I haven't gotten that far yet in my testing, but it's next on the list.

 

[artlessknave]

the chips listed on the supermicro compatibility site are literally the intergrated circuit chips that are known to work, not the DIMM chips. any company can make ram using those IC's. Crucial is a brand name of Micron, so you can use their compatibility lists pretty confidently. I bought straight from Crucial and the RAM i got worked perfectly.

flash media has limited writes and because freenas has moved to using zfs for boot, it can freely write to the boot volume. depending on your config, that could chew through the limited writes of flash media fast

VM freenas is possible, but not recomended, because its very easy to make a setup that is not at all reliable but appears so.
VM abstracts the storage hardware out to multiple levels, but zfs is specifically designed to manage the hardware directly. the abstraction can greatly confuse zfs and cause it to eat your pool trying to *fix* things.

if you REALLY want to virtualize freenas vt-d passthrough of HBA controllers (no raid) seems to be the most reliable as that gives zfs the direct control it expects

also, freenas10 includes hypervisor-like bhyve virtualization platform, so it kind of makes alot more sense to just use freenas to make any virtual machines you might want (docker is implemented with a VM)

 

[Victor0001]

the chips listed on the supermicro compatibility site are literally the intergrated circuit chips that are known to work, not the DIMM chips. any company can make ram using those IC's. Crucial is a brand name of Micron, so you can use their compatibility lists pretty confidently. I bought straight from Crucial and the RAM i got worked perfectly.

So actually, its most safe to use those Samsung DIMM's. I checked Crucial memory for the X11SSM-F but those are all not in stock and I can't find those types in The Netherlands. Only Kingston ValueRAM, but those are more expensive.

flash media has limited writes and because freenas has moved to using zfs for boot, it can freely write to the boot volume. depending on your config, that could chew through the limited writes of flash media fast

So another reason to go for SSD.

VM freenas is possible, but not recomended, because its very easy to make a setup that is not at all reliable but appears so.
VM abstracts the storage hardware out to multiple levels, but zfs is specifically designed to manage the hardware directly. the abstraction can greatly confuse zfs and cause it to eat your pool trying to *fix* things.

if you REALLY want to virtualize freenas vt-d passthrough of HBA controllers (no raid) seems to be the most reliable as that gives zfs the direct control it expects

Thanks, but I have already decided to use FreeNAS as my host OS.

also, freenas10 includes hypervisor-like bhyve virtualization platform, so it kind of makes alot more sense to just use freenas to make any virtual machines you might want (docker is implemented with a VM)

Exactly, cheers :)

 

[artlessknave]

So another reason to go for SSD.

um, ssd's ARE flash media. so are USB drives. ssd's are higher reliablity flash media, but they are still flash media, with the same eventual limit of writes.

thats why i said "flash media" not "usb drives"

 

[Victor0001]

Uch of course... I feel dumb.