First FreeNAS - Hardware Feedback (Overkill? / Compatibility)

[SeaWolfX]

Hi guys,

So I have been putting this on hold for a while, but I am finally planning on building my first FreeNAS server. I have some requirements for the server and I although I have read up on some stuff I would like to check with you guys here if everything is feasible with FreeNAS and get any comments/feedback you might have. I would also like your input on hardware selection. I typically overkill and would really appreciate any input you might have so that I do not overspend on unnecessary specs. E.g. memory type / how much, do I need read/write cache, motherboard etc.?

It is going to be a family server primarily used for media streaming and as a file server. There will be a common storage space (min. 10 TB, with option for future expansion) for media (movies, music, pictures, books, etc.) as well as personal storage spaces (min. 100 GB) for each member. Although not a primary objective I am planning for it to maybe host a couple if Windows/Linux VMs of a separate SSD disk. The server will be located at a location with internet speed 500/500. Both common and personal directories should be able to be mapped as "network drives" on the users personal computers outside the server local network, so that file handling can be done as if the user was operating on folders on the local computer (what solutions are available for this?).

Main objectives:
- Media Server (streaming HD movies and music) via Plex
- VM
- File Server
- Data Integrity & Backup

Number of users:
Total: 8
Max simultaneous access: 3-4 (e.g. streaming movies / transferring files)

Hardware:
I based my suggestions below on the FreeNAS recommendations and it will probably set me back around $2400 which is bit more than I was aiming for. So If you have some suggestions on what I might cut back on I would really appreciate it. Keep in mind that I would like the setup to be a bit flexible with regards to future expansions / upgrades. Cabinet and PSU i have from before.

Motherboard
Supermicro X11SSH-F

Processor
Intel Xeon E3-1230v6 3,5GHz Socket 1151 Tray

Memory
Crucial DDR4 2400MHz ECC 2x16GB (CT2K16G4WFD824A)

Harddrives
4 x WD Red WD60EFRX 64MB 6TB
1 x Samsung SSD 256 GB

UPS
APC Back-UPS Pro BR900GI
(for safe shutdown on power-loss)

 

[Beemer533]

I'm far from an expert, but I can make a couple of comments;

According to the drive calculator:
https://wintelguy.com/zfs-calc.pl

Using Z2 and accounting for the 20% free space minimum, you'll only end up with about 8TB

You can't put anything other than the os on the SSD so 256gb is somewhat of a waste.

I went with a 120gb as it was about the smallest name brand ssd I could get new.

Sent from my SAMSUNG-SM-G930A using Tapatalk

 

[SeaWolfX]

I'm far from an expert, but I can make a couple of comments;

According to the drive calculator:
https://wintelguy.com/zfs-calc.pl

Using Z2 and accounting for the 20% free space minimum, you'll only end up with about 8TB

You can't put anything other than the os on the SSD so 256gb is somewhat of a waste.

I went with a 120gb as it was about the smallest name brand ssd I could get new.

Sent from my SAMSUNG-SM-G930A using Tapatalk

@Beemer533 Thanks for your input. I am planning for a remote backup solution as well so I think that I should OK with a Z1 configuration.

The SSD was not intended for for the OS, but for VMs to be separated from the WD drive storage pool. I am planning on a external memory stick for the OS.

 

[charlie89]

Here are my thoughts:
As you mentioned, you use an memory stick for the OS: they are unreliable, so use 2 of them and mirror them via the webgui.

If you host VMs, 32GB may not be enough.

You intend to have the server running somewhere accessible over the internet? Never expose Freenas directly to the internet. There has to be a firewall (pfsense for example) between the internet and your freenas box. I would then run a vpn server on the firewall and allow all vpn users access to the nas. This way, you can use samba to have your storage mounted as network drives, even over the internet.

You said, one of the main purposes is video streaming. You're probably not streaming everything in original quality, so your cpu has to do transcoding. 3-4 transcodes of 1080p may be possible with this cpu, but with high quality 1080p or even 4k, I would use a faster cpu with a passmark score greater than the nearly 10.000 of your chosen cpu.

I personally would go with used parts. Something like Supermicro X9 board and DDR3 ram. This way you should get a faster cpu and more ram for the same money (at the expense of a little bit more power consumption).

 

[kdragon75]

Processor
Intel Xeon E3-1230v6 3,5GHz Socket 1151 Tray

This is a bad CPU for VMs. There are simply not enough cores.
https://itknowledgeexchange.techtarget.com/virtualization-pro/scheduling-virtual-cpus/

Crucial DDR4 2400MHz ECC 2x16GB (CT2K16G4WFD824A)

32GB of RAM is still on the low side for VMs and a media server. Doable but low.

The SSD was not intended for for the OS, but for VMs to be separated from the WD drive storage pool. I am planning on a external memory stick for the OS.

You won't have any redundancy on your VMs then. You will need to backup your VMs to your main pool. Backing up running VMs is not advisable as the backup copy will basically be like if you pulled the power cord. It works but you may have some corruption in the backup.

 

[kdragon75]

You intend to have the server running somewhere accessible over the internet? Never expose Freenas directly to the internet. There has to be a firewall (pfsense for example) between the internet and your freenas box. I would then run a vpn server on the firewall and allow all vpn users access to the nas. This way, you can use samba to have your storage mounted as network drives, even over the internet.

pfSense is amazing for this and I have setup several clients with pfSense/IPsec for remote backup and remote sites to access the main server.

 

[JohnK]

Both common and personal directories should be able to be mapped as "network drives" on the users personal computers outside the server local network, so that file handling can be done as if the user was operating on folders on the local computer (what solutions are available for this?).

I think you hardware looks fine. Read thread below for remote access.
https://forums.freenas.org/index.ph...r-freenas-server-remotely-and-securely.27376/

Personally, I prefer NextCloud. Also, if I was going to do a VPN tunnel, I prefer OpenVPN configured on my firewall

 

[kdragon75]

Both common and personal directories should be able to be mapped as "network drives" on the users personal computers outside the server local network, so that file handling can be done as if the user was operating on folders on the local computer (what solutions are available for this?).

For this,I'll revise my comment on IPsec, Use OpenVPN with the client exporter on pfSense. It will give you a single file to run for each client making setup dead simple.

 

[SeaWolfX]

Thanks for a lot of good input. However, I am sorry the focus became so much on the VM (a bit my fault I guess since I put it down under Main Objectives for the server). VMs is by no means the primary goal of this server and the hardware selection should not be focused around accommodating that. It is more a nice to have thing I intended to play around with :)

 

[JohnK]

Thanks for a lot of good input. However, I am sorry the focus became so much on the VM (a bit my fault I guess since I put it down under Main Objectives for the server). VMs is by no means the primary goal of this server and the hardware selection should not be focused around accommodating that. It is more a nice to have thing I intended to play around with :)

I use similar hardware and run 2 Win 7s, FreeNas , Plex and NextCloud in VMs. I have no trouble with performance.

One thing you do mention is Plex. Depending on what you are trying to stream/transcode, it could require more CPU. There are many post on this forum, but I have not seen comprehensive benchmarks. (Not that I have looked too much :) )

 

[SeaWolfX]

I use similar hardware and run 2 Win 7s, FreeNas , Plex and NextCloud in VMs. I have no trouble with performance.

One thing you do mention is Plex. Depending on what you are trying to stream/transcode, it could require more CPU. There are many post on this forum, but I have not seen comprehensive benchmarks. (Not that I have looked too much :) )

Good to hear!

I am a bit surprised regarding feedback from you and some of the others above on the CPU though. I selected the CPU from the 'Heavy Usage' section of the FreeNAS Community Hardware Guide and so I would have though it would be more than enough for at least 2 simultaneous 1080p transcodes (which I believe would be the maximum the server would be subjected to), but maybe I'm missing something. I'm definitely not an expert on this. Any suggestions on what type of CPU I should go for?

 

[JohnK]

Good to hear!

I am a bit surprised regarding feedback from you and some of the others above on the CPU though. I selected the CPU from the 'Heavy Usage' section of the FreeNAS Community Hardware Guide and so I would have though it would be more than enough for at least 2 simultaneous 1080p transcodes (which I believe would be the maximum the server would be subjected to), but maybe I'm missing something. I'm definitely not an expert on this. Any suggestions on what type of CPU I should go for?

I think you will be fine with that processor.
For example, when I stream 2 movies (h265) using plex with ultrafast transcoding to my computer and iPhone, I manage to tax the CPU at 20% usage. I believe once you go full on 4k, you might need more.

 

[SeaWolfX]

I think you will be fine with that processor.
For example, when I stream 2 movies (h265) using plex with ultrafast transcoding to my computer and iPhone, I manage to tax the CPU at 20% usage. I believe once you go full on 4k, you might need more.

Yeah, might be. At the moment i do not have any 4K movies though, mostly 720p and 1080p of varying quality.

Here are my thoughts:
As you mentioned, you use an memory stick for the OS: they are unreliable, so use 2 of them and mirror them via the webgui.

If you host VMs, 32GB may not be enough.

You intend to have the server running somewhere accessible over the internet? Never expose Freenas directly to the internet. There has to be a firewall (pfsense for example) between the internet and your freenas box. I would then run a vpn server on the firewall and allow all vpn users access to the nas. This way, you can use samba to have your storage mounted as network drives, even over the internet.

You said, one of the main purposes is video streaming. You're probably not streaming everything in original quality, so your cpu has to do transcoding. 3-4 transcodes of 1080p may be possible with this cpu, but with high quality 1080p or even 4k, I would use a faster cpu with a passmark score greater than the nearly 10.000 of your chosen cpu.

I personally would go with used parts. Something like Supermicro X9 board and DDR3 ram. This way you should get a faster cpu and more ram for the same money (at the expense of a little bit more power consumption).

I'll make sure I buy two of the same kind then. Are there any particular requirements for the memory sticks running FreeNAS OS?

I was planning on setting u a VPN to access the server from the outside. I am a lot on travel and I need access to it when I am away from home and I would like some of my family members to have access to it as well. That will mainly include access to a personal storage for file-backup as well as access to the media folders so that they can upload and download movies, music etc. I have been wondering about the best way to go about this that do not involve to much "high-tech" hassle. For instance would the it be best to set up a VPN on my home router or on a FreeNAS Jail? When you say firewall, do you mean like a separate device performing the firewall function besides my router?

 

[JohnK]

Y
I'll make sure I buy two of the same kind then. Are there any particular requirements for the memory sticks running FreeNAS OS?

Quoting straight from the hardware guide:
USB Flash Drives - The traditional boot device for FreeNAS has been a USB flash drive. These can be easily mirrored for increased reliability and are cheap. Unfortunately, reliability can be very bad and random write performance (which gets exercised during updates and other boot environment manipulations) is almost as bad.
These are certainly a workable solution, but not the ideal one. USB Flash Drives are most useful where using up an additional SATA port would add a significant cost (such as the cost of an HBA). Using drives from reputable manufacturers (SanDisk, Toshiba, Lexar) should improve the expected quality.

SSDs - More recently, SSDs have become popular boot devices. These solve all of the problems associated with USB flash drives, at the expense of a higher price tag and a SATA or M.2 port.
Cheap, low-end SSDs from reputable brands work very well.
SATA DOMs - A sort of compromise between SSD and USB flash drive, these employ a more compact form factor than most SATA SSDs and generally include provisions for being powered from compatible SATA ports (available on newer Supermicro motherboards, for instance). Their main disadvantage is cost, which tends to be much higher than that of SSDs

I prefer using SATA DOM for Freenas on my backup server and ESXI on my main server. Have read somewhere that people were having problems with SanDisk USB at some stage. I used the 16GB Sandisk Glide (USB 2) in the past and never lost one.

When you say firewall, do you mean like a separate device performing the firewall function besides my router?

Read the part about not exposing FreeNas to internet. Thought there are many ways to achieve your objective, I believe you will find that many FreeNas users also use PFSense as both firewall and router. And lots of other things :) You can set up OpenVPN with relative ease.

If you are using an off-the-shelf router, you might be able to do a lot if the same functionality, but don't always have the flexibility nor speed. You can imagine that those routers don't have the processing power of an Intel withAES New Instructions for encryption/decryption.

Personally, I run PFsense on its own hardware, Atom 2358, but you could virtualize it onto the same server as your FreeNas using ESXI.