My first FreeNAS build

[jbduncan]

Hi all,

I'm looking to build my first FreeNAS server (exciting stuff!), but after having done a lot of research over the last few days, such as reading the "FreeNAS Community Hardware Guide" and "FreeNAS Quick Hardware Guide" on this forum, and coming up with part of a build that I believe will fit my needs, I'm struggling to figure out how to fill the gaps in this build, and I'm not even sure if the parts I have selected so far are even suitable for my needs. So I was wondering if you could help me move forward.

My use case is as follows:

• Replace my existing openmediavault system, which is running on an Odroid HC2, with a system that is hopefully less likely to fail on me in the foreseeable future.
• Store a bunch of FLAC & MP3 music and other personal files (~160GB), accessible over SMB or Syncthing.
• Potentially store music belonging to my father, adding up to 15GB currently, but which may increase significantly in the future.
• Be able to stream all this music to our home SONOS system over SMB, or Plex or some other media server. Not sure if I'll need transcoding. If I use a media server, I'll probably install it as a plugin or Jail on the FreeNAS system.
• Have a daily backup process, preferably using Borgbackup and Borgmatic as that's my current solution. I understand that I can use a custom Jail for this.
• Potentially store ISOs for a separate Proxmox/XCP-ng hypervisor build that I'm considering.
• Keep the energy bill as low as reasonably possible. Other than keeping the FreeNAS machine off during the night, I understand that there exist FreeNAS builds out there that consume only 50-60 watts when running. If so, I'd love to achieve this.

Here is what my build looks like:

• Storage drives: 3 2TB Seagate Ironwolf hard drives, as part of a RAIDZ2 setup.
• OS drive: 120GB 2.5" Crucial BX500.
• Case: Fractal Design Node 804.
• Motherboard: ?
• RAM: 16GB ECC RAM, such as Kingston 16GB PC4-17000. Might choose another kind depending on which motherboard I go for.
• CPU: ?
• PSU: ?
• UPS: CyberPower UT650EIG.

I've read the hardware guides on what motherboard to go for, but I'm having trouble acting upon their advice. Specifically, they seem a little out-of-date in the sense that all the motherboards I've checked seem to be end-of-life now (although I'd love to be proven otherwise on this). Furthermore, to take the Supermicro X11SCH-F as an example, when I search for sellers of it here in the UK, its price range seems to vary wildly (anything between £280-400), and it doesn't seem to be available on pcpartpicker to compare with other mobos. My understanding so far is I need a Supermicro motherboard with: support for ECC RAM; IPMI; 4+ SATA 6GB ports, ideally 6 or more for hard disk expansion; and maybe support for Intel LGA1151 socket type. But entering those specs into pcpartpicker doesn't narrow down my options very much.

Regarding the CPU, the hardware guides' advice on this also seems a bit out-of-date because the ones I've checked under the sections "Light usage" and "Medium usage" also seem to be at end-of-life (but again I'd love to proven otherwise).

Regarding the PSU, I know that I shouldn't skimp on it because otherwise it may not be powerful enough to power all the other components. I've thought of using pcpartpicker to find a suitable one when I've chosen all the other components, but my research so far has shown that not all Supermicro motherboards are available to choose from there, so I won't be able to choose a PSU there reliably. I then saw that one of the hardware guides said that I can measure the power I need with the formula "(30W+TDP of other parts)*1.2", but I'm not entirely sure what TDP means in this context. Once I figure this out, I'm sure I can then choose a suitable PSU from the "FreeNAS Quick Hardware Guide".

So, here are my questions:
1) Any thoughts on which motherboard I should go for, or even better, any advice on how I can find a suitable one myself?
2) Any thoughts on which CPU would work best for this motherboard given my use case above, or even better any advice on how I can research this myself?
3) Does anyone know what "TDP" means in the context of a choosing a PSU?
4) Will this build consume 50-60 watts, or is it too hard to tell at this point?
5) Any obvious components that I'm missing, or anything that I can do to improve this build in general?

I look forward to your responses!

Best regards,
Jonathan

 

[warllo]

Hi all,

I'm looking to build my first FreeNAS server (exciting stuff!), but after having done a lot of research over the last few days, such as reading the "FreeNAS Community Hardware Guide" and "FreeNAS Quick Hardware Guide" on this forum, and coming up with part of a build that I believe will fit my needs, I'm struggling to figure out how to fill the gaps in this build, and I'm not even sure if the parts I have selected so far are even suitable for my needs. So I was wondering if you could help me move forward.

My use case is as follows:

• Replace my existing openmediavault system, which is running on an Odroid HC2, with a system that is hopefully less likely to fail on me in the foreseeable future.
• Store a bunch of FLAC & MP3 music and other personal files (~160GB), accessible over SMB or Syncthing.
• Potentially store music belonging to my father, adding up to 15GB currently, but which may increase significantly in the future.
• Be able to stream all this music to our home SONOS system over SMB, or Plex or some other media server. Not sure if I'll need transcoding. If I use a media server, I'll probably install it as a plugin or Jail on the FreeNAS system.
• Have a daily backup process, preferably using Borgbackup and Borgmatic as that's my current solution. I understand that I can use a custom Jail for this.
• Potentially store ISOs for a separate Proxmox/XCP-ng hypervisor build that I'm considering.
• Keep the energy bill as low as reasonably possible. Other than keeping the FreeNAS machine off during the night, I understand that there exist FreeNAS builds out there that consume only 50-60 watts when running. If so, I'd love to achieve this.

Here is what my build looks like:

• Storage drives: 3 2TB Seagate Ironwolf hard drives, as part of a RAIDZ2 setup.
• OS drive: 120GB 2.5" Crucial BX500.
• Case: Fractal Design Node 804.
• Motherboard: ?
• RAM: 16GB ECC RAM, such as Kingston 16GB PC4-17000. Might choose another kind depending on which motherboard I go for.
• CPU: ?
• PSU: ?
• UPS: CyberPower UT650EIG.

I've read the hardware guides on what motherboard to go for, but I'm having trouble acting upon their advice. Specifically, they seem a little out-of-date in the sense that all the motherboards I've checked seem to be end-of-life now (although I'd love to be proven otherwise on this). Furthermore, to take the Supermicro X11SCH-F as an example, when I search for sellers of it here in the UK, its price range seems to vary wildly (anything between £280-400), and it doesn't seem to be available on pcpartpicker to compare with other mobos. My understanding so far is I need a Supermicro motherboard with: support for ECC RAM; IPMI; 4+ SATA 6GB ports, ideally 6 or more for hard disk expansion; and maybe support for Intel LGA1151 socket type. But entering those specs into pcpartpicker doesn't narrow down my options very much.

Regarding the CPU, the hardware guides' advice on this also seems a bit out-of-date because the ones I've checked under the sections "Light usage" and "Medium usage" also seem to be at end-of-life (but again I'd love to proven otherwise).

Regarding the PSU, I know that I shouldn't skimp on it because otherwise it may not be powerful enough to power all the other components. I've thought of using pcpartpicker to find a suitable one when I've chosen all the other components, but my research so far has shown that not all Supermicro motherboards are available to choose from there, so I won't be able to choose a PSU there reliably. I then saw that one of the hardware guides said that I can measure the power I need with the formula "(30W+TDP of other parts)*1.2", but I'm not entirely sure what TDP means in this context. Once I figure this out, I'm sure I can then choose a suitable PSU from the "FreeNAS Quick Hardware Guide".

So, here are my questions:
1) Any thoughts on which motherboard I should go for, or even better, any advice on how I can find a suitable one myself?
2) Any thoughts on which CPU would work best for this motherboard given my use case above, or even better any advice on how I can research this myself?
3) Does anyone know what "TDP" means in the context of a choosing a PSU?
4) Will this build consume 50-60 watts, or is it too hard to tell at this point?
5) Any obvious components that I'm missing, or anything that I can do to improve this build in general?

I look forward to your responses!

Best regards,
Jonathan

Hi,

I'll do my best to answer your questions.

1) Given your use case and using new hardware I like the Supermicro X11SCL-F for mainstream builds. You get 6 sata ports, dual nics and an M.2 slot
2) Paired with the X11SCL-F check out the i3-9100f it fully supports ECC memory and should provide plenty of CPU provided you are not transcoding. Please make sure your Plex clients support direct stream otherwise you might need a more powerful CPU. Supermicro generally provides good documentation as to what is and isn't compatible with their products on their website.
3) My understanding of TDP is the maximum amount of heat generated by the device. Using the Node 804 you should be ok to choose a mainstream power supply. I really like the Seasonic brand but to each there own. For this build I personally would go with the Seasonic Core GX 650
4) I'll let someone more qualified try to estimate the power consumption but maxed out with 6 drives I'm pretty sure you would more than exceed 50-60 watts
5) A few tips that I have picked up along the way include: Use test memory modules which you can find on the motherboards product page, this will save you some potential headache not to say other memory wouldn't work. Also, make sure to avoid SMR drives they have at best poor performance with Freenas.

 

[joeschmuck]

Power consumption of 50-60 watts is awful low but not impossible. I can see that when the system is idle but not running full speed unless you bought a mobile/low power type CPU but then performance drops. On general each hard drive is 5 watts of power so three hard drives = 15 watts. Also, your current plan gives you ~4TB of storage, and that might be fine for you but if you plan to fill it up to 2TB immediately then consider more storage, maybe larger capacity drive or addition of another drive. I always tell someone to plan 5 years out for storage because a hard drives typically will last 5 years if you keep them cool and don't cycle power on it a lot.

Question: Is the sole purpose of this NAS to stream music and host a few files? If yes then you can buy a low power CPU, actually complete motherboard with CPU and it will draw minimal power. If you plan to stream video content and use Plex to transcode, you will need more CPU horsepower.

As previously stated, avoid SMR drives.

You didn't mention one important thing, how much can you spend? It's a huge factor.

Also, it's best to leave the hard drives running, they tend to last longer and the hard drives are overall the largest investment for a NAS, and must be replaced as they fail, and they will fail.

Did you read the "Hardware Recommendations Guide" in the resources section. you need to download the PDF to read it and I think it has all your answers. A lot of time was spent to create this guide so take the time to read it.

 

[jbduncan]

@warllo @joeschmuck Thank you very much for your enlightening responses! Sorry for taking so long to reply, it's been a busy week for me. I've addressed your questions and jotted down my thoughts to your posts below.

Please make sure your Plex clients support direct stream otherwise you might need a more powerful CPU.

Good point. I've gone and checked how SONOS works, and apparently it doesn't (and perhaps even can't) transcode music, and it's limited to playing <=16-bit music. This hasn't been a problem for me over the ~5 years we've had SONOS, but that may change in the future if I ever procure 24-bit music.

Video-wise, I've found out my Android phone isn't able to receive a stream of 1080p video from the Odroid HC2 over SMB without buffering (using the VLC Media Player Android app), but I only watch my personal video files occasionally and always on a capable PC.

Having said all this, I don't know if I will ever want to watch personal video on my phone or on e.g. a smart TV in the future, or if my father and other family members have music/video that would require transcoding. So it sounds like I should go for a CPU that can do transcoding to be on the safe side? If so, which CPU would you recommend?

...check out the i3-9100f...

On a related note, I'm reminded of a YouTube video I found about a 55 watt build that uses a Supermicro A2SDi-8C+-HLN4F with an embedded Intel Atom C3758 processor and a TDP of 25W. By my understanding, Atom processors are even weaker than i3s, so as much as the reduced energy usage appeals to me, I don't suppose this board would be suitable for my needs, would it?

My understanding of TDP is the maximum amount of heat generated by the device. Using the Node 804 you should be ok to choose a mainstream power supply. I really like the Seasonic brand but to each there own. For this build I personally would go with the Seasonic Core GX 650

Many thanks! Would the Seasonic Core GX 650 also be suitable for a transcoding-capable CPU?

I'll let someone more qualified try to estimate the power consumption but maxed out with 6 drives I'm pretty sure you would more than exceed 50-60 watts

Power consumption of 50-60 watts is awful low but not impossible. I can see that when the system is idle but not running full speed unless you bought a mobile/low power type CPU but then performance drops. On general each hard drive is 5 watts of power so three hard drives = 15 watts.

Good to know, so thank you both for enlightening me. If I have to live with above 60 watts when the FreeNAS server is idle, then so be it. :)

A few tips that I have picked up along the way include: Use test memory modules which you can find on the motherboards product page, this will save you some potential headache not to say other memory wouldn't work. Also, make sure to avoid SMR drives they have at best poor performance with Freenas.

As previously stated, avoid SMR drives.

Also, it's best to leave the hard drives running, they tend to last longer and the hard drives are overall the largest investment for a NAS, and must be replaced as they fail, and they will fail.

Awesome, those tips match my understanding! Re. the RAM, I went for the Kingston as an arbitrary example to highlight what sort of RAM I wanted; I definitely intend to use the Supermicro website to determine which RAM works well. Re. SMR drives, I'm glad then that I'm thinking of going for Seagate Ironwolfs, as I understand those are CMR drives.

Also, your current plan gives you ~4TB of storage...

Oh wow, really? I'm pleasantly surprised to hear that, because I thought my intended setup would only give me 2TB of storage. Good to know!

...that might be fine for you but if you plan to fill it up to 2TB immediately then consider more storage, maybe larger capacity drive or addition of another drive. I always tell someone to plan 5 years out for storage because a hard drives typically will last 5 years if you keep them cool and don't cycle power on it a lot.

Thanks for the tip! I don't intend to fill up to 2TB immediately. I'm not even sure that my personal files and ISOs and my father's music even add up to that much, so 4TB sounds like overkill for me. It depends if I decide to build a hypervisor server separately, I suppose, but even then I struggle to imagine all the VMs I'd want taking up that much space, assuming I decide to store the VMs on the FreeNAS somehow; this is where my understanding stops.

Am I right to infer then that 3 1TB hard drives in RAIDZ2 would be enough to give me 2TB of storage?

You didn't mention one important thing, how much can you spend? It's a huge factor.

Excellent question! My budget is £1000. I'm more concerned about the monthly energy cost to run the FreeNAS (for environmental reasons) than the initial upfront cost.

Did you read the "Hardware Recommendations Guide" in the resources section. you need to download the PDF to read it and I think it has all your answers. A lot of time was spent to create this guide so take the time to read it.

I did, or at least I thought I did at the time. I remember I was feeling stressed from going around in circles with my research, so I'm pretty sure I failed to read it completely, but by no means intentionally. And regarding the related "Quick Hardware Guide", I remember looking into a bunch of the recommended motherboards and CPUs and finding they were either end-of-life and unavailable, or that different retailers sold them at surprisingly different prices, so that didn't help my stress either. So thank you for querying me on this. :)

@warllo @joeschmuck Any further thoughts?

 

[joeschmuck]

Oh wow, really? I'm pleasantly surprised to hear that, because I thought my intended setup would only give me 2TB of storage. Good to know!

Sorry, I screwed up and used someone else's calculator than the one I normally use. 3 drives of 2TB as RAIDZ2 = 1.8TB of storage, minus about 6GB (2GB from each drive). Very sorry for the misinformation. On a more positive note, you are not storing much data at all so if 4TB is going to keep you happy for 5 years, I'd recommend using three 4TB hard drives which gives you ~3.6TB of storage.

Excellent question! My budget is £1000. I'm more concerned about the monthly energy cost to run the FreeNAS (for environmental reasons) than the initial upfront cost.

So that is basically $1340 USD, which isn't much over here, this is going to make me ask you to do a few things to ensure you can afford this project.

1) Price each of the major components that you think you want to buy:
a) Case
b) Fans (if any as many cases include some case fans)
c) Hard Drives/Solid State Drives
d) Power Supply
e) UPS

What is your total?

2) The following pricing is completely dependent on what Motherboard you choose:
a) Motherboard
b) CPU (if not built in, note that a built in CPU is typically a low power and Slow CPU)
c) CPU Fan (only if the CPU does not come with a boxed fan, and I prefer the OEM boxed fan which works great and keeps costs down)
c) RAM (16GB is not very expensive at this time and you should consider upgrading to 32GB if you can afford it, but 16GB is enough for now)

Now what is your total?

The parts would should never skimp (get cheap quality) on are: Power Supply, Mother Board, RAM, CPU. All other items like Hard Drives, Fans, UPS will and do fail over time. The case should be one that allows your component to stay cool which will allow those to last longer. If you buy a fanless Power Supply then you have just reduced one future failure of a fan which you might not be able to replace (depends on your skills).

Honestly, there are a lot of things to think about.

As for products nearing End Of Life, well those could be the ones you get a great deal on. But I personally would buy a newer model but I also wait until I can afford exactly what I want, if that means waiting 6 more months, then that is what I do.

Transcoding Music: To be honest I have no idea how much CPU horsepower is required to transcode music, I can't expect it to be much. Most of the time here on the forums when we are talking about transcoding, it's video transcoding. A mobile/low power CPU will not handle video transcoding.

Recommended CPU's : If I were buying a CPU it would be a boxed Xeon (boxed because it comes with the CPU fan). Even the E3-1220 is a good unit and would provide the horsepower needed to transcode your music easily. You could easily get away with I think a i3-9100 for what you are asking to do. And remember that in general, the newer the CPU, the more energy efficient it will be.

Recommended Motherboards: I prefer a Supermicro motherboard but there are less expensive ones out there. If you really want a solid performer with ECC RAM, then that is where I'd go. But I've been a rebel before and ran an AMD setup, it worked great except form one thing, I have to add an Intel NIC card, the AMD motherboard had a RealTek NIC and it was problematic.

Hey, two more big questions and this may sway your path forward:
1) Can you convert your music into a compatible format that all your devices can use, or at least most? I as because if you do not need transcoding then you could run a very inexpensive system.

2) Is there any reason you wouldn't want to use a pre-built NAS system like QNAP for example? QNAP TS-832X 2G Diskless 8-Bay Professional-grade NAS offers video transcoding using hardware acceleration, not sure if that also includes music, and it is about half of your budget. Then you just add some hard drives and you should be done. It's worth looking into is all I'm saying. I'm not trying to talk you out of FreeNAS, but I'm trying to find a solution that is low power, low cost, and gives you the support you want. FreeNAS is one of those things that you would have to keep on top of, it is not "Set and Forget", that will bite you in the ass.

Lets talk energy consumption briefly. While you can make a system that uses little power, it will not have sufficeint power to transcode, or buy something too slow and your data throughput overall could be super slow. Trying to design an ultra low power system is not easy and takes a lot of work and risk as if you choose wrong and it's too slow, then you have a system that makes you unhappy and you need to spend more money upgrading.

My main system running ESXi and FreeNAS on top of that, 64GB RAM, E3-1240 Xeon CPU, 4 Hard Drives, 1 Laptop HD, 1 SSD, Running Plex in a jail is pulling 63 watts idle. When I stream video content I hit points of up to 70 watts but to be honest, I don't think any real transcoding is going on since my devices will use the content as formatted. I have not used my cell phone to play content, the screen is too small when I have a perfectly good 65" TV to use. As for high use, Since I only run Plex on FreeNAS, the power use stays in the 60 watt area 99% of the time.

Also, you can sleep the hard drives if you setup your system properly and that will save you about 5 watts of power per drive, but the trade off is typically a premature hard drive failure due to constant spinup cycles. You can read more about that in the forums here, you just need to search for it and there is a script to run to ensure the drives do sleep.

I hope I gave you a few things to think about.
1) Can you afford to Proprely build a FreeNAS system?
2) Are there other options available such as QNAP?
3) Can I ensure my music format does not require transcoding and then I can buy a low power computer that will meet my needs?

I wish you luck, you have a lot to think about and I have no idea what components are available in the UK.

Forgot one topic, the case. Do you need a small case? The larger the case = Cooler component in general. A High Air Flow case is typically large but you can place a lot of items in it, including very large slow turning fans to move lots of air quietly. Your components will last longer as a rule vs. a compact hot case, unless the system was designed for those temperatures.

Later.

 

[jbduncan]

@joeschmuck Wow, you've given me lots to think about!

Apologies in advance for the wall of text. :)

My main system running ESXi and FreeNAS on top of that...

Ah, interesting you say this, because I was actually considering a FreeNAS-on-top-of-Proxmox-or-XCP-ng setup early in my research (mainly out of interest, but also so that I could install e.g. PiHole in a separate VM or container, without needing to do potentially irreversible changes on my Odroid running openmediavault). I decided against it because I remember having read somewhere that running FreeNAS on a hypervisor was a bad idea, I think due to needing the pass the hard disks straight through to the FreeNAS VM for ZFS, which can be tricky.

I'm also pleasantly surprised to hear that your setup only uses 63 watts when idle (by comparison my gaming PC uses 125-150 watts depending, and an old fashioned light bulb here in the UK uses 100 watts). So that sounds quite encouraging!

I'm now re-considering if I want FreeNAS running on bare metal after all. Is running FreeNAS on a hypervisor something worth pursuing? If so, do you happen to know of any resources that I could use to find out more, or do you have any other thoughts?

The rest of my post assumes I'd still be running FreeNAS on bare metal...

So that is basically $1340 USD, which isn't much over here, this is going to make me ask you to do a few things to ensure you can afford this project.

1) Price each of the major components that you think you want to buy:
a) Case
b) Fans (if any as many cases include some case fans)
c) Hard Drives/Solid State Drives
d) Power Supply
e) UPS

What is your total?

1)
a) Fractal Design Node 804 case (a microATX case): £99.98 (scan.co.uk)
b) Fans: The Node 804 has space for 7 fans other than the built-in ones. Not sure how many I need. 4x top fans, Fractal Silent Series R3 140mm: £37.96 (scan.co.uk)
c) 1x Crucial BX500 120GB SSD + 3x Seagate Ironwolf 4TB HDDs = £423.92 (scan.co.uk)
d) Seasonic Core GC 650 (I couldn't find a GX on Google, not clear what the difference is): £79.99 (scan.co.uk)
e) CyberPower UT650EIG UPS: £45.99 (ebay.co.uk)

Total so far: £687.84

2) The following pricing is completely dependent on what Motherboard you choose:
a) Motherboard
b) CPU (if not built in, note that a built in CPU is typically a low power and Slow CPU)
c) CPU Fan (only if the CPU does not come with a boxed fan, and I prefer the OEM boxed fan which works great and keeps costs down)
c) RAM (16GB is not very expensive at this time and you should consider upgrading to 32GB if you can afford it, but 16GB is enough for now)

Now what is your total?

2)
a) Motherboard: Supermicro X11SCL-F: £234.89 (ebay.co.uk)
b) CPU: Intel i3-9100: £105.98 (scan.co.uk)
c) CPU fan: I believe Scan sell a boxed version of the i3-9100, so no need for separate fan
d) RAM: Micron MTA18ADF2G72AZ-2G6E1 16GB ECC RAM: £185.00 (mouser.co.uk)

Grand total: £1213.71

Wow, I hadn't appreciated before how much money a FreeNAS would cost in individual parts! (And a hypervisor build would cost more!) But I'm not too bothered; for me, my budget of £1000 is only a soft goal, as I'm fortunate to have the savings to be able to spend significantly more. If it approached £2000, then I would've thought twice.

As for products nearing End Of Life, well those could be the ones you get a great deal on...

Oh, good to know!

Recommended CPU's : If I were buying a CPU it would be a boxed Xeon (boxed because it comes with the CPU fan). Even the E3-1220 is a good unit and would provide the horsepower needed to transcode your music easily. You could easily get away with I think a i3-9100 for what you are asking to do. And remember that in general, the newer the CPU, the more energy efficient it will be.

Also good to know! Especially if I decide to take the hypervisor route after all.

Recommended Motherboards: I prefer a Supermicro motherboard but there are less expensive ones out there. If you really want a solid performer with ECC RAM, then that is where I'd go...

Cool, that matches my understanding and what I've seen recommended on this corner of the internet as well as YouTube. :)

Now, onto your two big questions:

1) Can you convert your music into a compatible format that all your devices can use, or at least most? I as because if you do not need transcoding then you could run a very inexpensive system.

Good question, I hadn't considered that before! Yes, I'm sure I could convert my music to a suitable format if or when I find SONOS isn't able to play a new album in the future.

That being said, this might be moot if I decide to go for a Xeon as part of a FreeNAS-on-hypervisor build, because then I could run Plex and stream to SONOS without any buffering or compatibility problems.

2) Is there any reason you wouldn't want to use a pre-built NAS system like QNAP for example? QNAP TS-832X 2G Diskless 8-Bay Professional-grade NAS offers video transcoding using hardware acceleration, not sure if that also includes music, and it is about half of your budget. Then you just add some hard drives and you should be done. It's worth looking into is all I'm saying. I'm not trying to talk you out of FreeNAS, but I'm trying to find a solution that is low power, low cost, and gives you the support you want. FreeNAS is one of those things that you would have to keep on top of, it is not "Set and Forget", that will bite you in the ass.

There's a few reasons:
a) I used to run a Synology with 512MB RAM (it ran "DSM", Synology's proprietary distribution of Linux), but it lacked a good package manager to download and install Borgbackup and Borgmatic, my preferred backup solution, and it wasn't powerful enough to run a Docker container with Borgmatic on top, or any container for that matter.

b) I then bought an Odroid HC2 a few months ago, running openmediavault (so Debian, which has Borgmatic through apt), which met my needs. But then I learned about FreeNAS's ZFS and ECC RAM support, which the paranoid part of me really likes, since I quite like the idea of keeping my data as pristine as possible, or at least being warned if my data is ever changed unexpectedly; I'd like to avoid backing up corrupt data if possible.

I understand that a UPS will help prevent data corruption from abrupt shutdowns e.g. power cuts, but I also understand that cosmic radiation can cause bit flips and in turn cause data corruption, which ECC RAM and ZFS have a good chance of correcting.

The particular QNAP you linked doesn't seem to have ECC RAM; not sure about ZFS. But a quick Google search tells me that QNAP the company does do ECC RAM in some way, and that they have an OS called "QuTS" with ZFS support? But I don't know yet which of their NASs support both ECC and ZFS, and if any of them have a good old Linux or UNIX package manager. I consider package manager support to be important.

Thus I think FreeNAS would best suit my needs (although I'm happy to be corrected on this), whilst keeping my paranoia to a minimum. :)

I'm actually unclear what you mean by "Set and Forget". I have my openmediavault set up to send me an email if a S.M.A.R.T. check fails, and I have a healthchecks.io check that warns me if a backup fails or doesn't run. So unless I'd need to monitor the FreeNAS a lot more than that, then that should be perfectly fine!

Lets talk energy consumption briefly. While you can make a system that uses little power, it will not have sufficeint power to transcode, or buy something too slow and your data throughput overall could be super slow. Trying to design an ultra low power system is not easy and takes a lot of work and risk as if you choose wrong and it's too slow, then you have a system that makes you unhappy and you need to spend more money upgrading.

Okay, that's a very good point. In that case, I'm happy to let go of my desire for a low power system and aim for somewhere in the middle (or to not worry about it for a hypervisor build). Worst case scenario, we're in the process of changing to a green energy supplier anyway, so the environmental impact shouldn't be that bad. :)

Also, you can sleep the hard drives if you setup your system properly and that will save you about 5 watts of power per drive, but the trade off is typically a premature hard drive failure due to constant spinup cycles. You can read more about that in the forums here, you just need to search for it and there is a script to run to ensure the drives do sleep.

Good to know! Am I right to infer then that even just setting up the FreeNAS to shut down during the night would wear down the drives?

Do you need a small case? The larger the case = Cooler component in general. A High Air Flow case is typically large but you can place a lot of items in it, including very large slow turning fans to move lots of air quietly. Your components will last longer as a rule vs. a compact hot case, unless the system was designed for those temperatures.

Yes, I could potentially swap the Node 804 for a Fractal Define 7 (547 x 240 x 475 mm (LxWxH))
or a Define 7 XL (604 x 240 x 566 mm (LxWxH)). I assume a case of this size would be an absolute requirement for a hypervisor build for airflow reasons, right?

I look forward to your response!

 

[joeschmuck]

I decided against it because I remember having read somewhere that running FreeNAS on a hypervisor was a bad idea, I think due to needing the pass the hard disks straight through to the FreeNAS VM for ZFS, which can be tricky.

There was a mindset many years ago where a hypervisor was discouraged. I'm glad to say that is not the case now. Many folks prefer bare metal for FreeNAS, and it really depends on your goals.

I'm actually unclear what you mean by "Set and Forget". I have my openmediavault set up to send me an email if a S.M.A.R.T. check fails, and I have a healthchecks.io check that warns me if a backup fails or doesn't run. So unless I'd need to monitor the FreeNAS a lot more than that, then that should be perfectly fine!

All too often I will see a posting from a person who setup FreeNAS and then 3 years later either a hard drive will fail or worse, some other component will fail or there will be data corruption (like from a USB Flash Boot Drive as an example). They they start flip-flopping like a fish out of water becasue they forgot most of what it took to build and setup the system, now they are begging for help. My point was to pay attention to your system. Once I created a step by step hard drive replacement procedure specifically for my FreeNAS system, and if you have seen any of my older troubleshooting posts, you will note I number each step. It also helps when the person tells me step 7 worked but step 8 didn't. Now I have a better understanding of what is going on.

Am I right to infer then that even just setting up the FreeNAS to shut down during the night would wear down the drives?

Yes. The problem with the drives sleeping is they need to spin back up and this puts stress on the motor and the power driving components. Many drives fail earlier because the drive cannot spin back up. But it's a personal decision. Now you plan to turn off the whole system, now you are adding wear to the entire system when you power it all back on. Many computers are powered on and off each day and they last for many years. So it's again a personal choice. All I can say is the system will generally last longer when you do not cycle power to it. But if your system is in your bedroom, I would power it off at night or it would likely wake me up. And to be honest with you, right now my computers are in my bedroom and I do power them off at night. This is temporary until I can find a better place to put them in the house. In my old house I had a nice cool basement, I don't have that now.

Yes, I could potentially swap the Node 804 for a Fractal Define 7 (547 x 240 x 475 mm (LxWxH))
or a Define 7 XL (604 x 240 x 566 mm (LxWxH)). I assume a case of this size would be an absolute requirement for a hypervisor build for airflow reasons, right?

I have a really cheap DIYPC case ($15 USD) to hold my test ESXi server. Lots of room, ugly case, good airflow after a minor modification. I choose my case for function over looks but if I can get both, that is a bonus. I would recommend that you plan it all out. Remember that if you can install large 120mm or larger fans that turn slowly, they make virtually no noise and still move air. I tend to modify my cases. For example the case rear exhaust fan often has a grill (holes) that are punched into the case metal. I will draw a circle to correlate with the maximum size for a fan and then using a Dremel tool I will cut that perforated disk out of the case. I will install a wire fan guard. Now I have removed significant air restriction and improved my airflow, and I do this to the front of the case as well if needed, wherever I place a fan. I plan for the air flow I want and make it happen. You need to ensure the hard drives get some air flow, the motherboard heatsinks and power regulators, of course a light breeze across the CPU doesn't hurt. And the power supply needs air flow. I also have my air pulled into the front of the case (typically where the hard drives are located) and exit the Power Supply and rear of the case near the CPU. Also, never add more exhaust fans then intake fans or you could be restricting your power supply fan output air (fans fighting against each other).

I'm giving you an awful lot to think about. I don't want it to be more confusing but I'm trying to let you know that a properly designed system takes a lot of thought.

set up to send me an email if a S.M.A.R.T. check fails,

I recommend that people set a Daily SMART Short Test and a Weekly SMART Long/Extended Test.

my budget of £1000 is only a soft goal, as I'm fortunate to have the savings to be able to spend significantly more.

That is very good, but you are also being smart to choose your parts wisely. You might be able to purchase a prebuilt server with minimal RAM and upgrade it. There use to be a popular server that was low cost and by adding a few more dollars, you could have a good FreeNAS system. So while you look for parts, also pay attention to premade systems/workstations with ECC RAM. Just something else to think about.

I really hope I'm not confusing you and I'm helping you.

Good luck!

 

[KrisBee]

@jbduncan
UK prices and avalibility of server grade kit make the ideal FreeNAS build much less affordable than in the US. I not advocating using consumer grade m/boards and non-ECC RAM as you would find here for example: https://blog.briancmoses.com/2019/11/diy-nas-econonas-2019.html, but as you're data storage needs are small by the standards of this forum, I can see why a QNAP type system was suggested as an alternative to a full blown FreeNAS build.

I have an alternative suggestion for a FreeNAS build which could meet your needs and that's to look for a s/hand HP Microserver Gen8 with 16GB or ECC RAM and if possible an upgraded CPU (this is relatively cheap DIY fix). This model is no longer made but has held its s/hand value and are regularly found on Ebay UK. They sell for anything between £250 and £350 for a complete system. Audio processing doesn't require much processing power compared to video streaming/transcoding. My low powered E3-1220L has a pass mark of just 2050. The HP Microserver Gen8 also has iLo, the equivalent of Supermicro's IPMI remote access feature.

At this price you can then invest in four 4TB HDDs, which to my mind is better than using just 3 drives in a raidz2 pool. It's true the Microserver limits you to a max of 4 HDDs, but you will then have just under 8TB of usable storage. You can choose to create either a 4 drive raidz2 pool, or a pair of mirrors. The later may have some performance advantages, depending on your intended use. Please remember that the redundancy you can create in a zfs pool is not a substitute for proper backups.

If you're still intent on building a larger system, then here's a couple of relevant examples:

Supermicro in a Fractal node 804 - https://ramsdenj.com/2016/01/01/freenas-server-build.html
Supermicro x11 series in Fractal R5 (?) - https://home.apeconsulting.co.uk/adrian/freenas0-build-2/

FreeNAS running on a HP Microserver gen 8 can make a workable NFS storage backend for promox running on a separate box which could easily be built on consumer grade kit, for example this: https://xtremeownage.com/2020/07/24/closet-mini-server-build/ An all in one, FreeNAS on VM under Proxmox requires the use of a HBA for the drives passed through to the FreeNAS VM, a CPU with more than 4 cores is better, and RAM needs are at least 16GB but probably 32GB to 64GB depening what other VMs or containers are running. You'll also need a mix of SSD and HDD to get reasonable performance. Such a server is another kind of beast.

 

[joeschmuck]

HP Microserver Gen8

That is what I could not remember but the system I was thinking of and I see you have it so you are speaking from personal experience, fantastic. And great post, I'm sure it will be very helpful. I like that you recommended using four 4TB drives too, it makes the most sense.

 

[jbduncan]

Wow, thank you both @joeschmuck and @KrisBee for your thoughts on this!

@joeschmuck Admittedly, I was a bit demoralised after reading your otherwise helpful comments, just because of the sheer amount of information and because I realised, "Ah crud, if I want to do FreeNAS on a hypervisor, then I need to start my research all over again!". But @KrisBee's advice regarding getting a second-hand HP Microserver Gen8, which you've affirmed, looks like a good place to get my bearings straight again, and I've realised that trying to learn both FreeNAS and Proxmox or XCP-ng, and trying to get the right computer parts for such a build is just too much for me to handle at once. Besides, the only thing I really want to do other than storage and music streaming is to setup Pi-Hole, which I can easily do on my Odroid after I've migrated everything to FreeNAS. Furthermore, if I do find I ever need to do video transcoding in the future, I'm reasonably sure I could find a compatible Xeon processor in the future (if @KrisBee's current build is anything to go by).

Also, thank you for the clarification on what you meant by "Set and Forget". I hadn't truly appreciated what I'd have to do when my system breaks down (even for my Odroid), but I'm sure that after I get past the initial feeling of being a fish out of water, I'd be more than capable of Googling things and using the resources in this forum inc. your hard drive troubleshooting guide to diagnose and fix things. Thanks again!

@KrisBee Thank you also for your thoughts and blog post link on running Proxmox on a separate machine using consumer hardware. If I ever get around to investigating Proxmox/XCP-ng again in the future, I'm sure it will be helpful!

I'll report back here if or when I settle on a new build. :)

 

[jbduncan]

@KrisBee I see in your build that you have your SSD (which I assume is your OS drive) in your HP Microserver's optical disk drive bay. Any advice (or any online resources that you know of) on how to put it there? (All online resources I've found so far show how to duct-tape the SSD on top of the optical drive, rather than putting it inside.)

Edit: If you have your SSD duct-taped on the top as well because e.g. the optical drive can't be removed, then I'm happy to use duct tape. :)

 

[Clexp]

warllo

1) Given your use case and using new hardware I like the Supermicro X11SCL-F for mainstream builds.

2) Paired with the X11SCL-F check out the i3-9100f

Have you actually got this combo working somewhere? You suggested this to me but in this post - at the end I link to ref pages which I think indicate they should not work together. How did you find out that they worked? Do you have some special hardware-fu to try it despite the spec sheets?

 

[Constantin]

Good point. I've gone and checked how SONOS works, and apparently it doesn't (and perhaps even can't) transcode music, and it's limited to playing <=16-bit music. This hasn't been a problem for me over the ~5 years we've had SONOS, but that may change in the future if I ever procure 24-bit music.

I would think long and hard how to support Sonos on a FreeNAS. The older versions of Sonos required SMB1 NTLM v1, i.e. a version of SMB that is categorically unsafe, unsupported, and so on. Worse, you cannot just limit SMB1 to your Sonos share - Every SMB share on the NAS will be affected and any security issues will potentially escalate across the whole NAS.

It is possible that the newer S2 version of Sonos will some day catch up with Bluesound and other competitors re: SMB2+ support but I wouldn't count on it. NAS-related software development has not been a priority at Sonos in years. They have been focusing on streaming services instead. Other approaches serving up the data to your Sonos (such as hosting content via Plex) is certainly possible. One advantage of Plex over NAS-hosted iTunes or other "native" Sonos approaches is that Plex can host far more files than Sonos (65k+). But it also adds complexity and new attack vectors.

I prefer the simplicity of having one central database, no Plex, and so on. To address the security issues inherent with Sonos, I found it easiest to host my music content on a separate RPi + drive. That way, my FreeNAS will not get compromised due to SMB1 NTLM v1 insecurities, rather the Sonos server will be the canary in the coal mine if something does go wrong (i.e. use it as a IDS). Sonos has fought improving security hand and foot unless they were forced to by content providers like the streaming services out there.

I also suggest blackholing your Sonos equipment to limit the amount of telemetry going on and using SonoPhone as an App on the iPhone and SonoPad on the iPad, if you're into iOS. Only the Sonos SSL validator server has to remain accessible. Even internet radio works just fine with the 8.4 firmware, which prevents my dear CR100 controller from becoming a useless paperweight.

Anyhow, you could consider Plex to allow transcoding. Also, have a look at the following Flex-ATX motherboard: The Supermicro X10SDV-2C-7TP4F is storage oriented (i.e. up to 20 SATA drives), low power, yet offers lots of expansion opportunities. Only downside is the embedded CPU. In a Soho AFP/SMB setting with a focus on simple storage its likely better performing (i.e. higher clock speed) than the more expensive version of the motherboard I bought.

 

[KrisBee]

@KrisBee I see in your build that you have your SSD (which I assume is your OS drive) in your HP Microserver's optical disk drive bay. Any advice (or any online resources that you know of) on how to put it there? (All online resources I've found so far show how to duct-tape the SSD on top of the optical drive, rather than putting it inside.)

Edit: If you have your SSD duct-taped on the top as well because e.g. the optical drive can't be removed, then I'm happy to use duct tape. :)

Have a look at this example: https://www.admfactory.com/hp-microserver-gen8-install-ssd-in-odd-slot/

You can get the necessary power adapter from here: https://www.lindy.co.uk/cables-adap...m-power-adapter-sata-power-to-3-5-power-p6723

The HP Microserver gen8 has some boot limitations when using FreeNAS. It's inbuilt B120I RAID controller ( fake software RAID) has to be switched to AHCI mode to give zfs direct control of the HDDs. With this setting you can only directly boot from USB or the leftmost HDD slot. So its an indirect boot from an SDD mounted in the ODD bay by using a GRUB boot USB which points to the SSD. There's at least one thread about this on the forum .

If you going to consider a microserver, look for a late model with a PID number starting with 8 (near of chassis label) and it possible one with the redesigned front badge (light green rectangle) & logo as in this example: https://www.ebay.co.uk/itm/HP-ProLi...480546&hash=item3fe765e16e:g:DGcAAOSwCNJfJWfm

HP memory is not required for RAM upgrades and it tolerates a mix of ECC RAM spec, eg 10600E + 12800E or 12800E of different makes (hynix,samsung.crucial,micron). 14900E also works at the lower speed of 12800,E assuming CPU supports 12800E. As 2x 8GB is ideal, a machine with a single RAM slot populated is preferable to say on with 2 x 4GB when buying s/hand.

Upgrading the CPU is straight forward DIY job, just be careful when disconnecting the internal cables and use the pulls where provided. See for example: https://somoit.net/home-server/upgrade-gen8-cpu

Which CPU to use? There's a ton of refs on the web about this, and more info in threads at the https://homeservershow.com/forums/forum/88-microserver-gen-8/

The cheapest option in the UK right now is a 4 core E-1200v2 @ circa £25. While theorectically not in thermal spec, peope use if successfully without modding the internal passive heastsink and without having to bump the fan speed by much, if at all. See here: https://homeservershow.com/forums/topic/6415-gen8-xeon-e3-1220v2/ Of course its your choice and pocket.

EDIT: Forgot to mention ilo license. look for a example which has the full/advanced ilo license added.

 

[jbduncan]

I would think long and hard how to support Sonos on a FreeNAS. The older versions of Sonos required SMB1 NTLM v1, i.e. a version of SMB that is categorically unsafe, unsupported, and so on. Worse, you cannot just limit SMB1 to your Sonos share - Every SMB share on the NAS will be affected and any security issues will potentially escalate across the whole NAS.

It is possible that the newer S2 version of Sonos will some day catch up with Bluesound and other competitors re: SMB2+ support but I wouldn't count on it. NAS-related software development has not been a priority at Sonos in years. They have been focusing on streaming services instead. Other approaches serving up the data to your Sonos (such as hosting content via Plex) is certainly possible. One advantage of Plex over NAS-hosted iTunes or other "native" Sonos approaches is that Plex can host far more files than Sonos (65k+). But it also adds complexity and new attack vectors.

I prefer the simplicity of having one central database, no Plex, and so on. To address the security issues inherent with Sonos, I found it easiest to host my music content on a separate RPi + drive. That way, my FreeNAS will not get compromised due to SMB1 NTLM v1 insecurities, rather the Sonos server will be the canary in the coal mine if something does go wrong (i.e. use it as a IDS). Sonos has fought improving security hand and foot unless they were forced to by content providers like the streaming services out there.

I also suggest blackholing your Sonos equipment to limit the amount of telemetry going on and using SonoPhone as an App on the iPhone and SonoPad on the iPad, if you're into iOS. Only the Sonos SSL validator server has to remain accessible. Even internet radio works just fine with the 8.4 firmware, which prevents my dear CR100 controller from becoming a useless paperweight.

Anyhow, you could consider Plex to allow transcoding. Also, have a look at the following Flex-ATX motherboard: The Supermicro X10SDV-2C-7TP4F is storage oriented (i.e. up to 20 SATA drives), low power, yet offers lots of expansion opportunities. Only downside is the embedded CPU. In a Soho AFP/SMB setting with a focus on simple storage its likely better performing (i.e. higher clock speed) than the more expensive version of the motherboard I bought.

@Constantin Oh cool, I actually knew about most of this already, but the information on the Sonos apps, the Sonos telemetry and the mobo was new for me, so thank you!

I enabled SMB1 for my Odroid, and I decided at the time that that was a risk worth taking, as I don't expose it to the internet, and my Borgbackup process runs in "append only" mode protected by an SSH key. For my FreeNAS build, I intend to use Plex to eliminate that risk.

 

[jbduncan]

Have a look at this example: https://www.admfactory.com/hp-microserver-gen8-install-ssd-in-odd-slot/

You can get the necessary power adapter from here: https://www.lindy.co.uk/cables-adap...m-power-adapter-sata-power-to-3-5-power-p6723

The HP Microserver gen8 has some boot limitations when using FreeNAS. It's inbuilt B120I RAID controller ( fake software RAID) has to be switched to AHCI mode to give zfs direct control of the HDDs. With this setting you can only directly boot from USB or the leftmost HDD slot. So its an indirect boot from an SDD mounted in the ODD bay by using a GRUB boot USB which points to the SSD. There's at least one thread about this on the forum .

If you going to consider a microserver, look for a late model with a PID number starting with 8 (near of chassis label) and it possible one with the redesigned front badge (light green rectangle) & logo as in this example: https://www.ebay.co.uk/itm/HP-ProLi...480546&hash=item3fe765e16e:g:DGcAAOSwCNJfJWfm

HP memory is not required for RAM upgrades and it tolerates a mix of ECC RAM spec, eg 10600E + 12800E or 12800E of different makes (hynix,samsung.crucial,micron). 14900E also works at the lower speed of 12800,E assuming CPU supports 12800E. As 2x 8GB is ideal, a machine with a single RAM slot populated is preferable to say on with 2 x 4GB when buying s/hand.

Upgrading the CPU is straight forward DIY job, just be careful when disconnecting the internal cables and use the pulls where provided. See for example: https://somoit.net/home-server/upgrade-gen8-cpu

Which CPU to use? There's a ton of refs on the web about this, and more info in threads at the https://homeservershow.com/forums/forum/88-microserver-gen-8/

The cheapest option in the UK right now is a 4 core E-1200v2 @ circa £25. While theorectically not in thermal spec, peope use if successfully without modding the internal passive heastsink and without having to bump the fan speed by much, if at all. See here: https://homeservershow.com/forums/topic/6415-gen8-xeon-e3-1220v2/ Of course its your choice and pocket.

EDIT: Forgot to mention ilo license. look for a example which has the full/advanced ilo license added.

@KrisBee Oh, oops. I actually won an auction for an HP Microserver Gen8 with 16GB ECC RAM and a Xeon E3-1220v2 on ebay yesterday evening. Looking at the image of it, it specifically does not have a green logo on the front, unlike the one in your ebay link. I'm not clear where I'd find the chassis label and in turn the PID number. Is this a problem?

The description didn't mention anything about an iLO license, so if I need to buy an Advanced license separately, then so be it.

Regarding booting from GRUB on USB to the SSD, is this the thread you meant, by any chance? https://www.ixsystems.com/community/threads/unable-to-boot-freenas-on-ssd.74327/

Regarding installing the SSD in the ODD bay, it looks to me like the link you've given me shows how to put the SSD on top of the bay rather than inside it. I suppose realistically this is the only option, isn't it?

Thank you very much for all this information, quite helpful!

 

[KrisBee]

@jbduncan Label is on the back, as per last image of ebay sale I linked to. The "ilo Advanced license" can be picked up cheaply (mayboe for free) if you look around on ebay and the net. You found the "GRUB on USB to the SSD" thread I was thinking of.

Just google images for "fit ssd i mircoserver gen8" and you'll see some people put their SSD in a laptop caddy. It's your choice, but seemed unnecessary to me.

 

[jbduncan]

@KrisBee Many thanks! The machine I won has a PID that starts with "7", rather than an "8". Will this be a problem, or will the machine function just fine?

 

[KrisBee]

@KrisBee Many thanks! The machine I won has a PID that starts with "7", rather than an "8". Will this be a problem, or will the machine function just fine?

I don't know of any problems, It does mean it's an earlier production model, which may mean it has less working life left. I seem to remember the psu was uprated in the later production runs . The homeservershow forum has all the info on this. Definitely worth updating to latest ilo (Firmware Version 2.70, or later) which gives you HTML5 rather than just java for the remote console. Update the BIOS, if need be, at your convenience.

PS If you want to overdose on Microserver Gen8 info then there's the superthread at the German language website: https://www.hardwareluxx.de/communi...g2020t-i3-3240-e3-1220lv2-microserver.963207/

 

[jbduncan]

I don't know of any problems, It does mean it's an earlier production model, which may mean it has less working life left. I seem to remember the psu was uprated in the later production runs . The homeservershow forum has all the info on this. Definitely worth updating to latest ilo (Firmware Version 2.70, or later) which gives you HTML5 rather than just java for the remote console. Update the BIOS, if need be, at your convenience.

PS If you want to overdose on Microserver Gen8 info then there's the superthread at the German language website: https://www.hardwareluxx.de/communi...g2020t-i3-3240-e3-1220lv2-microserver.963207/

Awesome, this is all good information. Thank you very much again @KrisBee. :)

I'll report back here with my final build when I receive the server and assemble everything together, but in the meantime, my to-be-confirmed build is:

- HP Microserver Gen8
- Xeon E3-1220 v2
- 16GB ECC RAM (don't know yet what exact kind the server will have)
- Crucial BX500 120GB SSD
- 4 2TB Seagate Ironwolf NAS HDDs
- Some sort of USB stick to install GRUB onto
- CyberPower UT650EIG uninterruptible power supply

(That being said, if anyone does notice anything off about this build, please do let me know!)