Updating from Nexenta to a new, upgraded FreeNAS build

[Jatrabari]

Hello all,

This is my first post to this forum. My history regarding ZFS is from Nexenta and have now been getting inspired to update and move to FreeNAS that I had thought a few years ago already as my next server OS.

I have to say that the amount of information that has been generated here in the resources section, guides and FAQs is overwhelming. They have been a huge help in getting familiar with FreeNAS and its requirements... Thanks to all that have contributed to these resources.

So I thought that I could update and upgrade the whole thing and here are parts that I have now thought to be in the new build:

MB: Supermicro X10SRL-F
CPU: Intel Xeon E5-2630v3 8x 2.40GHz or Intel Xeon E5-1650 V4 6x 3.50GHz
Heatsink: Noctua NH-U9DX i4
Memory: 64 GB ECC 4x 16GB Samsung M393A2K40BB1 2x 32GB Crucial CT32G4RFD424A DDR4-2400 (Micron part number MTA36ASF4G72PZ-2G3A1)
HBA: LSISAS9201-16i SAS card (will be flashed into IT firmware)
HDDs:

• 4 TB WD Red x 6 as Raidz2 (primary pool) + tested cold spare
  
• 6 TB WD Red x 2 as mirror (backup pool)

Case: Full tower case with 12x 5,25” bays for up to 6x Icydock MB153SP-B 3 in 2 SATA Internal Backplane Modules for easy access and hotswap
Power: Enermax EMD625AWT Seasonic Prime Ultra Titanium 650W
Boot device: 16/32 GB SSD 2x Supermicro 32 GB SATA DOM mirrored

Main uses for my server would be:

• backing up of other computers in my home (Windows, MacOS)
• streaming HD video and music to at least 3 devices at the same time (Win, MacOS, PS3 and 4… thinking of using plex or something else, experiences?)
• storing a digital photo library and also viewing them easily directly from server
• general all-round file server
• life span for this build is planned to be 5-10 years

I haven’t been up-to-date on computer hardware in years so I would like to ask a few things from the more experienced people here.

My main questions are:

1. I would like to build a server that would last the next 5-10 years so what kind of CPU would be best so it has enough uumph to do the above things and some future proofing? I have narrowed it down to 2 options the latter being the most likely. Is either one overkill for a home server?
2. Is there anything that could be improved in this setup so it will have a long lifespan hardware malfunctions or breakages aside?

Update 250418

Final build configuration

MB: Supermicro X10SRL-F
CPU: Intel Xeon E5-1650 V4 6-core 3.50GHz
Heatsink: Noctua NH-U9DX i4
Memory: 64 GB ECC (2x 32GB Crucial CT32G4RFD424A DDR4-2400, Micron part number MTA36ASF4G72PZ-2G3A1)
HBA: LSISAS9201-16i SAS card (will be flashed into IT firmware)
HDDs:

• 4 TB WD Red x 6 as Raidz2 (primary pool) + cold spare
  
• 6 TB WD Red x 2, first as 2-way mirror + cold spare, later 3 disc Raidz1 when expansion is needed (backup pool)

Case: Fractal Design R6 with 5 additional HDD trays
Power: Seasonic Prime Ultra Titanium 650W
Boot device: 2x Supermicro 32 GB SATA DOM mirrored

Still coming: removable HDD cage for off-site backup, Noctua fans and UPS

 

[Ericloewe]

This power is from a old server and I am planning to repurpose it to this new build but I still have to calculate if it is enough and suited for this build and future expansion or just invest straight to a Seasonic…

I'd get a new PSU. Bad power is not worth the saved cash - just the time wasted troubleshooting will have you regretting the savings.

I would like to build a server that would last the next 5-10 years so what kind of CPU would be best so it has enough uumph to do the above things and some future proofing? I have narrowed it down to 2 options the latter being the most likely. Is either one overkill for a home server?

You'll never get 10 years doing the same job. 5 is a stretch, but doable without building a ship of Theseus.

That said the E5-1650 is the better option for FreeNAS.

Network: Intel D33682 PRO/1000 PT Dual Port Server Adapter (on old server ATM but MB has dual LAN ports so this is not necessarily needed

As you say, not needed. Store it safely or sell it.

HBA: LSISAS9201-16i SAS card (will be flashed into IT firmware)

You can save a lot of cash by going with an 8-port controller like the SAS 9211 or 9207. You have plenty of SATA connectivity on the motherboard to make up for it.

Also, make sure the heatsink fits. The board may use the narrow ILM socket, which isn't compatible with the standard LGA 2011 square ILM.

 

[Jatrabari]

Ericloewe, thanks for the reply.

I'd get a new PSU. Bad power is not worth the saved cash - just the time wasted troubleshooting will have you regretting the savings.

Considering the requirements in the future and also reading on these build and build report threads I am leaning towards getting a new PSU that can handle also future expansions... Already looked into Corsair and Seasonic PSUs in the 1000 W range...

You can save a lot of cash by going with an 8-port controller like the SAS 9211 or 9207. You have plenty of SATA connectivity on the motherboard to make up for it.

I already have the 16i card so no saving oppurtunity anymore there...

Also, make sure the heatsink fits. The board may use the narrow ILM socket, which isn't compatible with the standard LGA 2011 square ILM.

Ok, I will check that. Also try to find out any info about the clearance on RAM and the heatsink so it is possible to get all of the slots filled if needed.

 

[Jatrabari]

Hello,

have been researching more about my setup and reading other build threads in this forum. Haven't pulled the trigger yet concerning components.

Datakeeper confirmed that the heat sink will fit to the motherboard with good room for ram.

PSU is also still on the burner... Datakeeper reported that his system draws at peak 288 W. Do I remember correctly that PSU should be used over 20 % at the lowest consumption as the efficiency drops when going under 20 % load?

What I have been pondering is backups for the server. I don't have the money or the resources to build another server for backup but I have been wondering about a solution where my main pool would be for example 6 x 4 TB as raidz2 and then create a backup pool in the same system for example 2 x 6 TB mirrored. Not all of the data in the main pool has to be backed up and of course the amount of data will be first lower...

I know this is of course not recommended to have main pool and its backup in the same place but I think it is better and more convenient solution than using for example external drives.

I welcome comments and/or suggestions about this backup solution

 

[Ericloewe]

Do I remember correctly that PSU should be used over 20 % at the lowest consumption as the efficiency drops when going under 20 % load?

It's far more important to size it so that it can handle all the disks spinning up, plus the rest of the system booting.

I know this is of course not recommended to have main pool and its backup in the same place but I think it is better and more convenient solution than using for example external drives.

Convenient, yes, but it still leaves them subject to a number of things that can wipe out everything on your server.

 

[Jatrabari]

It's far more important to size it so that it can handle all the disks spinning up, plus the rest of the system booting.

Then I will go and read again the power sizing resource by jgreco...

Convenient, yes, but it still leaves them subject to a number of things that can wipe out everything on your server.

Thinking that I don't have external SATA ports in the motherboard or HBA, it's USB 3.0 then and some external solution. Thanks for the input.

 

[Jatrabari]

@Ericloewe You have in your servers Icy dock drive cages and Seasonic PSUs. Are the modular cables long and flexible enough to connect to the Icy Dock cages? How are the power connectors situated in the drive cages? In my cages they are situated on the side vertical on top of each other.

 

[Ericloewe]

Are the modular cables long and flexible enough to connect to the Icy Dock cages?

Barely, with lots of elbow grease.

In my cages they are situated on the side vertical on top of each other.

Right, on the right side of the chassis.

 

[Jatrabari]

Barely, with lots of elbow grease.

Ok. Thanks for the info. Just for clarification, do you have 1 or 2 modular SATA cables for 2 cages? If I remember correctly one modular cable has 4 SATA ports so it should be able to reach both cages...

 

[Ericloewe]

I use one.

 

[Jatrabari]

I have been trying to get some power consumption values for CPU and memory but I can't make heads or tails out of the spec sheets and all the voltage and current values galore. Can anyone help me what values I should search for to get values to my calculations?

 

[Ericloewe]

The CPU is easy, just use the TDP. As for RAM, 10W should cover it easily.

 

[Jatrabari]

Convenient, yes, but it still leaves them subject to a number of things that can wipe out everything on your server.

For example, what kind of scenarios do you mean?

 

[Ericloewe]

PSU frying everything in the server, someone kicking the server, drunk/sleepy/inattentive admin destroying everything, fire, etc.

 

[Jatrabari]

So I have now got to the end of my calculations for the PSU size. I calculated 2 options, with and without Caviar Blacks that I have now on my old server because I am still pondering about including them in the new server. Even with the WD Reds I will have plenty of space for my needs for years.

Some comments about the calculations:

• Couldn't find any idle value for the CPU but basically it is 0 when idle?
• Fan idle values are the same as peak as they are always on for cooling
• For those components that I didn't find any idle value I used 50 % of peak value (is it too much?)
  
• For MB and RAM the values are straight from Supermicro power budget that I got from their support
• The idle total wattage is a little bit shy from 20 % but the peak wattage and 12 V rail peak load are nicely in the green with room to spare

So I need another pair of eyes to have a glance at these numbers so I have not missed anything and the results are reasonable so any and all comments are welcomed.

Calculations are in the next post.

 

[Jatrabari]

Yeah, got it working...

 

[Jatrabari]

I was looking a new set of memory from Micron/Crucial to my build because the Samsung's that I planned to buy ran out when I found this site

https://www.insight.com/en_US/buy/partner/micron/memory/crucial-memory-selector.html

It converts Micron memory part numbers into long Crucial memory part numbers which you can paste straight to a online store search field. I tested it against the QVL memory list for X10SRL and also the memory recommended by Ericloewe in his guide and it worked like a charm and as far as I can see they panned out. You can also get the specific "short" or internal (?) Crucial part number when you fill out the compatibility form in the memory information page.

 

[Jatrabari]

@Chris Moore I noticed that you have in your Emily-NAS a backup pool in the same system as your primary pools. Is this also the case in your other NAS that you are updating now? Or are both backed up also elsewhere?

 

[Chris Moore]

@Chris Moore I noticed that you have in your Emily-NAS a backup pool in the same system as your primary pools. Is this also the case in your other NAS that you are updating now? Or are both backed up also elsewhere?

The backup pool is a weekly backup of all the data in the storage pool.
Normally the second NAS (currently offline) is synced with the primary NAS storage pool every hour so that if the primary was down for any reason I would lose access only to the most recent changes.
This normally gives me all of my data in three functionally separate pools, but it would be better to have one of them be offsite. I just don't have a good place to put it.

 

[ramar]

Thinking that I don't have external SATA ports in the motherboard or HBA, it's USB 3.0 then and some external solution. Thanks for the input.

i've spent an hour looking for FreeNAS documentation that states USB 3.0 isn't reliable, works intermittently or isn't recommended for FreeNAS but i can't find where i read it. the post was from a few years ago, as i recall, i was hoping USB3.0 might be reliable when used in FreeNAS at this point. is it?