BUILD 1st Build Hardware (Small Backoffice Server)

[Rafal Lukawiecki]

I made the error of buying a new, top-spec HPE MicroServer Gen10 a few days ago, which won't boot FreeNAS/FreeBSD, see here: https://forums.freenas.org/index.ph...nds-on-a-microserver-gen10.56809/#post-399023

I am now considering two options. Buying MicroServer Gen8 with a Xeon E3 (but I dislike the 16GB RAM limit) or, reluctantly, a custom build, but I am a bit lost in choice. Having read the useful detailed hw recommendations/FAQs/resources here, I still have too many non-matching options. Would anyone be able to suggest, please, a case and a fitting motherboard (SuperMicro?) that would have the following:

• Small case, basically looking like a HPE MicroServer or similar.
• 4 x 3.5" HDD, ideally hot-plug, or at least not requiring unscrewing cages/trays. Like https://www.supermicro.nl/products/chassis/tower/721/SC721TQ-250B maybe? It will house a mirror of 2 x WD Red 10TB initially, later 4 of those.
• Perhaps space for a small boot SSD, but I can live with a pair of USBs.
• Xeon E3 v5 support (several jails, one video grabbing/analysis, rest small backoffice functions and sync to AWS S3, might need to run one VM/bhyve).
• 32 GB ECC RAM support.
• Space for one PCI-E video capture card.

I think I like SuperMicro X11 micro-ATX boards but the cases I see are either huuge or have non-easily-swappable drives, or cases that take a mini-ITX and hot-plug drives, but I do not see a suitable mini-ITX board for the E3... I am lost. Please help if you have any suggestions. Many thanks.

PS. I wrote to iXSystems about buying a FreeNAS Mini but I have not heard back, and it does not look like they retail in Europe. Import from US is customs/tax-prohibitive for me.
PPS. Could I retrofit a sensible board into an old MicroServer Gen7?

 

[Stux]

SuperMicro sell the FreeNAS mini case separately. An alternative is the UNAS 800A (supports micro atx)

Another alternative is a Xeon d mini ITX board. If you go that way you can buy a premade server direct from supermicro.

https://tinkertry.com/superservers

 

[Rafal Lukawiecki]

SuperMicro sell the FreeNAS mini case separately. An alternative is the UNAS 800A (supports micro atx)

Another alternative is a Xeon d mini ITX board. If you go that way you can buy a premade server direct from supermicro.

https://tinkertry.com/superservers

Thank you, @Stux, for the recommendation on the UNAS case. I am looking at the smaller, NSC410, as it would fit my needs. Otherwise the SuperMicro case it will be.

I did not consider the Xeon Ds initially, but perhaps that is the route to take. I wonder if the D1518, apparently network-optimised and only 35W, would be powerful enough to run the video grabbing/analysis while looking after the NAS and other jailed, and simpler, office functions. I will research the alternative Xeon Ds, too, as having it all on a mini-ITX board is very attractive. Unfortunately, the tinkertry do not seem to sell in EU and the spec is a bit big for my needs—pity, as I would prefer to buy a ready unit. Thank you again.

 

[Rafal Lukawiecki]

I think I am narrowing the board choice down to the D1518/D1528 ones:

• https://www.supermicro.nl/products/motherboard/Xeon/D/X10SDV-4C-TLN4F.cfm
  
• https://www.supermicro.nl/products/motherboard/Xeon/D/X10SDV-4C_-TLN4F.cfm
  
• https://www.supermicro.nl/products/motherboard/Xeon/D/X10SDV-6C-TLN4F.cfm
  
• https://www.supermicro.nl/products/motherboard/Xeon/D/X10SDV-6C_-TLN4F.cfm

And either the U-NAS 4-bay case (awaiting their reply) or the SuperMicro https://www.supermicro.nl/products/chassis/tower/721/SC721TQ-250B.cfm case.

 

[Stux]

Be aware, you want a board with an active heatsink.

Often, but not always this is indicated by a + after the core count, Ie 4C+TLN4F

https://www.supermicro.nl/products/motherboard/Xeon/D/X10SDV-4C_-TLN4F.cfm

I personally have the X10SDV-TLN4F, and that has an active heat sink. (8 core)

Without the fan it will over heat as the passive one is designed for use in a high air flow 1U chassis.

So the 4C+ or 6C+ you linked. Or the 8C version which is mine
;)

https://www.supermicro.nl/products/motherboard/Xeon/D/X10SDV-TLN4F.cfm

The TLN4F boards have dual 10gbe and dual 1gbe Ethernet ports (and an IPMI port)

 

[Evertb1]

Perhaps space for a small boot SSD, but I can live with a pair of USBs.

In almost any case you can find the room for an SDD. My backup device (OMV box) is build in a small aluminium case (Sharkoon) with only room (officially) for 2 drives. I just taped the (boot) SSD with double sided tape to the inside of the case front (keeps it nice cool to). If you use the right tape you can remove it without a trace if needed.

 

[Rafal Lukawiecki]

Be aware, you want a board with an active heatsink.

Often, but not always this is indicated by a + after the core count, Ie 4C+TLN4F

https://www.supermicro.nl/products/motherboard/Xeon/D/X10SDV-4C_-TLN4F.cfm

I personally have the X10SDV-TLN4F, and that has an active heat sink. (8 core)

Without the fan it will over heat as the passive one is designed for use in a high air flow 1U chassis.

So the 4C+ or 6C+ you linked. Or the 8C version which is mine
;)

https://www.supermicro.nl/products/motherboard/Xeon/D/X10SDV-TLN4F.cfm

The TLN4F boards have dual 10gbe and dual 1gbe Ethernet ports (and an IPMI port)

Thank you for explaining the need for an active heat sink. I would not have thought of that, assuming the small case would have handled that... Ouch.

Out of interest, how low is the power draw of your board when somwhat idle and how high does it push when busy with video or compute intensive activites?

 

[Rafal Lukawiecki]

@Stux, are you using registered or unregistered memory with this board? I see from the spec that it supports both types and I wonder if there is a benefit to using a registered module from their list of tested ones—other than the maximum overall capacity.

 

[Ericloewe]

I wonder if there is a benefit to using a registered module from their list of tested ones—other than the maximum overall capacity.

On the contrary, latency is slightly higher. But capacity is always more important.

 

[Stux]

@Stux, are you using registered or unregistered memory with this board? I see from the spec that it supports both types and I wonder if there is a benefit to using a registered module from their list of tested ones—other than the maximum overall capacity.

I'm using registered.

 

[Rafal Lukawiecki]

On the contrary, latency is slightly higher. But capacity is always more important.

Thanks, I suppose the register being in the way has to take some time to do its job. Having said that, does this offloading of effort from the CPU side to the memory chip impact reliability of memory operations in a positive/negative or no noticeable way at all? Thanks for your help.

 

[Stux]

Ideal situation with a Xeon d board is to use a pair of 32GB registered 2400 DIMMs. This gets you max speed (dual channel) and max potential capacity (4x 32GB for 128GB).

or start with 32GB in one channel.

I had the 16GB DIMMs available after upgrading another server (to 128GB)

 

[Ericloewe]

There's no logical offloading.

Electrically, the RAM bus has a lighter load, which will naturally allow for the bus drivers to last longer (not that that is ever a serious constraint, since most other things in the system are going to fail before the RAM bus circuitry - unless there's a design flaw, such as the C2000 LPC bus deterioration issue).

It's definitely way down my list of concerns, somewhere between "Secret Nazi moon base" and "Godzilla attack (non-Sim City)".

 

[Stux]

I think you can't mix registered and unregistered. It you can mix 16 and 32GB DIMMs. If they are both registered.

 

[Rafal Lukawiecki]

Thank you for your suggestions. I will put together my proposed shopping list tomorrow, hope you don't mind criticising it for me. I am just awaiting pricing.

In the meantime, I am debating putting the boot OS, and perhaps the jails, on an SSD. I am considering:

• ZFS mirror of a pair of identical Intel 535 Series, 2.5", 120 GB SSD, SATA-3s 2.5"
• ZFS mirror of a pair of different SSDs: an M.2 and a SATADOM, Intel DC S3500 M.2 80GB SSD and SuperMicro 64GB SATA3 DOM to use cable-less ports and eek a touch of extra read performance
• Just a single, but a more resilient (adv power protection) Intel SSD like the above 80GB DC S3500 or a larger version, Intel DC S3500 M.2 160GB SSD.

The data is kept on two mirrored WD Red 10 TB. But having jails on an SSD should make them faster—unless FreeNAS and ZFS is clever enough to move the executables into RAM ARC. If so, I could consider going for 64GB RAM and keep everything except the boot on the spinning disks.

I've been told I would not benefit from a dedicated SLOG device in my case—SMB/AFP writes mainly—right? I suppose the local video grabber and poss VM would not be critical to the overall user experience, Otherwise, an Intel DC S3500 M.2 160GB could play the SLOG role.

Thank you again. Your help is much appreciated by this FreeNAS novice.

 

[Ericloewe]

ZFS mirror of a pair of identical Intel 535 Series, 2.5", 120 GB SSD, SATA-3s 2.5"

That's probably my favorite option of the bunch.

 

[Evertb1]

Thank you for your suggestions. I will put together my proposed shopping list tomorrow, hope you don't mind criticising it for me. I am just awaiting pricing.

In the meantime, I am debating putting the boot OS, and perhaps the jails, on an SSD. I am considering:

• ZFS mirror of a pair of identical Intel 535 Series, 2.5", 120 GB SSD, SATA-3s 2.5"
• ZFS mirror of a pair of different SSDs: an M.2 and a SATADOM, Intel DC S3500 M.2 80GB SSD and SuperMicro 64GB SATA3 DOM to use cable-less ports and eek a touch of extra read performance
• Just a single, but a more resilient (adv power protection) Intel SSD like the above 80GB DC S3500 or a larger version, Intel DC S3500 M.2 160GB SSD.

The data is kept on two mirrored WD Red 10 TB. But having jails on an SSD should make them faster—unless FreeNAS and ZFS is clever enough to move the executables into RAM ARC. If so, I could consider going for 64GB RAM and keep everything except the boot on the spinning disks.

I've been told I would not benefit from a dedicated SLOG device in my case—SMB/AFP writes mainly—right? I suppose the local video grabber and poss VM would not be critical to the overall user experience, Otherwise, an Intel DC S3500 M.2 160GB could play the SLOG role.

Thank you again. Your help is much appreciated by this FreeNAS novice.

I am not the most experienced Freenas user around, but if I understood things right then having a fast boot medium will only be beneficiary to you on boot time, because after the boot the OS lives in RAM. You will benefit of the fact that SSD's are way more reliable then USB sticks ( I lost a couple of them before I installed Freenas on a small SSD).
A short time ago I had a thread going on about having my jails on a SSD or just in my main storage pool
https://forums.freenas.org/index.ph...nefits-to-store-jails-in-separate-pool.55996/
Based on that discussion I decided to have my jails in the main storage pool. But there are different opinions around as you can see in that thread (and other trheads).

 

[MrToddsFriends]

In the meantime, I am debating putting the boot OS, and perhaps the jails, on an SSD. I am considering:

• ZFS mirror of a pair of identical Intel 535 Series, 2.5", 120 GB SSD, SATA-3s 2.5"
• [...]

You can either use a pool of mirrored SSDs as boot pool or as a dedicated jails pool, but not both at the same time.

 

[Rafal Lukawiecki]

You can either use a pool of mirrored SSDs as boot pool or as a dedicated jails pool, but not both at the same time.

Thank you. Is there no reasonable way of creating another ZFS dataset on the boot volume?

 

[MrToddsFriends]

Thank you. Is there no reasonable way of creating another ZFS dataset on the boot volume?

Not that I knew of. The documentation says "Installation to a hard drive is discouraged as that drive is then not available for data storage".
http://doc.freenas.org/11/intro.html#the-operating-system-device

 

[Evertb1]

Thank you. Is there no reasonable way of creating another ZFS dataset on the boot volume?

This is a often discussed theme in the Forum. It always come down to the fact that it is not possible or even desired. To me one of the most nice things of Freenas is the fact that in the worst case, when your boot device dies, you can reinstall Freenas and be back in business in no time at all. Just make sure that saving your configuration becomes a habit. Knowing that, I find it pretty convenient to leave my boot device as it is. No worries about other things on that drive.