FreeNAS on ESXi, Supermicro X11SDV (?)

[KMaes]

Hello,

After 8 years, it is time to do a technology refresh in our home. In order to conserve energy, we would like to consolidate several hardware machines in one platform, running ESXi.

1.FreeNAS : 6TB usable should be enough for the next 3 years. The FreeNAS will replace an obsolete QNAP
2. Firewall : pfSense
3. Smarthome server: OpenHAB. This one is 'home critical'. Basic functions like lighting, heating, blinds can be controlled without the server running but there is no TV or music without OpenHAB.
4. Linux homeserver with some goodies like Indexing server etc. Might install Plex on this server or on FreeNAS.
5. Windows 2016 server. Because I can and I'm still a system engineer at heart. :)
6. Linux server for each child, so they can start tinkering without impacting the grown ups.

We use max. three simultaneous Plex streams, one in 4K.

After some shopping, I came up with the following setup:

Motherboard and CPU : https://www.supermicro.com/products/motherboard/Xeon/D/X11SDV-8C-TP8F.cfm
FreeNAS Storage : three 6 GB Western Digital RED
VMWare boot from USB, 2 x M.2 SSD 512 GB
Memory : 64 GB ECC RAM. No idea what brand/model I should take.
Powersupply : Seasonic or Supermicro 300W. Not decided yet.
HBA : all tips are very welcome

Thanks for all the help!

 

[Inxsible]

DDR4 is at an all time high. Go with a X9 or some of the X10(that support DDR3) series boards. You will save a lot of money in terms of the board as well as the memory and maybe even in CPU.

1. Have you decided on the case?
2. Have you decided the configuration of the drives? If you are using 3x6TB drives -- how many are for FreeNAS. How many in total? See question 1
3. Your PSU seems woefully inadequate. Depending on your answer for 1 & 2, you will need to upgrade also taking into account how many VMs you have and a whole host of things that we can get into later.
4. If you are going to have only 3 storage drives, why do you need a HBA? Whatever board you select will most likely have at least 3 SATA ports. Again see question 1

 

[KMaes]

It is a hard choice to make. Without spectre/meltdown, the choice would be easier. Intel guarantees D2100 patches. I reached out to Supermicro but the situation of the D1500 was not clear yet.

1. No.
2. 6TB disks are all three for FreeNAS. VM runs from SSD in the M.2 slots.
3. Thanks. Any recommendations please?
4. Is passthrough of the HBA to FreeNAS not the recommended solution? The SoC for VMWare and a dedicated one for FreeNAS?
Thanks!

 

[Inxsible]

It is a hard choice to make. Without spectre/meltdown, the choice would be easier. Intel guarantees D2100 patches. I reached out to Supermicro but the situation of the D1500 was not clear yet.

Spectre/meltdown do make things difficult to choose. But if cost is no concern, then sure go for the latest. In fact, if you can wait some time, Intel is going to release the new generation later this year with the hardware fixes for Spectre variant 2 and Meltdown variant 3. You'd still need patches for other variants, if I am not mistaken.

Thanks. Any recommendations please?

That will depend on Q1 & Q2.
Will your case support more than 3 drives? Do you see yourself adding more drives if you case supports more than 3 drives? Once you have the answers to those questions, you can look up the PSU calculators to give you a ballpark. You would also need to give some room in the PSU so that it is not overloaded during peak usage.

 

[Inxsible]

I should have said this before, but if it were me, I'd keep pfSense on a separate box altogether. No point in having your firewall turned off in case there is an issue with your machine or you bring it down for maintenance.

I run pfSense 2.4.2 on a separate box as my router/firewall too.

 

[KMaes]

Thanks for the tip. I had planned to keep the old firewall in cold standby.

 

[KMaes]

Spectre/meltdown do make things difficult to choose. But if cost is no concern, then sure go for the latest. In fact, if you can wait some time, Intel is going to release the new generation later this year with the hardware fixes for Spectre variant 2 and Meltdown variant 3. You'd still need patches for other variants, if I am not mistaken.

I probably have to buy in early June so I can fix it during the summer holiday. The motherboard will only be available early may I was told

That will depend on Q1 & Q2.
Will your case support more than 3 drives? Do you see yourself adding more drives if you case supports more than 3 drives? Once you have the answers to those questions, you can look up the PSU calculators to give you a ballpark. You would also need to give some room in the PSU so that it is not overloaded during peak usage.

We have today 4TB usable data. I am still reading up on ZFS but in good old Raid5 days, 3 x 6TB would give approx. 12 TB. I don’t see me add additional ‘traditional’ 3,5 5400 disks during the lifespan of this machine.

The reason I prefer the FlexATX version over the miniITX version of the motherboard are the two PCI-e 3.0 slots and the network options. We can always add a 4 x M.2 module PCI-e card, so that is the logical way to expand the VMware storage in the future I think.




Sent from my iPad using Tapatalk

 

[KMaes]

Spectre/meltdown do make things difficult to choose. But if cost is no concern, then sure go for the latest. In fact, if you can wait some time, Intel is going to release the new generation later this year with the hardware fixes for Spectre variant 2 and Meltdown variant 3. You'd still need patches for other variants, if I am not mistaken.


That will depend on Q1 & Q2.
Will your case support more than 3 drives? Do you see yourself adding more drives if you case supports more than 3 drives? Once you have the answers to those questions, you can look up the PSU calculators to give you a ballpark. You would also need to give some room in the PSU so that it is not overloaded during peak usage.

Regarding the HBA : am I right passthrough is the way to go when presenting disks in ESX6.5 to FreeNAS please? This is all very new for me but the energy costs of my current QNAP are crazy so I really want to save some money if feasible.


Sent from my iPad using Tapatalk

 

[Inxsible]

We have today 4TB usable data. I am still reading up on ZFS but in good old Raid5 days, 3 x 6TB would give approx. 12 TB. I don’t see me add additional ‘traditional’ 3,5 5400 disks during the lifespan of this machine.

With a RaidZ1 configuration, you would get the same 3x6TB = 12 TB of space (minus some overhead related to filesystem etc.). Depending on how important your data is, most people tend to go with RAIDZ2, but you'd need at least 4 drives for it which would also give you 12TB. If 4 drives is the max you put in, then you can go with a number of rack mount cases that are available on ebay and such. I have a 1U case - CSE813MTQ-350B which has 4 hot swap drives. But I currently use Fractal Design Node 304 which allows 6 drives.

 

[Stubb]

I'm considering a similar tech refresh and that Supermicro board is on my short list. What did your system end up looking like? Are you happy with it?

 

[bdacasc]

I'm considering a similar tech refresh and that Supermicro board is on my short list. What did your system end up looking like? Are you happy with it?

Meetoo