FreeNAS build proposal (formerly ESXi all-in-one build proposal)

[baummer]

After weeks of research, and some playing around in a FreenNAS 9.10 VM, here is my proposed build for an ESXi all-in-one FreeNAS server that will function as a file backup server and Plex media server (one user only), as well as host a few VMs for web development purposes:

• CPU: Intel Xeon E3-1230v5 3.40GHz
• Motherboard: Supermicro MBD-X11SSM-F-O
• RAM: 2x16GB Crucial ECC RAM
• Storage:
  • 2x16GB Sandisk Cruzer Fit USB (FreeNAS boot drive mirroed)
  • 2x160GB Intel DC3500 SSD (as a datastore for FreeNAS mirrored installation)
  • 6x3TB WD Red (RAIDZ2 configuration for use in FreeNAS pool)
• HBA: IBM Serveraid M1015 (flashed to IT mode)
• PSU: ??? - gold rated, unsure
• Case: Fractal Node 804

Some questions:

1. I do need an HBA, right? Assuming yes:
   • The Sandisk Cruzer Fit USB boot drive - I assume this should be connected directly to the motherboard and not the HBA?
   • Should the rest of the storage (6x3TB WD Red and 2x160GB Intel DC3500 SSD) connect to the HBA or should some of it connect straight to the motherboard?
2. By my wattage calculation, I believe I can get by with a 300W or higher PSU. Do you agree? If so, any PSU brands to avoid?
3. Do I need a SLOG? If so, would adding another 160GB Intel DC3500 SSD work for that purpose? I've read conflicting information on this and would appreciate guidance here.
4. Anything else I am missing?

Appreciate your thoughts!

 

[Ericloewe]

The Sandisk Cruzer Fit USB boot drive - I assume this should be connected directly to the motherboard and not the HBA?

You'll find that the good people at the SATA-IO, INCITS and USB IF deliberately chose not to use the same connectors for both SAS and USB, so you'd have a hard time doing what you're suggesting shouldn't be done.

Should the rest of the storage (6x3TB WD Red and 2x160GB Intel DC3500 SSD) connect to the HBA or should some of it connect straight to the motherboard?

You'll be passing through the HBA to FreeNAS. The hypervisor will not be able to access it.

By my wattage calculation, I believe I can get by with a 300W PSU. Do you agree?

nope.avi
300W is woefully underpowered. Besides, there are exactly zero good units in that power range.
You should be looking at 450W minimum, 550W if you want to expand in the future.

If so, any PSU brands to avoid?

https://forums.freenas.org/index.php?threads/proper-power-supply-sizing-guidance.38811/

Do I need a SLOG?

If you need to ask, let's go with "no", for now.

 

[golfleep]

The datastore where the FreeNAS VM will be kept needs to accessible to ESXi. Because you will be passing the HBA to the VM, the HBA ports will not be visible to the hypervisor, so the DC3500's will need to be connected to the motherboard SATA ports. The red's connect to the HBA, as well as a SLOG if you choose to use one.

As for the the SLOG, it depends on whether or not you're planning to use your FreeNAS as a datastore for ESXi via NFS (or iSCSI if sync is enabled). A FreeNAS based datastore can be extremely slow due to the way ESXi writes to the datastore (synchronous), which a SLOG can help with. Not sure there is much use for a SLOG outside of this scenario, at least from what you've described as your use

 

[baummer]

You'll find that the good people at the SATA-IO, INCITS and USB IF deliberately chose not to use the same connectors for both SAS and USB, so you'd have a hard time doing what you're suggesting shouldn't be done.


You'll be passing through the HBA to FreeNAS. The hypervisor will not be able to access it.


nope.avi
300W is woefully underpowered. Besides, there are exactly zero good units in that power range.
You should be looking at 450W minimum, 550W if you want to expand in the future.

https://forums.freenas.org/index.php?threads/proper-power-supply-sizing-guidance.38811/


If you need to ask, let's go with "no", for now.

Thanks for your feedback. You're right about the USB and HBA - my mistake.

Regarding the HBA, I was led to believe from this blog post that you have to use an HBA so that FreeNAS controls the drives and you allocate VM storage from FreeNAS to ESXi via NFS or ISCSI: https://b3n.org/freenas-9-3-on-vmware-esxi-6-0-guide/. Did I misunderstand that post?

Regarding the PSU, I meant to say 300W and above PSU. Thanks for the link.

The datastore where the FreeNAS VM will be kept needs to accessible to ESXi. Because you will be passing the HBA to the VM, the HBA ports will not be visible to the hypervisor, so the DC3500's will need to be connected to the motherboard SATA ports. The red's connect to the HBA, as well as a SLOG if you choose to use one.

As for the the SLOG, it depends on whether or not you're planning to use your FreeNAS as a datastore for ESXi via NFS (or iSCSI if sync is enabled). A FreeNAS based datastore can be extremely slow due to the way ESXi writes to the datastore (synchronous), which a SLOG can help with. Not sure there is much use for a SLOG outside of this scenario, at least from what you've described as your use

Thanks! I got the idea to use FreeNAS to control the disks connected to the HBA from here: https://b3n.org/freenas-9-3-on-vmware-esxi-6-0-guide/. I also saw another post on this board about doing it that way. Am I missing something?

 

[Dice]

Did I misunderstand that post?

There are some really dodgy recommendations in that post, as per forum standards. Enough it would make most even moderately knowledgeable forum users TILT if read pre-morning coffee. Luckily I'm 3 cups deep atm.... ;)
I will say - that blogpost was very helpful (for me) to get some general ideas on what is going on. But should absolutely not be copied.
Here's the key, sent to you with all the best intentions:
Until you can spot the problematic settings and figure out which other are sort of wild settings in that blogpost, you're ...not yet ready to virtualize.

Wish you the best of luck to proceed on learning :)

 

[baummer]

There are some really dodgy recommendations in that post, as per forum standards. Enough it would make most even moderately knowledgeable forum users TILT if read pre-morning coffee. Luckily I'm 3 cups deep atm.... ;)
I will say - that blogpost was very helpful (for me) to get some general ideas on what is going on. But should absolutely not be copied.
Here's the key, sent to you with all the best intentions:
Until you can spot the problematic settings and figure out which other are sort of wild settings in that blogpost, you're ...not yet ready to virtualize.

Wish you the best of luck to proceed on learning :)

Thanks for your candor! Let's remove ESXi from the equation altogether then and focus on just a FreeNAS server:

1. Would I still need an HBA?
2. Would you recommend a mirrored FreeNAS boot installation? Is this typically done with two USB drives (since from what I've gathered, FreeNAS doesn't need more than 8GB of space) if so?
3. Any other changes to my proposed build?

Interested in thoughts from @golfleep and @Ericloewe as well regarding this change.

 

[Dice]

1. If you lack enough SATA ports, then yes. Preferably 1 free, to use when the inevitable happens and a new drive needs to be replaced without compromising redundancy.
2. If USB, yes. If SSD, no.

 

[baummer]

1. If you lack enough SATA ports, then yes. Preferably 1 free, to use when the inevitable happens and a new drive needs to be replaced without compromising redundancy.
2. If USB, yes. If SSD, no.

Curious, why don't you recommend a mirrored installation when using an SSD? Wouldn't anything beyond 8GB go unused on the SSD?

 

[Dice]

Reliability is sufficient with a SSD.

Wouldn't anything beyond 8GB go unused on the SSD?

correct.

It is tempting to get a cheap bang for the buck 120GB or 250GB drive, yet ....any 30GB would do just fine.

 

[Ericloewe]

Would I still need an HBA?

Only if you need more drives.

Would you recommend a mirrored FreeNAS boot installation?

Yes.

 

[Benjamin Bryan]

There are some really dodgy recommendations in that post, as per forum standards. Enough it would make most even moderately knowledgeable forum users TILT if read pre-morning coffee. Luckily I'm 3 cups deep atm.... ;)
I will say - that blogpost was very helpful (for me) to get some general ideas on what is going on. But should absolutely not be copied.
Here's the key, sent to you with all the best intentions:
Until you can spot the problematic settings and figure out which other are sort of wild settings in that blogpost, you're ...not yet ready to virtualize.

Wish you the best of luck to proceed on learning :)

Glad you found it useful, just curious, which parts did you find wild and dodgy?

 

[garym]

Glad you found it useful, just curious, which parts did you find wild and dodgy?

I also would like to hear the answer, for it is so easy to make a blanket statement such as that. I am suspecting that this is part of the keep Freenas pure, ban ESXi, sentiment I am seeing from some members in this forum. If one is seeing something that is wrong, spell it out! and prove me wrong.