ESXi and FreeNAS Pass-through Drives

[officialJCReyes]

Hello everyone,

I currently have a Dell R320 that was given to me at work.

CPU: E5-2430
RAM: 24GB ECC RAM
Storage: 1 x 8GB USB for ESXi, 1 x 500GB 850 EVO SSD for VMs, 7 x 4TB Seagate HDDS (not yet added)
Controller: PERC H710; RAID or HBA mode

Aside from adding the drives via PCI passthrough using the PERC as an HBA card, are there any other settings that need to be adjusted in order to ensure that FreeNAS has direct access to the drives. I will be using the FreeNAS VM to host Plex media files which will be sent to another VM on ESXi.

I will also have a dedicated NIC for FreeNAS and one for Plex.

Looking forward to being a part of this community!

 

[HoneyBadger]

The H710 is not an HBA, nor can it act closely enough like one to be considered safe for ZFS use. The proper solution would be to replace it with a different card like the PERC H200/H310, flashed to LSI IT firmware; or a similar model of supported HBA.

 

[officialJCReyes]

Thanks! I thought I had read somewhere that I could do HBA but when I search for that specific model, it turned up that it's not doable. :(

Do I have any options with my current hardware?

 

[HoneyBadger]

Thanks! I thought I had read somewhere that I could do HBA but when I search for that specific model, it turned up that it's not doable. :(

Do I have any options with my current hardware?

The only safe option is "replace the RAID card" I'm afraid.

Cards like the PERC H200/H310 are inexpensive to buy second-hand - if you were given that R320 for free, I'd consider the less than USD$50 needed to get one as money well spent.

 

[officialJCReyes]

The only safe option is "replace the RAID card" I'm afraid.

Cards like the PERC H200/H310 are inexpensive to buy second-hand - if you were given that R320 for free, I'd consider the less than USD$50 needed to get one as money well spent.

So I could replace the onboard controller with the following device? I may even be able to resell the H710 back on eBay.
https://www.ebay.com/itm/Dell-K09CJ...m=392154443310&_trksid=p2047675.c100005.m1851

So after getting this and getting in HBA mode, the rest of the setup should be pretty straightforward with freeNAS?

 

[HoneyBadger]

So I could replace the onboard controller with the following device? I may even be able to resell the H710 back on eBay.
https://www.ebay.com/itm/Dell-K09CJ-0K09CJ-H310-PERC-6GB-S-SAS-RAID-Controller-R320-R420-R520-R620-R720/392154443310?_trkparms=aid=555018&algo=PL.SIM&ao=1&asc=20131003132420&meid=348d47cca87b4f9fb6fadc450b14c31d&pid=100005&rk=4&rkt=12&sd=142881728556&itm=392154443310&_trksid=p2047675.c100005.m1851

So after getting this and getting in HBA mode, the rest of the setup should be pretty straightforward with freeNAS?

Unfortunately the H310 "mini/mono/integrated" style controllers can't be flashed over to the LSI firmware - they become fancy paperweights if you do. You'll need to use a full-size PCIe card model of either the H200 or H310.

But once you've obtained a full-size HBA and flashed it to the LSI firmware; yes, the setup is very straightforward.

 

[officialJCReyes]

Unfortunately the H310 "mini/mono/integrated" style controllers can't be flashed over to the LSI firmware - they become fancy paperweights if you do. You'll need to use a full-size PCIe card model of either the H200 or H310.

But once you've obtained a full-size HBA and flashed it to the LSI firmware; yes, the setup is very straightforward.

That's unfortunate.. I was hoping to keep the PCIe slots open for NICs and a potential TV tuner in the future. Guess I'll have to scrap the TV tuner for now. Thanks for your advice!

I ended up getting this card: https://www.ebay.com/itm/Dell-PERC-...S-RAID-Controller/183500429826?epid=523570514

Are these instructions valid for flashing HBA mode: https://tylermade.net/2017/06/27/ho...-lsi-9211-8i-firmware-hba-for-freenas-unraid/

 

[HoneyBadger]

That's unfortunate.. I was hoping to keep the PCIe slots open for NICs and a potential TV tuner in the future. Guess I'll have to scrap the TV tuner for now. Thanks for your advice!

I ended up getting this card: https://www.ebay.com/itm/Dell-PERC-...S-RAID-Controller/183500429826?epid=523570514

Are these instructions valid for flashing HBA mode: https://tylermade.net/2017/06/27/ho...-lsi-9211-8i-firmware-hba-for-freenas-unraid/

Looking at the instructions those are correct; just make sure to follow them step by step and come back here if anything goes off the rails.

You may also need slightly longer SAS cables as the ones inside your case probably won't reach all the way to the PCIe slot, versus the "mini" card slot where they were connected before. SFF-8087 is the connector name.

 

[officialJCReyes]

Looking at the instructions those are correct; just make sure to follow them step by step and come back here if anything goes off the rails.

You may also need slightly longer SAS cables as the ones inside your case probably won't reach all the way to the PCIe slot, versus the "mini" card slot where they were connected before. SFF-8087 is the connector name.

You read my mind! I noticed I would need longer cables but I couldn't find any on eBay. Thanks for the connector name. Knowing that I found it on Newegg and then searched eBay. Not that expensive!

Thanks for your help, I'll be sure to come back if anything else comes up during my install. Everything should be arriving mid next week, hopefully before Thanksgiving.

 

[Scharbag]

This is a good writeup for virtualizing FreeNAS on ESXi.

Cheers,

 

[officialJCReyes]

This is a good writeup for virtualizing FreeNAS on ESXi.

Cheers,

Great, thanks!
Out of curiosity, why do you have a separate NIC for storage on FreeNAS?

 

[Scharbag]

Great, thanks!
Out of curiosity, why do you have a separate NIC for storage on FreeNAS?

It is a virtual network - not a physical NIC. That way the storage network can use an MTU of 9000 for jumbo frames etc. It is all detailed in the link that I sent. That guy did a nice job of the writeup. He has a version that describes ESXi 6.5 with Coral too. Between those 2 write-ups, you should be good to go.

Cheers,

 

[officialJCReyes]

Unfortunately the H310 "mini/mono/integrated" style controllers can't be flashed over to the LSI firmware - they become fancy paperweights if you do. You'll need to use a full-size PCIe card model of either the H200 or H310.

But once you've obtained a full-size HBA and flashed it to the LSI firmware; yes, the setup is very straightforward.

Hey man, I just wanted to thank you again for your help on this. I got the necessary components and now I'm working on getting the drives. I was reading a post saying how the drive that will be storing the ESXi VMs should not be connected to the same controller that will be passed through to FreeNAS. I'm currently using a 500 GB SSD for that. How do I go about connecting that drive so that it is separate from the LSA controller, if that is even necessary?

Once again, I'm grateful for your advice!

 

[Scharbag]

Hey man, I just wanted to thank you again for your help on this. I got the necessary components and now I'm working on getting the drives. I was reading a post saying how the drive that will be storing the ESXi VMs should not be connected to the same controller that will be passed through to FreeNAS. I'm currently using a 500 GB SSD for that. How do I go about connecting that drive so that it is separate from the LSA controller, if that is even necessary?

Once again, I'm grateful for your advice!

You should connect your ESXI local storage to your motherboards on-board SATA ports. Then connect all of your FreeNAS drives to the HBA. Once you pass-through the HBA, ESXI will not have access to any drives connected to the HBA, only FreeNAS will be able to access them. Which is what you want.

ESXI really does not need much local storage, only enough to run FreeNAS. Once FreeNAS is running, you can use NFS or ISCSI to present storage for ESXI to use to host more machines. I have 2@250GB SSDs that I use with ESXI to launch FreeNAS and pfSense. All of my other VMs (lots) are running on NFS shares hosted by my FreeNAS system.

Hope that helps.

Cheers,

 

[HoneyBadger]

Hey man, I just wanted to thank you again for your help on this. I got the necessary components and now I'm working on getting the drives. I was reading a post saying how the drive that will be storing the ESXi VMs should not be connected to the same controller that will be passed through to FreeNAS. I'm currently using a 500 GB SSD for that. How do I go about connecting that drive so that it is separate from the LSA controller, if that is even necessary?

Once again, I'm grateful for your advice!

Unfortunately the backplane of the R320 is set up in such a manner that you can't connect just one bay to a separate controller. You'll have to find a place to rig up the SSD internally, or use an adapter for the optical drive (does yours have one?) to mount it.

But right now I'm looking at the 2.5" form factor of the R320's drive bays and noting that you wanted 4TB drives. I'm assuming you're thinking of the 4TB Seagate drives. Be careful. You'll want to specifically seek out the older 15mm ST4000LM0016 (aka the Samsung Spinpoint M10P) rather than the newer ST4000LM024. The former drive will be a bit slow due to having only 16MB of cache [1] but the latter is SMR (shingled) recording, so it will be unsuitable for RAID use.

1: https://www.servethehome.com/seagate-4tb-2-5-sata-hdd-124-get-ready-density/

 

[officialJCReyes]

Unfortunately the backplane of the R320 is set up in such a manner that you can't connect just one bay to a separate controller. You'll have to find a place to rig up the SSD internally, or use an adapter for the optical drive (does yours have one?) to mount it.

But right now I'm looking at the 2.5" form factor of the R320's drive bays and noting that you wanted 4TB drives. I'm assuming you're thinking of the 4TB Seagate drives. Be careful. You'll want to specifically seek out the older 15mm ST4000LM0016 (aka the Samsung Spinpoint M10P) rather than the newer ST4000LM024. The former drive will be a bit slow due to having only 16MB of cache [1] but the latter is SMR (shingled) recording, so it will be unsuitable for RAID use.

1: https://www.servethehome.com/seagate-4tb-2-5-sata-hdd-124-get-ready-density/

Thank you! I do have an ODD but I haven't had luck finding out how to connect mount an SSD there or power it/connect to the board. I appreciate the insight on the drives. I was unaware that there were two different versions and that one wasn't fit for RAID.

EDIT: I just found an adapter I can use for this purpose.

 

[officialJCReyes]

You should connect your ESXI local storage to your motherboards on-board SATA ports. Then connect all of your FreeNAS drives to the HBA. Once you pass-through the HBA, ESXI will not have access to any drives connected to the HBA, only FreeNAS will be able to access them. Which is what you want.

ESXI really does not need much local storage, only enough to run FreeNAS. Once FreeNAS is running, you can use NFS or ISCSI to present storage for ESXI to use to host more machines. I have 2@250GB SSDs that I use with ESXI to launch FreeNAS and pfSense. All of my other VMs (lots) are running on NFS shares hosted by my FreeNAS system.

Hope that helps.

Cheers,

Hey man,
Thanks for the info. The reason I have a 500 GB SSD is that I have 2 Plex VMs running, a test AD server VM and a PiHole/HomeBridge VM. I'd rather not have them depend on FreeNAS to load the VMs. I actually just found an optical disk drive to 2.5 adapter so I think I'll end up replacing the DVD drive with that.