SOLVED Help with SATA Controller Passthrough from ESXi 6.5 to FreeNAS

[toadman]

Glad you got it solved. :)

 

[jgreco]

I understand that virtualized FreeNAS is not an overly popular topic and I am stretching my skill set trying to pull this off, but I feel like I’m pretty close and I’m hoping that someone might be able to chime in with an idea (or tell me where I’ve gone terribly wrong). Thank you in advance for reading my post and for any assistance you can offer.

Well, it really is kind of a popular topic, and I'm happy to see a number of others stepped up while I was gone. The X8DTU PCH SCU (and PCIe passthru on 5500/5600 boards in general) are on my list of "this isn't expected to work." Basically support before the X9 series for PCIe passthru is dodgy at best, and even within the X9 boards, there have been some that don't seem to work quite right.

 

[ralphvanos]

Hi Dennis,

I got the same problem. I have almost the exact same hardware, same mb. The remark was made "Make sure your controller is set to ACHI in your system's BIOS", this solved it for me. Instead of showing a 4 port and a 2 port SATA adapter in ESXi, it now shows up as a 6 port adapter (ICH10 6 port SATA AHCI Controller). If a pass this to FREENAS all the harddrives are available

Reg. Ralph

 

[djwarwick]

Ralph,

That's interesting, I wonder if you have a different BIOS version on your machine, can you provide me with version number you are running? Mine was and always had been set to ACHI mode yet it continued to present as two separate controllers in ESXi. Even though I have the issue resolved with the HBA card, I might go back in and toggle some settings to see if it presents itself differently in the dashboard.

As a side note I'm somewhat glad that I didn't go that route because it would have eliminated the ability to use the DVD drive on the other VMs but then again it would have saved me a hundred bucks so I might have settled for that as a solution at the time!

Either way, thanks for chiming in with your results.

 

[ralphvanos]

Hi Dennis,

If have v02.67

Ralph

 

[AVSION]

Hi Guys, i have the same problem cant passthrough AHCI Controllers, i have X10SRI -F with 8 x HGST 4T one intel ssd run ESXi (see all my system below). in the bios total i have two groups SATA and eSATA which i can see that are showing plugged in, in the ESXi i have two Wellsburg AHCI Controllers greyed out which i assume are my bios SATA groups. both groups where already enabled before i installed the ESXi 6.7 and FreeNAS 11.2 BETA2 VM. in the ESXi CLI passthrou.map i added the AHCI IDs and rebooted, but it's not working still greyed out. is there any workaround to fix it for now with my current hardware as its my production machine, sorry i couldn't wait :) . i plan to get better SSD, NICs and if necessary PCI cards for the HDD soon.

also attached screenshots

Thank you in advance

Code:

# CUSTOMS

# Intel Wellsburg AHCI Controller

8086 8d02 d3d0 false

8086 8d62 d3d0 false

		

# /CUSTOMS

											

# Intel 82579LM Gig NIC can be reset with d3d0

8086  1502  d3d0	 default					

# Intel 82598 10Gig cards can be reset with d3d0

8086  10b6  d3d0	 default

8086  10c6  d3d0	 default

8086  10c7  d3d0	 default

8086  10c8  d3d0	 default

8086  10dd  d3d0	 default							

# Broadcom 57710/57711/57712 10Gig cards are not shareable

14e4  164e  default  false

14e4  164f  default  false

14e4  1650  default  false

14e4  1662  link	 false			

# Qlogic 8Gb FC card can not be shared

1077  2532  default  false		 

# LSILogic 1068 based SAS controllers

1000  0056  d3d0	 default

1000  0058  d3d0	 default

# NVIDIA				

10de  ffff  bridge   false

FreeNAS Shows


[root@freenas ~]# camcontrol devlis
<VMware Virtual disk 2.0> at scbus2 target 0 lun 0 (pass0,da0)
<SanDisk Cruzer Glide 1.00> at scbus4 target 0 lun 0 (pass1,da1)


 

[taaangy234]

Been reading this thread and many others said this controller has issues.
I didn't see anyone mention about enable DirectPath I/O vt-d in the BIOS along with the SATA in AHCI mode.
My board is an Intel S5520HC https://ark.intel.com/products/36599/Intel-Server-Board-S5520HC
I didn't give up, almost pulled the trigger and order the IBM Sata Controller.
I forgot I have a Synology DS214 laying around so I used the Synology NAS as my ESXI 6.7 datastore following the article below.
https://www.synology.com/en-us/know..._use_the_iSCSI_Target_service_on_Synology_NAS
https://www.synology.com/en-us/know...et_up_Synology_NAS_as_VMware_server_datastore

I was running the legacy FreeNAS 11.1-U6.3 on the bare metal box, exported the config.
Made a new VM on ESXI and install FreeNAS-11.2-RELEASE-U1 then import the config.

Everything came up as expected after I update the DHCP reservation for the new VM to get the same IP.
I'm running 2x3TB WD RED and 2x3TB HGST Ultrastar refurbished
So far it ran much smoother and easier to manage as my previous bare metal box is not on ECC ram which is a bit on my nerve.

My next project is getting a bigger UPS and configure everything to shut off properly when I get a power outage.

 

[toadman]

Glad you got it working.

I didn't see anyone mention about enable DirectPath I/O vt-d in the BIOS along with the SATA in AHCI mode.

I'm guessing no one mentioned it because pass through requires VT-d, so I'm sure everyone assumed it was enabled from the get go for anyone attempting pass through.

 

[phier]

hello,
i am running the latest esxi v7, supermicro x11ssl-f ;
sata mode in AHCI;
bios v2.7
vt-d enabled (within chipset configuration->System Agent configuration)
intel pentium g4400


But the sata controller is still grey out ... any idea why?

thanks

 

[joeschmuck]

@phier Did you ever solve this? If not, have you seen this? I have not tried it but just maybe it will work? Use at your own risk, and better is to do a little research on VMkernel.Boot.disableACSCheck to see what it really does, but several forums have discussed this to enable PCI passthrough for some cards.

- Toggle passthrough for your device to on (it should say require reboot)

- Put ESXi into maintenance mode

- Change Manage->Advance Settings->VMkernel.Boot.disableACSCheck from false to true

- Reboot and exit maintenance mode

 

[phier]

@joeschmuck havent tried that... how safe is that?
this is what i did - https://www.truenas.com/community/t...cro-from-esxi-7-to-truenas.102386/post-704490

and i am thinking to rather buy HBA controller... might be then 100% stable i hope?

 

[joeschmuck]

@joeschmuck havent tried that... how safe is that?
this is what i did - https://www.truenas.com/community/t...cro-from-esxi-7-to-truenas.102386/post-704490

and i am thinking to rather buy HBA controller... might be then 100% stable i hope?

I have never done it myself hence the use at your own risk and do a little research.

I read the link you provided. Here is my honest advice if this is just a home system and you are making periodic backups of your important data. Let the system run if it's working. Don't break it. If this were for my job, work system, then I'd pour the money needed into it to make the best reliable system I could, but if this is just a home system and you don't want to pour big money into it and the system appears to be working fine, then don't mess with it.

If you really want to buy a heat producing HBA to support the NAS, that is fine and there is no one going to tell you to stop. Maybe it's just for peace of mind or you need more speed (which doesn't always happen).

So to summarize: If this is a home system and it's working...
1) Let it run.
2) Ensure you are running frequent hard drive test via SMART (GUI). I like a daily Short test and a weekly Long/Extended test.
3) Monitor your Scrubs (default is ~30 days)
4) If you have any stability problems, it might not be the passthrough, could just be your hardware, BUT pay attention.

I've been running ESXi for a very long time now, I'm thinking about 8 years and FreeNAS/TrueNAS on top of that. I do not have the traditional HBA advertised here, see my hardware.

 

[phier]

I have never done it myself hence the use at your own risk and do a little research.

I read the link you provided. Here is my honest advice if this is just a home system and you are making periodic backups of your important data. Let the system run if it's working. Don't break it. If this were for my job, work system, then I'd pour the money needed into it to make the best reliable system I could, but if this is just a home system and you don't want to pour big money into it and the system appears to be working fine, then don't mess with it.

If you really want to buy a heat producing HBA to support the NAS, that is fine and there is no one going to tell you to stop. Maybe it's just for peace of mind or you need more speed (which doesn't always happen).

So to summarize: If this is a home system and it's working...
1) Let it run.
2) Ensure you are running frequent hard drive test via SMART (GUI). I like a daily Short test and a weekly Long/Extended test.
3) Monitor your Scrubs (default is ~30 days)
4) If you have any stability problems, it might not be the passthrough, could just be your hardware, BUT pay attention.

I've been running ESXi for a very long time now, I'm thinking about 8 years and FreeNAS/TrueNAS on top of that. I do not have the traditional HBA advertised here, see my hardware.

@joeschmuck thank, understood.

as hba costs ~50-100e it should be fine ... i mean other parts already reached 1000e.

2) Ensure you are running frequent hard drive test via SMART (GUI). I like a daily Short test and a weekly Long/Extended test.

i assume the results have to be evaluated by a human all the time?

4) If you have any stability problems, it might not be the passthrough, could just be your hardware, BUT pay attention.

what do u mean by stability problems?


i am still not sure whats going on with my setup.... is it possible that my supermicro is completely incompatible with the ESXi? I cant even passthrough usb controller... ;/

thanks

 

[joeschmuck]

i assume the results have to be evaluated by a human all the time?

Non and Yes. TrueNAS automatically evaluates SMART test results and will report to your email if you have any issues, but you need to have setup TrueNAS correctly to get this done. It's not difficult, just part of the original setup. Sometimes you can have errors that are not tragic that are flagged, but most times tragic errors require immediate attention, and sometime interpretation. For example: You have a drive that has 2 pending sector errors, no other errors and will pass an Extended SMART test. I would not reject that drive but I'd keep an eye on it. Or you have a lot of MultiZone issues with no other errors, same thing, keep an eye on it. Some drives will throw MulitZone Errors and it's harmless, some drives it will be your indication that a drive is absolutely failing.

Whenever in doubt, run a SMART Extended test immediately on the suspect drive. if it fails, the drive has surface or head damage in most cases.

Along with others, I have developed a script to generate a detailed report of your drives and it will flag problems. My next revision of the script is due to come one in the next few days, well so long as no more minor issues crop up during testing. I get an email everyday providing me the drives status and the Subject will tell you "Okay", "Warning", or "Critical". Pretty easy, and the message body will provide detailed information on each drive. You can also perform statistical analysis for those people who love to play with numbers.

what do u mean by stability problems?

The system if not stable can crash. For example: You build a system and when you run into a heavy workload the system periodically crashes and stop working, but works fine for light loads. Odds are you have a lack of cooling related issue. So we recommend that yo do a burn-in test where you would saturate the system with heat by running the CPU and motherboard components to their peak for a period of time (could be 1 hour or 1 day, your choice).

i am still not sure whats going on with my setup.... is it possible that my supermicro is completely incompatible with the ESXi? I cant even passthrough usb controller... ;/

Odds are you cannot pass through the USB controller because the motherboard has control over it. What I've found out is that ESXi want to be in full control of the component, in this case it would be the USB controller. I run ESXi on my Supermicro x11-SSM without issue. I can passthrough (well RDM) hard drives or/and I can passthough add-on boards. If you want to passthough the USB controller, I believe you will need to go into the BIOS and disable a USB controller, this should make it available for ESXi to take control of it.